AWS Live Subtitling Solution Reduces Streaming Latency

AWS Live Multilingual Subtitling

Executive Summary

• AWS described a reference solution for real-time multilingual subtitles for live events designed to avoid adding video streaming latency.
• The architecture separates video and subtitle processing into parallel paths and brings them together at the content delivery layer.
• The workflow uses WebVTT subtitle chunks and standard streaming formats (HLS and DASH), with subtitle segments stored and retrieved using timestamped records.
• An open source implementation is provided on GitHub, with deployment guidance and customization options.

Key Industry Developments

• Parallelized live workflow for subtitles and video
• The solution is organized as two concurrent processing paths: a video delivery path and a subtitle delivery path, designed to converge at the content delivery layer rather than inside the live encoding pipeline.
• The video path uses AWS Elemental services for live transport, encoding, and packaging, including AWS Elemental MediaConnect, AWS Elemental MediaLive, and AWS Elemental MediaPackage, with delivery via Amazon CloudFront.
• WebVTT chunk interception and modification at the edge
• The subtitle delivery path uses WebVTT (VTT) chunks and applies edge logic to intercept and modify those chunks before they reach viewers.
• Lambda@Edge is used as a reverse proxy to insert chunk creation time (timecode) into VTT chunks, enabling downstream synchronization behavior based on time metadata.
• Timestamped subtitle segment storage for synchronization
• Translated subtitle segments are stored in Amazon DynamoDB with precise timestamps, supporting real-time retrieval and synchronization against the live stream timeline.
• The workflow includes sentence-boundary detection in transcribed text prior to translation, with the stated goal of translating complete thoughts rather than word-by-word fragments.
• CloudFront distribution separation for media vs. subtitle processing
• The design creates two Amazon CloudFront distributions: one dedicated to VTT chunk interception/modification and another for traditional media delivery, separating subtitle manipulation concerns from standard video distribution.

Real-World Use Cases

• Live events requiring multilingual subtitles
• The solution targets live events where viewers need accurate subtitles in multiple languages while maintaining synchronization with spoken audio.
• Language switching is intended to work through standard video player controls, aligning with common subtitle track selection workflows.
• Live sports with rapid speech patterns
• For fast-paced commentary, the workflow can use AWS Elemental MediaConnect buffer delay to accumulate longer transcription segments, providing more context for transcription and translation.
• Corporate, news, and entertainment broadcasts
• The described audience includes broadcasters of live sports, corporate events, news, and entertainment content, indicating applicability across multiple live programming categories.

Why It Matters

• Latency management through parallel processing
• By separating subtitle processing from the primary video delivery path and converging at the CDN layer, the design aims to deliver multilingual subtitles without adding video streaming latency, while still aligning subtitles to the live stream.
• Operationally explicit synchronization primitives
• Storing translated subtitle segments in DynamoDB with precise timestamps provides a concrete synchronization mechanism: subtitle retrieval can be keyed to time metadata rather than relying only on client-side heuristics.
• Edge-based subtitle manipulation for WebVTT
• Using Lambda@Edge as a reverse proxy to intercept and modify VTT chunks places subtitle transformation close to viewers and decouples subtitle chunk rewriting from origin services.
• Reusable implementation
• The availability of an open source implementation on GitHub provides a starting point for teams that want to deploy and customize a real-time multilingual subtitling workflow using AWS-managed services.

Sources

• https://aws.amazon.com/blogs/media/scale-global-live-reach-with-aws-powered-real-time-webvtt-multilingual-subtitling/