Stackbook Logo
← Back to System Architectures

Designing a Video Streaming Platform

Design a video streaming platform with upload, transcoding to adaptive bitrate ladders, CDN delivery and a metadata service for 200M daily viewers.

4 min read

A video platform ingests creator uploads, converts them to formats that play on any device and network, and delivers them globally. Storage, transcoding compute and bandwidth dominate cost, so the design revolves around the CDN and an efficient transcoding pipeline.

Requirements

Functional

  • Upload videos up to several GB; resumable.
  • Playback on web, mobile and TV with adaptive quality.
  • Search and browse by metadata; view counts and likes.

Non-functional

  • 200M daily viewers, 5M new videos per day.
  • Playback start under 2 seconds at p95; rebuffer rate under 1 percent.
  • Uploaded video available for playback within minutes.
  • Durable storage: no uploaded original is lost.

Capacity estimation

  • Uploads: 5M per day, average 500 MB original, is 2.5 PB per day of raw ingest.
  • Transcoded renditions add about 1.5x, so roughly 6 PB per day total storage growth; cold-tiering and dropping unpopular renditions are essential.
  • Egress: 200M viewers watching 40 minutes at an average 3 Mbps is about 0.9 GB each, so roughly 180 PB per day, which is only feasible through CDN edges and ISP caches. Origin egress should stay under 5 percent of this.
  • Playback requests: manifests and segments of 4 seconds each give very high request rates, which suits CDNs and not origin servers.

High-level design

Creator -> Upload Service -> Object store (raw, chunked upload)
                                  |
                                  v
                      Job queue -> Transcoding workers (DAG)
                                  |
                       segments + manifests (HLS/DASH)
                                  v
                           Origin object store
                                  |
Viewer <- CDN edge <- regional cache <- origin shield

Metadata Service (SQL/NoSQL) <-> Search index; View/like counters via stream

Data model

videos(video_id PK, owner_id, title, description, status, duration, created_at)
renditions(video_id, profile, codec, bitrate, manifest_path)
view_events (stream) -> aggregated counters(video_id, views, likes)

Metadata is small and read-heavy, so a replicated relational or document store with a cache works. Media bytes never go through the database.

Deep dives

Upload. Clients upload in chunks directly to object storage using pre-signed URLs, so application servers do not carry the bytes. Chunks are resumable; on completion an event triggers processing. Checksums verify integrity.

Transcoding. Split the source into short groups of pictures (GOP-aligned chunks of a few seconds). A DAG of tasks runs in parallel across a worker fleet: decode, encode multiple renditions (for example 240p to 4K, in H.264 and VP9 or AV1), audio, thumbnails, then package. Parallel chunking turns hours of encoding into minutes. Encode the cheapest ladder first so the video is playable quickly and add higher quality later.

Adaptive bitrate streaming. Output is HLS or DASH: a manifest listing renditions plus 2 to 6 second segments. The player measures throughput and buffer, and switches rendition per segment. Short segments adapt faster but increase request counts.

CDN strategy. Popular videos follow a power law. Push or pre-warm new releases from known creators, and use pull-through caching for the long tail. A tiered cache (edge, regional, origin shield) collapses misses so the origin sees each segment once. Long-tail content that is rarely watched is served from the origin and can use cheaper storage classes.

DRM and access control. Premium content is encrypted with keys served by a license server. Signed URLs with short expiry prevent hotlinking.

View counts. Emit view events to a stream and aggregate in micro-batches. Display approximate counts and reconcile later, because exact real-time counts are costly.

Failure modes and trade-offs

  • Transcode worker failure: tasks are idempotent and retried per chunk, so a crash re-does seconds of work, not the whole video.
  • Poison videos: corrupt input can crash encoders. Limit retries, then mark failed and notify the creator.
  • CDN edge outage: DNS or anycast routing steers to another edge, with multi-CDN as a further safeguard.
  • Storage cost versus latency: more renditions improve quality but cost storage and compute. Generate expensive renditions (4K, AV1) only after a video passes a view threshold.
  • Thundering herd on a live or viral launch: pre-warm the cache and let the shield coalesce requests.

Key takeaways

  • Bandwidth, not compute, is the dominant cost, so the CDN decides economics.
  • Upload directly to object storage and process asynchronously.
  • Chunked, parallel transcoding makes processing fast and retries cheap.
  • Adaptive bitrate with HLS or DASH adapts to each viewer's network.
  • Defer expensive renditions until a video proves popular.