Design Netflix

This case study simulates a realistic FAANG system design interview for architecting a global, high-scale video-on-demand streaming service like Netflix, Prime Video, or Disney+. It details the separation of the control plane (management) and data plane (byte delivery), the parallel transcoding pipeline, and ISP-integrated CDNs.

1. Present the Interview Question

2. Clarifying Questions

The candidate clarifies the system metrics and streaming scope:

• ‍Candidate: What is the target active user base, and what is the peak concurrent streaming volume?
  Interviewer: We have 200 million registered users. At peak hours, we must support up to 15 million concurrent streams.
• ‍Candidate: What video quality tiers do we support?
  Interviewer: We support standard SD (480p), HD (1080p), and 4K Ultra HD resolutions.
• ‍Candidate: How large is the movie/show catalogue, and what is the typical ingestion rate?
  Interviewer: The catalogue contains ~10,000 titles. Ingestion is low-volume: only a few dozen new titles are uploaded per day by internal studio staff.
• ‍Candidate: Do we need to support user profile browsing and watch history sync?
  Interviewer: Yes. Synchronizing watch progress (e.g. "Resume Playback" across devices) is a core feature.

3. Functional Requirements

• Video Ingestion & Transcoding: Accept raw master media files, split them, and encode them into multiple bitrates and formats.
• Adaptive Bitrate Streaming: Serve video segments dynamically matching the client's network speed.
• Catalog Browsing: Allow users to browse movies/shows by categories, recommendations, and search metadata.
• Playback Progress Synchronization: Store and sync watch progress markers (last offset watched) across devices.

4. Non-Functional Requirements

• Minimal Buffering Latency: Playback must start in under 2 seconds.
• High Video Quality: Seamless transition between resolution qualities without stopping stream buffers.
• High Uptime: Catalog and streaming initiation services must target 99.99% availability.
• High Durability: Master uploads must be stored with 99.999999999% durability.

5. Capacity Estimation

1. Network Bandwidth (Peak Egress)

• Concurrent Streamers: 15 Million users.
• Assume average streaming bitrate (1080p stream) is 5 Megabits per second (Mbps).
• Peak Network Bandwidth Required: 15M * 5 Mbps = 75 Terabits per second (Tbps).
  Implication: No cloud data center has egress lines large enough to handle 75 Tbps. Video bytes must be offloaded to edge CDNs embedded directly inside ISPs.

2. Catalogue Storage

• Assume 10,000 video titles. A single raw master video is ~100 GB.
• Each title is transcoded into 5 resolutions (360p, 480p, 720p, 1080p, 4K) and 3 codecs (H.264, H.265, VP9).
• Total transcoded files per movie = ~100 GB.
• Total Catalogue Storage: 10,000 * 100 GB = 1 Petabyte (PB). (Stored easily in Amazon S3).

3. Watch Progress Sync Storage

Assume 200 Million active users stream 5 titles per day, updating playback offsets every 10 seconds.
Progress update size: user_id (16B), video_id (16B), offset_sec (4B), timestamp (8B) = ~44 bytes.
To avoid database locks, clients buffer offset progress locally and push updates every 10 seconds.
Peak concurrent streams: 15 Million.
Write QPS: 15,000,000 streams / 10 seconds = 1.5 Million write requests/sec (QPS).

6. Core Components

We partition the system into two distinct environments:

• Control Plane (AWS Cloud): Runs catalog metadata search, recommendation engines, authorization tokens, billing, and watch history synchronization (handling the 1.5M QPS progress writes).
• Data Plane (CDN Edge): Handles streaming video bytes directly to users. Uses Netflix Open Connect, an ISP-integrated caching server network.
• Transcoding Pipeline: A parallel, asynchronous system to split, transcode, and package video assets.

7. High-Level Architecture

The High-Level Design maps the separation of Control Plane services from the Data Plane CDN loops:

8. API Design

1. Playback Ingress Initialization

GET /api/v1/playback/init

Request Parameters:

• videoId: "mov_981a29d"
• deviceId: "smart-tv-903"

Response Payload (200 OK):

2. Playback Progress Sync

POST /api/v1/playback/progress

Request Payload:

Response Payload (200 OK):

9. Data Model

To handle 1.5 Million write QPS for watch offsets, we use a wide-column NoSQL schema partitioned by user:

10. Database Selection

‍Candidate:
- Catalog Database: I select Amazon DynamoDB for movie metadata because read access is heavily cached and schema fields (actors, links) are flexible.
- Playback Progress Database: I select Apache Cassandra.
Justification: Cassandra handles the write pressure of 1.5M QPS easily. Writing watch history offset does not require strong consistency or complex transactions. Partitioning by user_id maps all of a user's resume states to the same disk block, making read checks near-instant when the app initializes.

11. Deep Dive: Transcoding & Adaptive Streaming

1. The Transcoding Workflow

In order to support multi-bitrate adaptive streaming, raw movies are processed asynchronously:

1. Studio uploads 100 GB Master raw file (ProRes format) to Amazon S3.
2. Upload completion triggers a notification event to SQS.
3. Chunker Service downloads the video and cuts it into 2-second segments (e.g. chunk_001.mov). Chunks are split on keyframe boundaries.
4. Workers pull chunks from SQS and encode them in parallel across different resolutions (360p, 720p, 1080p, 4K) and formats (H.264, H.265, AV1).
5. Transcoded segments are saved back to S3.
6. Manifest Creator generates a Manifest File (e.g. index.m3u8 for HLS or index.mpd for DASH). The manifest maps chunk URLs to resolutions and bitrates.

2. Adaptive Bitrate Streaming (ABR)

12. Complete Request Lifecycle

1. Playback Initialization: Client opens the app and requests GET /api/v1/playback/init?videoId=123. The API Gateway forwards the request to the Playback Service.
2. Resume Offset Lookup: Playback service queries Cassandra for the user's last offset matching videoId, pulls catalog manifest metadata from Redis, and returns the manifest URL and offset (e.g., 2405s).
3. Geo-DNS Routing: The client reads the manifest URL and resolves the DNS. The DNS uses geo-routing (Anycast DNS) to route the client to the nearest local Internet Service Provider (ISP) network hosting a Netflix Open Connect cache node.
4. Streaming Chunks: The client player connects directly to the Open Connect edge server, requests video chunks starting at offset 2405 seconds, and begins playback.
5. Progress Update: Every 10 seconds of playback, the client pushes the current offset to /api/v1/playback/progress. The gateway routes this to Cassandra asynchronously.

13. Scaling Strategy: Open Connect CDN

To handle peak egress bandwidth (75 Tbps), Netflix avoids commercial CDNs (Akamai, Cloudflare) and builds its own physical hardware delivery network called Open Connect:

• Netflix constructs custom storage and routing boxes (Open Connect Appliances) holding up to 280 TB of flash storage.
• These boxes are shipped and installed directly inside local Internet Service Provider (ISP) racks globally for free.
• ISPs coordinate this because it keeps heavy video bytes localized within their own internal networks, saving them transit bandwidth costs.
• 95% of Netflix traffic is served from these ISP-integrated edge boxes, cutting latency to single-digit milliseconds and bypassing backbone congestion.

14. Bottleneck Analysis

• Transcoding Hotspots: Transcoding a 4K movie sequentially on a single server is extremely slow (hours/days).
  Mitigation: Chunker Service divides raw files into 2-second segments. These segments are distributed across thousands of AWS Spot instances, allowing a 2-hour movie to transcode in under 10 minutes.
• Cache Stampede for Popular Releases: When *Stranger Things* drops, millions of users initiate playback concurrently. Edge CDNs experience cache misses, causing a write storm to AWS master servers.
  Mitigation: Pre-stage popularity content. During off-peak hours (e.g. 2 AM), new releases are pre-stage copied to all Open Connect boxes globally before the release window opens.

15. Trade-off Discussion: DASH vs. Custom UDP Protocols

‍Candidate:
- Custom UDP: Reduces transport latency because UDP does not require connection handshakes or packet acknowledgments. However, UDP packets are often blocked by standard home routers and office firewalls. Building custom recovery, congestion control, and browser players is highly complex.
- DASH/HLS over HTTP/TCP: Runs over standard TCP port 80/443, ensuring it is never blocked by firewalls. It allows us to use standard HTTP proxies and cache nodes. Because video playback relies on pre-buffering (downloading chunks 30 seconds in advance), transport-level latency spikes do not impact playback smoothness.
Decision: The convenience and firewall traversal of DASH/HLS outweigh the latency benefits of UDP, making standard HTTP streaming the ideal production choice.

16. Failure Scenarios

How the control plane handles failures:

• Chaos Engineering (Chaos Monkey): Netflix runs automated daemons in production that randomly terminate microservice instances. This forces teams to build resilient architectures:
  - Services must default to graceful degradation (if the profile avatar service fails, return a default avatar, do not crash page loading).
  - Use Circuit Breakers to fail fast when dependency latency spikes.
• Regional AWS Outage: If US-East-1 goes offline, catalog browsing must stay online.
  Mitigation: Multi-Region Active-Active deployment. Traffic is redirected to US-West-2, reading database state replicated via Cassandra.

17. Security Design

• Digital Rights Management (DRM): Integrate licensing servers (Widevine for Android/Chrome, FairPlay for Apple, PlayReady for Windows) to decrypt video keys in secure hardware enclaves on devices.
• Secure Playback Tokens: CDN links are signed URLs containing the client IP, video ID, and expiration timestamp to prevent link sharing.

18. Monitoring & Observability

• SPS (Stream Starts Per Second): The primary high-level health metric. A drop in SPS indicates a connection or payment gateway outage.
• Rebuffer Rate: The percentage of playback sessions that experience stalls. Alert if this spikes above 0.5%.

19. Cost Optimization

Transcoding is highly CPU-intensive. To optimize costs:
1. Use AWS Spot Instances (spare AWS compute sold at a 90% discount). Since transcoding is chunk-based and stateless, if AWS terminates a spot instance, a worker node simply pulls that chunk from SQS and resumes processing on another node.
2. Apply Dynamic Bitrate Allocation (dynamic encoding). Simple scenes (e.g. news show talking head) are encoded at lower bitrates, while complex scenes (e.g. forest scenes with high movement) are encoded at higher bitrates, reducing storage costs by up to 20%.

20. Production Improvements

To optimize streaming quality in remote regions: support AV1 Codec encoding. AV1 provides up to 30% better compression than H.265/HEVC at the same visual quality, allowing HD streaming on limited cellular connections.

21. Interviewer Follow-up Questions

‍Candidate: We populate the boxes during off-peak hours (e.g. 2 AM to 5 AM local time):
1. The CDN controller predicts content popularity per region based on viewing history.
2. During off-peak windows, the controller pushes popular movies and new releases to the local ISP boxes.
3. This avoids saturating the ISP's network during peak viewing hours.

22. Final Architecture

The final optimized setup showing the transcoding pipeline, control plane, and Open Connect edge delivery:

23. Summary

We designed Netflix by decoupling the Control Plane (AWS-based metadata & watch sync) from the Data Plane (Open Connect CDN). Peak egress bandwidth (75 Tbps) is managed by embedding caching boxes directly inside ISP server racks globally. Raw videos are transcoded in parallel using SQS queues and spot instances, generating manifests for adaptive bitrate HLS/DASH streaming.

24. Cheat Sheet

25. Candidate Interview Evaluation