Design WhatsApp

This case study simulates a realistic FAANG system design interview for designing a massive, real-time messaging application like WhatsApp, Telegram, or Facebook Messenger. It walks through persistent connections, session routing, presence indicators, and group chat scaling challenges.

1. Present the Interview Question

2. Clarifying Questions

The candidate clarifies the system parameters with the interviewer:

• ‍Candidate: What is our target user scale? What is the concurrent connection count?
  Interviewer: We have 1 billion Daily Active Users (DAU). We need to support 100 million concurrent connected users at any given instant.
• ‍Candidate: Are messages stored permanently on our servers?
  Interviewer: No. Chat history is stored locally on the user's mobile device. Once a message is successfully delivered to the recipient, it should be deleted from our servers. If a recipient is offline, we queue the message on our servers until they connect.
• ‍Candidate: What is the limit of group chat membership?
  Interviewer: A group can contain up to 500 members.
• ‍Candidate: Do we need to support End-to-End Encryption (E2EE)?
  Interviewer: Yes. Assume the cryptographic key exchange (Diffie-Hellman) happens on the clients; the server acts as an encrypted blob router.

3. Functional Requirements

• 1:1 Real-time Messaging: Direct user-to-user text delivery with sub-second latency.
• Group Chat: Messages fanned out to up to 500 users.
• Message Status Receipts: Track sent (single tick), delivered (double grey tick), and read (double blue tick) milestones.
• Presence Tracking: Real-time "online" and "last seen" indicators.
• Media Sharing: Support sending photos, videos, and files.

4. Non-Functional Requirements

• Low Latency Delivery: Target end-to-end message delivery under 500ms.
• Zero Message Loss: Delivery must be guaranteed; offline messages must persist reliably until retrieved.
• High Connection Volume: Handle 100 million persistent TCP connections concurrently.
• Privacy: End-to-end encryption ensures servers cannot decrypt content.

5. Capacity Estimation

1. Messages Volume & QPS

• Assume 1 billion DAU send an average of 40 messages per day.
• Total Daily Messages: 1B * 40 = 40 Billion messages/day.
• Average QPS: 40 Billion / 86400 seconds = ~463,000 messages/sec (QPS).
• Peak QPS (2x): ~926,000 messages/sec.

2. Transient Offline Database Storage

Assume 10% of users are offline at any moment, and their messages must be queued. Average offline queue time is 1 day.
Offline message count per day: 40 Billion * 10% = 4 Billion messages queued/day.
Message metadata (sender_id, recipient_id, payload_blob, timestamp): ~300 bytes.
Daily storage needed: 4B * 300 bytes = ~1.2 TB of active SSD capacity.

3. Persistent Connection Servers Sizing

Standard Linux servers can terminate approximately 50,000 concurrent TCP sockets (WebSockets) before hitting file descriptor and memory limits.
Total concurrent connections: 100 Million.
Required Connection Nodes (Gateways): 100,000,000 / 50,000 = 2,000 servers globally.

6. Core Components

• WebSocket Gateways (Chat Servers): Maintain long-running, duplex TCP socket sessions with active clients.
• Session Store (Redis Cluster): Maps active user_id -> gateway_server_ip to locate where to route messages.
• Presence Service: Tracks online status via heartbeats.
• Message Router: A stateless service checking sessions and routing payloads to active gateways or writing them to the transient database.
• Transient Database: Cassandra/DynamoDB cluster queue for offline deliveries.
• Push Notification Gateways: Triggers Apple Push (APNs) or Google Cloud Messaging (FCM) when a recipient is offline.

7. High-Level Architecture

The high-level setup decouples message routing from persistent connection boundaries:

8. API Design

Because communications are duplex, we use WebSocket frame formats rather than standard HTTP requests:

1. Send Message frame (Client to Gateway)

2. Delivery Receipt Status (Gateway to Sender)

9. Data Model

Offline queues must support fast writes and deletions. We index columns to fetch records by recipient:

10. Database Selection

We choose Apache Cassandra for our offline message store.
Justification: Cassandra utilizes a Log-Structured Merge (LSM) Tree storage engine, enabling incredibly fast sequential disk writes. Partitioning by recipient_id ensures that when an offline user reconnects, we can perform a single partition scan query to pull all their waiting messages sequentially.

11. Deep Dive into Core Services

1. WebSocket Session Management

When Client A initiates a WebSocket connection, it connects to a Gateway server.
Upon connection, the Gateway writes a record to the Redis Session Store: session:user_123 -> gateway_server_45 (with a TTL of 1 hour).
To keep the connection alive, the client sends a small ping frame every 30 seconds. This updates the Redis session TTL, ensuring the mapping remains fresh.

2. Presence Management

A user's presence status (Online/Offline/Last Seen) is managed using heartbeats:

• When a client is active, it publishes a heartbeat to the Presence Service every 10 seconds.
• The Presence service updates Redis: presence:user_123 -> "online" with a TTL of 15 seconds.
• If the user shuts the app down, heartbeats stop. After 15 seconds, the Redis key expires.
• When a user's contact list opens their details, the client requests the current presence status from Redis.

12. Request Lifecycle

Happy Path (Recipient B Online)

1. Client A sends message frame to WebSocket Gateway 1 (WG1).
2. WG1 routes the payload to the Message Router.
3. Message Router queries Redis and finds Client B is connected to Gateway 2 (WG2).
4. Message Router forwards the payload to WG2.
5. WG2 pushes the message frame over Client B's persistent WebSocket connection.
6. Client B acknowledges receipt, sending a DELIVERED status frame back to WG2, which routes it to Client A.

Offline Path (Recipient B Offline)

1. Client A sends message frame to WG1.
2. Message Router queries Redis and finds no session mapping for Client B.
3. Message Router writes the message to Cassandra's offline_messages table.
4. Message Router sends a push trigger event to Apple Push (APNs) / Android Push (FCM).
5. Client B's phone displays a notification. When B opens the app, they establish a new WebSocket to WG2.
6. WG2 detects the connection, queries Cassandra for B's partition key, and streams all queued messages. B receives the messages, and their client acknowledges delivery, updating the status.

13. Scaling Strategy

To scale to 100M concurrent users:

• Load Balancing Persistent Sockets: Standard load balancers cannot handle persistent TCP redirection without running out of ports. We deploy layer 4 load balancers (e.g. AWS Network Load Balancer) that route connections using IP consistent hashing directly to Gateway pods.
• Internal Message Routing: We run a Redis Pub/Sub backplane. Every Gateway server subscribes to its own server IP channel. When the Message Router wants to send a message to WG2, it publishes the payload to the channel wg_2_channel, which WG2 intercepts and pushes to the client. This prevents compute nodes from talking to each other directly, decoupling gateways.

14. Bottleneck Analysis: Group Chats Fan-out

‍Candidate: We must avoid blocking the gateway thread during group deliveries. We implement a Group Chat Service integrated with an asynchronous processing pipeline:
1. When Client A sends a message to Group X, WG1 forwards it to the Group Chat Service.
2. The Group Chat Service writes the message to a Kafka queue and returns an immediate acknowledgment back to Client A.
3. A pool of worker instances consumes messages from Kafka, queries the Group Membership Store (hosted on DynamoDB, caching membership list on Redis), and reads all 500 member IDs.
4. The worker issues 500 distinct delivery jobs to the Message Router queue.
This decodes the process from synchronous gateway calls to asynchronous event-driven queues, protecting server threads from resource exhaustion.

15. Trade-off Discussion: Message Store Duration

Interviewer: Why did you choose transient storage on servers rather than archiving history on the cloud?
‍Candidate:
- *Cloud Archiving (Telegram model):* High server storage cost (peta-scale databases), complex synchronization logic across multiple devices, but offers immediate historical access when a user changes phones.
- *Local Storage (WhatsApp model):* Zero permanent server storage costs, highly private, but requires users to back up their data manually to third-party clouds (Google Drive/iCloud).
Decision: For WhatsApp, privacy and zero-trust E2EE are non-negotiable. Storing history locally and purging it from servers immediately upon delivery aligns with E2EE principles and keeps server costs low.

16. Failure Scenarios

Outage scenarios:

• Gateway Node Crash (Thundering Herd): If a Gateway node with 50,000 WebSocket connections crashes, those 50,000 clients will immediately try to reconnect, overloading our Load Balancer.
  Mitigation: Configure client apps with Exponential Backoff and Jitter. They must wait a randomized delay (e.g. 1s, 2s, 4s, 8s plus random milliseconds) before attempting to reconnect.
• Redis Cluster Failure: If our Session Redis goes offline, we cannot route active messages.
  Mitigation: Fallback to Cassandra. If a user session is missing from Redis, the router can query Cassandra directly, or route via push notifications as a failover backup.

17. Security Design

• End-to-End Encryption: Implemented via the Signal Protocol. Senders encrypt payloads using the recipient's public key. The server only sees the routing metadata (sender, recipient, message ID) and cannot decrypt the actual text or media.
• Client Rate Limiters: Deny socket requests if clients attempt to flood connection gateways with more than 10 messages/sec.

18. Monitoring & Observability

• Socket Health: Active WebSocket connection count per gateway node. Alert if CPU or Memory consumption exceeds 80%.
• Message Queues: Kafka ingestion delay lag. Alert if group deliveries experience processing bottlenecks.

19. Cost Optimization

Media files are expensive to stream. Instead of sending files through WebSockets:
1. Client uploads media to an Object Storage (S3) bucket and receives a hash reference.
2. Only the hash reference and metadata link travel through the real-time WebSocket messaging route.
3. The recipient pulls the media from the nearest CDN edge cache, saving application server bandwidth.

20. Production Improvements

To improve reliability under weak cellular networks: implement Message Chunking. Split media assets into 1MB chunks. If a client loses connection mid-upload, it can resume from the last successfully uploaded block rather than restarting the entire file upload.

21. Interviewer Follow-up Questions

‍Candidate:
1. When Client B opens the chat, the client app detects B has viewed the message from Client A.
2. Client B sends a WebSocket read-receipt event frame back to its gateway node: { "action": "READ", "messageId": "msg_f3b90d2e", "senderId": "user_123" }.
3. WG2 routes this to the Message Router.
4. Message Router checks Redis, finds Client A is online at WG1, and forwards the read-receipt event.
5. WG1 pushes the receipt frame to Client A. Client A's UI updates the ticks to double blue.

22. Final Architecture

The complete optimized architecture showing the group fan-out worker pipeline and E2EE media transfer routing:

23. Summary

WhatsApp requires scaling concurrent persistent connections. We terminated 100M active sockets across 2,000 Gateway instances using Envoy Layer 4 IP Hash routing. Messages are routed asynchronously through a Redis Pub/Sub layer, with transient offline queues stored on Cassandra, and E2EE encryption handled exclusively by the client devices.

24. Cheat Sheet

25. Candidate Interview Evaluation