Design Twitter (X)

This case study simulates a realistic FAANG system design interview for designing a highly scale-resilient social platform like Twitter or Threads. It walks through timeline feed aggregation, read/write fan-out, cache hydration budgets, and hot-key mitigation.

1. Present the Interview Question

2. Clarifying Questions

The candidate clarifies the features and scale limits with the interviewer:

• ‍Candidate: What is our user base scale?
  Interviewer: We have 300 million Daily Active Users (DAU).
• ‍Candidate: What is the ratio of posting tweets to reading timelines?
  Interviewer: The system is heavily read-heavy. We expect 100 million tweets posted per day, while users request timeline views/refreshes 3 billion times per day.
• ‍Candidate: Do tweets support image or video attachments?
  Interviewer: Yes, media is supported, but focus first on text metadata and the routing of timeline lists.
• ‍Candidate: What are the extreme follower limits? Are there celebrity accounts we need to handle?
  Interviewer: Yes. Normal users have an average of 200 followers, but celebrity accounts can have upwards of 100 million followers.

3. Functional Requirements

• Post a Tweet: Users can publish new text tweets (max 280 characters) with optional media attachments.
• Home Timeline: Users can view a chronological list of recent tweets from all accounts they follow.
• User Timeline: Users can view a list of their own tweets in chronological order.
• Follow Graph: Users can follow/unfollow other accounts.

4. Non-Functional Requirements

• Ultra-Low Read Latency: Loading the Home Timeline must take under 200ms globally.
• High Availability: Redirection/viewing feeds must be highly available (99.99% uptime target).
• Eventual Consistency: Senders' tweets can take up to 2-3 seconds to propagate to their followers' home timelines.

5. Capacity Estimation

1. Throughput (QPS)

• Write QPS (Tweets Posted): 100M / 86400 seconds = ~1,160 QPS (average).
• Read QPS (Timeline Views): 3 Billion / 86400 seconds = ~35,000 QPS (average).
• Peak Read QPS: Assume 2x average = ~70,000 QPS.

2. Disk Metadata Storage

• Tweet size (Id, text, userId, timestamps, metadata): ~500 bytes.
• Daily storage volume: 100M * 500 bytes = ~50 GB/day.
• Yearly storage volume: 50 GB * 365 = ~18.2 TB/year. (Easily managed on a distributed database cluster).

3. Cache RAM Sizing (Home Timelines)

To ensure low-latency loads, we keep precomputed chronological feeds in Redis memory.
Assume we cache timelines only for the 300 million active users who connected in the last 7 days.
For each active user, we store their home feed as a list containing the top 800 Tweet IDs (8-byte longs).
RAM requirements per user feed: 800 * 8 bytes = ~6.4 KB.
Total cache RAM required: 300M * 6.4 KB = ~1.92 TB of RAM. (Distributed across a Redis cluster using consistent hashing).

6. Identify Core Components

• Tweet Service: Ingests posts, writes to database, and triggers the fan-out queue.
• Social Graph Service: Stores follow/unfollow adjacency lists.
• Fan-out Service: Precomputes home feed lists by pushing tweet IDs into active follower caches.
• Timeline Service: Serves user home feeds, pulling list IDs from Redis and hydrating tweet metadata.
• Timeline Cache (Redis): Holds lists of tweet IDs per user.
• Tweet Cache (Redis/Memcached): Holds individual raw tweet body data (key: tweet_id).

7. High-Level Architecture

The high-level design maps the separate read timeline pipeline from the write posting pathway:

8. API Design

1. Post a Tweet

POST /api/v1/tweets

Request Payload:

Response Payload (201 Created):

2. Retrieve Home Feed

GET /api/v1/timeline/home

Request Parameters:

• limit: 30 (number of tweets to fetch).
• maxId: "tweet_89320104" (optional bookmark cursor for pagination).

Response Payload (200 OK):

9. Data Model

The social graph relationship must support fast query lookups for follow checks and lists of followed users:

10. Database Selection

We choose Apache Cassandra for tweet persistence, and a sharded relational database (PostgreSQL) or key-value database for the Social Graph (Follows).
Justification: Cassandra supports high-throughput writes. The social graph follows data size is massive but relational checks are simple. PostgreSQL sharded by follower_id provides relational checks with consistent replication indexes.

11. Deep Dive: Timeline Fan-out (Push vs. Pull)

1. Fan-out on Write (Push Model)

When user A posts a tweet:
1. The Fan-out service reads the list of A's followers.
2. For each follower, the service appends A's tweet_id to their timeline list in Redis (e.g. using LPUSH).
The Celebrity Problem: If Elon Musk tweets (100M followers), the system must execute 100 million Redis writes instantly. This blocks execution threads, causes massive write delay backlogs, and saturates network interfaces.

2. Fan-out on Read (Pull Model)

When user B wants to view their Home Timeline:
1. The system reads B's followed list (e.g. 500 accounts).
2. The system fetches the top 10 recent tweet IDs for all 500 accounts from the database.
3. The system merges the 5,000 tweets in memory and sorts them chronologically.
Problem: Fetching and merging data from hundreds of accounts on every page refresh generates heavy database read stress and CPU overhead, resulting in high read latencies.

3. The Hybrid Model (Recommended Solution)

We partition feed generation based on user follower counts:

• Standard Users (Followers < 25,000): Use Fan-out on Write (Push). Their tweets are pushed directly to their followers' precomputed Redis timelines.
• Celebrity Users (Followers >= 25,000): Use Fan-out on Read (Pull). When a celebrity tweets, the system writes it to the database and a separate Celebrity cache. We do NOT push it to their followers' timelines.
• Read Time Merge: When user B reads their feed:
  1. Pull the precomputed timeline list from B's Redis cache (populated with standard user tweets).
  2. Fetch the list of celebrity accounts B follows. Query their recent tweets from the Celebrity cache.
  3. Merge and sort standard and celebrity tweets chronologically in memory (typically <20 items) and return them.

12. Request Lifecycle

Standard User Posting Tweet (Push Path)

1. User A posts a tweet. The request arrives at the Tweet Service.
2. Tweet Service writes the post to Cassandra.
3. Tweet Service publishes a message event to the Kafka fanout-event topic.
4. Fan-out workers read the event, query the Social Graph Service for A's followers (e.g. 500 followers).
5. For each follower, workers execute LPUSH timeline:follower_id tweet_id and trim the list to 800 items using LTRIM timeline:follower_id 0 799 in Redis.

Home Timeline Retrieval Path

1. User B requests their home timeline. The request hits the Timeline Service.
2. Timeline Service fetches the precomputed tweet IDs from B's Redis list (e.g. top 30 IDs).
3. Timeline Service queries the Social Graph to identify celebrity accounts B follows. It pulls their recent tweets from the Celebrity cache.
4. Timeline Service merges both tweet lists chronologically.
5. Timeline Service hydrates the combined Tweet IDs (fetching tweet text and user profiles from the Memcached Tweet cache).
6. The formatted JSON timeline is returned to the client.

13. Scaling Strategy

• Redis Cluster Partitioning: Since all user timelines are stored in Redis lists, we partition the Redis cluster using Consistent Hashing on the user_id. This prevents single cache nodes from becoming hot spots.
• Hydration Cache (Memcached): When pulling feeds, fetching tweet text and user profile cards from databases is slow. We deploy a multi-replica Memcached layer caching individual tweet payloads (key: tweet_id) and profile blocks (key: user_id).

14. Bottleneck Analysis

• Celebrity Feed Ingress: When a celebrity posts, millions of users pull their tweet simultaneously. The Redis cache server hosting that celebrity's key can experience network bandwidth saturation.
  Mitigation: Replicate celebrity tweet cache keys across multiple Redis read replicas to distribute query load.
• Redis Memory Overflow: Sizing timelines without limits will exhaust RAM.
  Mitigation: Enforce a strict cap of 800 items per user list using LTRIM on every write. If users scroll past 800 items, we query historical tweets directly from Cassandra.

15. Trade-off Discussion: Cache Layout Sizing

‍Candidate:
- Caching Full Tweet Objects: Stores everything (text, media links, author info) in each follower's list. Timeline reads require a single fast Redis call. However, this duplicates tweet data across millions of follower lists. If 1,000 followers have the same tweet in their feeds, we store that tweet object 1,000 times, inflating memory requirements to ~120 TB of RAM.
- Caching Tweet IDs Only: We store only the 8-byte ID in follower lists. When reading, we pull the IDs, then fetch the full tweet bodies from a shared Memcached layer. Since a single tweet is cached only once in Memcached, we eliminate duplication. This drops total Redis memory requirements from 120 TB to just 1.92 TB (a 60x cost reduction), trading a small network hop for massive infrastructure savings.

16. Failure Scenarios

How the system handles database outages:

• Redis Timeline Eviction (Cache Miss): If an inactive user logs in after months, their Redis timeline list will have been evicted.
  Mitigation: Reconstruct the timeline lazily. The service queries the social graph, pulls the most recent tweet IDs for the user's followed accounts from Cassandra, merges them, and populates Redis.
• Follower Graph database partition: If the follow service goes down, we cannot calculate celebrity pulls.
  Mitigation: Fallback to serving the precomputed standard timeline list directly, omitting celebrity tweets temporarily rather than returning an error.

17. Security Design

• Input Sanitization: Block HTML injection in tweet payloads.
• Decoupled Media Uploads: Clients upload photos directly to S3 using presigned URLs, keeping the media traffic away from our core API servers.

18. Monitoring & Observability

• Fan-out Lag: Time from when a tweet is posted to when it appears in a follower's Redis feed. Alert if lag exceeds 5 seconds.
• Cache Latency: Monitor Redis response times. Alert if p99 latency spikes above 10ms.

19. Cost Optimization

Redis is expensive. To optimize costs, we apply a strict inactivity policy: if a user does not open the app for 7 days, we evict their precomputed timeline list from Redis.

20. Production Improvements

To improve feed quality, deploy a Feed Ranking Service (machine learning scoring pipeline). Instead of simple chronological sorting, the pipeline scores tweets based on user engagement metrics (likes, retweets) and user interests before rendering the final timeline.

21. Interviewer Follow-up Questions

‍Candidate: Unfollowing triggers an asynchronous removal job:
1. The Unfollow event is pushed to a Kafka topic.
2. Workers consume the event and read B's recent tweet IDs.
3. Workers execute LREM timeline:follower_id 0 tweet_id in Redis to delete B's tweets from A's feed list.

22. Final Architecture

The complete optimized architecture showing the hybrid fan-out pipeline, social graph services, tweet hydration, and CDN caching:

23. Summary

We designed a hybrid fan-out system to scale Twitter's read-heavy feeds. Standard user tweets are pushed into followers' Redis timelines, while celebrity tweets are pulled and merged in memory at read time. By caching only 8-byte Tweet IDs in Redis and hydrating content from Memcached, we reduced RAM costs by 60x while maintaining sub-200ms timeline load latencies.

24. Cheat Sheet

25. Candidate Interview Evaluation