Sharding & Partitioning
Splitting massive database sets horizontally across independent physical hardware nodes — covers range partitioning, hash partitioning, directory-based routing, and hotspot prevention strategies.
What you'll learn
- Shard Key
- Horizontal Partitioning (Sharding)
- Vertical Partitioning
- Hash-Based Sharding
- Range-Based Sharding
TL;DR
Splitting massive database sets horizontally across independent physical hardware nodes — covers range partitioning, hash partitioning, directory-based routing, and hotspot prevention strategies.
Visual System Topology
Horizontal Database Sharding
Concept Overview
Sharding (horizontal partitioning) splits a database into smaller, more manageable pieces called shards. Each shard contains a subset of the data and can be on a different server.
This is essential for scaling beyond what a single database server can handle, but introduces significant complexity.
Key Architectural Pillars
Shard Key
The field used to determine which shard data belongs to. Common choices: user_id, geographic region, hash of ID.
Horizontal Partitioning (Sharding)
Splitting rows across multiple databases. User 1-1000 on Shard1, 1001-2000 on Shard2.
Vertical Partitioning
Splitting columns across tables. User profile in Table1, user preferences in Table2.
Hash-Based Sharding
Use hash function on shard key to determine shard. Good distribution but hard to rebalance.
Range-Based Sharding
Split by ranges (IDs 1-1M on Shard1, 1M-2M on Shard2). Easy to add shards but can create hotspots.