Home / System Design / Sharding and Replication

๐Ÿงฉ Sharding and Replication

Fundamentals โฑ 7 min read

When a single database can't keep up, you scale it two complementary ways: replication (make copies) and sharding (split the data). They solve different problems and are often used together.

Replication โ€” copies of the data

Keep multiple copies of the database in sync. The common model is leader-follower (primary-replica): writes go to the leader, which replicates to followers that serve reads. This boosts read capacity and gives you failover if the leader dies.

  • Synchronous replication โ€” safer, no data loss, but slower writes.
  • Asynchronous replication โ€” faster, but followers can lag (stale reads).

Sharding โ€” splitting the data

Partition the data across multiple databases so each holds only a slice. Each shard handles its own reads and writes, so you scale writes and storage horizontally. The shard key decides which shard a row lives on.

Sharding strategies

  • Range-based โ€” split by ranges of the key (e.g., Aโ€“M, Nโ€“Z). Simple, but can create hotspots.
  • Hash-based โ€” hash the key to pick a shard. Even distribution, but range queries get harder.
  • Consistent hashing โ€” minimises data movement when nodes are added/removed. See consistent hashing.

The trade-offs

Sharding adds real complexity: cross-shard joins and transactions are hard, re-sharding is painful, and a bad shard key creates hotspots (one shard overloaded). Only shard when replication alone can't handle your write/storage load.

Frequently Asked Questions

What is the difference between sharding and replication?
Replication keeps full copies of the data for read scaling and failover. Sharding splits the data into partitions across nodes to scale writes and storage. They are often combined.
What is a shard key and why does it matter?
A shard key is the field used to decide which shard a record goes to. A good shard key spreads load evenly; a bad one creates hotspots where one shard gets most of the traffic.