Cloud & Infrastructure · Architecture Patterns
Database Sharding Strategies: Range, Hash, and Directory-Based, Explained
Sharding splits one database across many machines when a single server can't hold or serve your data anymore. The strategy you pick determines whether you get hot spots, painful resharding, or a system that actually scales. Here's how each one works.
Prathviraj Singh
6 min read
Sponsored
A single database server has a ceiling: storage capacity, and write throughput a single primary can absorb. Read replicas push that ceiling back by copying the same data to more machines for reads, but every replica still holds the entire dataset and still funnels through one primary for writes. Sharding is what you reach for when that ceiling is the actual constraint, not read load, and it works by splitting the data itself, not copying it.
The strategy you pick determines whether the result scales cleanly or creates a new bottleneck shaped exactly like the old one.
Hash-based sharding: even distribution, expensive ranges
Hash-based sharding applies a hash function to the shard key, usually a user ID or tenant ID, and routes the row to hash(key) % number_of_shards.
function getShardForUser(userId, shardCount) {
const hash = hashFunction(userId); // e.g. a consistent hash, not just userId itself
return hash % shardCount;
}
// user 8821 and user 8822 can land on completely different shards,
// even though their IDs are sequential
Because a well-chosen hash function distributes values close to uniformly, rows spread evenly across shards regardless of any natural clustering in the source data. No single shard ends up disproportionately large or busy just because a lot of users signed up in the same week.
The cost shows up on range queries. “Give me all orders from users 8000 to 9000” now has to query every shard, since the hash scattered those sequential user IDs randomly across the cluster, then merge the results in application code. If your access patterns are almost entirely single-key lookups (fetch this user’s data, fetch this order), hash-based sharding’s even distribution is close to a pure win. If range scans over the shard key are common, it’s a real tax on every one of them.
Range-based sharding: fast ranges, hot-spot risk
Range-based sharding assigns each shard a contiguous range of the shard key’s values, commonly a time range.
Shard 1: orders created 2026-01-01 to 2026-03-31
Shard 2: orders created 2026-04-01 to 2026-06-30
Shard 3: orders created 2026-07-01 to 2026-09-30
Range queries over the shard key become fast and cheap, since a query for “orders from Q2” hits exactly one shard instead of fanning out to all of them. This is the natural fit for time-series data, logs, orders, events, where “give me the recent window” is the dominant query pattern.
The risk is concentration. If new writes always land in the newest range, which is the common case for time-based sharding, one shard absorbs effectively all current write traffic while older shards sit nearly idle. You’ve distributed the data across machines, but not the load, which defeats a large part of the original point. Mitigating this usually means either sizing ranges by expected volume rather than fixed calendar periods, or accepting that the current-period shard needs to be provisioned for peak write throughput regardless of how many other shards exist.
Directory-based sharding: a lookup layer, and real flexibility
Directory-based sharding replaces the algorithmic mapping (a hash or a range boundary) with an explicit lookup: a separate table or service that records which shard holds which key.
// A lookup service, itself needs to be fast and highly available
async function getShardForTenant(tenantId) {
return shardDirectory.lookup(tenantId); // returns e.g. "shard-7"
}
// Moving a specific tenant to its own shard is just an update to the directory,
// no rehashing, no range boundary change for anyone else
await shardDirectory.reassign('tenant-4471', 'shard-12-dedicated');

The payoff is real operational flexibility. A single noisy tenant hammering shared infrastructure can be moved to a dedicated shard by updating one directory entry, without touching the hash function or range boundaries that every other key’s placement depends on. This matters a lot for multi-tenant SaaS, where load is rarely evenly distributed across tenants and the biggest customer is often disproportionately the busiest one.
The cost is an extra network hop on every query (the directory lookup itself) and a new critical-path component: if the directory service is slow or unavailable, every sharded query is blocked behind it, so it needs its own high-availability story, typically a small, heavily-cached, aggressively-replicated dataset since it’s just key-to-shard mappings.
Comparing the three
| Hash-based | Range-based | Directory-based | |
|---|---|---|---|
| Distribution | Even, by design | Depends on key distribution over ranges | Whatever the directory assigns |
| Range queries | Expensive, fans out to all shards | Cheap, hits one shard | Depends on underlying scheme |
| Hot-spot risk | Low | High if writes concentrate on one range | Low, individual keys can be rebalanced |
| Resharding | Requires rehashing most keys | Requires redrawing range boundaries | Update directory entries, more surgical |
| Extra infrastructure | None beyond the hash function | None beyond range config | A directory service that must be fast and available |
Reach for sharding last, not first
Sharding is usually the wrong first move. Read replicas solve read-heavy load without touching how data is stored. Caching and query optimization solve a large share of what looks like a capacity problem but is actually an inefficient-query problem. Vertical scaling, a bigger single machine, buys real headroom before it hits diminishing returns.
Sharding earns its complexity specifically when data volume exceeds what one machine can store, or write throughput exceeds what a single primary can absorb, regardless of hardware. Once you’re actually there, resharding a live system without downtime, moving data between shards while queries keep running against it, is genuinely one of the harder operations in distributed systems engineering. Pick the strategy that matches your actual query patterns up front, because migrating from hash-based to range-based sharding after the fact means rebuilding the same painful process you were trying to avoid by sharding in the first place.
Frequently asked questions
- What's the actual difference between sharding and read replicas?
- A read replica is a full copy of the same database, used to distribute read load across multiple identical copies; every replica holds all the data. Sharding splits the data itself, each shard holds a different subset of rows, so no single machine needs to store or serve the entire dataset. Read replicas solve read throughput on data that still fits on one machine. Sharding solves what happens when the data itself, or the write load, no longer fits on one machine at all.
- How does hash-based sharding decide where a row goes?
- A hash function is applied to the shard key, most commonly a user ID or tenant ID, and the resulting hash value modulo the number of shards determines which shard the row lives on. Because a good hash function distributes values roughly uniformly, this spreads rows evenly across shards and avoids the hot spots that come from natural clustering in the data. The tradeoff is that rows with related shard keys, like a range of user IDs signed up in the same week, end up scattered randomly, which makes range queries across the shard key expensive since they have to hit every shard.
- Why would range-based sharding cause hot spots?
- Range-based sharding assigns each shard a contiguous range of the shard key, for example one shard per calendar month if the key is a timestamp. That makes range queries, everything from March, fast, because it's confined to one shard. But if writes concentrate on the most recent range, all new orders being written to this month's shard, that one shard absorbs all the write traffic while older shards sit idle, recreating the exact bottleneck sharding was supposed to fix, just on a smaller piece of the cluster.
- What does a directory-based sharding lookup actually add?
- It adds a separate service or table that explicitly maps each shard key, or range of keys, to the physical shard holding it, rather than computing the mapping algorithmically from a hash or range. That extra lookup hop costs latency and adds a new component that itself needs to be highly available, since every query depends on it. What it buys back is flexibility: a specific key, a noisy tenant hammering one shard, can be moved to its own dedicated shard without touching the hash function or range boundaries every other key depends on.
- When should I actually reach for sharding instead of something simpler?
- After you've exhausted read replicas, caching, query optimization, and vertical scaling (a bigger single machine), and you're still hitting a wall on either data volume that doesn't fit on one machine's storage, or write throughput that a single primary can't absorb even with faster hardware. Sharding solves a real problem, but resharding a live production system, moving data between shards without downtime, is one of the more painful operations in distributed systems, so it's worth being certain simpler options are actually exhausted first.
Sponsored
More from this category
More from Cloud & Infrastructure
R.01 The AWS CloudFront Outage That Took Down Canvas, Blackboard, and Hugging Face at Once
R.02 Kubernetes 1.36 Haru: What's Actually Worth Upgrading For
R.03 AWS's Trillion-Dollar Billing Bug Is a Warning About Automated Cost Alerts
Sponsored
Discussion
Join the conversation.
Comments are powered by GitHub Discussions. Sign in with your GitHub account to leave a comment.
Sponsored