12.4 Shards

The built-in storage is divided into shards virtual shards (set once). The shard of a key is xxh3(serialised key) mod shards.
The "shard → nodes" map is stored in the coordinator. When a node is added or fails, shards are moved; calls are buffered during the move (up to REBALANCE_BUFFER) and redirected.

Implementation-defined. Raft group membership change (D-8055; DIV-020; D-4379). The Standard requires Raft replication per shard but does not prescribe the way the membership is changed. This implementation changes the membership in single steps (one node per change); joint consensus is not implemented. Reads from replicas and leader lease — see the note below.

Reads from std.kv are linearizable by default. kv.reading(.Follower) reads from the replica of one's own node and stays linearizable (the leader reports the read index, the replica waits until it is applied); kv.reading(.Stale, d) reads from a replica without asking the leader if it heard the leader no more than d ago: the result is not older than what the process read before (monotonic reads) but may miss the writes of the last d. If the node has no replica, is itself the leader, or the replica does not catch up within 2 s, the read goes through the leader. Writes, transactions, watch and the ORM always go through the leader (D-8065; D-8220, D-8228).

Implementation-defined. Limits of replica reads (D-8065; DIV-019; D-8220, D-8221). Optional features that this implementation does not contain: leader lease (linearizability is ensured by ReadIndex, no clocks are needed) and reading SQL tables of builtin:// (the ORM) from replicas.

12.4 Shards

See also