Skip to the solution
MediumSystem Design14 min read

What is the CAP theorem?

764 views

The problem

Question presented to candidate: "What is the CAP theorem? Tell me what C, A and P actually mean, why people say you can only have two, and how it changes the way you design a real replicated database or service."

What a strong answer should cover:

  • The precise statement (Brewer's conjecture in 2000, proved by Gilbert and Lynch in 2002): when the network partitions, a replicated system cannot be both linearizable (C) and available to every request (A)
  • Why "pick 2 of 3" is misleading: partitions are not optional in a distributed system, so the real choice is CP or AP during a partition; with no partition you can have both
  • What each choice looks like: CP rejects or blocks requests on the minority side; AP answers everywhere and must reconcile divergent data later (last-writer-wins loses updates, mergeable types such as CRDTs do not)
  • PACELC: even with no partition there is a trade-off between latency and consistency, which shapes everyday design far more often than partitions do
  • The choice is made per operation or per data type, not per company: inventory decrements CP, like counters and shopping carts AP
  • Concrete systems with care: etcd and ZooKeeper behave CP; Dynamo-style stores lean AP but are tunable; DynamoDB lets each read choose strong or eventual

Clarifying questions expected:

  • What is the data, and what does a stale or conflicting value cost the business?
  • Is the system in one region or several (how likely and how long are partitions, and how far apart are replicas)?

Code / implementation expected: Optional โ€” a small simulation of a partition showing CP rejecting requests and AP overselling; the example does exactly that.

  • cap-theorem

Try it yourself

Sketch your approach before you read the solution. That is what the interview tests.

Stuck? Get a hint.

A short nudge toward the approach. It will not give the answer away.

Solution

The solution is hidden so you can try first

14 min read ยท 13 sections

Read the code

CAP during a partition: CP refuses, AP oversells, mergeable data survives, PACELC latency
// CAP in one run: a 3-seat inventory replicated on A, B, C, then a partition {A} | {B, C}.
// CP: an operation needs a majority (2 of 3 replicas) reachable, or it fails.
// AP: any replica answers from what it knows and syncs with the others later.
const NODES = ["A", "B", "C"], SEATS = 3;
const script = [["buy", "B"], ["partition"], ["buy", "A"], ["buy", "B"], ["buy", "A"], ["buy", "C"],
  ["read", "A"], ["read", "B"], ["heal"], ["read", "A"]];

function simulate(mode) {
  const sales = { A: new Set(), B: new Set(), C: new Set() }; // sale ids each replica knows about
  let side = { A: 0, B: 0, C: 0 }, ok = 0, errors = 0, n = 0;
  const reachable = (x) => NODES.filter((y) => side[y] === side[x]);
  const left = (x) => SEATS - sales[x].size;
  console.log("-- " + mode + " --");
  for (const [op, at] of script) {
    if (op === "partition") { side = { A: 1, B: 0, C: 0 }; console.log("   partition: A | B C"); continue; }
    if (op === "heal") {
      const all = new Set([...sales.A, ...sales.B, ...sales.C]);
      for (const x of NODES) sales[x] = new Set(all);
      side = { A: 0, B: 0, C: 0 };
      console.log("   heal: replicas merge, " + all.size + " sales recorded for " + SEATS + " seats" + (all.size > SEATS ? " -> OVERSOLD by " + (all.size - SEATS) : ""));
      continue;
    }
    const peers = reachable(at);
    let res;
    if (mode === "CP" && peers.length * 2 <= NODES.length) { res = "ERROR no quorum (" + peers.length + " of 3 reachable)"; errors++; }
    else if (op === "read") { res = left(at) < 0 ? "oversold: " + -left(at) + " tickets have no seat" : left(at) + " seats left"; ok++; }
    else if (left(at) === 0) { res = "sold out"; ok++; }
    else {
      const id = "sale" + ++n;
      for (const p of peers) sales[p].add(id); // replicate to every replica this side can reach
      res = "sold, " + left(at) + " left on " + peers.join("");
      ok++;
    }
    console.log("   " + op + " at " + at + ": " + res);
  }
  console.log("   answered " + ok + " of " + (ok + errors) + " requests, " + errors + " errors");
}
simulate("CP");
simulate("AP");

console.log("-- AP with data that merges: a like counter, partition A | B C --");
// 10 likes before the split; then 9 likes arrive, each at one replica, and spread only within its side.
const likes = ["A", "B", "A", "C", "A", "B", "A", "C", "B"], side2 = { A: 1, B: 0, C: 0 };
const lww = {}, gc = {}; // lww: [value, timestamp]; gc: one count per replica (G-counter)
for (const x of NODES) { lww[x] = [10, 0]; gc[x] = { A: 10, B: 0, C: 0 }; }
likes.forEach((x, i) => {
  const ts = i + 1, val = lww[x][0] + 1, mine = gc[x][x] + 1;
  for (const y of NODES) if (side2[y] === side2[x]) { lww[y] = [val, ts]; gc[y][x] = mine; }
});
let newest = lww.A; // heal: last-writer-wins keeps the value with the newest timestamp
for (const x of NODES) if (lww[x][1] > newest[1]) newest = lww[x];
const merged = {}; // heal: G-counter takes the max of each replica slot, then sums
for (const k of NODES) merged[k] = Math.max(gc.A[k], gc.B[k], gc.C[k]);
const truth = 10 + likes.length, gTotal = merged.A + merged.B + merged.C;
console.log("   side A alone saw " + lww.A[0] + ", side B C saw " + lww.B[0]);
console.log("   true total " + truth + ", last-writer-wins keeps " + newest[0] + " (lost " + (truth - newest[0]) + "), G-counter merge gives " + gTotal);

console.log("-- No partition: the else-branch of PACELC (client next to A) --");
const rtt = { A: 1, B: 12, C: 80 }; // ms round trip from the client region to each replica
const sorted = NODES.map((x) => rtt[x]).sort((a, b) => a - b);
console.log("   ack from 1 replica (EL, may read stale): " + sorted[0] + " ms");
console.log("   ack from a majority, 2 of 3 (EC):        " + sorted[1] + " ms");
console.log("   ack from all 3 replicas:                 " + sorted[2] + " ms");

Discussion

Other approaches, follow-ups and how it went in real interviews.

Sign in to join the discussion.

No comments yet. Share how you would answer it.

Question 22 of 106 in System design

Keep your momentum going.

Mark this question solved to track your progress, then take the next one while it is fresh.

Up nextMedium

How would you design OpenTelemetry tracing for 200 microservices?

Continue