Skip to the solution
MediumSystem Design15 min read

What is the circuit breaker pattern?

331 views

The problem

Question presented to candidate: "One of our services calls a recommendations API that sometimes becomes slow or starts timing out, and when it does, our own service falls over too. What is the circuit breaker pattern, how does it work, and how would you configure one here?"

What a strong answer should cover:

  • The problem it solves: calls to a failing dependency tie up threads and connections until timeouts fire, so the caller runs out of resources and the failure cascades
  • The three states: closed (calls pass, results counted), open (calls fail fast without touching the dependency), half-open (a few trial calls decide whether to close or reopen)
  • Trip conditions: failure rate (and slow-call rate) over a sliding window, with a minimum number of calls so a single error does not trip it
  • What the caller does when open: a fallback (cached or default value, degraded feature) or a fast, clear error
  • How it fits with timeouts, retries with backoff, bulkheads and load shedding; typical tools (Resilience4j, service-mesh outlier detection)
  • Trade-offs: open-state wait too long delays recovery, too short hammers a sick service; breakers per dependency or per host

Code / implementation expected: Optional โ€” a small state machine is a plus; the example runs one on a fake clock and counts calls, timeouts and blocked time.

  • circuit-breaker

Try it yourself

Sketch your approach before you read the solution. That is what the interview tests.

Stuck? Get a hint.

A short nudge toward the approach. It will not give the answer away.

Solution

The solution is hidden so you can try first

15 min read ยท 12 sections

Read the code

Circuit breaker on a fake clock: closed, open, half-open probes, fast-fails and the cost of waiting on timeouts
// Circuit breaker on a fake clock: one request per second for 60 s; the dependency
// times out (2000 ms) from t=10 to t=34 and is healthy otherwise.
const CFG = { window: 10, minCalls: 5, failRate: 0.5, openMs: 5000, halfOpenPermits: 3 };
const OK_MS = 50, TIMEOUT_MS = 2000;
const healthy = (t) => t < 10 || t >= 35;

class CircuitBreaker {
  constructor(cfg) { this.cfg = cfg; this.state = "CLOSED"; this.results = []; this.openedAt = 0; this.probes = 0; this.log = []; }
  move(now, to, why) { this.log.push(`t=${String(now / 1000).padStart(2)}s ${this.state.padEnd(9)} -> ${to.padEnd(9)} ${why}`); this.state = to; }
  allow(now) {
    if (this.state === "OPEN" && now - this.openedAt >= this.cfg.openMs) { this.move(now, "HALF_OPEN", "wait over, letting probes through"); this.probes = 0; }
    if (this.state === "OPEN") return false;
    if (this.state === "HALF_OPEN") return this.probes < this.cfg.halfOpenPermits;
    return true;
  }
  record(now, ok) {
    if (this.state === "HALF_OPEN") {
      if (!ok) { this.openedAt = now; this.move(now, "OPEN", "probe failed"); return; }
      if (++this.probes === this.cfg.halfOpenPermits) { this.results = []; this.move(now, "CLOSED", `${this.probes} probes succeeded`); }
      return;
    }
    this.results.push(ok);
    if (this.results.length > this.cfg.window) this.results.shift();
    const fails = this.results.filter((r) => !r).length;
    if (this.results.length >= this.cfg.minCalls && fails / this.results.length >= this.cfg.failRate) {
      this.openedAt = now; this.move(now, "OPEN", `${fails}/${this.results.length} failed in window`); this.results = [];
    }
  }
}

function run(useBreaker) {
  const cb = new CircuitBreaker(CFG);
  let timeline = "", depCalls = 0, failed = 0, fastFails = 0, waitedMs = 0;
  for (let t = 0; t < 60; t++) {
    const now = t * 1000;
    const wasHalf = cb.state === "HALF_OPEN" || (cb.state === "OPEN" && now - cb.openedAt >= CFG.openMs);
    if (useBreaker && !cb.allow(now)) { fastFails++; timeline += "o"; continue; }
    depCalls++;
    const ok = healthy(t);
    waitedMs += ok ? OK_MS : TIMEOUT_MS;
    if (!ok) failed++;
    timeline += useBreaker && wasHalf ? (ok ? "?" : "!") : ok ? "." : "X";
    if (useBreaker) cb.record(now, ok);
  }
  return { cb, timeline, depCalls, failed, fastFails, waitedMs };
}

let health = "";
for (let t = 0; t < 60; t++) health += healthy(t) ? "." : "X";
const none = run(false), withCb = run(true);
console.log("legend: . ok  X timeout  o rejected by open breaker  ? probe ok  ! probe failed");
console.log(`dependency   ${health}`);
console.log(`no breaker   ${none.timeline}`);
console.log(`breaker      ${withCb.timeline}`);
console.log("\nstate transitions:");
for (const line of withCb.cb.log) console.log("  " + line);
console.log("\n                 calls to dep  timeouts  fast-fails  caller time blocked");
for (const [name, r] of [["no breaker", none], ["breaker", withCb]])
  console.log(`  ${name.padEnd(14)} ${String(r.depCalls).padStart(12)} ${String(r.failed).padStart(9)} ${String(r.fastFails).padStart(11)} ${String(r.waitedMs / 1000).padStart(14)} s`);

console.log("\nWhy it matters at 200 req/s (Little: busy workers = rate x time per call):");
for (const [name, ms] of [["healthy, 50 ms", OK_MS], ["failing, 2000 ms timeout", TIMEOUT_MS], ["breaker open, ~0 ms", 0]]) {
  const busy = (200 * ms) / 1000;
  console.log(`  ${name.padEnd(26)} busy workers = ${String(busy).padStart(3)}  pool of 100 ${busy > 100 ? "EXHAUSTED" : "ok"}`);
}

Discussion

Other approaches, follow-ups and how it went in real interviews.

Sign in to join the discussion.

No comments yet. Share how you would answer it.

Question 41 of 106 in System design

Keep your momentum going.

Mark this question solved to track your progress, then take the next one while it is fresh.

Up nextMedium

How does caching work end-to-end (where, what, when) โ€” cache-aside vs write-through?

Continue