EasySystem Design13 min read

What latency numbers and availability targets should every engineer know?

Q: What latency numbers and availability targets should every engineer know?

**Target Audience:** Beginners — no hardware or operations background assumed. **Difficulty:** Easy **Verified with:** Node 22.22 and Python 3.12 (the calculator prints the same output in both). > **How to read this doc:** Every idea is explained in plain language first, then tagged with `📌 Interview term:` so you also learn the exact words an interviewer expects. Every derived number was computed by the example program; the **Verified** section quotes its full output. ## 1. Why This Even Matters — A Story First The famous latency table uses nanoseconds, which mean nothing to a human. So stretch time: pretend that reading from the CPU's L1 cache — the fastest memory there is — takes **one second**. The example program does that conversion: ~~~text 0. If an L1 cache read (0.5 ns) took one second main memory read 200 seconds round trip in one datacenter 11.6 days disk seek 231.5 days California-Netherlands round trip 9.5

614 views

3 of 106

The problem

Question presented to candidate: "Without looking anything up: roughly how long does a memory read take compared with an SSD read, a network round trip inside a datacenter, and one across an ocean? And if a service promises three nines, or four, how much downtime is that a month? How do those numbers change the way you design?"

What a strong answer should cover:

The latency ladder in orders of magnitude: L1 cache about 0.5 ns, main memory about 100 ns, random SSD read about 150 µs, datacenter round trip about 0.5 ms, disk seek about 10 ms, California–Netherlands round trip about 150 ms (the classic, dated ~2012 table)
What follows: avoid sequential network calls and random disk I/O on the request path; batch, parallelise, cache, and put data near users
Percentiles and tail latency: with fan-out, the slowest backend sets the response time (1% slow per backend means 63% of requests slow at 100 backends)
The nines: 99.9% is about 43 minutes a month, 99.99% about 4 minutes, 99.999% about 26 seconds; and the vocabulary SLI, SLO, SLA, error budget
Composition: dependencies in series multiply availabilities (always lower), redundant copies in parallel raise it; availability = MTBF / (MTBF + MTTR), so faster recovery buys nines

Code / implementation expected: No — the arithmetic is the answer; the example computes every number quoted.

fundamentals
estimation

Try it yourself

Sketch your approach before you read the solution. That is what the interview tests.

Stuck? Get a hint.

A short nudge toward the approach. It will not give the answer away.

Solution

Target Audience: Beginners — no hardware or operations background assumed. Difficulty: Easy Verified with: Node 22.22 and Python 3.12 (the calculator prints the same output in both).

How to read this doc: Every idea is explained in plain language first, then tagged with 📌 Interview term: so you also learn the exact words an interviewer expects. Every derived number was computed by the example program; the Verified section quotes its full output.

1. Why This Even Matters — A Story First

The famous latency table uses nanoseconds, which mean nothing to a human. So stretch time: pretend that reading from the CPU's L1 cache — the fastest memory there is — takes one second. The example program does that conversion:

0. If an L1 cache read (0.5 ns) took one second
  main memory read                    200 seconds
  round trip in one datacenter        1

The solution is hidden so you can try first

13 min read · 11 sections

Read the code

Nines, dependency chains, error budgets and tail latency

Open in playground

// The arithmetic behind "nines", dependency chains, error budgets and tail latency.
// Year = 365 days, month = 30 days (state this when you quote downtime).
const round = (x) => Math.floor(x + 0.5);
function dur(sec) {
  let s = round(sec);
  const d = Math.floor(s / 86400); s -= d * 86400;
  const h = Math.floor(s / 3600); s -= h * 3600;
  const m = Math.floor(s / 60); s -= m * 60;
  const parts = [];
  if (d) parts.push(d + "d");
  if (h) parts.push(h + "h");
  if (m) parts.push(m + "m");
  if (s || !parts.length) parts.push(s + "s");
  return parts.join(" ");
}
const YEAR = 365 * 86400, MONTH = 30 * 86400, WEEK = 7 * 86400, DAY = 86400;

console.log("0. If an L1 cache read (0.5 ns) took one second");
const human = [["main memory read", 100], ["round trip in one datacenter", 500e3], ["disk seek", 10e6], ["California-Netherlands round trip", 150e6]];
for (const [what, ns] of human) {
  const sec = ns / 0.5;
  const text = sec < 3600 ? round(sec) + " seconds" : sec < 86400 * 365 ? (sec / 86400).toFixed(1) + " days" : (sec / YEAR).toFixed(1) + " years";
  console.log("  " + what.padEnd(36) + text);
}

console.log("1. Allowed downtime");
console.log("  target       per year    per month     per week   per day");
for (const a of ["99", "99.5", "99.9", "99.95", "99.99", "99.999"]) {
  const down = 1 - Number(a) / 100;
  console.log(
    "  " + (a + "%").padEnd(9) + dur(down * YEAR).padStart(12) + dur(down * MONTH).padStart(13) +
    dur(down * WEEK).padStart(13) + dur(down * DAY).padStart(10)
  );
}

console.log("2. Composition");
const chain = [["load balancer", 99.99], ["app", 99.95], ["database", 99.95], ["cache", 99.9]];
let serial = 1;
for (const [, a] of chain) serial *= a / 100;
console.log("  in series (all must work): " + chain.map((c) => c[1] + "%").join(" x ") +
  " = " + (serial * 100).toFixed(3) + "% (" + dur((1 - serial) * YEAR) + "/yr)");
for (const n of [1, 2, 3]) {
  const p = 1 - (1 - 0.999) ** n;
  console.log("  " + n + " x 99.9% in parallel (any one enough): " + (p * 100).toFixed(7) + "%");
}

console.log("3. Error budget (SLO 99.9%, 30 days, 2,000,000 requests/day) and repair time");
const total = 2e6 * 30, budget = total * (1 - 0.999);
console.log("  failed requests allowed: " + round(budget) + " of " + round(total));
console.log("  full-outage minutes allowed: " + ((1 - 0.999) * MONTH / 60).toFixed(1));
const used = 25 / ((1 - 0.999) * MONTH / 60);
console.log("  one 25-minute outage burns " + (used * 100).toFixed(1) + "% of the budget");

console.log("  MTBF 30 days, MTTR 1 hour: " + ((720 / (720 + 1)) * 100).toFixed(3) + "%; MTTR 6 minutes: " + ((720 / (720 + 0.1)) * 100).toFixed(3) + "%");

console.log("4. Latency budget for one user request (approximate numbers, ~2012 table)");
const NS = { mem: 100, ssd4k: 150e3, dcRtt: 500e3, diskSeek: 10e6, transatlantic: 150e6 };
const ms = (ns) => (ns / 1e6).toFixed(3) + " ms";
console.log("  6 sequential calls in one datacenter: " + ms(6 * NS.dcRtt));
console.log("  6 calls in parallel (one round trip): " + ms(NS.dcRtt));
console.log("  1 California-Netherlands round trip:  " + ms(NS.transatlantic));
console.log("  1,000 random SSD reads, sequential:   " + ms(1000 * NS.ssd4k));
console.log("  1,000 random disk seeks, sequential:  " + ms(1000 * NS.diskSeek));
console.log("  1,000 memory reads:                   " + ms(1000 * NS.mem));

console.log("5. Tail at scale: each backend is slow on 1% of calls");
for (const n of [1, 10, 100, 500]) {
  console.log("  fan-out to " + String(n).padStart(4) + " backends: " +
    ((1 - 0.99 ** n) * 100).toFixed(1).padStart(5) + "% of user requests hit a slow one");
}

Discussion

Other approaches, follow-ups and how it went in real interviews.

No comments yet. Share how you would answer it.

Question 3 of 106 in System design

Keep your momentum going.

Mark this question solved to track your progress, then take the next one while it is fresh.

All System design questions

Up nextMedium→

How do you design for high availability?

Continue