What is the difference between throttling and rate limiting?

Q: What is the difference between throttling and rate limiting?

**Target Audience:** Developers preparing for system design or backend interviews. No prior knowledge of rate limiting assumed. **Difficulty:** Medium **Verified with:** Node 22.22 and Python 3.12 (both print identical output). > **How to read this doc:** Every idea is explained in plain language first, then tagged with `📌 Interview term:` so you also learn the exact words an interviewer expects. All behaviour shown comes from running the example; the **Verified** section quotes the full output. ## 1. Why This Even Matters — A Story First A popular bakery can bake 4 loaves a minute. At opening time 10 people rush in. - **Bakery A** has a bouncer. He lets 4 people in, and tells the other 6: "Full. Come back in one minute." They leave and return later. Nobody waits inside, but people get turned away and must try again. - **Bakery B** has a queue with a rope. Everybody gets a ticket and waits in line; the counter serves one person every 15 seconds. Nobody is turned away (until the li

The problem

Question presented to candidate: "People use 'throttling' and 'rate limiting' as if they were the same thing. If you had to tell them apart for your team, what would you say? What happens to a request over the limit in each case, and when would you choose one over the other?"

What a strong answer should cover:

The useful distinction is what happens to excess traffic: a rate limiter rejects it (HTTP 429 Too Many Requests, ideally with Retry-After), a throttle delays or slows it (queue, pace, degrade) so it still completes later
Both enforce a rate; they differ in who carries the cost: rejection pushes the waiting to the client (retries), throttling pushes it onto the server (held requests, memory, added latency)
A throttle still needs a bound: an unbounded queue only turns overload into timeouts, so a throttle rejects once its queue is full
The same algorithm can do either: a token or leaky bucket can drop the excess or schedule it (nginx limit_req delays excess up to burst and rejects beyond it)
The words are used loosely: AWS API Gateway calls its 429 rejections "throttling", so define your terms in the room
Related ideas: client-side throttling (the caller paces itself), backpressure, load shedding

Code / implementation expected: Optional — the example runs the same burst through a rejecting limiter and a delaying throttle and compares what clients see.

throttling

Try it yourself

Sketch your approach before you read the solution. That is what the interview tests.

Stuck? Get a hint.

A short nudge toward the approach. It will not give the answer away.

Solution

Target Audience: Developers preparing for system design or backend interviews. No prior knowledge of rate limiting assumed. Difficulty: Medium Verified with: Node 22.22 and Python 3.12 (both print identical output).

How to read this doc: Every idea is explained in plain language first, then tagged with 📌 Interview term: so you also learn the exact words an interviewer expects. All behaviour shown comes from running the example; the Verified section quotes the full output.

1. Why This Even Matters — A Story First

A popular bakery can bake 4 loaves a minute. At opening time 10 people rush in.

Bakery A has a bouncer. He lets 4 people in, and tells the other 6: "Full. Come back in one minute." They leave and return later. Nobody waits inside, but people get turned away and must try again.
Bakery B has a queue with a rope. Everybody gets a ticket an

The solution is hidden so you can try first

13 min read · 10 sections

Read the code

Same burst, two policies: a rate limiter answers 429 + Retry-After, a throttle queues and delays

Open in playground

// Same burst, two policies: a rate limiter REJECTS the excess (429 + Retry-After),
// a throttle DELAYS it (a leaky-bucket queue that drains at a fixed pace).
// Fake clock in ms. Server capacity in both cases: 4 requests per second.
const arrivals = [];
for (let i = 0; i < 10; i++) arrivals.push({ id: "r" + i, t: 0 });
arrivals.push({ id: "r10", t: 3000 }, { id: "r11", t: 3000 });

// --- Policy 1: rate limiter, fixed window of 4 per 1000 ms; client honours Retry-After ---
function rateLimit() {
  console.log("== rate limiter: 4 per 1 s window, excess gets 429, client retries after Retry-After");
  const counts = new Map();
  let rejected = 0, worst = 0;
  for (const a of arrivals) {
    let t = a.t, tries = 0;
    const log = [];
    for (;;) {
      tries++;
      const win = Math.floor(t / 1000);
      const used = counts.get(win) || 0;
      if (used < 4) { counts.set(win, used + 1); log.push("200 at " + t); break; }
      const retryAfter = (win + 1) * 1000 - t; // ms until the window resets
      log.push("429 Retry-After " + retryAfter / 1000 + "s");
      rejected++;
      t += retryAfter;
    }
    worst = Math.max(worst, t - a.t);
    console.log("  " + a.id.padEnd(4) + "arrives " + String(a.t).padStart(4) + "  " + log.join(" -> "));
  }
  console.log("  429 responses sent: " + rejected + ", worst wait seen by a client: " + worst + " ms");
}

// --- Policy 2: throttle, a queue drained every 250 ms, at most 8 waiting ---
function throttle() {
  console.log("== throttle: queue drained one request every 250 ms, room for 8 waiting");
  let nextSlot = 0, rejected = 0, worst = 0, held = 0;
  const starts = [];
  for (const a of arrivals) {
    const start = Math.max(a.t, nextSlot);
    const waiting = starts.filter((s) => s > a.t).length; // still queued when this one arrives
    if (waiting >= 8) {
      rejected++;
      console.log("  " + a.id.padEnd(4) + "arrives " + String(a.t).padStart(4) + "  queue full -> 503");
      continue;
    }
    starts.push(start);
    nextSlot = start + 250;
    worst = Math.max(worst, start - a.t);
    held = Math.max(held, waiting + 1);
    console.log("  " + a.id.padEnd(4) + "arrives " + String(a.t).padStart(4) + "  held " + String(start - a.t).padStart(4) + " ms, 200 at " + start);
  }
  console.log("  rejections: " + rejected + ", worst added latency: " + worst + " ms, most requests waiting in the queue at once: " + held);
}

rateLimit();
throttle();

Discussion

Other approaches, follow-ups and how it went in real interviews.

Sign in to join the discussion.

No comments yet. Share how you would answer it.

The solution is hidden so you can try first

Keep your momentum going.