Skip to the solution
HardSystem Design17 min read

How do you build a sliding-window rate limiter with Redis + Lua?

645 views

The problem

Question presented to candidate: "Our API gateways run on 20 machines and every API key gets 100 requests per rolling minute. Build the limiter on Redis. Show me the data structure and the script, and explain why it has to be a Lua script rather than a few Redis commands from the gateway."

What a strong answer should cover:

  • Data structure: one sorted set per key, score = request timestamp in ms, member = a unique request id; trim with ZREMRANGEBYSCORE, count with ZCARD, record with ZADD, expire with PEXPIRE
  • Why Lua: the check and the write must be atomic; as separate commands from many gateways it is check-then-act and over-admits. Redis runs a script without interleaving other commands
  • Correctness details: members must be unique (two requests in the same ms would collapse), denied requests must not be recorded, compute Retry-After from the oldest entry
  • Operations: SCRIPT LOAD + EVALSHA with a NOSCRIPT fallback (or Redis Functions), which clock to trust (client time vs TIME), keys in one hash slot in Redis Cluster
  • Cost: memory grows with the limit (one entry per allowed request in the window); the sliding window counter variant uses two counters when the log is too big
  • Failure policy if Redis is slow or down: fail open or fail closed, with a timeout

Clarifying questions expected:

  • Must the limit be exact, or is an approximation within a few percent acceptable?
  • How large are the limits (10 per minute or 10,000 per minute), and how many active keys?
  • If Redis is unavailable, should requests be allowed or rejected?

Code / implementation expected: Yes โ€” the Lua script and how the gateway calls it; the example simulates the sorted set to show the race, member collisions and lockout, and the script itself was run on Redis 7.0.15.

  • rate-limiting
  • redis
  • lua

Try it yourself

Sketch your approach before you read the solution. That is what the interview tests.

Stuck? Get a hint.

A short nudge toward the approach. It will not give the answer away.

Solution

The solution is hidden so you can try first

17 min read ยท 14 sections

Read the code

Sliding-window log on a simulated Redis sorted set: atomic script vs check-then-act race, member collisions, counting denials
// A tiny in-process stand-in for a Redis sorted set, used to show why the sliding-window
// limiter must run as ONE atomic Lua script. Limit 5 per 1000 ms; fake clock in ms.
class MiniRedis {
  constructor() { this.z = new Map(); } // key -> Map(member -> score)
  set(key) { if (!this.z.has(key)) this.z.set(key, new Map()); return this.z.get(key); }
  zremrangebyscore(key, max) { const s = this.set(key); for (const [m, sc] of s) if (sc <= max) s.delete(m); }
  zcard(key) { return this.set(key).size; }
  zadd(key, score, member) { this.set(key).set(member, score); }
  oldest(key) { let best = Infinity; for (const sc of this.set(key).values()) best = Math.min(best, sc); return best; }
}

const WINDOW = 1000, LIMIT = 5;

// The body of the Lua script. Redis runs it without interleaving any other command.
function script(r, key, now, member) {
  r.zremrangebyscore(key, now - WINDOW);
  const count = r.zcard(key);
  if (count < LIMIT) { r.zadd(key, now, member); return [1, LIMIT - count - 1, 0]; }
  return [0, 0, r.oldest(key) + WINDOW - now];
}

console.log("== 1. the script, one client (same calls as the redis-cli run)");
let r = new MiniRedis();
let n = 0;
for (const now of [1000, 1100, 1200, 1300, 1400, 1500, 1600, 2001]) {
  n++;
  console.log("  now=" + now + "  [allowed remaining retry_ms] = " + script(r, "rl:{user42}", now, now + "-req" + n).join(" "));
}

console.log("== 2. four gateways, three rounds of simultaneous requests");
function race(atomic) {
  const r = new MiniRedis();
  let admitted = 0, id = 0;
  for (let round = 0; round < 3; round++) {
    const now = round * 10;
    if (atomic) {
      for (let g = 0; g < 4; g++) admitted += script(r, "k", now, "req" + id++)[0];
    } else {
      // check-then-act from the client: every gateway reads before any of them writes
      const seen = [];
      for (let g = 0; g < 4; g++) { r.zremrangebyscore("k", now - WINDOW); seen.push(r.zcard("k")); }
      for (let g = 0; g < 4; g++) if (seen[g] < LIMIT) { r.zadd("k", now, "req" + id++); admitted++; }
    }
  }
  return admitted;
}
console.log("  ZCARD then ZADD as separate commands: admitted " + race(false) + " of 12 (limit 5)");
console.log("  one Lua script per request:           admitted " + race(true) + " of 12 (limit 5)");

console.log("== 3. member = timestamp vs a unique member, 4 requests in each of 3 milliseconds");
for (const unique of [false, true]) {
  const r = new MiniRedis();
  let admitted = 0, id = 0;
  for (let ms = 0; ms < 3; ms++) {
    for (let g = 0; g < 4; g++) admitted += script(r, "k", ms, unique ? ms + "-req" + id++ : String(ms))[0];
  }
  console.log("  member " + (unique ? "now-requestId" : "now          ") + "  admitted " + admitted + " of 12, ZCARD " + r.zcard("k"));
}

console.log("== 4. a client sending every 100 ms for 3 s: should denied requests be recorded?");
for (const recordDenied of [false, true]) {
  const r = new MiniRedis();
  let admitted = 0;
  const perSecond = [0, 0, 0];
  for (let i = 0; i < 30; i++) {
    const now = i * 100;
    let ok;
    if (recordDenied) {
      r.zremrangebyscore("k", now - WINDOW);
      ok = r.zcard("k") < LIMIT ? 1 : 0;
      r.zadd("k", now, "req" + i); // logged even when denied
    } else ok = script(r, "k", now, "req" + i)[0];
    admitted += ok;
    perSecond[Math.floor(now / 1000)] += ok;
  }
  console.log("  " + (recordDenied ? "denied requests recorded" : "only allowed recorded   ") + "  admitted " + admitted + " of 30, per second " + perSecond.join(" "));
}

Discussion

Other approaches, follow-ups and how it went in real interviews.

Sign in to join the discussion.

No comments yet. Share how you would answer it.

Question 76 of 106 in System design

Keep your momentum going.

Mark this question solved to track your progress, then take the next one while it is fresh.

Up nextHard

How would you design a Web Crawler at scale?

Continue