Skip to the solution
MediumDSA11 min read

How does an LRU cache work internally?

710 views

The problem

Question presented to candidate: "Design a cache with a fixed capacity that throws away the least recently used item when it is full. Both get and put should be O(1). How does it work inside, and how does Python's own functools.lru_cache do it?"

What a strong answer should cover:

  • The policy: every get hit or put marks a key as most recently used; when the cache is full, the least recently used key is evicted.
  • The data structures: a hash map from key to node for O(1) lookup, plus a doubly linked list ordered by recency for O(1) move-to-front and evict-from-back; sentinel nodes remove edge cases.
  • Why both are needed: a dict alone cannot cheaply find the oldest key after reorderings (unless you use its insertion order), a list alone cannot find a key in O(1).
  • Python shortcuts: OrderedDict.move_to_end() and popitem(last=False), or a plain dict with pop and re-insert, since dicts keep insertion order.
  • How functools.lru_cache does it in CPython: a C type with a dict plus a circular doubly linked list around a root sentinel; arguments become the key (must be hashable); cache_info() reports hits and misses; maxsize=None skips the list entirely.

Clarifying questions expected:

  • What is the capacity, and what should get return on a miss?
  • Does put on an existing key count as a use? Is thread safety required?

Code / implementation expected: Yes โ€” an O(1) LRU class, usually with a dict plus a hand-written doubly linked list, or with OrderedDict if allowed.

  • lru-cache
  • data-structures

Try it yourself

Sketch your approach before you read the solution. That is what the interview tests.

Stuck? Get a hint.

A short nudge toward the approach. It will not give the answer away.

Solution

The solution is hidden so you can try first

11 min read ยท 10 sections

Run the code

Edit the code and run it here. Each console line with an expected value becomes a test.

PythonAn O(1) LRU cache from scratch: dict + doubly linked list
PythonOrderedDict, plain dict and functools.lru_cache versions

Discussion

Other approaches, follow-ups and how it went in real interviews.

Sign in to join the discussion.

No comments yet. Share how you would answer it.

Question 73 of 139 in Python

Keep your momentum going.

Mark this question solved to track your progress, then take the next one while it is fresh.

All Python questions
Up nextMedium

What are bound and unbound methods in Python?

Continue