What is a vector database and when do you actually need one?

Q: What is a vector database and when do you actually need one?

**Target Audience:** Engineers starting a RAG or semantic search project who keep being told to "add a vector database" and want to know what that is and whether they need one yet. **Difficulty:** Easy **Verified with:** Node 22.22.0 and Python 3.12.3 (the sizing, filtered search and sync simulation below). The per-core speed in the sizing table is a stated assumption, not a measurement. Product facts are cited inline with the date checked (October 2026). > **How to read this doc:** Every idea is explained in plain language first, then tagged with `📌 Interview term:` so you also learn the words an interviewer expects. All program output shown was produced by actually running the example; the **Verified** section quotes it in full. ## 1. Why This Even Matters — A Story First You have a box of 200 recipe cards. To find "something like lasagne" you flip through all 200; it takes a minute and you never miss one. Now you have 20 million recipe cards. Flipping through all of them takes w

The problem

Question presented to candidate: "Everyone says you need a vector database for RAG. What is a vector database actually, what does it do that a list of vectors in memory does not, and how would you decide whether our project needs one?"

What a strong answer should cover:

A vector database stores vectors with their IDs and metadata and answers "the k nearest vectors to this one, matching this filter", using an approximate nearest neighbour index, plus the normal database jobs: inserts, updates, deletes, persistence, replication and access control
The options form a ladder: exact search in application memory or a library, a vector extension in the database you already run (such as pgvector in Postgres), or a dedicated vector database or managed service
The decision is sizing arithmetic: vectors x dimensions x 4 bytes of memory, and vectors x dimensions multiply-adds per exact query, against the latency target and query rate
Exact search over tens of thousands of vectors, or over a filtered per-tenant subset, is fast and perfectly accurate; ANN indexes matter at millions of vectors or high query rates
A separate vector store adds a second copy of your data that must be kept in sync (deletes, permission changes, updates); keeping vectors next to the rows avoids that drift
Signals that justify a dedicated system: very large or fast-growing collections, strict latency at high QPS, heavy filtered search, built-in hybrid search, or a team that does not want to operate it

Code / implementation expected: Optional — the memory and compute arithmetic, or a short exact top-k loop with a filter, is a strong signal.

vector-database
pgvector
infrastructure
rag

Try it yourself

Sketch your approach before you read the solution. That is what the interview tests.

Stuck? Get a hint.

A short nudge toward the approach. It will not give the answer away.

Solution

Target Audience: Engineers starting a RAG or semantic search project who keep being told to "add a vector database" and want to know what that is and whether they need one yet. Difficulty: Easy Verified with: Node 22.22.0 and Python 3.12.3 (the sizing, filtered search and sync simulation below). The per-core speed in the sizing table is a stated assumption, not a measurement. Product facts are cited inline with the date checked (October 2026).

How to read this doc: Every idea is explained in plain language first, then tagged with 📌 Interview term: so you also learn the words an interviewer expects. All program output shown was produced by actually running the example; the Verified section quotes it in full.

1. Why This Even Matters — A Story First

You have a box of 200 recipe cards. To find "something like lasagne" you flip through all 200; it takes a minut

The solution is hidden so you can try first

14 min read · 11 sections

Read the code

Sizing arithmetic, a filtered exact search, and the stale result a separate vector store can return

Open in playground

// Do you need a vector database? (1) sizing arithmetic, (2) a filtered exact
// search in plain code, (3) what goes wrong when vectors live in a second store.
let seed = 11;
const rand = () => { seed = (seed * 1664525 + 1013904223) % 4294967296; return seed / 4294967296; };

// 1) sizing: raw float32 vectors and brute-force work per query
// ASSUMPTION: one CPU core does about 10 billion multiply-adds per second in a
// tight SIMD loop. Change it to your own measurement.
const DIMS = 1024, MADDS_PER_SEC = 1e10;
console.log(`1) sizing at ${DIMS} dims, float32, exact scan on one core`);
console.log("   vectors        raw size    multiply-adds/query  est. ms/query");
for (const n of [50000, 1000000, 10000000, 100000000]) {
  const gb = (n * DIMS * 4) / 1e9, madds = n * DIMS, ms = (madds / MADDS_PER_SEC) * 1000;
  console.log(`   ${n.toLocaleString("en-US").padEnd(14)} ${(gb.toFixed(1) + " GB").padEnd(11)} ${madds.toLocaleString("en-US").padEnd(20)} ${ms.toFixed(1)}`);
}

// 2) exact search with a filter: no index, no database, just a loop
const D = 32, N = 5000, TENANTS = ["acme", "globex", "initech", "umbrella"];
const rows = [];
for (let i = 0; i < N; i++) {
  const v = []; for (let d = 0; d < D; d++) v.push(rand() - 0.5);
  rows.push({ id: i, tenant: TENANTS[i % 4], vec: v });
}
const dot = (a, b) => { let s = 0; for (let d = 0; d < D; d++) s += a[d] * b[d]; return s; };
function search(store, q, tenant, k) {
  let scanned = 0; const hits = [];
  for (const r of store.values()) {
    if (r.tenant !== tenant) continue; // the WHERE clause runs first: exact pre-filter
    scanned++; hits.push([dot(q, r.vec), r.id]);
  }
  hits.sort((a, b) => b[0] - a[0] || a[1] - b[1]);
  return { ids: hits.slice(0, k).map((h) => h[1]), scanned };
}
const query = rows[42].vec.map((x) => x + (rand() - 0.5) * 0.2); // a paraphrase of row 42 (tenant initech)
const r2 = search(new Map(rows.map((r) => [r.id, r])), query, "initech", 5);
console.log("2) exact filtered search, tenant = initech, top 5");
console.log(`   ids ${r2.ids.join(", ")}   scanned ${r2.scanned} of ${N}   all initech: ${r2.ids.every((i) => rows[i].tenant === "initech")}`);

// 3) two stores: rows in the main DB, vectors copied to a vector store by an async sync job
const mainDb = new Map(rows.map((r) => [r.id, r]));
const vectorStore = new Map(rows.map((r) => [r.id, r]));
const outbox = []; // change events waiting for the sync job
function deleteDoc(id, colocated) {
  mainDb.delete(id);
  if (colocated) vectorStore.delete(id); // same transaction: both or neither
  else outbox.push({ op: "delete", id }); // applied later
}
const syncJob = () => { while (outbox.length) { const e = outbox.shift(); if (e.op === "delete") vectorStore.delete(e.id); } };
const show = (label) => {
  const r = search(vectorStore, query, "initech", 3);
  const stale = r.ids.filter((id) => !mainDb.has(id));
  console.log(`   ${label.padEnd(26)} top 3: ${r.ids.join(", ").padEnd(16)} stale ids: ${stale.length ? stale.join(", ") : "none"}`);
};
console.log("3) user deletes doc 42 (separate vector store, async sync)");
deleteDoc(42, false); show("before the sync job runs"); syncJob(); show("after the sync job runs");
console.log("   same delete with vectors in the main DB (one transaction)");
deleteDoc(r2.ids[1], true); show("immediately");

Discussion

Other approaches, follow-ups and how it went in real interviews.

Sign in to join the discussion.

No comments yet. Share how you would answer it.

The solution is hidden so you can try first

Keep your momentum going.