Question 1

What is a Large Language Model and how does it generate text?

Accepted Answer

**Target Audience:** Beginners to AI engineering — you can write code and call an API, but you have never looked inside a language model.
**Difficulty:** Easy
**Verified with:** Node 22.22.0 and Python 3.12.3 (the toy model in the example; no real model is called). Product and paper facts are cited inline with the date they were checked (October 2026).

> **How to read this doc:** Every idea is explained in plain language first, then tagged with `📌 Interview term:` so you also learn the exact w

Question 2

What is a vector database and when do you actually need one?

Accepted Answer

**Target Audience:** Engineers starting a RAG or semantic search project who keep being told to "add a vector database" and want to know what that is and whether they need one yet.
**Difficulty:** Easy
**Verified with:** Node 22.22.0 and Python 3.12.3 (the sizing, filtered search and sync simulation below). The per-core speed in the sizing table is a stated assumption, not a measurement. Product facts are cited inline with the date checked (October 2026).

> **How to read this doc:** Every idea

Question 3

Context windows in 2025-26: how do you use 128k-1M tokens without drowning?

Accepted Answer

**Target Audience:** Engineers who have a 1M-token model available and need to decide when to use the window and when to keep retrieving.
**Difficulty:** Easy
**Verified with:** Node 22.22.0 and Python 3.12.3 (cost arithmetic, chunk reordering and a needle-test harness with canned answers; no real model is called). Model window sizes, prices and published guidance are cited inline with the date checked (October 2026); these change often, so check the provider docs for the model you use.

> **How

Question 4

What are tokens and why do they matter for cost and limits?

Accepted Answer

**Target Audience:** Beginners who are about to ship (or pay for) their first LLM feature and keep seeing "tokens" on every pricing and limits page.
**Difficulty:** Easy
**Verified with:** Node 22.22.0 and Python 3.12.3 (a tiny BPE tokenizer and the cost arithmetic; no real tokenizer or model is called). Product facts and prices are cited inline with the date they were checked (October 2026).

> **How to read this doc:** Every idea is explained in plain language first, then tagged with `📌 Inter

Question 5

What is a context window and what happens when you exceed it?

Accepted Answer

**Target Audience:** Beginners building chat features, RAG or agents who have hit (or will hit) "context too long" errors.
**Difficulty:** Easy
**Verified with:** Node 22.22.0 and Python 3.12.3 (a simulated provider with a 200-token window; no real model is called). Provider behaviour is cited inline from documentation and forum reports, checked October 2026.

> **How to read this doc:** Every idea is explained in plain language first, then tagged with `📌 Interview term:` so you also learn the

Question 6

How do you choose the right model for a task?

Accepted Answer

**Target Audience:** Engineers adding their first LLM feature who see a long list of models and want a repeatable way to pick one.
**Difficulty:** Easy
**Verified with:** Node 22.22.0 and Python 3.12.3 (the selection harness below; the models in it are deterministic stubs and the prices are illustrative). Product and paper facts are cited inline with the date they were checked (October 2026).

> **How to read this doc:** Every idea is explained in plain language first, then tagged with `📌 Inter

Question 7

How do you build resilient LLM calls with rate limits, retries and fallbacks?

Accepted Answer

**Target Audience:** Engineers putting an LLM feature in front of real users who have seen their first 429 or 529 in production and want a resilience layer that is more than "wrap it in a try/catch and try again".
**Difficulty:** Medium
**Verified with:** Node 22.22.0 and Python 3.12.3 (both examples below run on a fake clock with stub providers, so the timing logic is real but no API is called). Provider behaviour (status codes, headers, SDK defaults, rate-limit algorithms) is quoted from the A

Question 8

How does streaming work in LLM APIs and why does it matter for UX?

Accepted Answer

**Target Audience:** Engineers building a chat or assistant feature who want to know what `stream: true` really changes, and what it breaks.
**Difficulty:** Medium
**Verified with:** Node 22.22.0 and Python 3.12.3 (both examples below, identical output). Provider event names were checked in October 2026 against the Anthropic streaming docs (platform.claude.com, "Streaming messages") and the type definitions of the official `openai` npm package (v7.36.0).

> **How to read this doc:** Every idea i

Question 9

How do you get reliable structured JSON output from an LLM?

Accepted Answer

**Target Audience:** Engineers whose LLM feature has to hand JSON to ordinary code (a parser, a database, an API) and who want it to stop breaking.
**Difficulty:** Medium
**Verified with:** Node 22.22.0 and Python 3.12.3 (the constrained-decoding simulation below). Provider API shapes were checked against the vendor docs and announcements cited inline, in October 2026.

> **How to read this doc:** Every idea is explained in plain language first, then tagged with `📌 Interview term:` so you also

Question 10

Embeddings in 2025-26: how do you choose model, dimensions, and distance?

Accepted Answer

**Target Audience:** Engineers setting up retrieval who must commit to an embedding model, a vector size and a metric, and want a defensible way to choose rather than copying a blog post.
**Difficulty:** Medium
**Verified with:** Node 22.22.0 and Python 3.12.3 (the truncation and quantization program below, on seeded synthetic vectors). Model facts, prices and limits are cited inline with the source and the date checked (October 2026); they change often, so treat them as examples of the mechanis

AI ENGINEERINGDECODED.

Easy

What is a Large Language Model and how does it generate text?

What is a vector database and when do you actually need one?

Context windows in 2025-26: how do you use 128k-1M tokens without drowning?

What are tokens and why do they matter for cost and limits?

What is a context window and what happens when you exceed it?

How do you choose the right model for a task?

Medium

How do you build resilient LLM calls with rate limits, retries and fallbacks?

How does streaming work in LLM APIs and why does it matter for UX?

How do you get reliable structured JSON output from an LLM?

Embeddings in 2025-26: how do you choose model, dimensions, and distance?

What are reasoning models and when should you use them?

What is the difference between fine-tuning, LoRA and prompt engineering?

Reranking in RAG: when and how to add a cross-encoder?

What is prompt caching and how does it reduce cost and latency?

Hard

How should an agent handle tool errors, timeouts and retries?

How do you evaluate a RAG pipeline end to end?

Guardrails for LLM apps: how do you implement input/output safety in 2025?

How do you evaluate and debug agent trajectories?

How would you design a production RAG pipeline end-to-end in 2025-26?

How do you version, test and safely roll out prompt changes?

pgvector vs Pinecone vs Qdrant: how do you choose a vector DB in 2025?

Fine-tune vs RAG vs prompt engineering: how do you choose in 2025-26?