Question 1

What is a Large Language Model and how does it generate text?

Accepted Answer

**Target Audience:** Beginners to AI engineering — you can write code and call an API, but you have never looked inside a language model.
**Difficulty:** Easy
**Verified with:** Node 22.22.0 and Python 3.12.3 (the toy model in the example; no real model is called). Product and paper facts are cited inline with the date they were checked (October 2026).

> **How to read this doc:** Every idea is explained in plain language first, then tagged with `📌 Interview term:` so you also learn the exact w

Question 2

What are embeddings and what are they used for?

Accepted Answer

**Target Audience:** Engineers who keep hearing "it runs on embeddings" (search, RAG, recommendations) and want to know what an embedding actually is before building with one.
**Difficulty:** Easy
**Verified with:** Node 22.22.0 and Python 3.12.3 (the toy embedding program below; its 6-number word vectors are hand-written stand-ins for a learned model). Product facts are cited inline with the date they were checked (October 2026).

> **How to read this doc:** Every idea is explained in plain lan

Question 3

What belongs in the system prompt versus the user message?

Accepted Answer

**Target Audience:** Engineers wiring their first chat-API feature who are unsure which text belongs in which role.
**Difficulty:** Easy
**Verified with:** Node 22.22.0 and Python 3.12.3 (a request builder with a simulated byte-exact prefix cache; no real model is called). Provider API facts are cited inline from their docs or forum posts, checked October 2026.

> **How to read this doc:** Every idea is explained in plain language first, then tagged with `📌 Interview term:` so you also learn th

Question 4

What is a vector database and when do you actually need one?

Accepted Answer

**Target Audience:** Engineers starting a RAG or semantic search project who keep being told to "add a vector database" and want to know what that is and whether they need one yet.
**Difficulty:** Easy
**Verified with:** Node 22.22.0 and Python 3.12.3 (the sizing, filtered search and sync simulation below). The per-core speed in the sizing table is a stated assumption, not a measurement. Product facts are cited inline with the date checked (October 2026).

> **How to read this doc:** Every idea

Question 5

Context windows in 2025-26: how do you use 128k-1M tokens without drowning?

Accepted Answer

**Target Audience:** Engineers who have a 1M-token model available and need to decide when to use the window and when to keep retrieving.
**Difficulty:** Easy
**Verified with:** Node 22.22.0 and Python 3.12.3 (cost arithmetic, chunk reordering and a needle-test harness with canned answers; no real model is called). Model window sizes, prices and published guidance are cited inline with the date checked (October 2026); these change often, so check the provider docs for the model you use.

> **How

Question 6

What is RAG and why use it instead of fine-tuning for knowledge?

Accepted Answer

**Target Audience:** Beginners meeting retrieval-augmented generation for the first time, and engineers who need to defend "RAG, not fine-tuning" for a knowledge feature in an interview.
**Difficulty:** Easy
**Verified with:** Node 22.22.0 and Python 3.12.3 (a keyword retriever plus deterministic stub models; no real model or vector database is called). Papers cited inline were checked in October 2026.

> **How to read this doc:** Every idea is explained in plain language first, then tagged with

Question 7

What are tokens and why do they matter for cost and limits?

Accepted Answer

**Target Audience:** Beginners who are about to ship (or pay for) their first LLM feature and keep seeing "tokens" on every pricing and limits page.
**Difficulty:** Easy
**Verified with:** Node 22.22.0 and Python 3.12.3 (a tiny BPE tokenizer and the cost arithmetic; no real tokenizer or model is called). Product facts and prices are cited inline with the date they were checked (October 2026).

> **How to read this doc:** Every idea is explained in plain language first, then tagged with `📌 Inter

Question 8

What are evals and why does every LLM feature need them?

Accepted Answer

**Target Audience:** Anyone building on top of an LLM API who has never written an eval, and candidates who need to explain why "it looked good in the playground" is not a test.
**Difficulty:** Easy
**Verified with:** Node 22 and Python 3.12 (an eval harness over two stubbed prompt versions, and a sampling simulation with Wilson intervals). No model is called; the stub replies stand in for a real model. Vendor guidance is quoted with its source, checked October 2026.

> **How to read this doc:**

Question 9

What are hallucinations and what causes them?

Accepted Answer

**Target Audience:** Anyone new to building with LLMs who has seen a model state something false with total confidence and wants to understand why, in plain terms.
**Difficulty:** Easy
**Verified with:** Node 22.22.0 and Python 3.12.3 (a counting toy, an expected-score calculation and a string-matching grounding check; no model is called). Paper and case facts are cited inline with the date they were checked (October 2026).

> **How to read this doc:** Every idea is explained in plain language f

Question 10

AI observability: how do you trace and log LLM calls in prod?

Accepted Answer

**Target Audience:** Developers who have an LLM feature or agent in production and want to see each model call, tool call and its timing; no prior observability experience assumed.
**Difficulty:** Easy
**Verified with:** Node 22.22.0 and Python 3.12.3 (the in-process tracer below, identical output in both). OpenTelemetry convention names, the W3C header format and SDK options were checked with web searches in October 2026; where only secondary sources were found, the text says so.

> **How to re

AI ENGINEERINGDECODED.

Easy

What is a Large Language Model and how does it generate text?

What are embeddings and what are they used for?

What belongs in the system prompt versus the user message?

What is a vector database and when do you actually need one?

Context windows in 2025-26: how do you use 128k-1M tokens without drowning?

What is RAG and why use it instead of fine-tuning for knowledge?

What are tokens and why do they matter for cost and limits?

What are evals and why does every LLM feature need them?

Medium

How do you build resilient LLM calls with rate limits, retries and fallbacks?

How does streaming work in LLM APIs and why does it matter for UX?

What is the difference between a base model, an instruction-tuned model and a chat model?

How do you get reliable structured JSON output from an LLM?

How do you build a golden dataset for evaluating an LLM feature?

What is the Model Context Protocol (MCP) and what problem does it solve?

Embeddings in 2025-26: how do you choose model, dimensions, and distance?

When do you use code-based graders versus model-based graders?

Hard

How should an agent handle tool errors, timeouts and retries?

How do you sandbox an agent and enforce least-privilege tool access?

How do you evaluate a RAG pipeline end to end?

Guardrails for LLM apps: how do you implement input/output safety in 2025?

What is semantic caching and when is it safe to use?

When should you split work across multiple agents instead of one?

How do you evaluate and debug agent trajectories?

How would you design a production RAG pipeline end-to-end in 2025-26?