Question 1

What belongs in the system prompt versus the user message?

Accepted Answer

**Target Audience:** Engineers wiring their first chat-API feature who are unsure which text belongs in which role.
**Difficulty:** Easy
**Verified with:** Node 22.22.0 and Python 3.12.3 (a request builder with a simulated byte-exact prefix cache; no real model is called). Provider API facts are cited inline from their docs or forum posts, checked October 2026.

> **How to read this doc:** Every idea is explained in plain language first, then tagged with `📌 Interview term:` so you also learn th

Question 2

Context windows in 2025-26: how do you use 128k-1M tokens without drowning?

Accepted Answer

**Target Audience:** Engineers who have a 1M-token model available and need to decide when to use the window and when to keep retrieving.
**Difficulty:** Easy
**Verified with:** Node 22.22.0 and Python 3.12.3 (cost arithmetic, chunk reordering and a needle-test harness with canned answers; no real model is called). Model window sizes, prices and published guidance are cited inline with the date checked (October 2026); these change often, so check the provider docs for the model you use.

> **How

Question 3

When should you use batch APIs instead of real-time LLM calls?

Accepted Answer

**Target Audience:** Engineers who run LLM work that nobody watches live (backfills, enrichment, evals) and want to cut its cost and stop it competing with user traffic.
**Difficulty:** Easy
**Verified with:** Node 22.22.0 and Python 3.12.3 for the simulation below (identical output). Provider facts were checked in October 2026: Anthropic's "Batch processing" docs (platform.claude.com, fetched directly), the `openai@7.36.0` npm package type definitions, and, where only search results were reacha

Question 4

How does streaming work in LLM APIs and why does it matter for UX?

Accepted Answer

**Target Audience:** Engineers building a chat or assistant feature who want to know what `stream: true` really changes, and what it breaks.
**Difficulty:** Medium
**Verified with:** Node 22.22.0 and Python 3.12.3 (both examples below, identical output). Provider event names were checked in October 2026 against the Anthropic streaming docs (platform.claude.com, "Streaming messages") and the type definitions of the official `openai` npm package (v7.36.0).

> **How to read this doc:** Every idea i

Question 5

When do you use code-based graders versus model-based graders?

Accepted Answer

**Target Audience:** Engineers writing their first eval suite who need to decide, check by check, whether to write a function or ask a model, and candidates who should be able to defend that choice.
**Difficulty:** Medium
**Verified with:** Node 22 and Python 3.12 (five code graders and a stub judge over ten outputs, and a judge-versus-human validation with corrected pass rates). Judge verdicts are canned stand-ins for model calls; token prices in the cost line are assumptions. Vendor tooling is

Question 6

Beyond Ragas: how do you eval LLMs with LLM-as-judge, DeepEval, and human eval?

Accepted Answer

**Target Audience:** Engineers who have one eval framework running (often Ragas on a RAG endpoint) and now need to evaluate the rest of an LLM product: summaries, support replies, agents, anything without a neat retrieval score.
**Difficulty:** Medium
**Verified with:** Node 22.22.0 and Python 3.12.3 (the judge-calibration and review-routing example below). DeepEval and G-Eval details were checked against the DeepEval docs, PyPI and the G-Eval paper via web search in October 2026; where only sec

Question 7

What is context engineering and how does it differ from prompt engineering?

Accepted Answer

**Target Audience:** Engineers moving from single prompts to RAG systems and agents, who need to explain what changes when "the prompt" becomes a pipeline.
**Difficulty:** Medium
**Verified with:** Node 22.22.0 and Python 3.12.3 (a context assembler run over simulated candidate items with given token counts; no real model is called). Origins of the term and published guidance are cited inline, checked October 2026.

> **How to read this doc:** Every idea is explained in plain language first, the

Question 8

Streaming LLM responses to the UI: SSE, chunking, abort, and resume?

Accepted Answer

**Target Audience:** Frontend and full-stack engineers wiring a streamed LLM answer into a real UI that has to survive slow networks, Stop buttons and dropped connections.
**Difficulty:** Medium
**Verified with:** Node 22.22.0 and Python 3.12.3 (both examples below, identical output). AI SDK details were read from the published `ai@5.0.277` and `@ai-sdk/react@2.0.280` packages and the docs shipped inside `ai@7.0.143`; OpenAI SDK details from `openai@7.36.0` type definitions; all checked in Octob

Question 9

What is hybrid search and why combine BM25 with vector search?

Accepted Answer

**Target Audience:** Engineers whose embedding-only RAG search misses exact terms, and anyone who needs to explain lexical vs semantic retrieval and why teams combine them.
**Difficulty:** Medium
**Verified with:** Node 22.22.0 and Python 3.12.3 (BM25 written from scratch, a deterministic stub embedding, and RRF fusion on an eight-article knowledge base; no model or search engine is called). Engine facts were checked in each vendor's documentation in October 2026.

> **How to read this doc:** Ev

Question 10

Vercel AI SDK 5 streamText: how do you stream LLM chat to React?

Accepted Answer

**Target Audience:** React and Next.js developers building a chat page with the Vercel AI SDK 5 who want to understand what flows between `streamText` and `useChat`, not just copy the starter.
**Difficulty:** Medium
**Verified with:** Node 22.22.0 and Python 3.12.3 for the protocol simulation below (identical output). AI SDK facts were read in October 2026 from the published npm packages `ai@5.0.277` and `@ai-sdk/react@2.0.280` (type definitions and source) and the migration guides shipped in `a

AI ENGINEERINGDECODED.

Easy

What belongs in the system prompt versus the user message?

Context windows in 2025-26: how do you use 128k-1M tokens without drowning?

When should you use batch APIs instead of real-time LLM calls?

Medium

How does streaming work in LLM APIs and why does it matter for UX?

When do you use code-based graders versus model-based graders?

Beyond Ragas: how do you eval LLMs with LLM-as-judge, DeepEval, and human eval?

What is context engineering and how does it differ from prompt engineering?

Streaming LLM responses to the UI: SSE, chunking, abort, and resume?

What is hybrid search and why combine BM25 with vector search?

Vercel AI SDK 5 streamText: how do you stream LLM chat to React?

Hard

What is semantic caching and when is it safe to use?

How do you version, test and safely roll out prompt changes?

What is quantization and when would you use a quantized model?

What is red-teaming an LLM application and how is it done?