Question 1

Context windows in 2025-26: how do you use 128k-1M tokens without drowning?

Accepted Answer

**Target Audience:** Engineers who have a 1M-token model available and need to decide when to use the window and when to keep retrieving.
**Difficulty:** Easy
**Verified with:** Node 22.22.0 and Python 3.12.3 (cost arithmetic, chunk reordering and a needle-test harness with canned answers; no real model is called). Model window sizes, prices and published guidance are cited inline with the date checked (October 2026); these change often, so check the provider docs for the model you use.

> **How

Question 2

What are tokens and why do they matter for cost and limits?

Accepted Answer

**Target Audience:** Beginners who are about to ship (or pay for) their first LLM feature and keep seeing "tokens" on every pricing and limits page.
**Difficulty:** Easy
**Verified with:** Node 22.22.0 and Python 3.12.3 (a tiny BPE tokenizer and the cost arithmetic; no real tokenizer or model is called). Product facts and prices are cited inline with the date they were checked (October 2026).

> **How to read this doc:** Every idea is explained in plain language first, then tagged with `📌 Inter

Question 3

What is the difference between zero-shot, one-shot and few-shot prompting?

Accepted Answer

**Target Audience:** Beginners writing their first production prompts, and anyone who needs to explain in-context learning precisely in an interview.
**Difficulty:** Easy
**Verified with:** Node 22.22.0 and Python 3.12.3 (a ticket-classifier harness with a deterministic stub model; no real model is called). Paper results are cited inline, checked October 2026.

> **How to read this doc:** Every idea is explained in plain language first, then tagged with `📌 Interview term:` so you also learn the

Question 4

What is a context window and what happens when you exceed it?

Accepted Answer

**Target Audience:** Beginners building chat features, RAG or agents who have hit (or will hit) "context too long" errors.
**Difficulty:** Easy
**Verified with:** Node 22.22.0 and Python 3.12.3 (a simulated provider with a 200-token window; no real model is called). Provider behaviour is cited inline from documentation and forum reports, checked October 2026.

> **How to read this doc:** Every idea is explained in plain language first, then tagged with `📌 Interview term:` so you also learn the

Question 5

How do you choose the right model for a task?

Accepted Answer

**Target Audience:** Engineers adding their first LLM feature who see a long list of models and want a repeatable way to pick one.
**Difficulty:** Easy
**Verified with:** Node 22.22.0 and Python 3.12.3 (the selection harness below; the models in it are deterministic stubs and the prices are illustrative). Product and paper facts are cited inline with the date they were checked (October 2026).

> **How to read this doc:** Every idea is explained in plain language first, then tagged with `📌 Inter

Question 6

When should you use batch APIs instead of real-time LLM calls?

Accepted Answer

**Target Audience:** Engineers who run LLM work that nobody watches live (backfills, enrichment, evals) and want to cut its cost and stop it competing with user traffic.
**Difficulty:** Easy
**Verified with:** Node 22.22.0 and Python 3.12.3 for the simulation below (identical output). Provider facts were checked in October 2026: Anthropic's "Batch processing" docs (platform.claude.com, fetched directly), the `openai@7.36.0` npm package type definitions, and, where only search results were reacha

Question 7

How do you build resilient LLM calls with rate limits, retries and fallbacks?

Accepted Answer

**Target Audience:** Engineers putting an LLM feature in front of real users who have seen their first 429 or 529 in production and want a resilience layer that is more than "wrap it in a try/catch and try again".
**Difficulty:** Medium
**Verified with:** Node 22.22.0 and Python 3.12.3 (both examples below run on a fake clock with stub providers, so the timing logic is real but no API is called). Provider behaviour (status codes, headers, SDK defaults, rate-limit algorithms) is quoted from the A

Question 8

How does streaming work in LLM APIs and why does it matter for UX?

Accepted Answer

**Target Audience:** Engineers building a chat or assistant feature who want to know what `stream: true` really changes, and what it breaks.
**Difficulty:** Medium
**Verified with:** Node 22.22.0 and Python 3.12.3 (both examples below, identical output). Provider event names were checked in October 2026 against the Anthropic streaming docs (platform.claude.com, "Streaming messages") and the type definitions of the official `openai` npm package (v7.36.0).

> **How to read this doc:** Every idea i

Question 9

How do you get reliable structured JSON output from an LLM?

Accepted Answer

**Target Audience:** Engineers whose LLM feature has to hand JSON to ordinary code (a parser, a database, an API) and who want it to stop breaking.
**Difficulty:** Medium
**Verified with:** Node 22.22.0 and Python 3.12.3 (the constrained-decoding simulation below). Provider API shapes were checked against the vendor docs and announcements cited inline, in October 2026.

> **How to read this doc:** Every idea is explained in plain language first, then tagged with `📌 Interview term:` so you also

Question 10

How do you build a golden dataset for evaluating an LLM feature?

Accepted Answer

**Target Audience:** Engineers who know why evals matter and now have to assemble the dataset behind them, and candidates asked to design one in an interview.
**Difficulty:** Medium
**Verified with:** Node 22 and Python 3.12 (a dataset builder over 200 simulated traces, and an inter-annotator agreement calculation). The traces and labels are synthetic and stated as such; no model is called.

> **How to read this doc:** Every idea is explained in plain language first, then tagged with `📌 Intervi

AI ENGINEERINGDECODED.

Easy

Context windows in 2025-26: how do you use 128k-1M tokens without drowning?

What are tokens and why do they matter for cost and limits?

What is the difference between zero-shot, one-shot and few-shot prompting?

What is a context window and what happens when you exceed it?

How do you choose the right model for a task?

When should you use batch APIs instead of real-time LLM calls?

Medium

How do you build resilient LLM calls with rate limits, retries and fallbacks?

How does streaming work in LLM APIs and why does it matter for UX?

How do you get reliable structured JSON output from an LLM?

How do you build a golden dataset for evaluating an LLM feature?

Embeddings in 2025-26: how do you choose model, dimensions, and distance?

When do you use code-based graders versus model-based graders?

What are reasoning models and when should you use them?

How do multimodal models handle images, audio and other non-text inputs?

Hard

How should an agent handle tool errors, timeouts and retries?

How do you evaluate a RAG pipeline end to end?

Guardrails for LLM apps: how do you implement input/output safety in 2025?

What is semantic caching and when is it safe to use?

When should you split work across multiple agents instead of one?

How do you evaluate and debug agent trajectories?

How would you design a production RAG pipeline end-to-end in 2025-26?

MCP authoring: how do you build a Model Context Protocol server in 2025-26?