Question 1

Context windows in 2025-26: how do you use 128k-1M tokens without drowning?

Accepted Answer

**Target Audience:** Engineers who have a 1M-token model available and need to decide when to use the window and when to keep retrieving.
**Difficulty:** Easy
**Verified with:** Node 22.22.0 and Python 3.12.3 (cost arithmetic, chunk reordering and a needle-test harness with canned answers; no real model is called). Model window sizes, prices and published guidance are cited inline with the date checked (October 2026); these change often, so check the provider docs for the model you use.

> **How

Question 2

What are evals and why does every LLM feature need them?

Accepted Answer

**Target Audience:** Anyone building on top of an LLM API who has never written an eval, and candidates who need to explain why "it looked good in the playground" is not a test.
**Difficulty:** Easy
**Verified with:** Node 22 and Python 3.12 (an eval harness over two stubbed prompt versions, and a sampling simulation with Wilson intervals). No model is called; the stub replies stand in for a real model. Vendor guidance is quoted with its source, checked October 2026.

> **How to read this doc:**

Question 3

How do you choose the right model for a task?

Accepted Answer

**Target Audience:** Engineers adding their first LLM feature who see a long list of models and want a repeatable way to pick one.
**Difficulty:** Easy
**Verified with:** Node 22.22.0 and Python 3.12.3 (the selection harness below; the models in it are deterministic stubs and the prices are illustrative). Product and paper facts are cited inline with the date they were checked (October 2026).

> **How to read this doc:** Every idea is explained in plain language first, then tagged with `📌 Inter

Question 4

How do you get reliable structured JSON output from an LLM?

Accepted Answer

**Target Audience:** Engineers whose LLM feature has to hand JSON to ordinary code (a parser, a database, an API) and who want it to stop breaking.
**Difficulty:** Medium
**Verified with:** Node 22.22.0 and Python 3.12.3 (the constrained-decoding simulation below). Provider API shapes were checked against the vendor docs and announcements cited inline, in October 2026.

> **How to read this doc:** Every idea is explained in plain language first, then tagged with `📌 Interview term:` so you also

Question 5

How do you build a golden dataset for evaluating an LLM feature?

Accepted Answer

**Target Audience:** Engineers who know why evals matter and now have to assemble the dataset behind them, and candidates asked to design one in an interview.
**Difficulty:** Medium
**Verified with:** Node 22 and Python 3.12 (a dataset builder over 200 simulated traces, and an inter-annotator agreement calculation). The traces and labels are synthetic and stated as such; no model is called.

> **How to read this doc:** Every idea is explained in plain language first, then tagged with `📌 Intervi

Question 6

What is the Model Context Protocol (MCP) and what problem does it solve?

Accepted Answer

**Target Audience:** Engineers who have used tool calling and keep hearing "just ship an MCP server", and anyone preparing for an AI engineering interview.
**Difficulty:** Medium
**Verified with:** Node 22.22.0 and Python 3.12.3 (both examples, identical output). Protocol details were checked in October 2026 against the published SDK packages themselves: `@modelcontextprotocol/sdk` 1.32.1 (the v1 line, spec up to 2025-11-25), `@modelcontextprotocol/server` and `@modelcontextprotocol/core` 2.3.1

Question 7

Embeddings in 2025-26: how do you choose model, dimensions, and distance?

Accepted Answer

**Target Audience:** Engineers setting up retrieval who must commit to an embedding model, a vector size and a metric, and want a defensible way to choose rather than copying a blog post.
**Difficulty:** Medium
**Verified with:** Node 22.22.0 and Python 3.12.3 (the truncation and quantization program below, on seeded synthetic vectors). Model facts, prices and limits are cited inline with the source and the date checked (October 2026); they change often, so treat them as examples of the mechanis

Question 8

When do you use code-based graders versus model-based graders?

Accepted Answer

**Target Audience:** Engineers writing their first eval suite who need to decide, check by check, whether to write a function or ask a model, and candidates who should be able to defend that choice.
**Difficulty:** Medium
**Verified with:** Node 22 and Python 3.12 (five code graders and a stub judge over ten outputs, and a judge-versus-human validation with corrected pass rates). Judge verdicts are canned stand-ins for model calls; token prices in the cost line are assumptions. Vendor tooling is

Question 9

How do multimodal models handle images, audio and other non-text inputs?

Accepted Answer

**Target Audience:** Engineers who have used text-only LLM APIs and now need to send screenshots, scans, photos or audio.
**Difficulty:** Medium
**Verified with:** Node 22.22.0 and Python 3.12.3 (the patch-projection demo and the token estimators below; no real model is called). Provider token rules and paper details are cited inline with the source and the date checked (October 2026).

> **How to read this doc:** Every idea is explained in plain language first, then tagged with `📌 Interview te

Question 10

How do you estimate and control LLM costs in production?

Accepted Answer

**Target Audience:** Engineers asked "what will this cost per month?" before an LLM launch, and anyone whose first invoice was a surprise.
**Difficulty:** Medium
**Verified with:** Node 22.22.0 and Python 3.12.3 (a cost estimator and a metering ledger). All prices in the examples are a **hypothetical price card** with round numbers so the arithmetic is easy to follow; they are not any vendor's current prices. Discount mechanics (cache read and write multipliers, batch discounts) are quoted from

AI ENGINEERINGDECODED.

Easy

Context windows in 2025-26: how do you use 128k-1M tokens without drowning?

What are evals and why does every LLM feature need them?

How do you choose the right model for a task?

Medium

How do you get reliable structured JSON output from an LLM?

How do you build a golden dataset for evaluating an LLM feature?

What is the Model Context Protocol (MCP) and what problem does it solve?

Embeddings in 2025-26: how do you choose model, dimensions, and distance?

When do you use code-based graders versus model-based graders?

How do multimodal models handle images, audio and other non-text inputs?

How do you estimate and control LLM costs in production?

Cost optimization for LLM apps: how do you keep the bill predictable?

Hard

How do you evaluate a RAG pipeline end to end?

When should you split work across multiple agents instead of one?

How would you design a production RAG pipeline end-to-end in 2025-26?

Why are BLEU and ROUGE poor metrics for LLM outputs, and what replaced them?

How do you version, test and safely roll out prompt changes?

How do you monitor LLM quality drift in production?

What techniques reduce LLM latency at each stage of a request?

What is quantization and when would you use a quantized model?