Question 1

What is a Large Language Model and how does it generate text?

Accepted Answer

**Target Audience:** Beginners to AI engineering — you can write code and call an API, but you have never looked inside a language model.
**Difficulty:** Easy
**Verified with:** Node 22.22.0 and Python 3.12.3 (the toy model in the example; no real model is called). Product and paper facts are cited inline with the date they were checked (October 2026).

> **How to read this doc:** Every idea is explained in plain language first, then tagged with `📌 Interview term:` so you also learn the exact w

Question 2

What belongs in the system prompt versus the user message?

Accepted Answer

**Target Audience:** Engineers wiring their first chat-API feature who are unsure which text belongs in which role.
**Difficulty:** Easy
**Verified with:** Node 22.22.0 and Python 3.12.3 (a request builder with a simulated byte-exact prefix cache; no real model is called). Provider API facts are cited inline from their docs or forum posts, checked October 2026.

> **How to read this doc:** Every idea is explained in plain language first, then tagged with `📌 Interview term:` so you also learn th

Question 3

What are tokens and why do they matter for cost and limits?

Accepted Answer

**Target Audience:** Beginners who are about to ship (or pay for) their first LLM feature and keep seeing "tokens" on every pricing and limits page.
**Difficulty:** Easy
**Verified with:** Node 22.22.0 and Python 3.12.3 (a tiny BPE tokenizer and the cost arithmetic; no real tokenizer or model is called). Product facts and prices are cited inline with the date they were checked (October 2026).

> **How to read this doc:** Every idea is explained in plain language first, then tagged with `📌 Inter

Question 4

What are evals and why does every LLM feature need them?

Accepted Answer

**Target Audience:** Anyone building on top of an LLM API who has never written an eval, and candidates who need to explain why "it looked good in the playground" is not a test.
**Difficulty:** Easy
**Verified with:** Node 22 and Python 3.12 (an eval harness over two stubbed prompt versions, and a sampling simulation with Wilson intervals). No model is called; the stub replies stand in for a real model. Vendor guidance is quoted with its source, checked October 2026.

> **How to read this doc:**

Question 5

What are hallucinations and what causes them?

Accepted Answer

**Target Audience:** Anyone new to building with LLMs who has seen a model state something false with total confidence and wants to understand why, in plain terms.
**Difficulty:** Easy
**Verified with:** Node 22.22.0 and Python 3.12.3 (a counting toy, an expected-score calculation and a string-matching grounding check; no model is called). Paper and case facts are cited inline with the date they were checked (October 2026).

> **How to read this doc:** Every idea is explained in plain language f

Question 6

AI observability: how do you trace and log LLM calls in prod?

Accepted Answer

**Target Audience:** Developers who have an LLM feature or agent in production and want to see each model call, tool call and its timing; no prior observability experience assumed.
**Difficulty:** Easy
**Verified with:** Node 22.22.0 and Python 3.12.3 (the in-process tracer below, identical output in both). OpenTelemetry convention names, the W3C header format and SDK options were checked with web searches in October 2026; where only secondary sources were found, the text says so.

> **How to re

Question 7

How do you choose the right model for a task?

Accepted Answer

**Target Audience:** Engineers adding their first LLM feature who see a long list of models and want a repeatable way to pick one.
**Difficulty:** Easy
**Verified with:** Node 22.22.0 and Python 3.12.3 (the selection harness below; the models in it are deterministic stubs and the prices are illustrative). Product and paper facts are cited inline with the date they were checked (October 2026).

> **How to read this doc:** Every idea is explained in plain language first, then tagged with `📌 Inter

Question 8

When should you use batch APIs instead of real-time LLM calls?

Accepted Answer

**Target Audience:** Engineers who run LLM work that nobody watches live (backfills, enrichment, evals) and want to cut its cost and stop it competing with user traffic.
**Difficulty:** Easy
**Verified with:** Node 22.22.0 and Python 3.12.3 for the simulation below (identical output). Provider facts were checked in October 2026: Anthropic's "Batch processing" docs (platform.claude.com, fetched directly), the `openai@7.36.0` npm package type definitions, and, where only search results were reacha

Question 9

How do you build resilient LLM calls with rate limits, retries and fallbacks?

Accepted Answer

**Target Audience:** Engineers putting an LLM feature in front of real users who have seen their first 429 or 529 in production and want a resilience layer that is more than "wrap it in a try/catch and try again".
**Difficulty:** Medium
**Verified with:** Node 22.22.0 and Python 3.12.3 (both examples below run on a fake clock with stub providers, so the timing logic is real but no API is called). Provider behaviour (status codes, headers, SDK defaults, rate-limit algorithms) is quoted from the A

Question 10

How does streaming work in LLM APIs and why does it matter for UX?

Accepted Answer

**Target Audience:** Engineers building a chat or assistant feature who want to know what `stream: true` really changes, and what it breaks.
**Difficulty:** Medium
**Verified with:** Node 22.22.0 and Python 3.12.3 (both examples below, identical output). Provider event names were checked in October 2026 against the Anthropic streaming docs (platform.claude.com, "Streaming messages") and the type definitions of the official `openai` npm package (v7.36.0).

> **How to read this doc:** Every idea i

AI ENGINEERINGDECODED.

Easy

What is a Large Language Model and how does it generate text?

What belongs in the system prompt versus the user message?

What are tokens and why do they matter for cost and limits?

What are evals and why does every LLM feature need them?

What are hallucinations and what causes them?

AI observability: how do you trace and log LLM calls in prod?

How do you choose the right model for a task?

When should you use batch APIs instead of real-time LLM calls?

Medium

How do you build resilient LLM calls with rate limits, retries and fallbacks?

How does streaming work in LLM APIs and why does it matter for UX?

How do you get reliable structured JSON output from an LLM?

How do you build a golden dataset for evaluating an LLM feature?

When do you use code-based graders versus model-based graders?

How do multimodal models handle images, audio and other non-text inputs?

How do you estimate and control LLM costs in production?

Beyond Ragas: how do you eval LLMs with LLM-as-judge, DeepEval, and human eval?

Hard

How do you evaluate a RAG pipeline end to end?

Guardrails for LLM apps: how do you implement input/output safety in 2025?

How do you evaluate and debug agent trajectories?

Why are BLEU and ROUGE poor metrics for LLM outputs, and what replaced them?

How do you version, test and safely roll out prompt changes?

What is prompt injection and how do you defend against it?

How do you validate and repair structured LLM output at runtime?

Fine-tune vs RAG vs prompt engineering: how do you choose in 2025-26?