Question 1

What are tokens and why do they matter for cost and limits?

Accepted Answer

**Target Audience:** Beginners who are about to ship (or pay for) their first LLM feature and keep seeing "tokens" on every pricing and limits page.
**Difficulty:** Easy
**Verified with:** Node 22.22.0 and Python 3.12.3 (a tiny BPE tokenizer and the cost arithmetic; no real tokenizer or model is called). Product facts and prices are cited inline with the date they were checked (October 2026).

> **How to read this doc:** Every idea is explained in plain language first, then tagged with `📌 Inter

Question 2

What is a context window and what happens when you exceed it?

Accepted Answer

**Target Audience:** Beginners building chat features, RAG or agents who have hit (or will hit) "context too long" errors.
**Difficulty:** Easy
**Verified with:** Node 22.22.0 and Python 3.12.3 (a simulated provider with a 200-token window; no real model is called). Provider behaviour is cited inline from documentation and forum reports, checked October 2026.

> **How to read this doc:** Every idea is explained in plain language first, then tagged with `📌 Interview term:` so you also learn the

Question 3

When should you use batch APIs instead of real-time LLM calls?

Accepted Answer

**Target Audience:** Engineers who run LLM work that nobody watches live (backfills, enrichment, evals) and want to cut its cost and stop it competing with user traffic.
**Difficulty:** Easy
**Verified with:** Node 22.22.0 and Python 3.12.3 for the simulation below (identical output). Provider facts were checked in October 2026: Anthropic's "Batch processing" docs (platform.claude.com, fetched directly), the `openai@7.36.0` npm package type definitions, and, where only search results were reacha

Question 4

How do you build resilient LLM calls with rate limits, retries and fallbacks?

Accepted Answer

**Target Audience:** Engineers putting an LLM feature in front of real users who have seen their first 429 or 529 in production and want a resilience layer that is more than "wrap it in a try/catch and try again".
**Difficulty:** Medium
**Verified with:** Node 22.22.0 and Python 3.12.3 (both examples below run on a fake clock with stub providers, so the timing logic is real but no API is called). Provider behaviour (status codes, headers, SDK defaults, rate-limit algorithms) is quoted from the A

Question 5

Cost optimization for LLM apps: how do you keep the bill predictable?

Accepted Answer

**Target Audience:** Engineers and tech leads who already know roughly what their LLM features cost, but whose monthly bill keeps surprising finance, and who need it to land within a known range.
**Difficulty:** Medium
**Verified with:** Node 22.22.0 and Python 3.12.3 (a 30-day spend simulation with and without controls, and a worst-case and reserved-capacity calculator). All prices and traffic in the examples are **hypothetical**; provider budget and capacity features are quoted from the Anthro

Question 6

How do you design good tool definitions for an LLM?

Accepted Answer

**Target Audience:** Engineers turning a backend (REST endpoints, database queries, internal scripts) into tools for an LLM or agent, and anyone who has watched a model pick the wrong tool.
**Difficulty:** Medium
**Verified with:** Node 22.22.0 and Python 3.12.3 (a tool-definition linter and a tool-selection eval). Vendor guidance is quoted from the Claude docs ("Define tools", "Handle tool calls"), Anthropic engineering posts and other sources as cited, checked October 2026.

> **How to read th

Question 7

Evaluating RAG with Ragas: faithfulness, answer relevancy, context precision?

Accepted Answer

**Target Audience:** Engineers who run or are about to run Ragas on a RAG system and want to know what each number means, how it is computed, and what to fix when it moves.
**Difficulty:** Medium
**Verified with:** Node 22.22.0 and Python 3.12.3 (a hand-computed version of the four formulas below). Metric definitions were checked against the Ragas documentation pages for v0.2 to v0.4 via web search in October 2026; API details that only appeared in third-party guides are marked as such.

> **How

Question 8

How do you handle PII and sensitive data when calling LLM APIs?

Accepted Answer

**Target Audience:** Engineers putting a hosted LLM behind a product that touches customer data (support, CRM, HR, health, finance) and who need a design legal and security can sign off.
**Difficulty:** Medium
**Verified with:** Node 22.22.0 and Python 3.12.3 (the redaction pipeline below). Provider retention terms were checked against the Anthropic docs and OpenAI's data-controls guide as summarised in search results, in October 2026. This is engineering guidance, not legal advice.

> **How to

Question 9

Streaming LLM responses to the UI: SSE, chunking, abort, and resume?

Accepted Answer

**Target Audience:** Frontend and full-stack engineers wiring a streamed LLM answer into a real UI that has to survive slow networks, Stop buttons and dropped connections.
**Difficulty:** Medium
**Verified with:** Node 22.22.0 and Python 3.12.3 (both examples below, identical output). AI SDK details were read from the published `ai@5.0.277` and `@ai-sdk/react@2.0.280` packages and the docs shipped inside `ai@7.0.143`; OpenAI SDK details from `openai@7.36.0` type definitions; all checked in Octob

Question 10

What is the ReAct pattern and why did it become the default for agents?

Accepted Answer

**Target Audience:** Engineers who know what an agent loop is and want to understand the pattern most agent frameworks are built on, including its history and what is still true today.
**Difficulty:** Medium
**Verified with:** Node 22.22.0 and Python 3.12.3 (the example below, identical output). Paper details were checked in October 2026 against the arXiv listing (2210.03629), the ICLR 2023 proceedings listing and the Google Research blog post on ReAct (November 2022).

> **How to read this doc:

AI ENGINEERINGDECODED.

Easy

What are tokens and why do they matter for cost and limits?

What is a context window and what happens when you exceed it?

When should you use batch APIs instead of real-time LLM calls?

Medium

How do you build resilient LLM calls with rate limits, retries and fallbacks?

Cost optimization for LLM apps: how do you keep the bill predictable?

How do you design good tool definitions for an LLM?

Evaluating RAG with Ragas: faithfulness, answer relevancy, context precision?

How do you handle PII and sensitive data when calling LLM APIs?

Streaming LLM responses to the UI: SSE, chunking, abort, and resume?

What is the ReAct pattern and why did it become the default for agents?

Hard

How do you sandbox an agent and enforce least-privilege tool access?

Guardrails for LLM apps: how do you implement input/output safety in 2025?

How would you design a production RAG pipeline end-to-end in 2025-26?

What is prompt injection and how do you defend against it?

Guardrails for AI agents: rate limits, budget caps, and HITL?

pgvector vs Pinecone vs Qdrant: how do you choose a vector DB in 2025?