Question 1

How does streaming work in LLM APIs and why does it matter for UX?

Accepted Answer

**Target Audience:** Engineers building a chat or assistant feature who want to know what `stream: true` really changes, and what it breaks.
**Difficulty:** Medium
**Verified with:** Node 22.22.0 and Python 3.12.3 (both examples below, identical output). Provider event names were checked in October 2026 against the Anthropic streaming docs (platform.claude.com, "Streaming messages") and the type definitions of the official `openai` npm package (v7.36.0).

> **How to read this doc:** Every idea i

Question 2

How do you design human-in-the-loop approval for agent actions?

Accepted Answer

**Target Audience:** Engineers adding approval steps to agents that can spend money, message people or change data, and candidates asked to design that step.
**Difficulty:** Medium
**Verified with:** Node 22.22.0 and Python 3.12.3 (both examples, identical output). Framework APIs were checked in October 2026: AI SDK `ai` 7.0.143 type definitions (read from the published package), the LangGraph and OpenAI Agents SDK docs (via web search), and the `@modelcontextprotocol/server` 2.3.1 doc comments.

Question 3

What are guardrails and how do you implement them around a model?

Accepted Answer

**Target Audience:** Engineers adding an LLM feature to a product who need to explain, and build, the checks that run around the model call.
**Difficulty:** Medium
**Verified with:** Node 22.22.0 and Python 3.12.3 (both examples below, identical output). Product facts (Llama Guard 4, Prompt Guard 2, NeMo Guardrails rail types, Guardrails AI on-fail actions, the OpenAI moderation endpoint, Amazon Bedrock ApplyGuardrail) were checked by web search in October 2026 and are cited where they appear.

Question 4

Guardrails for LLM apps: how do you implement input/output safety in 2025?

Accepted Answer

**Target Audience:** Engineers who own a production LLM feature and have to make its safety checks fast, reliable and measurable, not just present.
**Difficulty:** Hard
**Verified with:** Node 22.22.0 and Python 3.12.3 (both examples below, identical output). Vendor facts were checked by web search in October 2026: the Amazon Bedrock user guide ("Configure streaming response behavior to filter content") and API reference (`ApplyGuardrail`), Microsoft Learn (Prompt Shields), Meta model cards (Lla

Question 5

What is semantic caching and when is it safe to use?

Accepted Answer

**Target Audience:** Engineers with repetitive LLM traffic (support bots, FAQ assistants, internal help desks) who have been told a semantic cache will halve the bill, and want to know when that is true and when it serves wrong answers.
**Difficulty:** Hard
**Verified with:** Node 22.22.0 and Python 3.12.3 (a semantic cache with safety rails, and a threshold sweep on labelled pairs). The embedding in both examples is a **toy stand-in** (words mapped to concepts) so the similarity numbers are exp

Question 6

How do you evaluate and debug agent trajectories?

Accepted Answer

**Target Audience:** Engineers building tool-using agents who can already evaluate a single LLM response and now need to evaluate, and debug, multi-step runs with real side effects.
**Difficulty:** Hard
**Verified with:** Node 22.22.0 and Python 3.12.3 (the trajectory scorer below). The τ-bench definitions and results, and the LangChain agentevals match modes, were checked via web search in October 2026; the agentevals per-mode rules were only found in secondary sources and are marked as such.

Question 7

MCP authoring: how do you build a Model Context Protocol server in 2025-26?

Accepted Answer

**Target Audience:** Engineers asked to expose a product or internal system to AI hosts through MCP, and candidates who need to talk through the build end to end.
**Difficulty:** Hard
**Verified with:** Node 22.22.0 and Python 3.12.3 (both examples, identical output). SDK facts were read in October 2026 from the published packages: `@modelcontextprotocol/server` 2.3.1, `@modelcontextprotocol/core` 2.3.1, `@modelcontextprotocol/node` 2.1.1, `@modelcontextprotocol/sdk` 1.32.1 and the Python `mcp`

Question 8

Why are BLEU and ROUGE poor metrics for LLM outputs, and what replaced them?

Accepted Answer

**Target Audience:** Engineers who have seen BLEU or ROUGE in a dashboard or a paper and need to explain why they mislead for LLM features, and candidates expected to know the modern metric stack.
**Difficulty:** Hard
**Verified with:** Node 22 and Python 3.12 (BLEU, ROUGE-1 and ROUGE-L implemented from scratch, scored on hand-written answers next to a key-fact check, then against different references). Published results are cited by paper, checked October 2026.

> **How to read this doc:** Ever

Question 9

How do you monitor LLM quality drift in production?

Accepted Answer

**Target Audience:** Engineers and tech leads running an LLM feature in production who need to know when it is getting worse without anyone having changed it; also anyone preparing for an AI engineering interview.
**Difficulty:** Hard
**Verified with:** Node 22.22.0 and Python 3.12.3 (the 21-day drift monitor simulation below, identical output in both). The published incidents cited were checked with web searches in October 2026.

> **How to read this doc:** Every idea is explained in plain lang

AI ENGINEERINGDECODED.

Medium