Question 1

What are evals and why does every LLM feature need them?

Accepted Answer

**Target Audience:** Anyone building on top of an LLM API who has never written an eval, and candidates who need to explain why "it looked good in the playground" is not a test.
**Difficulty:** Easy
**Verified with:** Node 22 and Python 3.12 (an eval harness over two stubbed prompt versions, and a sampling simulation with Wilson intervals). No model is called; the stub replies stand in for a real model. Vendor guidance is quoted with its source, checked October 2026.

> **How to read this doc:**

Question 2

How do you manage conversations that outgrow the context window?

Accepted Answer

**Target Audience:** Engineers running a chat assistant or agent whose sessions get long, and who need a history strategy that is cheaper than "send everything" without making the bot forget.
**Difficulty:** Medium
**Verified with:** Node 22.22.0 and Python 3.12.3 (the conversation replay below). Provider compaction features were checked against the Anthropic docs and OpenAI documentation summaries cited inline, in October 2026.

> **How to read this doc:** Every idea is explained in plain langu

Question 3

How do you catch regressions when changing a prompt or upgrading a model?

Accepted Answer

**Target Audience:** Engineers who own an LLM feature in production and must decide whether a prompt edit or a model upgrade is safe to ship.
**Difficulty:** Medium
**Verified with:** Node 22.22.0 and Python 3.12.3 (the regression gate below, with a seeded simulator). The promptfoo CI integration and the "Adding Error Bars to Evals" paper were checked via web search in October 2026.

> **How to read this doc:** Every idea is explained in plain language first, then tagged with `📌 Interview term:

Question 4

MCP authoring: how do you build a Model Context Protocol server in 2025-26?

Accepted Answer

**Target Audience:** Engineers asked to expose a product or internal system to AI hosts through MCP, and candidates who need to talk through the build end to end.
**Difficulty:** Hard
**Verified with:** Node 22.22.0 and Python 3.12.3 (both examples, identical output). SDK facts were read in October 2026 from the published packages: `@modelcontextprotocol/server` 2.3.1, `@modelcontextprotocol/core` 2.3.1, `@modelcontextprotocol/node` 2.1.1, `@modelcontextprotocol/sdk` 1.32.1 and the Python `mcp`

Question 5

What is prompt injection and how do you defend against it?

Accepted Answer

**Target Audience:** Engineers building LLM features that read outside content (web pages, emails, documents, tool results) or take actions, and anyone asked to explain why "just tell the model to ignore it" is not a defence.
**Difficulty:** Hard
**Verified with:** Node 22.22.0 and Python 3.12.3 (both examples below, identical output). Research and standards cited were checked by web search in October 2026: OWASP Top 10 for LLM Applications 2025 (via reference summaries), the OpenAI instruction

Question 6

What is red-teaming an LLM application and how is it done?

Accepted Answer

**Target Audience:** Engineers and tech leads asked to "red-team the assistant before launch" who want a method rather than a list of famous jailbreak prompts.
**Difficulty:** Hard
**Verified with:** Node 22.22.0 and Python 3.12.3 (the example below, identical output). Research, tools and standards mentioned were checked by web search in October 2026: Anthropic's Constitutional Classifiers announcement (February 2025), the Crescendo paper (Russinovich, Salem and Eldan; USENIX Security 2025), sec

AI ENGINEERINGDECODED.

Easy