Question 1

What are embeddings and what are they used for?

Accepted Answer

**Target Audience:** Engineers who keep hearing "it runs on embeddings" (search, RAG, recommendations) and want to know what an embedding actually is before building with one.
**Difficulty:** Easy
**Verified with:** Node 22.22.0 and Python 3.12.3 (the toy embedding program below; its 6-number word vectors are hand-written stand-ins for a learned model). Product facts are cited inline with the date they were checked (October 2026).

> **How to read this doc:** Every idea is explained in plain lan

Question 2

Context windows in 2025-26: how do you use 128k-1M tokens without drowning?

Accepted Answer

**Target Audience:** Engineers who have a 1M-token model available and need to decide when to use the window and when to keep retrieving.
**Difficulty:** Easy
**Verified with:** Node 22.22.0 and Python 3.12.3 (cost arithmetic, chunk reordering and a needle-test harness with canned answers; no real model is called). Model window sizes, prices and published guidance are cited inline with the date checked (October 2026); these change often, so check the provider docs for the model you use.

> **How

Question 3

What is RAG and why use it instead of fine-tuning for knowledge?

Accepted Answer

**Target Audience:** Beginners meeting retrieval-augmented generation for the first time, and engineers who need to defend "RAG, not fine-tuning" for a knowledge feature in an interview.
**Difficulty:** Easy
**Verified with:** Node 22.22.0 and Python 3.12.3 (a keyword retriever plus deterministic stub models; no real model or vector database is called). Papers cited inline were checked in October 2026.

> **How to read this doc:** Every idea is explained in plain language first, then tagged with

Question 4

What are evals and why does every LLM feature need them?

Accepted Answer

**Target Audience:** Anyone building on top of an LLM API who has never written an eval, and candidates who need to explain why "it looked good in the playground" is not a test.
**Difficulty:** Easy
**Verified with:** Node 22 and Python 3.12 (an eval harness over two stubbed prompt versions, and a sampling simulation with Wilson intervals). No model is called; the stub replies stand in for a real model. Vendor guidance is quoted with its source, checked October 2026.

> **How to read this doc:**

Question 5

AI observability: how do you trace and log LLM calls in prod?

Accepted Answer

**Target Audience:** Developers who have an LLM feature or agent in production and want to see each model call, tool call and its timing; no prior observability experience assumed.
**Difficulty:** Easy
**Verified with:** Node 22.22.0 and Python 3.12.3 (the in-process tracer below, identical output in both). OpenTelemetry convention names, the W3C header format and SDK options were checked with web searches in October 2026; where only secondary sources were found, the text says so.

> **How to re

Question 6

Embeddings in 2025-26: how do you choose model, dimensions, and distance?

Accepted Answer

**Target Audience:** Engineers setting up retrieval who must commit to an embedding model, a vector size and a metric, and want a defensible way to choose rather than copying a blog post.
**Difficulty:** Medium
**Verified with:** Node 22.22.0 and Python 3.12.3 (the truncation and quantization program below, on seeded synthetic vectors). Model facts, prices and limits are cited inline with the source and the date checked (October 2026); they change often, so treat them as examples of the mechanis

Question 7

What is the difference between fine-tuning, LoRA and prompt engineering?

Accepted Answer

**Target Audience:** Engineers who can call an LLM API and have heard "just fine-tune it", but have never trained a model or seen what LoRA actually changes.
**Difficulty:** Medium
**Verified with:** Node 22.22.0 and Python 3.12.3 (a toy linear layer trained with plain gradient descent; no real model is trained or called). Paper and library facts are cited inline with the date they were checked (October 2026).

> **How to read this doc:** Every idea is explained in plain language first, then tag

Question 8

Beyond Ragas: how do you eval LLMs with LLM-as-judge, DeepEval, and human eval?

Accepted Answer

**Target Audience:** Engineers who have one eval framework running (often Ragas on a RAG endpoint) and now need to evaluate the rest of an LLM product: summaries, support replies, agents, anything without a neat retrieval score.
**Difficulty:** Medium
**Verified with:** Node 22.22.0 and Python 3.12.3 (the judge-calibration and review-routing example below). DeepEval and G-Eval details were checked against the DeepEval docs, PyPI and the G-Eval paper via web search in October 2026; where only sec

Question 9

How do you manage conversations that outgrow the context window?

Accepted Answer

**Target Audience:** Engineers running a chat assistant or agent whose sessions get long, and who need a history strategy that is cheaper than "send everything" without making the bot forget.
**Difficulty:** Medium
**Verified with:** Node 22.22.0 and Python 3.12.3 (the conversation replay below). Provider compaction features were checked against the Anthropic docs and OpenAI documentation summaries cited inline, in October 2026.

> **How to read this doc:** Every idea is explained in plain langu

Question 10

Reranking in RAG: when and how to add a cross-encoder?

Accepted Answer

**Target Audience:** Engineers who already understand what a reranker is and now have to decide whether to put one into a real RAG system, which one, how deep, and inside what latency budget.
**Difficulty:** Medium
**Verified with:** Node 22.22.0 and Python 3.12.3 (the depth sweep and timeout simulation below). The relevance scores are simulated with a seeded generator and the reranker latency comes from a stated cost model, so the numbers show the *method*, not a benchmark of any product. Vendo

AI ENGINEERINGDECODED.

Easy

What are embeddings and what are they used for?

Context windows in 2025-26: how do you use 128k-1M tokens without drowning?

What is RAG and why use it instead of fine-tuning for knowledge?

What are evals and why does every LLM feature need them?

AI observability: how do you trace and log LLM calls in prod?

Medium

Embeddings in 2025-26: how do you choose model, dimensions, and distance?

What is the difference between fine-tuning, LoRA and prompt engineering?

Beyond Ragas: how do you eval LLMs with LLM-as-judge, DeepEval, and human eval?

How do you manage conversations that outgrow the context window?

Reranking in RAG: when and how to add a cross-encoder?

What chunking strategies exist and how does chunk size affect RAG quality?

How do you reduce hallucinations in a production LLM application?

What is reranking and why add it after retrieval?

Hard

How do you evaluate a RAG pipeline end to end?

How would you design a production RAG pipeline end-to-end in 2025-26?

What are common RAG failure modes and how do you debug them?

pgvector vs Pinecone vs Qdrant: how do you choose a vector DB in 2025?

Fine-tune vs RAG vs prompt engineering: how do you choose in 2025-26?

What techniques reduce LLM latency at each stage of a request?

What is red-teaming an LLM application and how is it done?

How do agents maintain memory across steps and sessions?