Skip to the solution
EasyAI Engineering16 min read

What is the difference between zero-shot, one-shot and few-shot prompting?

192 views

The problem

Question presented to candidate: "People talk about zero-shot, one-shot and few-shot prompting. What do those terms mean, how does the model actually use the examples you give it, and how do you decide how many examples to put in a production prompt?"

What a strong answer should cover:

  • Definitions: zero-shot is an instruction with no worked examples; one-shot adds one input-output example; few-shot adds several. The model learns the pattern from the prompt itself (in-context learning), with no weight updates
  • What examples really teach: the output format, the label space and the kind of input to expect; research found the exact correctness of example labels mattered less than expected, which is why format and coverage of examples matter so much
  • Which examples beats how many: examples that cover the confusable boundary cases help; unbalanced or unrepresentative examples skew the output (majority-label and recency biases, order sensitivity)
  • Cost: every example is paid for on every call; a static example block can be prompt-cached, while examples chosen per request (dynamic few-shot by similarity) cannot share that prefix
  • Modern instruction-tuned models are strong zero-shot, so start zero-shot, measure on an eval set, and add examples when format or edge cases fail; move to fine-tuning when the example list grows past what a prompt can carry

Code / implementation expected: Optional โ€” building a few-shot prompt, choosing examples by similarity, or comparing strategies on a small labelled set is a strong signal.

  • few-shot
  • zero-shot
  • in-context-learning
  • prompting

Try it yourself

Sketch your approach before you read the solution. That is what the interview tests.

Stuck? Get a hint.

A short nudge toward the approach. It will not give the answer away.

Solution

The solution is hidden so you can try first

16 min read ยท 11 sections

Read the code

Compare zero-, one- and few-shot prompts on a ticket classifier with a stub model
// Zero-, one- and few-shot prompts for a ticket classifier, with a stub "model".
// The stub imitates the most similar example it was shown (a stand-in for
// in-context learning); with no examples it can only use keywords.
const POOL = [
  ["I was charged twice this month", "billing"],
  ["Please refund my last payment", "billing"],
  ["What is the price of the team plan", "billing"],
  ["The app crashes when I open settings", "bug"],
  ["The payment page throws an error", "bug"],
  ["Export button is broken on reports", "bug"],
  ["I forgot my password", "account"],
  ["Please change the email on my invoices", "account"],
  ["My two-factor code never arrives", "account"],
  ["Invoice PDF download fails with an error", "bug"],
  ["Upgrade my plan and send a receipt", "billing"],
  ["Lock my account after a suspicious login", "account"],
];
const TEST = [
  ["The invoice page shows an error after I pay", "bug"],
  ["Refund the duplicate charge please", "billing"],
  ["Update the email address on my invoices", "account"],
  ["The two-factor code never arrives by sms", "account"],
  ["The app freezes on the dashboard", "bug"],
  ["How much is the price for ten seats", "billing"],
  ["I cannot log in after the update", "account"],
  ["The receipt shows the wrong price", "billing"],
];
const INSTRUCTION = "Classify the support ticket as billing, bug or account. Reply with the label only.";
const STOP = new Set("i the a an my on of is to and for by in after what when how please".split(" "));
const words = (s) => new Set(s.toLowerCase().split(/[^a-z0-9-]+/).filter((w) => w && !STOP.has(w)));
const KEYWORDS = [
  ["billing", ["invoice", "invoices", "charge", "charged", "refund", "payment", "price", "pay"]],
  ["bug", ["error", "crash", "crashes", "broken", "freezes"]],
  ["account", ["password", "login", "log", "email"]],
];

function keywordGuess(text) {
  const w = words(text);
  for (const [label, keys] of KEYWORDS) if (keys.some((k) => w.has(k))) return label;
  return "unknown";
}
function jaccard(a, b) {
  let inter = 0;
  for (const x of a) if (b.has(x)) inter++;
  return inter / (a.size + b.size - inter);
}
// Stub model: copy the label of the most similar shown example (later one wins a tie).
function stubModel(shots, text) {
  if (shots.length === 0) return keywordGuess(text);
  const w = words(text);
  let best = -1, label = "unknown";
  for (const [ex, lab] of shots) {
    const s = jaccard(w, words(ex));
    if (s >= best) { best = s; label = lab; }
  }
  return best > 0 ? label : keywordGuess(text);
}
function buildPrompt(shots, text) {
  const lines = [INSTRUCTION, ""];
  for (const [ex, lab] of shots) lines.push(`Ticket: ${ex}`, `Label: ${lab}`, "");
  lines.push(`Ticket: ${text}`, "Label:");
  return lines.join("\n");
}
const approxTokens = (s) => Math.ceil(s.length / 4);
// Dynamic few-shot: pick the k pool examples most similar to this ticket.
function similarShots(text, k) {
  const w = words(text);
  return POOL.map((p, i) => ({ p, i, s: jaccard(w, words(p[0])) }))
    .sort((a, b) => b.s - a.s || a.i - b.i).slice(0, k)
    .reverse().map((x) => x.p); // most similar placed last, next to the question
}

const STRATEGIES = [
  ["zero-shot", () => []],
  ["one-shot", () => POOL.slice(0, 1)],
  ["3-shot, first 3", () => POOL.slice(0, 3)],
  ["3-shot, balanced", () => [POOL[0], POOL[3], POOL[6]]],
  ["3-shot, similar", (t) => similarShots(t, 3)],
  ["12-shot, all", () => POOL],
];
const PRICE_PER_M = 2.0; // illustrative $ per 1M input tokens
console.log("strategy           acc   avg prompt tok  $ per 1M calls");
for (const [name, pick] of STRATEGIES) {
  let correct = 0, tok = 0;
  for (const [text, gold] of TEST) {
    const shots = pick(text);
    tok += approxTokens(buildPrompt(shots, text));
    if (stubModel(shots, text) === gold) correct++;
  }
  const avg = tok / TEST.length;
  const acc = `${correct}/${TEST.length}`;
  console.log(`${name.padEnd(18)} ${acc.padEnd(5)} ${avg.toFixed(3).padStart(14)}  $${(avg * PRICE_PER_M).toFixed(2)}`);
}
console.log("\nper ticket (b=billing, g=bug, a=account, ?=unknown); columns in strategy order");
for (const [text, gold] of TEST) {
  const row = STRATEGIES.map(([, pick]) => ({ billing: "b", bug: "g", account: "a" })[stubModel(pick(text), text)] ?? "?");
  console.log(`  ${gold.padEnd(8)} ${row.join(" ")}  ${text}`);
}
const t = TEST[0][0];
console.log("\nfirst test ticket:", t);
console.log("  zero-shot guess:", stubModel([], t), "(keyword 'invoice' wins)");
console.log("  similar shots:", similarShots(t, 3).map((p) => p[1]).join(", "));
console.log("  3-shot similar guess:", stubModel(similarShots(t, 3), t));
console.log("\nprompt for one-shot:\n" + buildPrompt(POOL.slice(0, 1), TEST[1][0]));

Discussion

Other approaches, follow-ups and how it went in real interviews.

Sign in to join the discussion.

No comments yet. Share how you would answer it.

Question 11 of 83 in AI engineering

Keep your momentum going.

Mark this question solved to track your progress, then take the next one while it is fresh.

Up nextEasy

What is a context window and what happens when you exceed it?

Continue