The glossary

The agent claims to have acted without doing so, for example by giving an invented ticket number. The answer's text proves nothing.

SeeVerify the effect, not the claim

Architecture Decision Record: a page that records a technical decision, the option set aside, and the numeric measurement that would change it.

A program that uses a language model to decide what to do (search, call a tool, answer, refuse), in a loop, with stopping rules.

SeeThe agent loop and its exits

The cycle message → model → tool call → result → model, repeated until an answer or a planned exit.

SeeThe agent loop and its exits

Testing a method on the past: you act as if you didn't know how recent cases turned out, you predict, then you compare with what really happened.

SeeEvaluate without lying to yourselfHacktoberfest 2026 · Week 2

The “normal” proportion in the data. A model that always repeats the most frequent answer looks good without understanding anything.

SeeEvaluate without lying to yourselfHacktoberfest 2026 · Week 2

A tool that validates its arguments before acting, refuses by naming the valid values, and stays read-only by default.

SeeThe tool as the model sees it

The maximum number of calls an agent loop may make, checked before each call, never after.

SeeThe agent loop and its exits

An 80% score is well calibrated if, over many cases scored 80%, about 8 in 10 happen. A single case can't prove it.

SeeEvaluate without lying to yourselfHacktoberfest 2026 · Week 2

A piece of a document that retrieval can return. A passage that's too short can lose the sentence that answers.

SeeThe right page must arrive

The ID of the document that supports an answer. A correct citation doesn't prove the sentence comes from the document.

SeeThe model's output is untrusted dataEvaluate without lying to yourself

The slowness of the first call to a service that just started. You measure it: it's a number, not a failure verdict.

The set of documents the agent is allowed to rely on.

SeeThe right page must arrive

A single second attempt where the model gets back the error from its first one. Beyond that, a flagged refusal.

SeeThe model's output is untrusted dataHacktoberfest 2026 · Week 2

When the model sees by mistake a piece of information it wouldn't have had at prediction time (for example the payment date): the score looks excellent, but it's false.

SeeEvaluate without lying to yourselfHacktoberfest 2026 · Week 2

The score from which you say “yes”. You choose it on a separate slice of data, never on the final test data.

SeeEvaluate without lying to yourselfHacktoberfest 2026 · Week 2

The exact list of lines added and removed in a file. It's what counts, not what the assistant announces.

SeeVerify the effect, not the claim

When in doubt, the guard blocks rather than lets through. A guard that checks nothing when the model doesn't call the tool “fails open”.

SeeRead the tool call, not the answerWhat must always happen is enforced in codeHacktoberfest 2026 · Week 2

The category a failure goes into before you fix it: retrieval, tool choice, instruction not followed, format.

SeeRead the tool call, not the answer

A deterministic model with scripted answers. It tests the circuit, not the judgment: a green test on it proves the plumbing.

SeeEvaluate without lying to yourself

Cloning your own repo into an empty folder and following only the README, like a stranger, to see if it works.

SeeVerify the effect, not the claimHacktoberfest 2026 · Week 2

A small set of reference questions, each labeled with the expected behavior: answer, cite, or refuse.

SeeEvaluate without lying to yourself

The fact that an answer is actually supported by the document it cites.

SeeThe right page must arriveEvaluate without lying to yourself

A filter applied to the model's inputs and outputs: harmful content, personal data, denied topics. It's tested like code, false positives included.

SeeData is never an orderHacktoberfest 2026 · Week 2

Everything around the model: the loop that runs it, the instructions, the tools, the traces, the evaluations. A managed harness runs the loop for you, with less control.

SeeThe agent loop and its exits

The share of questions whose expected document is among the top k search results.

SeeThe right page must arrive

Code triggered at a precise moment in the agent's life, before or after a call, to enforce a rule the prompt doesn't guarantee.

SeeWhat must always happen is enforced in code

Choosing the method case by case: for example a model for a new client, and the client's history otherwise.

SeeWhat must always happen is enforced in codeHacktoberfest 2026 · Week 2

Running an action again doesn't change the result: a return already started isn't started twice, an existing resource is reused.

SeeWhat must always happen is enforced in code

The page downloads the model once, then the computer's graphics card runs it directly in the browser: nothing to install, nothing is sent.

SeeHacktoberfest 2026 · Weekend

A set of documents, split and indexed, that the agent queries before answering. An unsynced base returns nothing, with no error.

SeeThe right page must arrive

Large Language Model: the language model that generates text and chooses which tools to call.

A model that scores another model's answers against a criterion. It reads the text, but sees neither the tool calls nor what was written to the database.

SeeEvaluate without lying to yourself

What survives from one session to the next, filed per user: facts, preferences, summaries. Without an owner or a limit, it leaks and contaminates tests.

SeeMemory has an ownerHacktoberfest 2026 · Week 1

Model Context Protocol: a standard for exposing tools and data sources to any agent. An MCP tool is discovered by its name, description and schema.

SeeThe tool as the model sees it

Data published by an institution, that anyone can download and reuse for free, often under an open licence.

SeeHacktoberfest 2026 · Week 1

Open weights don't mean “free to sell”: a model can be free for a prototype and off-limits in a paid product.

SeeHacktoberfest 2026 · Week 2

Every write carries the version read just before it; if the version changed in the meantime, the write is refused. Two agents can't silently overwrite each other.

SeeWhat must always happen is enforced in code

In a multi-agent system, the agent that never answers itself: it routes the work to specialist agents.

SeeThe agent loop and its exits

Also refusing what the agent could have answered. Safe, but less useful; you measure it as much as errors.

SeeData is never an orderRefusal is a state

A displayed confidence higher than the sources justify, for example 1.0 on a single document.

SeeThe model's output is untrusted data

The code that reads the model's output and rejects it if it doesn't match the schema: format, exact fields, types, bounds.

SeeThe model's output is untrusted data

What was asked, written down before the action exists. Every check compares the action to that pin, never to itself.

SeeWhat must always happen is enforced in code

An order hidden in a text (document, page, tool output, product name) that tries to steer the model. You treat it as data, never as an instruction.

SeeData is never an order

Retrieval-Augmented Generation: search for passages in a base, then answer from them, citing them.

SeeThe right page must arrive

Replacing a secret (key, token, email) with a label carrying a fingerprint before writing a log, without deleting the field. Same secret, same label.

SeeRead the tool call, not the answer

An explicit "I can't answer": no citation, zero confidence, human review. A state, not a crash.

SeeRefusal is a state

What a memory keeps, why, how to correct it, when it expires, and what it refuses to keep. Written before the first record.

SeeMemory has an owner

The step that searches the corpus for passages useful to a question, before calling the model. Nothing found: refuse without a call.

SeeRefusal is a stateThe right page must arrive

A score between 0.5 (chance) and 1 (perfect) that measures whether the model ranks risky cases ahead of the others.

SeeEvaluate without lying to yourselfHacktoberfest 2026 · Week 2

Going back to the last version that worked, described in advance by an action, a number and a unit.

A program the machine starts on its own at a set time, for example every morning.

SeeWhat must always happen is enforced in codeHacktoberfest 2026 · Week 1

The required shape of a piece of data: its fields, their types, their bounds.

SeeThe model's output is untrusted dataHacktoberfest 2026 · Weekend

The single point of contact between the agent and any model: one method, and you switch providers without touching the agent.

The history of one conversation, which disappears when it ends.

SeeMemory has an owner

An error that looks like a normal answer, like "no results" when there are some. The worst kind of error.

SeeRefusal is a state

An instruction sheet the assistant loads only when the task matches. It's judged by the difference it makes, with evidence.

Filling a tool's parameters during the conversation: keep what's given, ask for one missing field at a time, act as soon as everything is there.

SeeThe tool as the model sees it

A quick test run right after a deployment, designed to be able to say "this deployment is bad".

Server-Side Request Forgery: getting a tool that fetches URLs to call an internal or private address on the attacker's behalf.

SeeThe tool as the model sees it

Asking the model to answer in a fixed format (JSON), so the interface always shows the same thing, in the same place.

SeeThe model's output is untrusted dataHacktoberfest 2026 · Weekend

The fixed instructions given to the model before any conversation. A rule there reduces a behavior without eliminating it, and a prompt that's too long is followed less well.

SeeData is never an orderWhat must always happen is enforced in codeHacktoberfest 2026 · Week 1

A model already trained on millions of synthetic tables: you give it a small table of examples and it predicts the next rows, without long training.

SeeHacktoberfest 2026 · Week 2

The usage statistics a piece of software sends to its publisher. Some tools send them by default; you can turn them off.

SeeHacktoberfest 2026 · Week 2

The setting for randomness in generation. At 0, answers and tool calls become more stable, without becoming perfect.

A maximum waiting time. Past it, you abandon the call and return a readable refusal, instead of waiting forever.

SeeRefusal is a state

The model's request to run a tool with specific arguments. That's what you read in the trace, not the final answer.

SeeThe tool as the model sees itRead the tool call, not the answerHacktoberfest 2026 · Week 1Hacktoberfest 2026 · Week 2

The log of a run: every step (search, model call, tool call, decision) with its arguments and result.

SeeRead the tool call, not the answer

The list of allowed (state, event) pairs and the state each one leads to. Any missing pair is refused, and the state doesn't move.

SeeThe agent loop and its exits

A check that decides, on the string alone and before any request, whether an address is allowed: protocol, public address, permitted domain.

SeeThe tool as the model sees it