BAA 101 · Chapter 01
Verify the effect, not the claim
The principle
What an AgentA program that uses a language model to decide what to do (search, call a tool, answer, refuse), in a loop, with stopping rules.See in the glossary → says it did is not proof. Only the effect observed in the system counts: a row in a database, a file, a DiffThe exact list of lines added and removed in a file. It's what counts, not what the assistant announces.See in the glossary →, a transaction.
Why it matters
A model produces plausible text. It can announce a ticket created, a fix applied, a check run, without any of it having happened. The conversation looks right, and so does the score: neither of them looks at the system.
The mechanism
- For each action, name the observable effect it must leave: a row, a file, a diff, a balance.
- Check that effect through another path than the agent itself: a database query,
git diff, a read of the blockchain. - Cross-check: the same ID must appear in the conversation and in the system.
- Build the report from the effect you read, not from what the agent announces.
- When the rule matters, write it in the tool’s code, not only in the prompt.
In the field
- AWS Scholars, chapter 1: the chatbot gave out invented ticket numbers, without ever calling the tool. The evaluation showed 0.92; only a scan of the DynamoDB table revealed the problem.
- Dev3Pack, Session 1: the coding assistant announced a fix that wasn’t in the file.
git diffshowed it, and a small script proved it before and after the fix. - Dev3Pack, Gecko: a purchase receipt is written from two reads of the blockchain, before and after, never from the send’s response.
The question to ask
Where, in the system, can I see that it really happened?