Agent evidence: what should come back with the work?
Stefan-Iulian Tesoi · · 6 min read
A coding agent should hand back four things: the commands it ran with their exit codes, the files it changed, which acceptance criterion each change satisfies, and an explicit list of what it could not verify. Everything else in a submission is commentary on those four.
Agent evidence is the part of a submission that survives disbelief. A reviewer who trusts none of the prose should still reach a decision from what came back, and most submissions fail that while reading perfectly well.
What counts as agent evidence?
Anything the reviewer can re-run, open or diff themselves. Evidence is reproducible; a description is not, however accurate.
That matters because the summary comes from the same process that produced the work, carrying the same misunderstandings. An agent that misread a criterion writes a confident account of having satisfied it, and nothing in the prose separates that from an agent that read it correctly. The reviewer inherits the misreading.
The software supply chain reached this conclusion earlier. SLSA's provenance format exists so a build is described by a signed statement of what ran rather than by the builder's assurance. AI agent verification is the same problem one layer up, with a diff in place of a binary.
Why is a summary not enough?
Because a summary is a claim about evidence, and the two fail in opposite directions. "All tests pass" and pnpm run ci exiting 0 occupy the same line in a review queue, but the first cannot be checked without redoing the work and the second names an invocation anyone can repeat in thirty seconds.
A summary is written by the thing that did the work, about whether the work is right. Evidence is produced by something with no stake in the answer: a test runner, a compiler, Git. Only the second kind can contradict its author.
The common failure is dull. An agent runs the one test file it touched, sees green, and reports that tests pass. pnpm test src/lib/export.test.ts exiting 0 is true; "tests pass" is what gets read, and the integration suite the change breaks runs an hour later in someone else's branch. Nobody lied. The submission named an outcome instead of a command.
What should every submission carry?
Four fields, in the same order every time, each with a reviewer action attached.
| Field | What it holds | What the reviewer does with it |
|---|---|---|
| Commands and exit codes | The exact invocations, each with its status | Re-runs the cheapest and compares |
| Files changed | The path list, as git diff --stat | Decides how closely to read |
| Criterion to change | Each acceptance criterion, with the change satisfying it | Looks for rows with one side empty |
| Not verified | What the agent could not check | Works through it by hand |
The third field is the one teams add last and get the most from. Two defects surface in seconds once the mapping exists: a criterion with no change against it, which is unfinished work, and a change satisfying no criterion, which is scope nobody asked for. Both are near-invisible in a diff and obvious in a two-column list. Writing criteria that can be mapped at all is the subject of acceptance criteria an agent can verify.
Test evidence is the cheapest of the four to produce and the easiest to weaken by accident, so the rule is to record the command rather than the conclusion. Teams reach for this under several names — proof of work AI agent submissions, verification gates, evidence-based done — all asking what lets a reviewer disbelieve the summary and still decide.
What about work that cannot be tested automatically?
It gets named in the submission, as a list, or it becomes nobody's job. The unverifiable half does not shrink because it went unmentioned; it stops being tracked.
Three kinds of work reliably land there:
- Anything that leaves the process. An email arriving, a webhook reaching a third party, a payment clearing in a sandbox that accepts inputs a real processor would reject.
- Anything whose correctness is a judgement. Whether a layout reads well on a phone, whether an error message helps, whether a migration's edge cases match the data.
- Anything the environment cannot reproduce. Production-shaped volumes, a rate limit that appears only under load, a race needing two machines.
An agent reporting only successes hands back a green wall and no map of the gaps. The honest form is one line: "Criterion 3 not verified — the invoice PDF renders in CI, but nobody has opened one." That sentence tells a reviewer where their attention is worth more than the machine's.
Laimonade makes this a field rather than a convention people remember: an item reaches In Review carrying the checks that ran and a separate list of what a person must still look at, and a person — never the agent — moves it further. The handover tools are in MCP tools; where it sits in the week is in the sprint workflow.
How do you make evidence reviewable in two minutes?
Fix the order, and put the unverified list first because it is the only part a reviewer cannot skip.
- Unverified items at the top. That list is the work being handed over, and burying it under green checkmarks is how it gets missed.
- Commands verbatim, with exit codes.
pnpm run ciexiting 0, not "CI is green". - Paths with line numbers in the criterion mapping, so each claim opens in one click.
- The commit SHA, anchoring the evidence to a tree rather than to a branch that has moved.
- Output only where it failed. Four hundred lines of passing log is the same evidence with the signal removed.
Two minutes is a budget, not a figure of speech. A reviewer facing 23 changed files spends either two minutes or twenty, and the submission's structure decides which before they read a word of the diff. The alternative is approval by fatigue.
The standard it is measured against is definition of done when an agent wrote the code. Recurring bad-submission shapes are collected in troubleshooting.
Frequently asked questions
Should the agent paste full test output?
No. Paste the command, its exit code, and the failing assertions when there are any. Full output of a passing suite adds length without information, and length is what makes reviewers skim. The log is reproducible on demand and need not travel with the submission.
What if the test suite is slow?
Record what ran and what was skipped, with the reason. A submission saying the unit suite passed in 40 seconds and the 20-minute integration suite was skipped is useful. "Tests pass" after running a tenth of them converts a known gap into an assurance nobody revisits.
Does screenshot evidence count?
For anything visual, yes, and it is often the only evidence available. A screenshot proves a render happened and shows what it looked like; it does not prove the state behind it was real. Pair it with the fixture that produced that state, and treat it as the start of a human check.
What stops an agent from fabricating evidence?
Reproducibility, not good faith. A pasted exit code can be wrong; a command and a commit SHA can be re-run against the same tree by anyone who doubts it. That is the reason to record invocations rather than outcomes: evidence cheap to check usually gets checked.