When an Agent Says Done, What Is Actually True?

A production agent finished a run and marked its work complete. The artifact existed, and every internal step looked successful. A person still had to determine whether the result reached its intended destination and whether it resolved the original job. Until that check, “done” described the run, not the responsibility.

I kept seeing versions of this handoff while operating agent systems: files written but not published, pull requests opened but not deployed, and messages sent while decisions stayed unresolved. The governing question became: when an agent says done, what is actually true?

Note: This comes from operating more than 30 production agent instances at Atlan, building Waldo, and reading current work on agent evaluation. It is a working model, not a solved one.

A run is not an outcome

I use five words to keep the layers separate:

• A session is a period of model and tool activity.
• An artifact is what the session produced: code, a document, a message, or a plan.
• Evidence is what lets us inspect a claim about that artifact.
• An outcome is the change the person actually wanted.
• An open loop is whatever still needs attention, judgment, or follow-through.

Take a small release. An agent edits the code, runs the tests, and opens a pull request.

The session is finished. The artifact exists. But “customers can use the fix” depends on review, merge, deployment, and live behavior. A green build supports one part of that chain. It does not prove the whole thing.

intent → work → artifact → evidence → acceptance

Every arrow can break.

What should “done” include?

For important work, a completion claim should answer a few simple questions:

1. What changed?
2. Where did it change? A local checkout, production, an inbox, a calendar?
3. What did we observe?
4. How recent is that observation?
5. What is still uncertain or waiting on someone?

The strength of the evidence should match the consequence.

Renaming a private note needs little ceremony. Publishing a claim, moving money, deleting data, or changing production needs much more.

This is also why verification and acceptance are different. A system can verify that a deployment is live. The founder may still reject it because the experience is wrong.

Evidence is not one thing

Different checks answer different questions.

• A unit test checks a contract under known conditions.
• A screenshot checks appearance at one moment and viewport.
• A delivery receipt checks that a message left the system.
• A production check sees whether something is live now.
• A person decides whether the result was actually useful or acceptable.

Evidence gets stronger when it is direct, recent, and independent of the agent making the claim. It also gets stronger when it covers failure and recovery, not only the happy path.

None of this creates certainty. It simply makes uncertainty easier to see.

The UK AI Security Institute’s research agenda treats agent evaluation as a wider problem than short task success. OpenAI’s account of monitoring internal coding agents makes a similar point from operations: understanding agent behavior requires looking across actions and reasoning, not only at the final answer.

The interface should follow the responsibility

Most agent products are organized around chats and runs because those are easy to record.

People think in responsibilities:

• ship the release;
• submit the application;
• prepare for the meeting;
• follow up with the person;
• decide what to do next.

A session-first interface asks, “What did the agent do?”

An outcome-first interface asks:

• What was I trying to make true?
• What changed?
• What evidence supports that?
• What still needs me?

That is the direction I am exploring with Waldo. The point is not to create a prettier dashboard of agent activity. It is to preserve the thread between intention, work, evidence, and what remains.

Open loops need an ending

An open loop is not always a failure.

It may be waiting for another person. It may depend on a date. It may need a decision the agent should not make. Or the goal may no longer matter.

So closure needs more than a checkmark. A person should be able to say:

• this is complete;
• remind me when something changes;
• move this responsibility elsewhere;
• I no longer want to carry this.

That last option matters. A memory system that keeps every unfinished thought alive becomes a guilt machine.

What production taught me

At Atlan, I helped build and operate more than 30 production agent instances. The number is less interesting than the pattern.

The hard work was often outside the model: assembling context, setting tool boundaries, handling permissions, recovering from failure, and deciding what evidence was enough to continue.

The harness changed whether model capability became dependable behavior.

OpenAI’s discussion of running Codex safely shows the same principle from a security angle. Sandboxing, network controls, permissions, and review are not decorative layers around an agent. They shape what the agent actually is.

The Waldo test

My current hypothesis is simple:

If a personal agent preserves intent, shows its evidence, and carries unresolved work across time, people should spend less effort reconstructing what happened.

That should be measurable.

I would expect to see:

• fewer sessions or raw artifacts opened per accepted outcome;
• fewer interruptions per accepted outcome;
• less time between delegation and confident acceptance;
• fewer outcomes reopened because key evidence was missing;
• successful correction when the agent misunderstood the person.

These are proposed tests, not product results.

The thesis fails if Waldo becomes another surface to supervise. It also fails if the product decides what matters, silently turns every observation into a task, or makes its own interpretation harder to challenge.

The person has to remain the authority over meaning. The agent can propose, gather evidence, and remember. The person must still be able to correct, defer, transfer, or let go.

Agents will keep getting better at producing work. The harder question is whether they can help us know what the work actually changed.