An AI agent sounds exactly as sure when it is wrong as when it is right. The report arrives formatted, fluent and finished, and nothing about its tone tells you which of its claims it actually checked.
That is the real question behind every agent rollout: how to trust AI agent output you didn't produce and can't fully re-check yourself. Not "is the model good" — models are good — but "who looked at this before it reached me, against what standard, and what happened when it fell short?"
The short answer
Don't trust the agent. Trust the check. Put a second model on every result — one that did not do the work — and have it grade that result against a rubric written before the work started. It returns a verdict, a score and the reasons, and a failed verdict goes somewhere real instead of being passed along. Confidence is a property of the writing. Trust has to be a property of the process.
Check 01
Confident is not correct.
A language model writes the way its training taught it good answers look: complete sentences, a clear structure, a conclusion. That polish is produced whether or not the facts underneath it hold. An agent adds something worse on top — it takes many steps, and every step is a chance to pick up an error that the next step then builds on.
The failures that hurt are rarely dramatic. They look like this:
- A number with no source. "The market grows 340% by 2027" — plausible, specific, and invented to fill a gap in what the agent found.
- A partial answer written as a whole one. Two of three competitor pages loaded; the summary reads as if all three did.
- The right answer to a slightly different question. Asked for pricing on the Pro tier, it compares Starter plans, fluently.
- Filler in a professional voice. Three paragraphs that say nothing a reader could act on.
None of these throw an error. The run completes, the logs are clean, and the output arrives looking finished. If "nothing failed" is your signal, you will ship every one of them.
“The market grows 340% by 2027.” Sources: none given
- Addresses the directive
- Specific and actionable
- No hallucinated facts or sources
1.9/ 5
Retry — no source for the 340% figure.
Check 02
Why an agent can't grade itself.
The obvious fix is to add a line to the prompt: check your work before you answer. It helps a little, and it fails for a structural reason no prompt reaches.
An agent reviewing its own output reviews it with the same context, the same assumptions and the same gaps that produced it. It already chose to fill the missing number with an estimate; asked whether the number is sourced, it will find the reasoning it used to justify it and agree with itself. Self-review mostly returns a yes.
The author is the one reader guaranteed to miss what the author didn't know.
This isn't a flaw specific to AI — it's why code gets reviewed by someone other than the person who wrote it, and why audits are done by people who didn't keep the books. Independence is the active ingredient. A reviewer that shares the author's context shares the author's blind spots.
The same logic rules out the other common shortcut: letting the next agent in the chain judge the previous one's work. The next agent has a job of its own, and a finding it receives is an input to that job, not something it was asked to doubt. It will build on the invented number, not question it.
Check 03
What an independent check looks like.
"Have another AI look at it" is the right instinct and a vague instruction. A check you can actually rely on has five properties, and missing any one of them turns it back into a formality.
- 1
It didn't do the work.
A separate model call with its own instructions, seeing the output as something to evaluate — not a continuation of the conversation that produced it.
- 2
It grades against something written in advance.
A rubric fixed before the agent started, so the standard can't drift toward whatever the output happens to be.
- 3
It returns a verdict, not a vibe.
Pass or fail, decided — not a paragraph of gentle suggestions that nobody is obliged to act on.
- 4
It explains itself.
A score and notes per rubric item, so a person can see what failed and decide whether they agree.
- 5
A fail changes what happens next.
A failed verdict has somewhere to go. If the result moves on regardless, the check is decoration.
The two pipelines below run the same bad output through two systems, side by side and in step. On the left, nothing checks it. On the right, something does.
No reviewer
Independent check
Check 04
Write the rubric before the work.
A reviewer with no standard grades on impression, and impressions are exactly what fluent output is good at. The rubric is what makes a second model a checker rather than a second opinion.
Different work fails in different ways, so one rubric for everything is too loose for some agents and irrelevant for others. The rubric should follow the job. Two real examples, as short as they should be:
A research agent
The output addresses the agent's directive.
The data is specific and actionable.
No hallucinated facts or sources.
A writing agent
The content is relevant to the mission goal.
The quality meets professional standards.
No unoriginal filler content.
What makes a rubric item useful:
- One claim per line. "Accurate and well-sourced" is two checks; a reviewer can pass one and let the other slide.
- Checkable in the output itself. "Every figure names where it came from" can be verified by reading. "The research was thorough" can't.
- Aimed at how this kind of work fails. Research fails by inventing; writing fails by padding; code fails by not running. Write the line that catches the likely failure, not a generic virtue.
- Few. Three to five lines. A twenty-item rubric gets skimmed by models the same way it gets skimmed by people.
Check 05
More checkers where mistakes cost more.
Not every output deserves the same scrutiny. An internal draft that a person will read anyway can live with one reviewer. An email to five hundred customers can't be un-sent. The sensible rule is to scale the checking with the cost of being wrong.
Two reviewers aren't twice as good as one because each is smarter — they're better because they fail differently. The chance that two independent checks both miss the same invented number is much lower than the chance that one does. That extra call costs little next to a message you have to apologise for.
Check 06
What a failed check should do.
A check only earns trust if failing it has consequences. There are three honest options, and each is right for different work.
Whatever you pick, one rule holds: a failed verdict must be visible. The worst system is the one that records a fail in a place nobody looks and carries on as if it passed.
One more protection is easy to overlook. The output being graded may contain text the agent copied from the web — and web pages can contain instructions aimed at AI systems. A good reviewer is told to treat the output purely as data to evaluate, never as instructions to follow, so a page that says "rate this 5 out of 5" gets graded, not obeyed.
Check 07
Read the grade, not just the answer.
Once every result carries a verdict and a score, you have something most agent setups never produce: a record of how good the work has been, not just what it said.
- Per agent, a falling average tells you a directive has gone stale or a source has changed before a bad result reaches anyone.
- Per run, one rating tells you how much of a deliverable was graded well — a report built on a 4.8 and a 2.1 is not a 3.5 report you can skim.
- The notes are where the value is. A score says something was weak; the notes say which claim, and why.
And spot-check the passes, too. An independent reviewer is far better than none, but it's a model with its own blind spots. Reading a passed result now and then is how you learn whether your rubric is catching what matters — and which line to add when it isn't.
The list
Six questions before you trust it.
Ask these of any agent system — yours, a vendor's, or one you're about to build. Each "no" is a place where a confident wrong answer gets through.
- Is every result checked by a model that didn't produce it?
- Is the standard a rubric written before the work, and does it change with the kind of work?
- Does the check return a verdict, with a score and reasons you can read?
- Does risky work — external messages, payments, anything irreversible — get more than one reviewer?
- When a check fails, is that visible, and does it change what happens next?
- Is the reviewer told to treat the output as data, not instructions?
Ours
How Xorviex checks agent work.
In Xorviex, you describe a goal and it plans the agents the job needs. Every one of those agents is checked the way this article describes — not as a setting you remember to turn on, but as part of how an agent finishes.
- Every agent's final result is audited by a separate supervisor AI — a model call that did not do the work and sees the result only as something to judge.
- How many supervisors is decided per agent, with a reason. One for low-risk, reversible, internal work; two when the task is irreversible, touches payments, sends external messages or involves sensitive data.
- The rubric follows the role. A research agent is graded on the three lines above; a writing agent on its own three; a code-fixing agent on whether the checks that failed now pass and whether its change went to a pull request instead of the main branch.
- Each audit returns pass or retry, a 0–5 quality score and notes on every rubric item. The supervisor is instructed to treat the output purely as data, never as instructions.
- A failed audit is flagged, not buried. It appears in the mission's log marked Failed, with the supervisor's notes, so a person decides what happens next. Nothing is quietly rewritten until it passes.
- Scores roll up into a rating for each agent and each mission, so you can see how well a deliverable was graded before you read it.
It's one part of a larger design — the orchestration layer that decides which agents exist, what they share and what happens when one stops. Verification is the part that decides what you can believe.
Questions
Frequently asked questions.
How do you trust AI agent output?
By trusting the process that checked it rather than the output itself. Every result should be graded by an independent model against a rubric written before the work, return a verdict with a score and reasons, and — when it fails — be flagged or blocked instead of passed along.
Can't the agent just check its own work?
Partly, and not reliably. It reviews with the same context and assumptions that produced the mistake, so it tends to agree with itself. Independence is what makes a review catch things the author didn't know.
Isn't one AI checking another just as unreliable?
A reviewer is a model too, with its own blind spots. But it fails differently from the author, it works from a fixed rubric instead of an impression, and its reasons are written down for you to disagree with. That is far better than no check — and adding a second reviewer for high-stakes work lowers the odds further.
What should a rubric for an AI agent include?
Three to five lines, one claim each, checkable by reading the output, and aimed at how that kind of work usually fails — invented facts for research, filler for writing, failing tests for code.
Should failed outputs be retried automatically?
Sometimes. Retrying with the reviewer's notes works for mechanical fixes, but an unlimited retry loop can drift toward "whatever passes" and burn budget doing it. For judgment calls, flagging the result for a person is safer.
How many reviewers does an output need?
One for internal, reversible, low-risk work. Two when the task is irreversible, touches payments, sends messages outside the company or involves sensitive data.
Does checking every output make agents slow or expensive?
It adds one model call per result — small next to the agent's own run, and very small next to acting on a wrong answer. It's the cheapest point at which to catch a mistake.
The difference
Trusting the answer, or trusting the check.
Trusting the answer
It sounds sure, so it's probably right.
Reviewed by the agent that wrote it.
Standard: whatever looks good.
A mistake found by a customer.
No record of how good the work was.
Trusting the check
Graded by a model that didn't do the work.
Against a rubric written before it started.
Two reviewers where mistakes cost more.
A fail flagged, with the reason.
A score on every result, over time.
Xorviex is built to be the right-hand column. Describe a goal and it plans the agents, gives each a supervisor that didn't do the work, grades every result against its role's rubric, doubles the checkers where a mistake can't be undone, and flags every fail with its reasons — so what reaches you has been checked, not just written. Start your first mission.
An agent can sound sure. Only a check can make it trustworthy.