Every SOX program that starts using AI to review evidence eventually asks the same question: how much should the AI actually be trusted to decide? The honest answer is that AI is very good at reading documents quickly and consistently, and it should never be the one who decides a control passed or failed. That distinction, between doing the reading and owning the conclusion, is the whole design problem. Get it wrong and you have either a tool nobody trusts or, worse, one people trust too much.
This post lays out the principles SOXLayer is built around when it comes to AI and control testing: human in the loop as a non-negotiable default, citations instead of bare verdicts, honest confidence scoring, escalation when the AI is unsure, and treating evidence documents as data rather than as instructions the AI should follow. None of this is theoretical. It shapes how the product behaves sample by sample.
Human in the loop is not a feature toggle
It is tempting to frame "human in the loop" as a setting that a cautious admin turns on and a confident one turns off. That framing is wrong. In a control testing workflow, a person accepting, editing, or overriding every AI proposal is not a safety net bolted onto the process. It is the process. The AI proposes a result, and a person disposes of it. Removing that step does not make testing faster in any way that matters, because it removes the accountability that makes the test conclusion defensible in the first place.
In practice this means the reviewer always sees three things together: what the AI concluded, why it concluded that, and where in the document it looked. The reviewer's job is not to redo the AI's work from scratch. It is to confirm the AI's reasoning holds up, or to catch the cases where it does not.
Why citations matter more than the verdict itself
A verdict on its own, "Pass" or "Fail" with nothing behind it, asks the reviewer to take the AI's word for it. That is the opposite of what control testing is supposed to produce. The value of AI here is not that it can render a judgment. It is that it can read a fifty page evidence packet in seconds and point to the exact place that supports a conclusion.
That is why every attribute the AI tests should come back with the exact quote and page it relied on, not just a label. When the AI marks an attribute as Pass, the reviewer should be able to click straight to the sentence on the page that justified it. SOXLayer pairs this with annotated evidence: labelled highlights drawn directly on the source document, so the reviewer is looking at the same words the AI looked at, not a paraphrase of them.
This changes what review actually is. Instead of re-reading the whole document to check the AI's work, the reviewer is verifying a specific, cited claim. That is a faster review and a more rigorous one at the same time, which is a combination you rarely get.
Confidence thresholds and what "Insufficient" really means
Every AI proposal comes with a confidence score, and that score should mean something specific: how strongly the evidence in front of the AI supports the conclusion it reached. A high confidence Pass means the document contains clear, unambiguous support for the attribute being tested. A low confidence result means the AI found something that looks relevant but is not conclusive on its own.
"Insufficient" is a distinct verdict, not a low-confidence Pass or Fail. It means the AI could not find enough in the evidence to support any conclusion, whether that is because the document is missing the relevant detail, the attribute requires information the evidence simply does not contain, or the document does not match what the sample calls for. Insufficient is a useful, honest answer. An AI that always forces itself into Pass or Fail, even when the evidence does not support either, is less trustworthy than one that says plainly "I don't have enough here."
Confidence floors are configurable by admins, along with whether the AI is allowed to auto-fill Pass or Fail results at all. A program testing high-risk controls might set the floor high and require human acceptance on every result regardless of confidence. A lower-risk area might allow bulk acceptance of high-confidence results, with a person still reviewing each one before accepting, just faster.
Start your 14-day free trial
Bring one control and its evidence. See the AI test it in minutes.
Escalation: when low confidence should trigger a second opinion
A single model reading a single document once is not always enough, especially when the evidence is ambiguous or when two parts of the evidence seem to disagree with each other. Low-confidence or conflicting results should escalate automatically to a stronger model, with the reasons for escalation listed alongside the result rather than hidden.
This matters for two reasons. First, a harder model pass on a genuinely ambiguous case catches things a faster first pass misses, without paying the cost of running every single sample through the most expensive model available. Second, and just as important, it gives the reviewer a documented reason the result needed a second look. That reasoning becomes part of the evidence trail for the test itself, not just an internal implementation detail.
The goal is not to escalate everything. It is to reserve the extra step for the cases where the first pass genuinely could not resolve the question, and to be transparent about which cases those were.
Prompt injection in evidence documents: evidence is data, not instructions
Evidence documents are not written by people trying to help the AI test them correctly. Some are messy, some are incomplete, and every so often one contains text that looks like an instruction aimed at whoever, or whatever, reads it next. An AI reading evidence has to treat every word in that document as content to evaluate, never as a command to follow. A PDF does not get to tell the reviewer what conclusion to reach, and it does not get to tell the AI that either.
Evidence as data, not instructions
During testing, a document contained an embedded "system note" instructing the reviewer to mark everything as Pass. The AI flagged it as a prompt injection attempt and ignored the instruction rather than following it. The note was treated the way any other line of text in the document should be treated: as something to read and report on, not as a command to obey. The AI surfaced what it found so a human could see it, and moved on with testing the actual attributes the sample called for.
This is a narrow but important design rule. The AI's job when reading a document is to extract facts relevant to the attributes being tested. Nothing inside that document, however it is phrased or however authoritative it sounds, changes what the AI is supposed to do. If a document contains language that reads like an instruction to the system, the right response is to flag it, not comply with it, and definitely not to quietly act on it.
Why AI must never change a test conclusion by itself
There is a meaningful difference between an AI that proposes a result and an AI that finalizes one. SOXLayer's AI output never changes a status by itself. A person always has to Accept, Edit, or Override the AI's proposal, and an Override requires a reason. That reason becomes part of the record, so anyone looking back at the test later can see not just what the AI thought, but what a person decided and why, when those two things differed.
The AI can draft the test conclusion, the narrative summary that ties the attribute results together, but even that draft sits in front of a person before it becomes final. Bulk acceptance exists for high-confidence results so reviewers are not clicking through hundreds of obvious Pass results one at a time, but bulk acceptance is still a person choosing to accept, not the system deciding on its own that no one needs to look.
Admins control this at the program level: AI on or off, confidence floors, whether AI can auto-fill Pass or Fail at all, a daily AI run quota, and whether human acceptance is required on key controls regardless of confidence. Prompt versions are tracked, so if the AI's behavior on a given attribute changes because the underlying prompt changed, that change is visible and attributable rather than silent.
What AI should do versus what it should not do
| AI should | AI should not |
|---|---|
| Read evidence (PDFs, images, spreadsheets) and test each attribute against it | Change a test status without a person accepting, editing, or overriding it |
| Propose Pass, Fail, Insufficient, or N/A with a confidence score | Force a Pass or Fail when the evidence does not clearly support one |
| Cite the exact quote and page it relied on for every proposal | Assert a verdict without pointing to where in the document it came from |
| Escalate low-confidence or conflicting results to a stronger model with reasons listed | Silently settle on a low-confidence answer without flagging the uncertainty |
| Treat every word of an evidence document as content to evaluate | Follow instructions embedded in an evidence document, however they are phrased |
| Draft a test conclusion for a reviewer to read and finalize | Publish a final conclusion on its own |
A note on admin controls
None of the above works as a blanket promise. It works because it is configurable and auditable: confidence floors an admin sets, a daily run quota that caps how much the AI processes unattended, required human acceptance on key controls, and tracked prompt versions so behavior changes are visible over time. The design principle is the default; the admin controls are what let a program tune it to its own risk tolerance.
What this looks like day to day
For a reviewer, this feels less like trusting a black box and more like working with a fast, well-organized junior tester who always shows their work. Every proposed result comes with the passage it is based on, highlighted on the actual document. Every low-confidence or conflicting case has already been kicked up to a stronger model, with the reason written down. Every Insufficient result is an honest signal that something in the evidence is missing, not a forced guess.
For a program manager or the person answering to the CFO's office, the AI's role in testing stays fully accountable. Nothing gets marked Pass because a model said so. Every conclusion traces back to a person who accepted, edited, or overrode a proposal, with a reason on record when they overrode it.
Key takeaways
- Human in the loop is not optional: a person always accepts, edits, or overrides every AI-proposed result.
- Citations, the exact quote and page, matter more than the verdict itself, because they let a reviewer verify a specific claim instead of re-reading the whole document.
- "Insufficient" is an honest, distinct verdict for when the evidence does not support a conclusion, not a forced Pass or Fail.
- Low-confidence or conflicting results should escalate automatically to a stronger model, with the reasons for escalation documented.
- Evidence documents are data to be read, never instructions to be followed, even when they contain text written to look like a command.
- AI can draft a test conclusion, but it never finalizes one: only a person's Accept, Edit, or Override (with a reason) closes out a result.
