Designing the human checkpoint so people actually read it

What ninety-eight per cent approval actually tells you

On a lending workflow we instrumented, reviewers approved 98.4 per cent of agent-drafted recommendations, with a median review time of six seconds. The control was documented, the audit trail was complete, and the oversight was fictional. Nobody was being negligent — they had four hundred items a day and no signal about which ones deserved attention.

This is the failure mode that worries me most, because it is invisible in every artefact. The policy says a human reviews each decision. The logs say a human reviewed each decision. Only the timing data gives it away, and almost nobody looks at the timing data.

Route less to the checkpoint

The instinct when a system is new is to review everything, and it is understandable. But attention is a fixed budget, and spreading it across four hundred items produces six seconds each. Concentrating it on the forty that are genuinely consequential or genuinely uncertain produces sixty seconds each, and sixty seconds is enough to think.

Uncertainty is the harder half of that filter and it is worth doing properly: retrieval that found weak support, a case unlike anything in the evaluation set, an amount near a policy boundary, internal disagreement between reasoning steps. Everything else can pass with sampled audit, which is how the rest of the risk profession has always worked.

Present the decision, not the output

Most review interfaces show the generated text and an approve button. That asks the reviewer to reconstruct the reasoning before they can judge it, in the time they do not have. The alternative is to show them what the decision hinges on: the recommendation, the two or three pieces of evidence it rests on, what is unusual about this case, and what the system was uncertain about.

When we rebuilt one review screen along those lines, median time went up to fifty-one seconds and the override rate went from 1.6 per cent to 9 per cent. Both numbers moving is the point. The reviewers had not become more diligent; they had finally been given something to be diligent with.

Put the disagreement rate on the dashboard

Override rate is the health metric of a human checkpoint, and it should be visible to the people who own the control. Too low and the checkpoint is nominal. Very high and the system is not ready for the volume it is handling. The trend matters more than the level — a rate drifting steadily towards zero over a quarter is the signature of fatigue setting in.

Capture the override reasons as structured data too, not free text at the end of a form. They are the best training and evaluation signal the programme will ever produce, and they come from exactly the people whose judgement you are trying to encode.

Name the authority, not just the step

The last thing, and the one that governance documentation usually gets wrong: a checkpoint is only oversight if the person at it has the authority and the information to say no, and knows they do. We have seen reviewers who believed their role was to confirm the system rather than to judge it, because nobody had told them otherwise and their manager’s dashboard measured throughput.

So write down who holds the decision, what they may change, and what happens when they disagree — then check that the incentives around them agree with that document. If the queue metric punishes careful review, the queue metric wins. It always does.

The uncomfortable truth