A policy. A disclaimer. A slide deck that says, ‘Humans are in the loop.’
Then a decision gets challenged — a hiring rejection, a pricing outcome, a fraud flag, a credit limit change, an eligibility decision — and suddenly the business can’t explain what happened in a way a normal person would accept.
That’s the real risk. Not ‘AI’ as a concept, but a decision process that moves fast and leaves no receipts.
The UK Information Commissioner’s Office (ICO) currently has a consultation open on draft guidance about automated decision-making (including profiling), with the consultation closing on 29 May 2026. That deadline isn’t just for lawyers. It’s a forcing function for operators to ask a more practical question:
> If someone opts out or challenges an outcome, can we route them into a real human review process — and prove what happened?
If you can’t, you don’t have governance. You have theatre.
Why ADM compliance fails in the real world
Most teams treat ADM as a definition they might fall into. So they do two things:
1) They rename the system (‘recommendations’, not ‘decisions’)
2) They add a vague promise (‘humans are involved’)
But what makes something feel automated to the person affected isn’t the internal label. It’s the lived experience:
- They can’t see the basis of the outcome
- They can’t trigger a meaningful review
- They can’t get a human to re-assess the case
- The business can’t reconstruct what happened later
In other words: ADM is a workflow property.
If you want to be safe, you don’t start with wording. You start with workflow controls.
The uncomfortable test: where does override actually live?
Here’s a test I use because it cuts through debate quickly.
Pick one automated decision surface in your business. For many firms it will be one of these:
- recruitment screening
- fraud detection and account restrictions
- dynamic pricing or discounting
- credit/eligibility checks
- customer support triage that escalates or deprioritises
Now answer three questions in plain English:
1) Who can override the outcome? (Name a role, not a department.)
2) How do they get the case? (Queue, ticket, inbox, workflow state.)
3) What evidence do we keep? (What did the system decide, what did the human see, and what did the human decide?)
If those answers are fuzzy, your ‘human involvement’ is probably cosmetic.
And cosmetic human review is worse than none, because it creates a false sense of safety.
The ADM Control Bundle (ship this before you scale automation)
This is the smallest bundle of controls that turns ‘ADM compliance’ into something you can operate. It’s designed to be cheap to implement and hard to argue with.
1) Decision log (the receipt)
You need one record per decision event. Not a quarterly report, not a summary. A durable receipt.
A minimal decision log entry can capture:
- a unique decision ID
- the input snapshot (what data was used at that moment)
- the system output (score/recommendation/classification)
- the action taken (approve/deny/route/restrict/change)
- who approved it (human or system identity)
- timestamp
This is what allows you to reconstruct history when a decision is challenged later. It also gives you the substrate to tune false positives and process drift.
2) Notice + opt-out (the escape hatch)
If someone is subject to an automated decision surface, they need a practical way to trigger a non-automated path.
The key word is practical. ‘Email [email protected]’ is not a workflow. It’s a black hole.
An opt-out mechanism can be as simple as:
- a form that creates a ticket
- an in-product ‘request review’ action
- a reply keyword that triggers a case
But it must have three properties:
- it’s easy to find
- it creates a case with an owner
- it links back to the original decision ID
3) Human review queue (owned work, not vibes)
An opt-out is meaningless unless it routes into a real human review queue.
A human review queue is not ‘someone can look at it if they have time’. It’s owned work with an SLA.
The simplest pattern is a state machine:
- Pending Review
- Reviewed
- Overridden / Confirmed
- Closed (with reason)
This does two important things:
- It makes it obvious when work is stuck
- It makes it auditable when work is completed
4) Contestation path (what changes after a challenge)
If someone challenges an outcome, what can actually change?
This is where teams fall into a trap: they route the case to a human, but the human has no mechanism to override the system in a way that is recorded and replayable.
So the person experiences ‘review’ as another automated rejection.
A contestation path should include:
- the ability to override (with reason code)
- a record of what changed
- a link to the decision log entry
If you can’t diff the decision outcome, you can’t defend it.
Why ‘humans in the loop’ often fails
People say ‘human in the loop’ as if the existence of a human is the control.
It isn’t.
The control is the override mechanism plus the receipt.
Here are the common failure patterns I see in real businesses:
- Rubber-stamping: humans click approve because the system is treated as authoritative
- No override: humans can review, but they can’t change the outcome
- No evidence: humans can change the outcome, but nothing is logged
- No ownership: cases exist, but nobody is accountable for clearing them
Those patterns aren’t malicious. They’re what happens when teams ship capability and forget the operating model.
Implementation notes (how to do this without a giant programme)
You do not need a six-figure governance project to implement these controls. You need a small, disciplined data model and a few workflow primitives.
Treat ‘review’ as an event stream
Every review case should emit events:
- created
- assigned
- reviewed
- overridden/confirmed
- closed
Those events can be written to the same store you use for automation logs. The point is to make the workflow replayable.
Use reason codes (not essays)
Reason codes are underrated. They make analysis possible without forcing reviewers to write paragraphs.
Examples:
- insufficient evidence
- data mismatch
- exception approved
- false positive
- policy override
Reason codes give you a way to tune thresholds and improve inputs later.
Make opt-outs and high-risk cases the same queue
You’ll likely want two routes into the queue:
- explicit opt-outs
- high-risk edge cases (where you want human review by default)
Treating them as one queue with different priority rules keeps the operating model simple.
Store a snapshot, not just a pointer
If the input data can change later, you need to capture what was true at the moment of decision. Otherwise you’ll be reconstructing history from moving targets.
What to do before 29 May 2026
If you want a practical plan that doesn’t turn into a compliance rabbit hole, I’d do this in order:
1) Inventory decision surfaces where automation influences outcomes (start with recruitment, credit, pricing, fraud).
2) For each one, answer the override questions: who, how, evidence.
3) Implement the ADM Control Bundle for the top 1–2 highest risk surfaces.
4) Only then think about model upgrades, better prompts, or fancier scoring.
The goal is not to remove automation. The goal is to make automation accountable.
The position I’ll defend
If your workflow can’t route an opt-out into a real human review queue with a logged decision, you’re already doing automated decision-making — no matter what your policy document says.
That’s not a legal take. It’s an operational one. When decisions move fast, the only thing that keeps you safe is a workflow that can be challenged, overridden, and replayed.