Skip to main content
VantaSoft
Back to Insights

Human in the Loop for AI Agents: What to Approve and What to Let Run

Decide approvals one action at a time. Four oversight modes, a placement table, a review worksheet, and five questions that show whether a vendor really enforces a gate.

Rollout and oversight 20 min read

Human in the loop for AI agents means a person approves, reviews or is told about specific actions the agent takes, instead of trusting it from end to end. Decide it one action at a time rather than once per agent: which actions the agent does on its own, which it reports, which wait for your approval, and which it never touches at all.

Most vendor conversations treat this as a checkbox. Yes, there is a human in the loop. That answer hides the two ways oversight fails. If a person reviews everything, the agent turns into a suggestion machine feeding a queue nobody clears. If nobody reviews anything, you find out what the agent misunderstood after it has cost you something. Below is a method for placing each action, a way to test whether a tool or provider actually enforces the gates it promises, and the part most pages skip, which is designing the review so the person stays a real check instead of approving by reflex.

Human in the loop is a decision you make per action

An AI agent in a business workflow takes a series of separate actions. It reads a message, sorts it, updates a record, drafts a reply, sends it, closes a ticket or moves money. Those actions carry different risk, so they need different oversight. Each one can sit in one of four modes.

ModeWhat the agent doesWhat the person doesFits when
Act and logCompletes the action and records what it did and whyReviews a sample of the record on a scheduleThe action is easy to reverse, stays internal, and happens often
Act and notifyCompletes the action and tells a named personReads the notice and can reverse it inside a short windowThe action is reversible but someone should know it happened
Approve beforePrepares the action with its evidence and stopsApproves, edits, or rejects, usually with one clickThe action is hard to undo or reaches customers, money, access, or people
NeverCannot perform the action, because the permission was not grantedDoes the action themselves, with the agent's preparationThe action is irreversible and consequential, or outside the agent's job

Most of the agent's value comes from the first two modes. Approve before is where the agent earns trust with the people who depend on it, and the never list is what keeps the worst week survivable.

OpenAI's practical guide to building agents names two triggers that warrant human intervention: exceeding a failure threshold, such as too many retries, and high-risk actions that are "sensitive, irreversible, or have high stakes," which it says should trigger human oversight "until confidence in the agent's reliability grows." That last clause matters. It makes approve before a starting position for an action rather than a life sentence.

The same structure already ships in the tools engineers use. Anthropic's Claude Code permission system asks before it changes a file or runs a command, lets an approval be saved as a standing rule, and reserves its no-questions mode for sandboxes where the agent cannot do damage. Per action oversight is a product default there, not an add-on.

How to decide which mode an action gets

Placing an action does not require understanding the model. It takes two answers that a business can give in a meeting.

The first is reversibility. Can the action be undone, by whom, at what cost and inside what window? Relabeling an email can be undone in a second. A sent email cannot be undone at all.

The second is reach. Who and what does the action touch? A note on an internal record touches one row. A message to a customer touches your reputation. A changed payment detail touches money, a change to someone's access touches security, and a decision about a person's employment touches their life.

OpenAI's guide suggests rating each tool an agent can use as low, medium or high risk on these same factors, including "read-only vs. write access, reversibility, required account permissions, and financial impact," then using the rating to pause or escalate. The table below applies that idea for the person who owns the workflow.

ReversibilitySmall reach: internal, one recordMedium reach: a team's work, a vendor, an internal messageLarge reach: customers, money, access, employment, legal
Undone in seconds by anyoneAct and logAct and notifyApprove before
Undone with effort, or only inside a windowAct and notifyApprove beforeApprove before
Cannot be undoneApprove beforeApprove before, or neverNever at launch

Here is where common actions land. Sorting an inbound message and assigning an owner in the CRM are act and log. Posting a daily status summary to an internal channel is act and notify. Drafting a reply to a customer needs no gate, because a draft changes nothing, but sending that reply is approve before, since the send is the action. Issuing a refund or changing a vendor's bank details is never at launch. The agent prepares the case and a person does it.

Which systems the agent can reach at all is a separate decision, covered in what access an AI agent should have to your business systems. This article assumes access is settled and looks at what happens inside it.

The actions that stay behind a person, whatever the table says

Anything that commits money, changes access or permissions, deletes records, sends something a customer or vendor will treat as a promise, or affects someone's employment or legal position belongs on the human side of the line at launch. That is the default in our discovery sessions, and it matches the approval line in the rollout guide.

The agent can still help with these actions. It gathers the evidence and drafts the action, and a person commits it. In a law office, for example, the agent gathers the records for the monthly trust reconciliation and a lawyer moves the money, one of the splits in our guide to the office jobs a small law firm can hand an AI agent. In most workflows, the preparation is where the hours were going in the first place. Where that line landed for us in practice, including two jobs that went back to a person because they went out in a named person's name, is in when to use AI agents and when not to.

Design the review so it stays real

Vendor pages usually skip this half of human in the loop, and it decides whether your design keeps working or slowly stops meaning anything.

Name a person as the approver

An approver has to be a person, so "operations approves" does not count. Name the approver and a backup, and write both down. On our own task board, the longest waits were requests that named no one, as what happened to 101 handoffs from our agents to people shows. If nobody can be named, the action is not ready for approve before and belongs in never.

Use the channel the approver already works in

Approval requests go to the tool the approver already has open, whether that is a chat channel or an inbox. They arrive complete, with the proposed action, the evidence the agent used, its reasoning in a line or two, and the options to approve, edit or reject. If checking an approval means opening four systems, it will be approved without checking.

A deadline and a default

Every approve before action needs a response window and a stated outcome if the window closes with no answer. The safe default is almost always to do nothing and escalate to the backup. A default of "proceed if nobody objects" turns the gate into a timer, which is no oversight at all. Who approves, in what tool and by when is one of the seven inputs the implementation guide asks you to bring to a first working session.

A review load a person can actually carry

Most designs ignore this constraint, and the research on people supervising machines is blunt about it. In their review of the empirical studies, Parasuraman and Manzey found that automation complacency shows up when the supervised task competes with other work for attention, that it appears in novices and experts alike, and that it "cannot be overcome with simple practice." Automation bias, acting on the machine's suggestion when it is wrong, "cannot be prevented by training or instructions." (Human Factors, 2010)

So if you route hundreds of low-stakes approvals a day to one busy person, you get hundreds of clicks and very few real reviews. A more diligent person will not fix that. A shorter approve before list will. Put the reversible, internal, high-volume actions in act and log, keep approve before for the actions that need it, and treat the daily approval count as a number you check every week.

Sample the autonomous tier

Act and log still means somebody looks. A person checks a sample on a schedule instead of every instance in real time. Pull a fixed number of logged actions each week, check them against what a person would have done, and write down the miss rate. That miss rate tells you whether the action belongs in its current mode. Before an action gets into act and log at all, it helps to score the job itself: which recurring jobs can finish without anyone watching sets out the six questions we use, and the first two are how fast a wrong result shows up and how cheaply it undoes.

Enforce the gate in the system, not in the instructions

An instruction in the agent's setup such as "never send an email without approval" is only a request. OWASP's guidance on excessive agency names excessive autonomy as one of three root causes of damaging agent actions, and its fix is specific. It calls for a person to approve high-impact actions before they happen, built either into the system that performs the action or into the tool the agent calls, and for authorization to be checked by those systems rather than left to the agent to decide. In plain terms, the send button waits for a person because the sending tool is built that way, whatever the agent has been told.

We hold our own agents to that. Nothing any agent of ours writes to someone outside the company goes out on its own. The message is drafted, a named person reads that exact text, and only an approval sends it. None of them can spend money, change a price or touch a login without an owner approving it first. The rule lives in the tool that performs the action rather than in the agent's instructions, and we learned the difference the hard way when one of our own publishing jobs shipped work it had been told to leave alone: what went wrong when we gave AI agents real work. Our bid system goes a step further and will not record an owner's go-ahead on a bid until it reads back that owner's own message, as our piece on using AI for government contracting explains.

Keep a record and cap the rate

OWASP lists two measures that limit the damage of a bad action without preventing it: log and monitor what the agent's tools do, and limit how often they can run so monitoring catches a bad pattern before it does real harm. Anthropic's guidance on building effective agents makes the same point from the other side, noting that autonomy brings "the potential for compounding errors." Every action in every mode should leave a record of the input, the decision, the evidence and the approver if there was one, and every action type should have a ceiling per hour. If an agent starts doing the wrong thing two hundred times, the ceiling should stop it at ten.

How to tell whether a tool or provider really enforces approvals

Ask to see an approval happen on a live action instead of on a slide. A gate is real when the system that performs the action refuses to perform it without a recorded human decision. If the only thing between the agent and a sent email is a sentence in its setup, the gate is a preference rather than a control.

Five questions separate the two, and each one has an answer that means yes.

Ask themThe answer that means yes
Which actions can this agent never take, and what stops it?A named list, with the permission left off the connected app, instead of a rule written into the agent
Show me an approval being rejected. What happens next?The action does not happen, the rejection is recorded, and the agent reports back instead of trying again
What can the agent read and change in each app we connect?Read-only where reading is enough, and only the records the job needs
Whose login does the agent use?An account of its own with its own limits, not a shared admin login that can reach everything
What gets recorded, and can I open it myself?Every action, with the input, the decision, the evidence and the approver, in a place you can check without asking them

The third and fourth answers come from published security guidance rather than from our own preferences. OWASP's advice tells builders to limit what an agent can reach to the minimum the job needs, enforced by the permissions on the system itself, and to run actions in the context of a specific user with the fewest privileges necessary. A vendor who cannot tell you which account the agent uses has not done that work.

Two asks turn a demo into evidence. Have them point the agent at the one action you said it may never take, in front of you, and watch what happens. Then have them open yesterday's record for an action it already took and show you who approved it. Those two minutes tell you more than the rest of the hour.

There is one more reason the send stays behind a person. OWASP's own example is an assistant with access to a mailbox that receives a hostile email telling it to search the inbox for sensitive information and forward it to a stranger, and the fixes it names are read-only access to the mail and a human pressing send on every message the assistant writes. Your agent reads messages that come from outside your business, so the things it reads can try to give it orders.

Move actions between modes on evidence, in both directions

For most actions, approve before is meant to be temporary, so you need a rule for what earns a move.

Use the evidence from the review design above. For an action in approve before, track how often the approver changed or rejected what the agent proposed. For an action in act and log, track the sampled miss rate. When both the edit rate and the miss rate have stayed low across enough consecutive cycles to mean something for that workflow, move the action one step: approve before to act and notify, or act and notify to act and log. As an illustration rather than a benchmark, here is the kind of rule to write down: an internal action graduates after a full month of cycles with no approver edits and no sampled misses. Set your own numbers and put them in writing before launch.

Moves go the other way too. If the workflow's rules change, a connected app changes, the underlying model changes or the edit rate climbs, the action goes back a step until the evidence recovers. Customer-facing sends and anything touching money do not graduate on schedule alone. They graduate when a named person signs off on the evidence, and some of them never do.

Anthropic's guidance describes agents that "pause for human feedback at checkpoints or when encountering blockers." Graduation is how you decide, deliberately and with a record, where those checkpoints sit this quarter.

A worked example: an agent that triages a shared inbox

Take a common role: an agent that watches a shared inbound inbox, keeps it organized and prepares replies (how to build one). Here is one reasonable launch setup. Your actions and owners will differ.

ActionMode at launchWhy
Read new messages and sort them by typeAct and logInternal, reversible in seconds, high volume
Assign an owner in the helpdesk or CRMAct and logInternal, reversible, the owner sees it immediately
Post a morning summary of open items to the team channelAct and notifyInternal message, useful for everyone to see, easy to correct
Draft a reply to a customerNo gate neededDrafting changes nothing until someone sends it
Send a reply to a customerApprove beforeReaches a customer, cannot be unsent
Close a ticket as resolvedAct and notifyReversible, and the owner should know
Issue a refund or creditNever at launchMoney, and the case is prepared for a person

The review design that goes with it has the support lead approving customer sends, with the operations manager as backup. Requests arrive in the team's chat tool with the customer's message, the draft and the reason for the proposed reply. The window is the same business day, and if it closes, the item goes to the backup and nothing is sent. Each week the lead checks a fixed sample of the sorting and assignments and records the miss rate. After a month of cycles with no approver edits on a narrow class of replies, such as order status confirmations that quote the record exactly, that class becomes a candidate to move to act and notify, and the lead decides. To work out which of your own replies belong in that narrow class, sort your last 100 customer messages before you set the gates.

The review design worksheet

Fill in every field for each action before launch. If a field cannot be completed, the action is not ready for any mode except never. Here is the support example filled in:

FieldSend customer reply
Mode at launchApprove before
Approver and backupSupport lead, ops manager
ChannelTeam chat
Deadline and defaultSame business day, then escalate and hold
What is recordedMessage, draft, evidence, decision, approver
Evidence to move itNo edits across a full month of cycles, lead signs off
Next review dateFirst of next month

This worksheet is the human side of the scorecard in the AI agent use cases guide. A provider should ask you for it rather than write it for you.

What this looks like in a managed service

In VantaSoft's Agent Service, every agent gets a defined job, approved systems, scoped permissions, explicit human approval points and escalation rules. Those decisions are made in discovery, where we work out the business objective, the roles, the workflows, the apps to connect, the volume and your oversight rules together, and they are written into the Order Form rather than left to whoever sets up the agent. Approval gates are explicit for anything that reaches customers, money, hiring, legal or access. We build the agents and connect them around those rules, and monitoring, routine updates and issue investigation continue after launch.

Whether you buy a managed service or build your own, the approval rules are yours to own. Nobody outside your business can decide what your customers should hear from a machine, or who signs off on a refund.

Frequently asked questions

Does human in the loop make an AI agent slower than doing the work ourselves?

Only if you gate the wrong actions. Approve before on a customer send costs one click on a draft that is already written and already checked against the record. Approve before on every message that gets sorted costs a person their afternoon and produces reflex approvals. If you gate by reversibility and reach, most of the workflow runs without waiting.

Can the agent decide for itself when to ask a person?

An agent can be told to escalate when it is unsure, and it should be, but that is a request inside its setup rather than a control. The actions that must wait for a person should wait because the system enforces it, which is the authorization principle OWASP describes. Use the agent's own judgment as a second safety net and never as the only one.

What should I ask a vendor about approvals?

Which actions the agent can never take and what stops it, what happens when an approval is rejected, what it can read and change in each connected app, whose login it uses, and what gets recorded where you can see it. Then ask for two live demonstrations: the agent attempting a forbidden action, and yesterday's record of an approved one.

Can a message from outside trick the agent into doing something?

It can try. An agent that reads email, forms or tickets is reading text written by people outside your business, and that text can contain instructions. OWASP's example is a hostile email that tells an assistant to forward sensitive information to a stranger. The defenses are the same ones in this article: read-only access where reading is enough, a person pressing send, and a record you check. It is the first entry in our register of AI agent security risks, which pairs each risk with a control.

What is the difference between human in the loop and human on the loop?

People use the terms loosely. Broadly, in the loop means a person approves an action before it happens, and on the loop means the action happens while a person watches the results and can step in. In the four mode model, that is approve before versus act and notify, or act and log with sampling. Most real workflows use both, for different actions.

When can we remove an approval step?

When the evidence supports it: a low edit rate on the approvals and a low miss rate on the samples, sustained across enough cycles for that workflow, with a named person signing off. Write the rule down before launch, move one step at a time and keep the ability to move back.

Is approve before enough on its own?

No. It is one of three layers. The agent's access has to be scoped so it cannot reach what it should not, every action has to leave a record and have a ceiling, and the approval gates sit on top of both. A gate does little if the agent behind it can reach everything.

Decide the gates before the tools

Whether to have a human in the loop is rarely the open question. What you need to settle is which actions get which kind of oversight, who checks them, and what evidence would change that. Answer it per action, write down the review design, and look at it again each month. Your agent will do more on its own over time, and every expansion will have a record behind it.

If you want to work through this for a specific workflow, that is what our discovery session does. It starts with 30 minutes on your workflows, and the human oversight rules are one of the things we settle together before anything is built.

Start Automating

Five quick questions about where your team's time goes. You'll get the one agent we'd start with, and what it would take off your plate.

Take the quiz

Put your first agentto work.

We build custom AI agents around your processes and the tools you already use. It starts with 30 minutes on your workflows.