What “human in the loop” actually means for AI agents, and three ways products implement it
A definition of human in the loop for AI agents, and a fair comparison of the three ways products implement it: gate by default, per-tool toggle, opt-in workflow node.
Last updated
8 minute read · Written by the team that builds UtilitySpaces. More about UtilitySpaces.
Human in the loop, for an AI agent, means a person decides before the agent’s action takes effect in a real system. Products implement that in three broad ways: an approval gate that is on by default for every write, a per-tool toggle an administrator sets, or an opt-in review node the person building a workflow wires in. Each is legitimate. They differ in who owns the safety property and in what happens when nobody remembers to set it.
The phrase is doing a lot of work in vendor copy right now, and it is worth pulling apart, because two products can both say “human in the loop” and mean things that behave completely differently on a bad day.
Why is the phrase so slippery?
Because it predates agents. In machine learning it has long meant a person labelling data or correcting model output during training. In operations it often means a person reviews what the system did, afterwards, on a dashboard. And in agent products it has come to mean a person approves an action before it happens. Those are three different moments — before training, after the fact, and at the moment of action — and only the last one stops a wrong write.
So the first question to ask any product is when the human is in the loop. A review log you read on Friday is valuable and it is not a control. A person who had to click before the invoice went out is a control. The rest of this piece is only about the third kind, the approval-before-action kind, because that is the one that determines blast radius.
Model one: the approval gate is on by default
Every action that writes to a real system stops for a person unless the product has decided, categorically, that it is safe. There is no setting that turns the gate off. The safety property belongs to the vendor: it is true on the first day, in every workspace, whether or not anyone configured anything.
The strength is that it cannot be forgotten. A new agent, a new connector, a new team member who has never read the documentation — the write still stops. The weakness is that it cannot be relaxed either. A workspace that genuinely wants an agent to send a hundred routine confirmations unattended cannot have that, and will either accept a hundred approvals or take the job elsewhere. Default-on trades flexibility for a guarantee.
UtilitySpaces starts on the vendor-owned side of this for connector writes, with one admin lever: reads are never gated, every new connection starts on “Ask a person first”, and an admin can set a whole connection to run without asking or to refuse writes, a change that is itself recorded — a delete waits for a person whatever the policy. Merges and knowledge changes do stop for a person every time, and so does finished work unless the agent is on the trusted tier. What a workspace can change is the shape of the work — what the agent is asked to do, which apps it is connected to, which tier it is on — and, for an admin, each connection’s write policy. The mechanics are on the approvals page.
Model two: a per-tool toggle
Each tool or action carries a setting that says whether it needs review, and an administrator can raise or lower it. Sending mail might ship as “always ask”; looking up a contact might ship as “never ask”; and both can be changed. The safety property belongs to the administrator.
This is the most flexible of the three, and for a team with a thoughtful administrator it is arguably the best: the gate is exactly where the risk is and nowhere else, and approval fatigue is tunable rather than structural. The cost is that the guarantee is now a configuration state. Somebody can lower a toggle on a busy afternoon to clear a backlog and forget to raise it, and the product will do exactly what it was told. Auditing the current state of every toggle becomes part of the security review, which is a real job.
Some products also pair the toggle with a stake or risk level per tool, so the default leans safe and only an explicit change loosens it. That is a sensible refinement and it does not change the ownership: the answer to “will this write stop?” is “check the setting.”
Model three: the opt-in workflow node
The product gives the person building a workflow a node or step that pauses for a decision, and leaves it to them to put it in the right place. Nothing gates an action unless the builder wired review in. The safety property belongs to whoever drew the flow.
For a workflow builder this is the correct design, not an omission. The whole premise of a builder is that the person drawing the flow is responsible for every step; a mandatory gate would be as out of place there as a mandatory confirmation dialogue on every line of a script. Engineers who maintain pipelines want the pause exactly where they put it and nowhere else.
The weakness is the obvious one. A flow without a review step runs unattended, and the product cannot tell the difference between “the builder decided this needs no review” and “the builder forgot.” In a team where one person builds and many people depend on the result, the safety of a write depends on a decision made by someone else, some time ago, that nobody else can see without opening the flow.
How do the three compare?
| Model | Who owns the safety property | Fails how | Where it genuinely wins |
|---|---|---|---|
| Gate on by default | The vendor | Closed: unwanted approvals, not unwanted writes | Teams that want the guarantee without configuring it, and work where one bad write is expensive |
| Per-tool toggle | The administrator | Open if a toggle is lowered and left there | Teams with an engaged admin and a mix of trivial and serious actions |
| Opt-in workflow node | The builder | Open if the node was never added | Engineer-maintained pipelines where the builder is accountable for every step |
“Fails closed” means the mistake costs you attention; “fails open” means it costs you a write.
Which one should you want?
Ask who in your organisation you want holding the property. If nobody will audit toggles and nobody will review flows, default-on is the only one that stays true. If you have an administrator who will own it, a per-tool toggle gives you a better attention budget. If engineers build and maintain your automations and are accountable for them, the opt-in node is the right shape and a mandatory gate would only annoy them.
The honest framing is that the three models sit on a line from guarantee to flexibility, and every product picks a point. The comparison pages say which point each named product has chosen, in that product’s own words with a source, rather than characterising them here.
What does default-on look like in practice?
In UtilitySpaces a connector write on a connection left on the default is queued, not run; what reaches a person is eight kinds of request — task output, document update, message draft, tool permission, knowledge proposal, external data operation (the connector write), skill import and model marketplace — and every one of them passes through a single review function, so a decision cannot be applied by one route and skipped by another. Requests appear in the workspace approval queue, in the inbox beside the run that raised them, and as a notification in the channel where the work is happening; the decision and any reviewer comment join the same thread.
Reviewing requires the workspace admin role. Members can be given work and can raise requests, but the decision itself is an admin action, which is the only way the gate means anything in a workspace with more than a few people. And an approval does something rather than merely changing a status: executors run inside the review, so approving a delivery moves the card and rejecting it sends the work back for the agent to try again. Every decision is written to an append-only audit log whose entries each carry the hash of the one before, which is what lets you answer “who approved this and when” six months later with a sequence number rather than a recollection.
The related design question — which actions deserve a gate at all, and how to stop people approving on reflex — is its own piece: designing the moment an agent stops and asks.
Where default-on costs you, and what UtilitySpaces will not do
Default-on has real costs and this is the place to state them. Each waiting connector write is its own approval, so a job that needs a hundred identical writes belongs in a workflow builder, and a flow step that waits does not resume the rest of its run on its own. The only relaxation for connector writes is an admin setting a whole connection to run without asking — Chatwoot and WAHA connections made before the policy existed were kept that way, so writes through them run with nobody approving until an admin changes it — and there is no per-tool setting. Approvals are an admin action, so a workspace with one admin has one bottleneck, and an agent on the trusted tier skips the finished-work review. And the gate governs actions inside the workspace; it does nothing for writes made by other tools against the same applications. If your requirement is a tunable gate or unattended bulk writes, pick model two or three and do not feel bad about it.
When the work is deterministic and high-volume, the right answer is usually a workflow builder with an opt-in review node placed by an engineer who owns it; the full decision guide is in agentic workspace or workflow builder.
What to ask a vendor
- When is the human in the loop — before training, after the fact, or before the action takes effect? Only the last is a control.
- Who owns the setting? The vendor, an administrator, or the person who built the flow?
- Can it be turned off, and if so, who can turn it off and is that change logged?
- What does a write look like when nobody has configured anything? That is the answer that matters on day one and in the workspace nobody is watching.
- What does the record show afterwards — the tool, the arguments, the person, the time — and can the record be edited?
If you want to see what a default-on gate feels like before deciding whether you can live with it, open a demo workspace: it runs the full product for about an hour with no card, long enough to give an agent a job and watch what it does. Plans are on the pricing page, and support@utilityspaces.com replies within one business day.