Autonomy is not authority: where agent decisions end and the right to act begins

In the Controlled Autonomy Lab, I implemented the same incident-analysis task using six architectures with different degrees of control and autonomy. The experiment's most autonomous pattern receives five tools and decides which to consult, in what order, and when there is enough evidence to conclude.

That freedom was deliberately placed inside a narrow boundary:

The five tools are read-only;
The code limits execution to six steps and eight tool calls, validates each tool against an allowlist, and restricts access to the active incident;
There is no shell, restart, rollback, configuration change, or any production write operation.

The lab began as an investigation into who determines the next execution step: code or model. After working with that separation, a second dimension became clearer to me. A system can give an agent considerable freedom to decide how to perform a task while still strongly limiting the effects that execution can produce.

I came to treat that difference as autonomy and authority.

Choosing the path does not imply the power to produce any effect#

Think of an operations agent receiving an alert about increased latency. It can consult metrics, recent deployments, dependencies, runbooks, and previous incidents. It is unnecessary to predetermine whether the investigation must start with metrics or recent changes. The evidence found at each step can guide the next query.

Now add two tools to the same agent:

The cognitive trajectory remains similar, but the consequence of a bad decision changes substantially: whereas an incorrect query produces a worse diagnosis, an incorrect rollback changes a shared environment and may worsen the incident the agent was supposed to resolve.

This difference tends to disappear when everything is modeled solely as tool calling. To the LLM, consulting metrics and restarting a service may be two functions described in the same catalog. To the architecture, they represent very different levels of authority.

The point becomes even clearer in other domains. A customer-service agent can check a charge's status, prepare a refund request, or actually approve the refund. A coding agent can read a repository, generate a patch, open a pull request, merge, and start a deployment. The fact that these operations are technically accessible to the runtime does not mean they should be equally available to every execution.

Capability and authority need to appear separately in the design#

I have been using a simple separation to reason about this. Capability describes what the platform can do. Authority describes what a particular execution may do in that context, using that identity and under those conditions.

A payments API may offer the capability to transfer money. The authority granted to an agent may be much narrower: consult data, prepare transaction parameters, and request approval. The service remains capable of moving funds, but that capability does not need to be directly available to the component interpreting natural language.

This distinction also helps evaluate tool catalogs. A set with search_logs, get_metrics and read_ticket exposes a small operational surface. Another with delete_customer, transfer_funds and deploy_release changes the risk profile even if both use exactly the same tool-calling protocol.

In agent systems, therefore, the tool catalog also acts as a form of policy. It defines part of the action space the model can reach.

Read and write help, but the real boundary lies in the effect#

Separating read and write tools is a good start, but still a coarse distinction. Some write operations merely prepare something for review, while others immediately produce an external effect.

This difference is useful because it allows some autonomy to be retained without handing over all the authority available in the process along with it.

The agent can formulate the action, assemble parameters, gather evidence, and explain why it believes that is the appropriate next step. Execution of the effect may depend on another decision, made by deterministic policy, an identity with specific privileges, or human approval.

This design does not eliminate bad model decisions. It limits how far a bad decision can travel without encountering another boundary.

Human approval must be bound to what was approved#

Human-in-the-loop often appears in diagrams as a block between agent and action. In implementation, the important detail is exactly what that approval authorizes.

Imagine a transfer prepared by the agent with an amount, recipient, and source account. The user reviews and approves those details. If any parameter subsequently changes, the previous approval should not automatically remain valid.

A state such as:

is insufficient to represent that relationship. Approval must be associated with the concrete action, the presented parameters, the approver, and, where appropriate, a deadline or specific execution.

The same applies to retries. A transport timeout should not allow authorization created for one attempt to be reused indefinitely or applied to a different payload. The closer an agent gets to financial effects, production changes, or external communication, the more important it becomes to treat approval as a verifiable artifact rather than a boolean lost in conversation state.

A control failure should not increase authority#

There is a fallback pattern that works well for availability and is dangerous when applied to authorization. A component fails and the system tries a more permissive path to keep operating.

If a policy service is unavailable, for example, executing the action directly preserves availability at the expense of the control intended to decide whether that action was permitted. The same problem appears when a specific identity cannot be obtained and the system falls back to a broader service account, or when a restricted tool fails and the agent receives access to a generic tool with greater privileges.

For significant side effects, I prefer the opposite property: the less confidence there is in identity, policy, or context, the less authority is available to that execution.

This brings the design closer to fail-closed behavior. A failure to query metrics may result in an incomplete analysis. A failure in the component deciding whether a transfer is authorized should prevent the transfer, not open a broader alternative route.

The model can propose; another component can authorize#

One way to make this boundary explicit is to separate proposal, policy decision, and execution of the effect.

The agent remains responsible for interpreting context and suggesting the next action. It can produce something structured such as:

The policy decision evaluates information that need not be under the model's control: identity, role, financial limit, environment, segregation of duties, operation risk, transaction state, and applicable policy version.

The component holding the credential capable of executing the refund checks that decision before producing the effect.

This separation is especially useful because a plausible model response ceases to be equivalent to authorization. Prompt injection, hallucination, incomplete context, and incorrect classification remain risks, but the potential impact also comes to depend on boundaries implemented outside the model.

Broad credentials undermine part of this separation#

The architecture becomes inconsistent when the prompt restricts behavior while the identity used by the runtime remains excessively privileged.

A system prompt may say the agent should only read data. If the credential available to the tool also allows modification and deletion, the most important restriction is being implemented in the least deterministic component of the chain.

Where infrastructure allows, I think it is better to reduce the capability actually available: read-only credentials, short-lived tokens, resource restrictions, specific scopes, allowlists, and endpoints designed for bounded operations.

The prompt still has a role. It guides the agent's decisions and helps produce coherent behavior. It simply should not be the only barrier between a probabilistic interpretation and a sensitive effect.

Autonomy makes more sense when assigned to the action#

I have also avoided classifying an entire application as “autonomous” or “with HITL”. Within the same agent, different actions can justify different levels of control.

This gradation seems more useful than applying a single level of autonomy to the entire system. Consulting documentation or producing a diagnosis can happen without approval, while changing configuration, executing a rollback, or moving funds crosses different operational boundaries.

Improving reasoning, tool use, or planning may justify handing more complex problems to the agent. It does not imply expanding its permissions in the same proportion.

The bounded agent ended up illustrating this separation#

In the Controlled Autonomy Lab, the initial objective was to compare execution trajectories and observe what changed when part of the choice of next step moved from code to model.

The bounded agent ultimately represented this separation well. It chooses which evidence to consult and in what order, while permissions, budgets, and action capabilities remain defined by the application. Autonomy lies in the investigation; the authority boundary remains explicit.

This does not mean production agents must remain read-only. In many processes, value appears precisely when the system can complete an action. The point is to treat the passage from decision to effect as an architectural boundary of its own, with controls proportional to that action's impact.

From there another question emerges. If an action needs authorization, which identity is actually exercising that authority: the user, the agent, its hosting workload, or a delegated authority?

That is the next problem I want to explore.