How to Govern AI Agents That Can Take Autonomous Actions

Gartner expects that by 2027, four in ten enterprises will demote or decommission an autonomous AI agent because of governance gaps that only surfaced after a production incident. That is not a forecast about model quality. It is a forecast about permissions, approvals, and audit trails that nobody designed before the agent went live.

Agentic AI changed the governance problem in one specific way. An agent does not hand a human an output to check. It takes an action inside a live system: it issues the refund, closes the ticket, revokes the access, pushes the config change. That makes it an actor in your control environment, and most AI policies written in 2024 and 2025 still treat it like a chatbot. What follows are the controls that hold up for autonomous agents, how they map to the NIST AI Risk Management Framework and ISO/IEC 42001, and where US requirements are heading.

What counts as an autonomous action, and why the definition drives everything

Gartner’s May 2026 analysis put its finger on the failure mode precisely: organizations conflate an agent’s ability to act with the scope of access it has been granted. Those are two different variables, and governing only one of them produces either a locked-down agent nobody uses or a permissive one nobody can explain after the fact.

Separating them starts with classifying what the agent is actually allowed to do. Five action classes cover most enterprise deployments:

• Read. The agent retrieves data and returns it to the requester. Risk is data exposure and output accuracy.
• Recommend. The agent produces a decision a human then executes. Risk shifts toward automation bias.
• Reversible write. The agent changes state in a system where the change can be undone cheaply: updating a CRM field, drafting a document, tagging a ticket.
• Irreversible write. The action cannot be cleanly rolled back: sending an external email, moving money, deleting records, deprovisioning an account.
• External commitment. The agent binds the organization to a third party by accepting terms, placing an order, or responding to a regulator or customer on the record.

Governance effort should track the action class and the blast radius, not the sophistication of the underlying model. A retrieval agent summarizing internal policy documents and a procurement agent placing purchase orders may run on identical infrastructure. Governing them identically is how organizations end up with the worst of both outcomes: friction where it buys nothing, and no friction where the money moves.

These operational controls work best when they are implemented as part of a broader agentic AI governance strategy that connects governance frameworks, risk management, accountability, technical controls, and ongoing oversight.

Why your current AI governance policy does not cover agents

Most enterprise AI policies were built around a review checkpoint that agents remove. The assumption baked into them is that a person sees the output before anything happens. Take that assumption away and four control gaps open at once.

The first is identity. NIST’s National Cybersecurity Center of Excellence published a concept paper in February 2026, Accelerating the Adoption of Software and AI Agent Identity and Authorization, that stated the problem plainly: agents are commonly deployed as generic service accounts with no dedicated identity, authorization, or accountability controls. When an agent authenticates as svc-automation-prod, nobody can answer which agent took an action or on whose authority.

The second is assessment scope. Model risk assessments evaluate a model. Agent risk lives in the workflow: which tools the agent can call, which parameters it can pass, what happens when a tool returns something unexpected. Assessing the model and skipping the tool graph misses the entire attack surface that OWASP’s 2026 Top 10 for Agentic Applications catalogues under goal hijacking, tool misuse, and identity and privilege abuse.

The third is evidence. Analysis published by the Non-Human Identity Management Group in 2026 found roughly half of organizations cannot track and audit the data their AI agents access, which leaves them unable to answer a breach investigator or an auditor. The fourth is deployment velocity versus control velocity. One 2026 study of PyPI download data found agent framework packages pulling around 483 million downloads in May 2026 against 5.8 million for agent security and guardrail packages, a ratio that had widened by roughly 40 percent in five months. Teams are shipping agents far faster than they are shipping the controls around them.

Before assigning permissions or increasing an agent’s autonomy, organizations should conduct an AI agent risk assessment to understand its potential blast radius, reversibility of actions, failure modes, and the consequences of incorrect decisions.

Start with an inventory that records actions, not just models

You cannot scope controls to consequence if you do not know what your agents are allowed to touch. Standard AI inventories fail here because they record the model, the vendor, and the use case, and stop. An agent inventory needs eleven fields, and the last four are the ones that matter during an incident:

1. Agent name and the named human owner accountable for its behavior.
2. Business process it operates inside, and the process owner who signed off.
3. Every tool, API, and system it can call, with the specific operations enabled on each.
4. Data classifications it can read and write, including anything regulated under HIPAA, GLBA, or state privacy law.
5. Action class from the five above, and the highest-consequence action it is permitted.
6. Autonomy tier and the approval threshold that triggers human review.
7. The identity it authenticates as, and whether that identity is exclusive to this agent.
8. Credential type and rotation interval.
9. Escalation path when the agent fails or produces an unexpected result.
10. Who holds stop authority, and the mechanism they use.
11. Review date, and the events that force an off-cycle review.

Two practical notes from implementation work. Shadow agents are common and rarely malicious: a business analyst wires up a workflow tool with an API key because it solves a real problem this quarter. Amnesty periods work better than prohibition. Register what exists, then bring it under control. And build the inventory so it can be queried by action class, because that is the question a regulator or a board risk committee will ask first.

Give every agent a distinct, revocable identity

Shared service accounts collapse accountability. When five agents and two batch jobs authenticate as the same principal, an audit log entry tells you an action happened but not who caused it, and revoking access to contain one agent breaks the other six.

The direction NIST is signalling through the NCCoE project is to adapt existing identity infrastructure rather than invent a parallel one. That means workload identity issued per agent instance, delegated authorization that records the human principal the agent is acting on behalf of, and short-lived credentials rather than static keys. The practical checklist:

• One identity per agent, never shared with another agent or a human.
• Credentials with a lifetime measured in minutes or hours, issued at task start and expired at task end.
• An on-behalf-of claim that names the human or process whose authority the agent is exercising, carried through every downstream call.
• Tool allowlists enforced at the gateway, not in the prompt. Instructions are guidance; gateway policy is a control.
• A revocation drill run quarterly, with the elapsed time recorded as a metric.
• Lifecycle triggers that decommission the identity when the agent, its owner, or its business process goes away.

Non-human identities already outnumber human ones by a wide margin in most cloud estates, and agents that can spawn sub-agents are the first identity category in enterprise history that reproduces itself. Periodic access reviews were never designed for that. Provisioning has to be event-driven, and deprovisioning has to be automatic.

Scope permissions by consequence, not by convenience

The most common permission mistake is inheritance. An agent gets built to help a senior operations manager, so it is given that manager’s access, and now a language model with a prompt injection vulnerability holds the same privileges as a person who can approve six-figure transactions. Scope permissions to the narrowest set the task requires, then tier the oversight against the consequence of getting it wrong.

Agent autonomy tiers, matching controls, and human role

The tiers are useful because they give architects and risk teams a shared vocabulary. An agent moving from Tier 3 to Tier 4 is a governance event that requires reassessment, not a configuration change a developer makes on a Friday afternoon.

Put human oversight where the consequences are

Human oversight fails in two directions. Too little, and irreversible actions execute unreviewed. Too much, and reviewers approve hundreds of low-stakes actions a day until approval becomes reflex. The second failure is more common and harder to detect, because the control looks like it is working right up until the moment it matters. OWASP’s agentic risk taxonomy names this explicitly as human-agent trust exploitation.

Design the checkpoint around a threshold rather than a category. Instead of “all outbound emails need approval,” write “outbound emails to external recipients flagged as regulators, media, or customers with open complaints require approval; all others are sampled at 5 percent weekly.” Thresholds keep reviewer attention where it changes outcomes.

Four oversight mechanisms worth building deliberately. Pre-action approval gates, for anything in Tier 4 above threshold. Dry-run mode, where the agent produces the exact action it would have taken without executing it, which is the fastest way to build evidence during pilots. Standing stop authority, held by a named role with the technical means to use it. And post-action sampling with a documented exception rate that feeds management review.

US organizations with European exposure have a further consideration. Articles 14 and 15 of the EU AI Act apply human oversight, accuracy, and cybersecurity obligations to autonomous systems in high-risk domains, and the duty falls on the party relying on the system. The Digital Omnibus deferred most Annex III high-risk obligations to December 2027, which extends the runway but does not change the control themes worth building toward now.

The audit trail is the control, not the paperwork

A KPMG survey of large-enterprise leaders in 2026 found that three quarters cite security, compliance, and auditability as the most critical requirements for agent deployment. Auditability is the one teams underbuild, because it produces no visible benefit until an incident, a customer dispute, or an external audit forces reconstruction.

Application logs are not sufficient. Agent traces need to capture the reasoning path and the authority chain, not just the API call. At minimum, log the triggering input and its source, the plan or tool sequence the agent selected, every tool invoked with its parameters, the identity and credential used, the human principal the agent acted on behalf of, the approval record where one applied, the outcome returned, and any reversal.

Three design decisions determine whether that log is worth keeping. Make it append-only, because a mutable trace is worthless for non-repudiation. Redact at write time rather than logging sensitive payloads and cleaning them later, which keeps you compliant with HIPAA and state privacy law without losing the trace structure. And align retention with the longest applicable obligation across SOC 2 commitments, sector regulation, and litigation hold, then automate the expiry.

There is a simple test for whether the trail works. Pick an agent action from six weeks ago at random and answer four questions inside one hour: which agent took it, under whose authority, with what inputs, and who approved it. Teams that cannot do this find out during an investigation, which is the expensive way.

Map agent controls to the frameworks you already run

Agent governance does not need a separate compliance program. Every control described so far has an anchor in a framework most US compliance and security teams already operate, which matters because it means the evidence you produce serves multiple audits.

Agent control mapping across NIST, ISO/IEC 42001 and US regimes

The practical value of this mapping is audit efficiency and vendor accountability. An agent control that already produces SP 800-53 evidence does not need a separate justification to a security team, and an ISO/IEC 42001 management system gives you the review cadence, the documented roles, and the corrective action process that agent governance otherwise has to invent from scratch.

Multi-agent systems break single-agent assumptions

One agent with clear boundaries is a solvable problem. An orchestrator delegating to specialized sub-agents, each holding its own credentials and calling its own tools, is a different class of problem, and it is where a growing share of enterprise deployments now sit.

Three failure patterns show up repeatedly. Cascading failure, where a bad output from one agent becomes trusted input to the next and error compounds across the chain with no checkpoint between steps. Privilege accumulation through delegation, where each handoff is individually reasonable but the end of the chain holds a combination of permissions no single approver would have granted. And the confused deputy problem, where an agent with legitimate authority is manipulated into exercising it on an attacker’s behalf, which is prompt injection with real-world consequences attached.

Controls that address these directly:

• Delegation depth limits. Cap how many hops a task can travel. Three is a defensible starting point for most workflows.
• Non-inheritable permissions. A sub-agent receives its own scoped grant rather than the parent’s full set.
• Inter-agent authentication. Agents verify each other cryptographically. An agent that accepts instructions from any caller is an open relay.
• Action and spend budgets. Cap the total number of actions and total value a workflow can execute before it must stop and ask.
• Circuit breakers. Halt the chain automatically on repeated errors, anomalous action volume, or a tool call outside the expected sequence.
• A single accountable owner for the workflow, not one owner per agent, so nobody can point at the agent upstream.

The need for agent-specific controls exists because traditional AI governance approaches can fall short for autonomous agents that independently call tools, interact with external systems, modify records, and execute multi-step actions.

A 90-day sequence to get agent governance in place

Programs stall when they try to build the full framework before touching a live agent. Sequence it so that the highest-risk agents get controls first and the framework gets built around what you learn.

Days 1 to 30: find out what you actually have

12. Run an inventory sweep across API gateway logs, identity provider grants, and SaaS integration lists to surface agents nobody registered.
13. Populate the inventory fields above for every agent found, and classify each by action class.
14. Identify every Tier 4 and Tier 5 agent. That is your first control scope; everything else waits.
15. Assign a named owner to each one. An agent without a named human owner gets paused, not documented.

Days 31 to 60: control the irreversible actions

16. Replace shared service accounts with per-agent identities for Tier 4 and Tier 5 agents.
17. Move tool allowlists out of prompts and into gateway policy.
18. Define and implement approval thresholds as numbers, with the approver role named.
19. Turn on action-level logging with the eight fields listed earlier, and confirm the log is append-only.
20. Run a stop-authority drill on one production agent and record the elapsed time.

Days 61 to 90: make it repeatable

21. Add an agent gate to your change process so a tier change or a new tool grant triggers reassessment.
22. Map your implemented controls to NIST AI RMF functions and ISO/IEC 42001 clauses so audit evidence is reusable.
23. Set the review cadence, with event-driven triggers for new tools, new data classes, and model version changes.
24. Run the reconstruction test on a random historical action and fix whatever gaps it exposes.
25. Report tier distribution, exception rate, and mean stop time to the risk committee as standing metrics.

Where US requirements are heading

No federal statute currently regulates AI agents as a distinct category in the United States, but the standards infrastructure is moving quickly. NIST’s Center for AI Standards and Innovation launched the AI Agent Standards Initiative on February 17, 2026, following a January 2026 Request for Information on securing AI agent systems. The NCCoE identity and authorization project runs alongside it, and the COSAiS work extends SP 800-53 overlays to AI systems. None of these are binding, and all of them will shape what auditors and federal customers expect.

State law is the nearer-term pressure. Colorado’s amended AI legislation, SB 26-189, takes effect January 1, 2027 and reaches consequential decisions in employment, lending, housing, healthcare, and insurance. An agent that autonomously screens applicants or adjusts credit terms falls inside that scope regardless of whether anyone internally called it a high-risk system. New York City’s Local Law 144 already requires bias audits for automated employment decision tools, and sector regulators are applying existing authority rather than waiting for new statutes.

The through-line across all of it is consistent: identity, authorization, logging, human oversight, and demonstrable accountability for actions taken. Organizations that build those five now will not be scrambling when the requirement becomes explicit. Teams working through this systematically often start with the GAICC Agentic AI Governance Professional credential, which covers agent-specific control design, or with ISO/IEC 42001 training for the management system that holds it together.

Frequently asked questions

What is the difference between an AI agent and an AI assistant in governance terms?

An assistant produces output a human reviews before anything happens; an agent takes the action itself. That single difference removes the review checkpoint most AI policies depend on. Governance for assistants can focus on output quality and data handling. Governance for agents must add identity, permission scoping, approval thresholds, action logging, and a tested way to stop it mid-task.

Do AI agents need their own identities, or can they use existing service accounts?

They need their own. NIST’s NCCoE concept paper on agent identity and authorization identifies generic service accounts as a core enterprise gap, because shared credentials make it impossible to attribute an action to a specific agent or revoke one agent’s access without breaking others. Issue one workload identity per agent, with short-lived credentials and a recorded on-behalf-of claim.

Which agent actions should always require human approval?

Anything irreversible or externally binding above a defined threshold: moving funds, changing access rights, deleting records, sending communications to regulators or customers, and accepting contractual terms. Set the threshold as a number rather than a category so it is auditable, and use post-action sampling for lower-consequence actions instead of approving everything.

Does ISO/IEC 42001 cover agentic AI?

ISO/IEC 42001 does not name agents specifically, but its management system requirements apply directly. Clause 5 gives you accountable ownership, Clause 6 covers risk assessment for the agent’s workflow, Clause 8 covers operational control including tool access, and Clause 9 provides the audit and review cadence. The standard supplies the structure; you supply agent-specific controls inside it.

How do you log agent actions without capturing sensitive data?

Redact at write time rather than logging full payloads and cleaning them afterward. Store the structure of the action, the tool called, the parameter schema, the identity used, and the outcome, with sensitive values replaced by tokenized references. This preserves reconstruction ability for audits and investigations while keeping the log itself compliant with HIPAA and state privacy requirements.

What is a kill switch for an AI agent, and who should hold it?

It is a tested mechanism that halts a running agent and revokes its credentials immediately, not a policy statement that someone could shut it down. Assign it to a named role in security operations with 24-hour coverage, give the business owner the authority to invoke it, and drill it quarterly with the elapsed time recorded as a governance metric.

Are AI agents regulated in the United States right now?

Not as a distinct federal category. NIST’s AI Agent Standards Initiative and the NCCoE identity project are shaping expectations without binding force. State law bites sooner: Colorado SB 26-189 takes effect January 1, 2027 for consequential decisions, and NYC Local Law 144 already covers automated employment decision tools. Sector regulators apply existing authority to agent-driven decisions today.

How do you govern agents built by business teams without slowing them down?

Tier the controls. Retrieval and advisory agents get lightweight requirements: registration, a named owner, scoped read access, and logging. Reserve approval gates, dual control, and formal reassessment for agents that take irreversible or externally binding actions. Offer an amnesty window for unregistered agents so teams surface what they built rather than hiding it.

Closing thought

The organizations that get this right are not the ones with the strictest agent policy. They are the ones that can say, for any action an agent took, which agent did it, under whose authority, with what permissions, and who could have stopped it. That capability is built from five things: an inventory keyed to actions, per-agent identity, permission scoping tied to consequence, oversight set at numeric thresholds, and an append-only audit trail. Start with your Tier 4 agents, the ones taking irreversible actions today, and work outward. If you want the structured version of this, GAICC’s certification programs cover agentic governance and ISO/IEC 42001 implementation end to end.