Artificial intelligence is changing rapidly. While many organizations are still working to govern large language models (LLMs) and generative AI tools, a new challenge is emerging: agentic AI.
Unlike traditional AI systems that generate content or answer questions, agentic AI can take action. These systems can access data, interact with applications, make decisions and execute tasks with varying degrees of autonomy. As organizations explore the possibilities, many are discovering that existing governance frameworks aren’t designed to manage the risks that come with autonomous AI.
In a recent webinar, Info-Tech Research Group Advisory Fellow Valence Howden discussed what he calls the “Agentic Risk Gap” and why organizations need to rethink how they approach AI governance before autonomous systems become embedded in critical business processes. The gap is the distance between governance models built for AI outputs and the controls needed for AI systems that can act autonomously.
Key Takeaways
- Agentic AI can take actions autonomously, posing new governance challenges for organizations managing AI risks.
- The Agentic Risk Gap highlights the need for governance models that address the risks of autonomous AI systems.
- Organizations must assess risks and define accountability for AI agents, tailoring oversight based on risk levels.
- Implement foundational controls, prepare for failures, and expand testing practices to improve AI governance.
- Successful organizations will set clear boundaries and align governance with risk to manage AI outcomes effectively.
Table of Contents
What Makes Agentic AI Different?
Traditional generative AI typically operates within a simple model: a user provides a prompt and receives an output. The human remains responsible for evaluating the response and deciding what happens next.
Agentic AI changes that relationship.
These systems can pursue goals, execute multi-step workflows, access external systems through APIs and coordinate with other agents to complete tasks. Rather than assisting with work, they increasingly perform work.
Some examples include:
- Regulatory monitoring agents that gather and categorize compliance updates
- IT agents that automate routine operational tasks
- Decision-support agents that recommend next actions based on business rules
- Multi-agent workflows where specialized agents collaborate to complete complex processesÂ
The more autonomy organizations grant these systems, the more governance challenges emerge.
The Agentic Risk Gap
Many of the risks associated with generative AI still exist in agentic systems, including hallucinations, biased outputs and poor data quality. However, agentic AI introduces an entirely new category of risk because it can act independently.
According to Howden, the concern is no longer limited to incorrect outputs. Organizations must also consider:
Autonomous Actions
An agent with access to systems and data can execute tasks without direct human involvement. If the agent makes a flawed decision, the consequences may extend far beyond a single inaccurate response.
Cascading Failures
In multi-agent environments, one poor decision can influence downstream agents. A faulty assumption or incorrect action can propagate through an entire workflow, creating a chain of unintended consequences.
Lack of Traceability
Many organizations struggle to understand how an agent arrived at a particular decision. Without comprehensive logging and observability, it becomes difficult to investigate incidents, validate outputs or demonstrate compliance.
Irreversible Outcomes
One of the most significant concerns is allowing agents to perform actions that cannot easily be undone. Organizations need clear boundaries around what autonomous systems can and cannot do.
Why Human Oversight Still Matters
As AI autonomy increases, organizations often assume that human involvement can decrease. The reality is more nuanced.
Howden argues that oversight models should be determined by risk and criticality, not simply by the technology itself. High-risk use cases require stronger governance controls and greater human involvement.
Organizations typically move through three levels of oversight:
Human in the Loop
Humans review and approve actions before they occur. This approach offers the highest level of control but becomes difficult to scale when large volumes of decisions are involved.
Human Over the Loop
Agents operate independently, but humans monitor for exceptions and intervene when unusual behavior occurs. This model balances efficiency and governance for many use cases.
Minimal Human Oversight
This approach may be appropriate for low-risk, routine activities where errors have limited consequences. However, organizations should avoid applying fully autonomous models to highly sensitive processes without significant safeguards.
The key takeaway is simple: the higher the risk, the stronger the oversight should be.
The Critical Role of AI Governance Councils
One of the most effective ways to establish accountability is through a dedicated AI governance council.
These cross-functional groups help organizations:
- Define acceptable AI use cases
- Establish risk thresholds
- Inventory AI systems and agents
- Clarify accountability
- Align AI governance with existing data, IT and compliance governance programs
- Translate policy into operational controlsÂ
Without clear ownership, organizations often find themselves in situations where no one is responsible when an agent causes harm, violates policy or creates compliance issues.
Why Traditional Governance Frameworks Break Down
Many organizations attempt to govern agentic AI using governance models originally designed for compliance programs or traditional software systems.
The problem is that agentic AI behaves differently.
According to Howden, existing governance approaches often fail because they are:
- Highly structured and hierarchical
- Focused primarily on compliance requirements
- Built for static controls rather than adaptive systems
- Too disconnected from day-to-day operations
- Lacking clear accountability structures for autonomous systemsÂ
Effective AI governance must be embedded throughout the lifecycle, from design and deployment to monitoring and incident response. Governance cannot exist solely as a policy document.
A Risk-Based Approach to Agentic AI Governance
Rather than applying the same level of scrutiny to every AI use case, organizations should align governance requirements with risk.
A tiered model helps organizations focus resources where they matter most:
Low-Risk Agents
- Basic monitoring
- Defined access controls
- Structured outputs
- Standard validation processes
Medium-Risk Agents
- Enhanced oversight
- Human review by exception
- Kill switches
- Incident response procedures
High-Risk Agents
- Mandatory human involvement
- Comprehensive logging and traceability
- Continuous monitoring
- Formal testing and validation
- Regulatory alignment and compliance reviews
This approach helps organizations avoid two common mistakes: over-governing low-risk activities and under-governing high-risk ones.
Five Actions Organizations Should Take Now
Organizations don’t need perfect AI governance before exploring agentic AI. However, they do need a foundation.
Howden recommends starting with five priorities:
1. Inventory Your Agents
You can’t govern what you don’t know exists. Create visibility into all deployed AI agents and understand their purpose, autonomy level and risk profile.
2. Define Accountability
Every agent should have a clearly identified owner who is accountable for its outcomes and ongoing management.
3. Expand Testing Practices
Move beyond basic functionality testing. Use scenario planning, red teaming and purple teaming exercises to identify potential failures before they occur.
4. Implement Foundational Controls
At a minimum, organizations should apply:
- Least-privilege access
- Zero-trust principles
- Comprehensive logging
- Access controls
- Continuous monitoringÂ
5. Prepare for Failure
Build incident response plans, establish kill switches and define escalation procedures before an issue arises. The time to determine how you’ll stop a problematic agent is before you need to stop it.
The Path Forward
Organizations are moving beyond questions of whether to use AI and focusing instead on how to use it responsibly.
Agentic AI offers real opportunities to improve efficiency, automate complex workflows and accelerate decision-making. But as autonomy increases, so does the need for governance structures that can keep pace.
The organizations that succeed won’t be the ones that move the fastest or impose the most restrictions. They’ll be the ones that establish clear boundaries, align governance with risk and build accountability for the outcomes autonomous systems produce.
Want to go deeper on the Agentic Risk Gap? Watch the on-demand webinar, The Agentic Shift, featuring Valence Howden of Info-Tech Research Group and Michelle Randall, CMO at Onspring, for practical guidance on governing AI responsibly.
