A Human Approach to Agent Governance
Securing agents may be more effective by treating them like humans than software
Agents—autonomous logic that can control the environment— have arrived in the enterprise, offering incredible promise while creating an entirely new risk surface.
In order for security and risk leaders to be ready for agents, we have to think of them as a new software paradigm and approach security and governance from a first principles perspective.
As we introduce this new class of non-deterministic worker into our organizations, we need to recognize that agents operate much more like humans than the traditional software and applications we have secured in the past.
This presents a new governance challenge: How do we enable the creative potential of these agents while managing their unpredictability within our risk appetite?
There is hope, as it’s been done before quite successfully with the other non-deterministic agents already in the enterprise—we call them employees.
More Human Than Machine
Traditional software is entirely predictable in the way it achieves its expected outcomes. We know when it works, we know when it doesn’t, and when there’s a bug, we can find the error in the code and fix it. When someone figures out how to make software behave unpredictably, we call that a vulnerability, and we have clear tooling and processes to address and mitigate that risk.
Agents, on the other hand, are non-deterministic. We’re never quite sure what they’re going to do, and we can’t understand exactly why they made a decision––however, that unpredictability is what makes them amazing!
When a Waymo is deciding how to drive from A to B, it’s the fact that it doesn’t pick the same path every time that makes it valuable. The Waymo has learned about rush hour and which streets are slower at certain times of the day. It knows what other Waymos are seeing on the roads as they drive. Of the millions of decisions a Waymo has to make on the way to its destination, it is its adaptability as an agent that makes it truly useful and not simply a train on tracks.
In the enterprise, when a marketing agent seeks top prospects for a new campaign, it doesn’t just pull a static list of “VPs” from the CRM–the agent adapts its path in real-time. It may learn that a prospect's recent blog visit is a stronger buying signal than their title, or see a funding announcement and instantly elevate that entire company to the top of its list. This ability to synthesize live signals, not just follow programmed rules, is what makes an agent valuable and not simply a report on rails.
All of this is to say: Agents aren’t chatbots, APIs, non-human identities, or scripts. Agents have much more human-like capabilities and characteristics than other software.
Non-Determinism is a Feature, Not a Bug
If we take a step back and think about agents from a first principles perspective, we can recognize that agents are indeed a unique form of software–they are non-deterministic.
With their newness, we are very unfamiliar with the risk of non-deterministic software. If we try to apply the same approaches to securing agents as we do other software, we can get stuck quickly trying to control agents at the prompt-level. As we all know, prompt-injection and jailbreaking chatbots can be trivial, and for very risk averse and highly regulated organizations, the potential exploitation of non-determinism can be a non-starter.
What if we assumed, however, that agents could never be secured at the prompt and that we could never really know what an LLM was “thinking” or why it “thought” it? How might we secure agents knowing that they are non-deterministic, and we could never stop them from saying or thinking a bad thing?
Well, rather than looking at how we secure regular software, what if we looked at how we secure the other non-deterministic agents in the enterprise: humans.
Humans are quite non-deterministic in their behavior. We can’t easily stop them from thinking something, saying something, or hearing something, but we can stop them from engaging in behavior that might create risk for the organization.
In the enterprise, we celebrate human non-determinism by telling employees to “be creative” and “think outside the box” and “innovate.” It is this non-determinism that gives companies their competitive advantages.
Likewise, we create rules and governance around this non-determinism. Our security and compliance teams help the organization allow the “good” non-deterministic behavior that leads to new revenue, products, and services, while prohibiting “bad” behavior that creates unacceptable risk to the organization.
To mitigate this human risk, we have numerous tools and policies:
Identity and Access Management (IAM/IGA) to control what they can reach.
HR Policies and Codes of Conduct to govern acceptable behavior.
Auditing and Logging (SIEM, UEBA) to create a record of actions for accountability.
Data Loss Prevention (DLP) to protect sensitive information.
Legal contracts and compliance frameworks to enforce the rules.
While these tools, processes, and policies are quite effective at governing human behavior and predictable software, they do not easily extend to agents. These tools today struggle with attribution, tracking agent activity, stopping unwanted agent behavior in real-time, and being able to control a new scale of unpredictable activity.
Just like humans, non-deterministic agents are a huge asset to the enterprise and capable of generating enormous value–we just need to work to govern and control them in ways that empower them to be autonomous while still following the rules.
A New Security Playbook for a New Workforce
We are at the precipice of creating a new playbook for securing this new agent workforce. Like humans, we can’t just focus on preventing agents from thinking or saying something we don’t want them to. Instead, we need to start focusing on how to govern and control the behaviors agents take in our businesses.
Agent capabilities are very complex and modeling all the potential risks of a non-deterministic entity poses unique challenges compared to other software. To help, Ken Huang and others continue to do incredible research around agent threat modeling with his MAESTRO (Multi-Agent Environment, Security, Threat Risk, and Outcome) framework.
In addition to modeling, we need new ways of controlling agents at scale, with these core principles:
Model Agent Risk: Before we onboard a human, we vet them, and we need to do the same for agents.
Deep Observability: We need to see and log every agent action and its entire trajectory
Behavior-based Policies: We need to define and maintain rules to govern agent behavior
Real-time Enforcement: We need to be able to stop agents from engaging in non-compliant behavior in real-time
We are at the beginning of a new transformation of the enterprise as a new autonomous workforce comes online. As security practitioners, we can see the immense potential they bring but likewise the commensurate risk that introducing non-deterministic agents brings to the enterprise.
If we can see that we’ve capably handled the daunting challenge of securing human behavior in the enterprise, we can be optimistic that together we will find the right path forward to put effective guardrails around agents as well.