How Do You Stop Agents from Doing Dangerous Things?

Article 4 of 6 — “Governing AI Agents in the Enterprise: A Practical Architecture Guide”

This article covers preventive governance: the controls that stop an agent from taking an unauthorized, dangerous, or cascading-failure action — before it happens, not after. It covers four questions: policy enforcement, agent identity, execution sandboxing, and reliability engineering.

Q4.1 -- How do we stop an agent from taking a dangerous or non-compliant action in the first place -- not just detect it afterward?

The pattern: Runtime policy enforcement at the action boundary.

Evaluation and telemetry are largely detectivecontrols — they tell you something went wrong, often after it did. For high-consequence actions you need a preventivecontrol: a deterministic policy engine that sits between the agent’s decision and the real-world action, and that can permit, deny, narrow, or escalate for approval before the action executes.

Critically, this enforcement must be rule-based and deterministic — you do not want a second probabilistic model deciding whether the first one is allowed to act. Policy decisions need to be fast (well under a millisecond), auditable, and predictable.

A tool like the Microsoft Agent Governance Toolkit (AGT) provides a policy engine where policies are written declaratively — in YAML, OPA Rego, or Cedar — and evaluated against every tool invocation. The engine integrates with an agent framework’s callback hooks, so every tool call is checked against policy before it executes. A policy can outright deny an action, require human approval for it, rate-limit it, or constrain its arguments. Each decision is recorded in the audit log.

A representative policy: deny any database tool call whose query contains a destructive keyword; require named human approval before any tool that sends data externally if the payload contains personal data; rate-limit the web-search tool per agent instance.

Real-world examples

Q4.2 -- We have a swarm of agents and sub-agents. How do we know which agent did what, and on whose authority?

The pattern: Cryptographic agent identity and inter-agent trust.

In a multi-agent system, “the agent did it” is not an accountable statement — there are many agents, they call one another, and they act on behalf of different users and systems. Each agent needs a verifiable, non-spoofable identity, and inter-agent calls need mutual authentication, so that every action can be attributed to a specific agent acting under a specific authority.

The AGT identity module gives each agent a cryptographic identity — a decentralized identifier signed with strong keys — and supports mutual TLS between agents. An inter-agent trust protocol lets a calling agent and a called agent verify each other before exchanging work. Identities have a lifecycle: they can be issued, suspended, and revoked, including cascade revocation if a parent identity is compromised.

This identity stack is cloud-agnostic — it does not depend on any cloud provider’s directory — so it works in multi-cloud, hybrid, and sovereign deployments.

Real-world examples

Q4.3 -- Our agents execute code and call real systems. When one goes wrong, how do we contain the blast radius?

The pattern: Execution sandboxing — isolate what runs.

A policy engine decides whether an action is allowed. It does not control what damage that action can do once it is running. If an agent can execute code — through a code-interpreter tool, a shell tool, a Python runtime, or a third-party tool that itself runs code — then a malicious or hallucinated payload that the policy permitted can still read files, open network connections, or consume resources it should not. You need execution isolation.

The AGT runtime module provides layered execution sandboxing — graduated isolation rings — for code-executing tools. Code runs with constrained file-system, network, and resource access, so even a policy-permitted action cannot reach beyond its intended boundary.

This module is mandatory from day one for any agent with a code-executing tool. It is not a scale-up option, because policy enforcement alone does not contain runtime behavior.

Real-world examples

Q4.4 -- One stuck or looping agent threatens to take down the whole workflow. How do we stop cascading failure and hit an emergency stop?

The pattern: Reliability engineering for agents — circuit breakers, SLOs, and a kill switch.

In a multi-agent workflow, one failing, looping, or degraded agent can fan out and exhaust budgets, rate limits, and downstream capacity across the entire system. Agents need the same reliability controls mature distributed systems have: circuit breakers that isolate a failing dependency, service-level objectives that quantify acceptable failure, and — non-negotiably — a kill switch that can halt an agent or fleet immediately.

The kill switch is both an engineering control and increasingly a regulatory expectation.

The AGT reliability module provides circuit breakers between agents, SLO and error-budget tracking, chaos-testing support, and an emergency kill switch. Wired into the agent runner and the transitions between sub-agents, it prevents one bad actor in the graph from cascading into a system-wide outage.

For any multi-agent application, this module is core — not optional.

Real-world examples

The distinction that matters

Questions 5 through 8 all live in the governance layer, but they address different threat surfaces:

All four are needed. They are not substitutes for one another. The next article covers the accountability side of governance — how to prove that all of this was in place when a regulator or auditor asks.