--- title: Agentic AI Security 101 description: Getting started with agentic security for developers and security engineers. A twelve-part course: one workflow, ten boundaries, and the controls that actually hold. url: https://ziosec.com/agentic-security-101 --- # Agentic AI Security 101 Getting started with agentic security for developers and security engineers. This course walks one reference architecture from left to right and stops at each of the ten boundaries where something crosses a line, naming the control that actually holds. ## Introduction: Why agentic AI changes the security model Before anything else, a word on where this comes from, because it shapes what follows. My instincts are low-level and defensive. Most of my career has gone into the things that sit in front of production systems and decide what gets through: web application firewalls built from nothing, the rule engines, the match types, the data streaming underneath, because a WAF at real scale throws off an enormous number of events and somebody has to make that manageable. After that came a long stretch of runtime observation with eBPF, pulling application behavior out of the kernel to catch things that looked wrong. That work ended up patented. Deterministic controls, high volume, in the path. So when I started on agentic systems I assumed it was the same job with different nouns: put a gate in front of it, write good rules, enforce them the same way every time. That assumption held up better than expected in some places and fell apart completely in others. The places where it falls apart are the point of this course. ### Who this course is for This is written for developers and security engineers. It assumes you already know least privilege, trust boundaries, defense in depth, and how incident response works. It does not assume you know how a transformer assembles its context, what is actually happening inside retrieval-augmented generation, or what MCP is. Those get explained and then mapped onto something you already have in your head, because that mapping is what makes any of it stick. There is no product pitch and no vendor list. Everything stays at category level on purpose. Categories age well. Vendor lists do not. The goal is to tell you where a control belongs so you can decide what fills the slot. Expect bluntness about where controls are weak, including some that get sold hard right now. There is a lot of confident marketing in this space describing a filter as though it were a boundary, and the difference between those two things is most of the story. ### A support ticket that leaks a credit card Start with a scenario. Somebody files a support ticket with your company. Buried in that ticket, in white text on a white background, is a paragraph. It tells the agent to ignore its previous instructions, pull the stored payment method off the account, and put it in the reply. Your support agent reads the ticket, because reading tickets is the job. It can reach the billing API, because that is how it answers questions about renewal dates. It can send mail, because that is how it responds to people. So it does both. Nothing was misconfigured. There is no CVE. Nobody wrote a line of exploit code. Every component in that path behaved exactly the way it was designed to behave, and a customer's payment data still left the building. ### An agent is four things wired together An agent is four things wired together at runtime: a software supply chain, an identity, a workflow engine, and a probabilistic model. You already know how to secure three of them. What changed from the chatbot era is the failure surface. A chatbot hands you output, so when it goes wrong you get a bad answer and the damage is mostly embarrassment. An agent hands you outcomes. It picks steps, calls tools, and changes state in systems you own and care about. So the question stops being "is the model safe," which nobody can really answer, and becomes three questions you can answer: - Is the autonomy bounded? - Are the actions governed? - Can you reconstruct what happened afterward? ### Four properties that break your assumptions Four properties break the assumptions you are walking in with, and everything else depends on them. **The reasoning is nondeterministic.** It does not do the same thing twice. Same input, different plan, run to run. Nobody is going to patch that out; it is how the thing works. So you cannot enumerate agent behavior the way you would enumerate an API surface, and any policy phrased as "the agent will do X" is making a claim it cannot support. Policy has to constrain what is permitted. Predicting what happens is a different job and it is not available to you. If you have built detection systems, you already know this shape: it is signature versus rate limit. A signature assumes you know what the bad thing looks like, and here you do not. **Instructions and data share one channel.** This is the one that matters most. Every injection class you have ever dealt with has a fix, and every one of those fixes is the same fix wearing different clothes: separate the control plane from the data plane. Parameterized queries for SQL. Output encoding for cross-site scripting. An argv array instead of a shell string for command injection. Different languages, same idea. There is no parameterized prompt. The model takes all of it and flattens it into one stream: your system instructions, the user's question, whatever retrieval dragged in, your tool descriptions, messages from other agents. Every one of those sources gets an equal shot at influencing what the model says next. Nobody implemented that badly. It is how the architecture works, which means prompt injection is not solved today and is not getting solved at the model layer either. Everything that follows exists because of that. **Your trust boundary count went up and nobody sent a memo.** A traditional app has a user, a database, maybe a payment processor. Count them for an agent: a user channel, a set of documents retrieval searches through, a memory store, an external inference provider, a tool runtime, some number of MCP servers, peer agents, and your own enterprise APIs. Every one of those is a place untrusted content gets in or privileged action gets out. There is a shorthand for the first three worth carrying. Simon Willison calls it the lethal trifecta: private data, exposure to untrusted content, and a way to communicate externally. Any agent with all three can be made to leak. Take away any one and the attack usually dies. When you are triaging which agents to worry about first, that is your filter. **Tools mean code execution as a feature.** A file-write tool is arbitrary file write. A shell tool is remote code execution somebody filed a ticket requesting. A browser tool is server-side request forgery with a user agent attached. You are not preventing execution here. You are deciding which execution is allowed and containing whatever happens inside it. ### The reference architecture There is a single map that everything hangs on, and it reads left to right. On the left is everything that feeds the agent: user channels, your document collections (the ones retrieval searches through), memory, and other agents. Assume everything on that side can be influenced by someone who does not work for you. In the middle is the reasoning core: the input gateway, the thing that builds the prompt, the planner, a guard on the request going out, the model provider, and a judge checking what comes back. This is where the unpredictability lives. On the right are the side effects: the policy gate that authorizes actions, the runtime where tools actually execute, and your enterprise APIs. This is where language turns into consequence. Running underneath all three is the evidence layer: telemetry, audit, red teaming, incident response. It spans everything, because you cannot govern what you cannot reconstruct. One thing to be clear about: these control points are architectural. Any gate here might live in application code, in platform middleware, in a gateway, or in features your provider already ships. The point is where the controls go. What you buy to fill each slot is your call. ### Four control modes, not one wrapper This is where most programs go sideways, so name it plainly. People treat guardrails as one wrapper around the model: one thing you buy, one place you configure, done. That wrapper cannot see your retrieval pipeline, cannot see memory writes, cannot see tool arguments, and cannot roll anything back. You need four modes: - **Prevent:** block untrusted intent before planning ever starts. - **Constrain:** limit data, tools, identity, egress, and how much autonomy the thing has. - **Detect:** watch for drift, strange tool chains, and policy bypass. - **Recover:** kill a run, quarantine state, revoke credentials, replay what happened, roll back what changed. That is defense in depth, applied to a workflow instead of a network. ### Three frameworks you actually need You are going to get asked which framework you are aligned to, and the honest answer is that you need three of them. **OWASP ASI (the Top 10 for Agentic Applications).** OWASP published this in December 2025. It is the ASI list, built with over a hundred contributors, and it is the best risk taxonomy available. It tells you what can go wrong. **MITRE ATLAS.** This is the other half, and it has moved fast. It is structured like ATT&CK, so if you already run ATT&CK you can read it on day one. Through late 2025 and into 2026 it added a wave of agent-specific techniques: context and memory poisoning, agent configuration tampering, credential harvesting from tools, and exfiltration through tool invocation. The February update added Publish Poisoned AI Agent Tool and Escape to Host. Where OWASP gives you the risk categories, ATLAS gives you the adversary behaviors, with case studies. If your detection engineers want something to build against, that is the one to hand them. Worth knowing what ATLAS deliberately does not cover: it is about attacks on AI systems, not attackers using AI as a weapon. AI-assisted phishing and reconnaissance sit outside it, and MITRE says so directly. **AIUC-1.** This is the one that reaches you through procurement whether you go looking for it or not. It came out of the Artificial Intelligence Underwriting Company, built with more than a hundred Fortune 500 CISOs: 51 requirements, 130 controls, across six pillars (security, safety, reliability, accountability, data and privacy, and society). People describe it as SOC 2 for AI agents, and that is roughly the right shape. Where ISO 42001 governs your whole organization, AIUC-1 certifies a specific agent deployment. It is the artifact your enterprise customers are going to start asking for, and the one your vendors will wave at you. Two things about it deserve attention: it updates quarterly instead of annually, and the certificate requires technical testing every three months to stay valid. That is a framework built around the assumption that a point-in-time assessment goes stale fast. This quarter's refresh focused on MCP security, third-party risk, and agent identity and permissions, which tells you where the industry thinks the pressure is. Regulation is the piece that, for a lot of teams, actually unlocks budget. The specific dates matter and they moved recently, so the version most people are working from is now wrong. Those get treated separately, on their own. ### Where this leaves you Nothing here asks you to throw away what you know. Least privilege still applies, trust boundaries still apply, input validation and segmentation and audit all still apply. What changes is where those boundaries sit. And one more thing to hold onto. There is a component sitting in the middle of your workflow that behaves differently run to run. It cannot enforce its own rules. And it cannot tell your instructions apart from the ones an attacker hid in a document it read. That is why the work starts at the first boundary: everything the agent reads. --- ## Boundary 1: Intake and the prompt boundary In June 2025, researchers at Aim Security disclosed a vulnerability in Microsoft 365 Copilot called EchoLeak (CVE-2025-32711, CVSS 9.3). An attacker sends an ordinary-looking email with a hidden prompt tucked into an HTML comment or white text on a white background. The user never sees it and never clicks anything. Later, that user asks Copilot something routine. Copilot's retrieval pulls the old email into context, follows the hidden instructions, and exfiltrates data out through a URL. Microsoft patched it server-side and confirmed no exploitation in the wild. It has been called the first zero-click attack on an AI agent, but the reason it matters is not the CVE. It is that the underlying class applies to any assistant doing retrieval over data you did not write. ### Where you are This is the far left of the reference architecture: user and channels coming into your input and API gateway. This boundary stops before anything reaches the planner. There is a distinction here that people collapse constantly and you cannot afford to. - **Direct prompt injection** is the user typing the hostile instruction themselves. That is an insider or a compromised account, and it resembles the authenticated abuse cases you have handled for years. - **Indirect prompt injection** is the instruction arriving inside content your agent reads on somebody's behalf: documents, web pages, email bodies, ticket text, calendar invites, code comments in a repo, messages from peer agents, PDF metadata, HTML comments, image alt text. Indirect is the one that breaks your model of how attacks work, because the attacker never touches your system. They leave content somewhere your agent will eventually read, and then they wait. Structurally that is stored cross-site scripting, the same shape exactly, except the interpreter is a language model and there is no encoding function you can run that neutralizes it. At the token level, the separation between instruction and content does not exist. ### The controls Three controls belong here. **1. Separate user intent from untrusted content.** Pull out what the user actually asked for and hold it as a structured object. Everything else becomes context to reason over, labeled untrusted, never treated as instruction. You cannot do this with prompt phrasing. The code assembling the request has to enforce it, and validation downstream has to back it up, because to a probabilistic system a label is a hint. Sometimes a strong one. Never a guarantee. **2. Detect the carriers.** Jailbreak patterns, hidden instructions, secrets riding in the input. Be honest about what this buys you, because vendors will not be: detection at this layer is probabilistic and evadable. Think of it as a filter. It is useful as signal and useful as a rate limiter against unsophisticated attempts, and it is not the thing that stops somebody who is actually trying. What stops that person is everything downstream, where even a fully successful injection can only reach actions your policy gate was already going to permit. The carriers are worth listing out: - Invisible characters - Look-alike letters (a Cyrillic A standing in for a Latin one) - Encoded text like base64 - White text on a white background - HTML comments - The title text on a markdown link - Formula cells in a spreadsheet - Metadata buried in photos and documents - Code comments, if your agent reads repositories **3. Normalize, redact, route, or block.** Notice that is four options where most implementations offer two. - **Normalization** strips encoding tricks so you are seeing what is actually there. - **Redaction** pulls secrets and regulated data before any of it enters the context window. - **Routing** sends risky input into a constrained workflow with fewer tools instead of refusing outright. - **Blocking** is the last resort, because blocking is the one your users notice and escalate. And keep the original artifact, unmodified. You will want it for incident reconstruction, and the normalized copy is not evidence. ### Walking it through An attacker files a ticket with a hidden instruction block, and your agent retrieves it to summarize. With no intake controls, that instruction lands in planning context carrying the same weight as your system prompt, because it is the same channel. Now add the controls: - The content gets normalized, so the hidden text comes to the surface. - It gets classified, and it scores high for instruction-like patterns sitting in an untrusted field. - It gets tagged. - The run gets routed to a workflow that only holds read-only tools. The injection still lands. It is sitting right there in the context window. It just cannot reach anything worth reaching. That is the lesson underneath everything here: stop trying to make injection impossible, and aim instead at making a successful one worthless. ### Why this boundary matters Everything downstream inherits what happens here. If untrusted text reaches the planner carrying the same weight as your instructions, then every other control you build is compensating for a problem you allowed in at the door. And you will pay for it twice, because the injection that lands here is the one that shows up in memory three weeks later with no trail back. If you do one thing after this, do this one: get your team to show you where user intent gets separated from untrusted content in the code. Not in the prompt. In the code. If the answer is "the system prompt tells it to ignore instructions in documents," you do not have this control, and you now know that in about ninety seconds. ### Questions to take back - Can you produce logs of inputs that were blocked, rewritten, or escalated, with the original artifact sitting next to each one? - Which content sources feed your agents with no human reviewing them first? Write that list out, because it runs longer than people expect. - Has anyone tested indirect injection specifically, or does the red team report only cover direct? - When normalization fails, does it fail open or closed? Go read the code on that one, because nobody documents it. --- ## Boundary 2: Retrieval and data ingestion Someone edits a wiki page. Six weeks go by. An agent retrieves that page while working on something completely unrelated, and it follows the instruction that got left behind in there. There is no log entry anywhere connecting those two events. Hold onto that scenario, because it is the shape of every problem at this boundary. ### Where you are This is the data and RAG sources on the left of the reference architecture, feeding your prompt and context builder. Before anything else, kill an assumption a lot of programs are quietly resting on: internal does not mean trusted. Think about what is actually sitting in those document collections. Content from outside contributors. Scraped web pages. Forwarded mail. Files your customers uploaded. Documents that predate whatever controls you have now. And if you are multi-tenant, possibly another customer's data. Retrieval also has a problem specific to how it works. It is relevance-ranked, and relevance is manipulable. Craft a document to score well against a query somebody cares about, and retrieval pulls it into context. That is the system working correctly. It is SEO pointed at your vector index, and it costs almost nothing to do. On top of that, embedding runs one direction only. Once hostile content is in the index it is in the index, and you cannot grep an embedding for a payload. The design principle underneath all of it: retrieval relevance and instruction authority are different properties, and RAG systems collapse them. A chunk being relevant to the query tells you nothing about whether its contents should influence behavior. ### The controls - **Allowlist your sources and keep provenance on every chunk.** Provenance just means a record of where that piece of text came from: source URI, owner, tenant, sensitivity classification, ingestion timestamp, content hash, trust tier, all of it, on every chunk. Provenance is also what gives you incident response later. Find a poisoned document and the content hash plus lineage tells you every run that ever touched it. Without it, your remediation plan is re-index everything and hope, which is not a plan. - **Filter on tenant and purpose at query time, before the model sees anything.** Filter after retrieval and you are too late. The content already made it into the context window. - **Sanitize at ingestion.** Scan for hidden instructions, secrets, hostile markup, and encoding tricks at the moment content arrives. That is the cheapest place to intervene, and the only place where you still have the original document in front of you. - **Mark retrieved content structurally as reference material, and let the trust tier of the chunk constrain what the run can do.** Low-trust source comes back, high-impact tools go away for that run. ### Walking it through End to end, the chain runs like this. Someone crafts a document and gets it into a source you allow but rated low-trust. A user asks something that matches. The document comes back, injection and all, and lands in context. And because that chunk carries a low trust tier, the write action it was reaching for is not on the table for this run. Retrieval succeeds, the action does not. Same pattern as the previous boundary, and you will see it in every one after this. ### Why this boundary matters Retrieval is where an attacker gets persistence without ever touching your systems. They write a document, they wait, and your own index delivers it. Because it arrived from an approved internal source, every trust signal you have says it is fine. The highest-leverage control here is provenance. Not sanitization, not filtering. Provenance, because it is what turns a discovered poisoning into a bounded incident. Without it, somebody asks which decisions that document influenced and your honest answer is that you do not know and cannot find out. That is the sentence that turns a contained problem into a disclosure conversation. One more thing to keep in perspective: retrieval is per-run, so the damage is bounded to that run. That stops being true the moment the agent writes what it learned into memory, and the injection outlives the session that created it. That is the next boundary. ### Questions to take back - What is in those collections that you did not write yourself, and who has write access? - Given one specific document, can you list every run it influenced? If you cannot, you have no incident response path for a poisoned document at all. - Is tenant filtering happening before retrieval or after? Go read the code on that one. - Does chunk trust tier actually change what the agent can do, or is it metadata that gets written and never read again? --- ## Boundary 3: Memory and state Picture this one. An injection lands on a Monday. The agent summarises that conversation and writes the summary into long-term memory, which is exactly what you built it to do. Thursday comes: different session, different user, and the agent acts on what it stored. The Monday session was logged fine. Nothing in your telemetry connects the two. ### Where you are This is the Memory and State boundary on the reference architecture, the place where a temporary problem turns permanent. It is where you should look first if you are already running agents in production. The key distinction: every other boundary fails within a single run, but memory fails across runs, across users, and across time. Memory is a privileged datastore, and the only thing standing between untrusted input and its write path is a language model. Compare that to something you would never approve: a database where any user can write, with no schema validation, no access control on reads, and a query planner that decides on its own which rows are relevant enough to hand back. Nobody would ship that deliberately. A lot of agent memory implementations amount to exactly it. This is not hypothetical. Johann Rehberger demonstrated a technique called SpAIware, a persistent Copilot backdoor that combines memory tool poisoning with automatic exfiltration on every future session. He pairs it with something called Delayed Tool Invocation: plant instructions during a harmless interaction, trigger them in a later session. That defeats any guardrail that only inspects the current context, which is most of them. Microsoft patched the specific issue, but the primitives transfer to any assistant with memory and document ingestion. The asymmetry is what makes this serious. One successful injection produces one memory write, and that write buys indefinite persistence plus reach into sessions the attacker was never part of. Detection is genuinely hard, because the session where the bad thing happens contains nothing bad. The malicious input arrived days earlier, in a conversation that already closed cleanly. ### The controls **Validate writes before they commit.** Every memory write is a privileged operation, not a side effect of the conversation. That means: - Schema validation and content scanning on the proposed write. - Provenance capture: which session produced this, which inputs fed it, what trust tier were those inputs, and was any of it corroborated. - Treat the agent proposing a write as a proposal, not authorisation. The agent asking to write is not permission to perform one. **Segment the store.** Segment by user, by tenant, by task, and by trust tier. Shared memory across tenants is a cross-tenant data flow with no access control in front of it, and you would never accept that in a database. Segmentation is not only a leak control: it bounds how far a poisoned entry can travel once it exists. **Add expiry, quarantine, and rollback.** Memory quarantine is the pattern worth taking away. Unverified memory is allowed to exist and allowed to inform low-risk behaviour, but it cannot influence a high-impact action until something corroborates it or a human signs off. That is two trust tiers inside the same store. Rollback matters more than people expect, because deletion alone is insufficient: if a poisoned entry fed fifty downstream summaries, deleting the original leaves all fifty in place, still carrying whatever it planted. You need lineage, a trail connecting an entry to everything built on top of it, all the way down. ### Walking it through Follow the chain end to end. An injection arrives, gets summarised, and gets written. Days later a different session reads it and acts. Now put quarantine in place. The write still happens and the entry still exists, but it is flagged low-trust, so any run that reads it loses access to high-impact tools. The injection landed, but it cannot reach anything that matters. And when you find it, rollback removes the entry plus everything derived from it. ### Why this boundary matters Memory deserves budget ahead of most things on this list. Every other boundary fails within a single run. Memory fails across runs, across users, and across time. One injection buys an attacker persistence measured in weeks, and the session where the damage happens contains nothing suspicious at all. Your detection is looking at the wrong day. Start with segmentation. If memory is scoped per tenant and per task, a poisoned entry is contained to one blast radius even before you build validation or quarantine. Segmentation is also the cheapest of the three to retrofit, because it is a data model change instead of a new control plane. ### Questions to take back - Is memory scoped per tenant, or is there one shared pool everything reads from? - What validates a memory write, and can somebody show you a rejected one? If there is no rejection log, there is probably no validation. - Given a poisoned entry, can you find everything derived from it? - Can you roll memory back to a known good state, and has anyone on your team actually tried it? --- ## Boundary 4: The inference request Think about what actually happens on a single call. Your agent assembles a prompt. Inside it: the user's question, four retrieved documents, six memory entries, a set of tool descriptions, and your system instructions. That whole payload gets posted to a third party, over the internet. So here is a question worth taking seriously. What exactly was in it? ### Where you are This is the LLM Request Guard sitting between your agent and the Provider Inference API. Two separate concerns meet here, and they are worth keeping apart, because they need different controls. The first is that this is an egress event. Every inference call ships context to an external system, possibly across a jurisdiction. The data classification question is not what the user typed. It is what the entire assembled context contained, which your code built dynamically about four milliseconds ago from four different sources. That is data loss prevention against a payload that did not exist when you wrote the policy. The second is that this is your last deterministic checkpoint. After this call returns you are reasoning about probabilistic output. Anything you want to be certain of needs to be established before you hit send. ### The controls Start with how you build the request. Use structured fields. Do not glue strings together. A string-concatenated prompt is the sprintf of agentic systems. Conceptually the fix is parameterised queries, with one important caveat: the model does not enforce the separation the way a SQL parser does. What you get is typed assembly, auditability, and the ability to reason about what is in the payload. What you do not get is a guarantee. That is precisely why response validation exists downstream. The envelope wants distinct fields: - **System instructions**, pinned and versioned and never assembled from anything a user can influence. - **User intent**, extracted and structured. - **Untrusted context**, labelled, carrying its provenance. - **Tool descriptions**, pinned and versioned ahead of time. - **Your constraints**. Then schema, policy tags, and least context. Least context is the principle worth naming out loud, because it inverts how people think about prompts. Every additional token in that window is additional attack surface and additional egress. If a retrieved chunk is not needed for this specific step, it should not be in this specific request. Context is not free and it is not neutral. Policy tags ride along with the content (sensitivity, tenant, trust tier) and those drive redaction now and action authorisation later. Keep safe system instructions separate from untrusted user content. Pin them, version them, and never build them from user-influenceable input. Tool descriptions deserve special attention, because in MCP-style setups they arrive from an external server, which makes them a live attack surface pointed straight at a trusted position in your context window. Then the provider boundary itself, where architects have more control than they usually use: - Set data residency and retention. - Allowlist model IDs and pin versions, so a silent provider-side model swap does not quietly change your behaviour underneath you. - Set rate limits and cost budgets per agent and per session, because a runaway loop is simultaneously a spend incident and a denial of service against your own quota. - Push for zero-retention or in-tenant deployment wherever classification demands it. - Make your egress logging capture the request envelope itself. Knowing a call happened tells you very little. The one thing worth doing first is the cheapest thing on the list. Log the request envelope. Not that a call happened, the actual assembled payload. It takes an afternoon, it answers the data classification question permanently, and it is the artifact you will want the first time somebody asks what left the building. ### Why this boundary matters Here is the framing that reaches your legal and compliance people. This is a continuous, automated export of internal data to a third party. The payload gets assembled at runtime, by code. No human reviews what goes into it. Say it in those words and the conversation about controls gets much shorter. ### Questions to take back - Can you produce the exact payload sent for any given run, on demand? - What classification of data is permitted to enter a context window bound for an external provider, and what actually enforces that rather than documenting it? - Are model IDs pinned, or are you inheriting provider defaults? - Is there a per-agent cost ceiling, with something sensible happening when it is hit? This is the last thing you fully control before the model speaks. So now you treat what comes back as untrusted. --- ## Boundary 5: Validating the model response Imagine you are looking at a tool call the model just produced. It is well-formed JSON against a valid schema. The arguments look reasonable. It is calling a tool the agent legitimately has and uses every day. Every check you wrote passes. And it is entirely the wrong action. Forty turns ago a retrieved document told the agent to take it, and nothing since has questioned that. ### Where you are This boundary sits at the response judge and parser, the point where the model has emitted output and your application decides what, if anything, to do with it. The principle is one you already hold in a different context: model output is untrusted input to your application. You have never trusted the client, and this is the same rule. The reflex is usually missing because the model feels like part of your system, when really it is a third party that just handed you a payload. What is genuinely new is the failure mode: plausibility without correctness. Traditional validation catches malformed input, and this input is beautifully formed. Schema passes. Types pass. The action is wrong anyway, and no amount of structural validation will tell you that. Two named failure modes live here. OWASP calls the human side Human-Agent Trust Exploitation (ASI09): agents projecting unwarranted confidence and offering plausible but wrong reasoning that people accept because it reads well. It is automation bias turned into an attack surface, and it matters again when you reach human approval. The other is intent drift, the version that catches teams out. Over a long run the proposed action gradually stops corresponding to what the user actually asked for. Nothing in a schema detects that. To catch it you have to compare the proposed action against the original intent, which is why intent has to be captured as structured data all the way back at intake. Leave it implicit in a conversation transcript and you have nothing to compare against. ### The controls Treating output as untrusted until validated has one concrete implication that matters more than the principle: - **The model never invokes a tool. It emits a proposal.** Your orchestration code parses that proposal, validates it, and decides. That indirection is the entire control. If your framework lets the model call functions directly, you have no gate here, and everything in the next boundary has nowhere to run. - **Strict schemas plus value-level policy.** This goes past type checking. Is this identifier inside the user's tenant? Is this path under the permitted root? Is this recipient on an allowed domain? Is this amount within limits? The schema tells you the field is a string. Policy tells you which strings. - **A second policy or judge step, chosen with eyes open.** A rules engine is deterministic, auditable, and testable, and it cannot catch anything you did not anticipate. A judge model catches semantic problems like intent drift, and is itself probabilistic and injectable. A judge that shares context with the primary model can be compromised by the very same payload, a detail people miss. The shape that works: deterministic rules for anything you can express as a rule. A judge model for semantic checks only. That judge runs on the minimum context it needs. And the judge can never approve something the rules engine denied. It can veto. It cannot override. Then fail closed. Invalid JSON, an ambiguous action, an unexpected tool, an intent mismatch, or a judge that times out should all stop the run or escalate to a human. One warning. In production everything pushes toward failing open, because a fail-closed agent is an agent that stops being useful at three in the afternoon and generates a ticket. Resist it. An agent that halts is recoverable. An agent that proceeds on unvalidated output is an incident you will reconstruct later without the evidence you need. ### Why this boundary matters This is the boundary people skip, because everything looks fine here. The output is well-formed, the schema validates, and nothing errors. Teams see green and move on. Meanwhile this is your last chance to catch a plan that has been quietly wrong for thirty turns. The single highest-value change is architectural, and you can find out in one conversation whether you have it. Ask whether the model can invoke a tool directly. If it can, you have no gate here, and the policy gate that comes next has nowhere to live. Everything else in this boundary is refinement on top of that one answer. ### Questions to take back - Can your model invoke a tool directly, or does everything route through parse, validate, decide? - What happens on malformed output, specifically: retry, fail open, or fail closed? - Does anything at all compare the proposed action against the original user intent? - If you run a judge model, does it share context with the primary model, and has anyone tried injecting through it? The proposal survived validation. That leaves the question of who decides whether it is authorized, and why that decision has to live somewhere the model cannot reach. --- ## Boundary 6: The tool and action policy gate The agent proposes deleting forty records. The call is well formed. The schema is valid. It uses a tool the agent touches every day without incident. Every upstream control did its job and passed the proposal through. The only question left is whether the action is allowed, and the only thing that can answer it is something sitting outside the model. This is the pivot point. Everything before this gate is text. Everything after it is a side effect on a real system. ### Where you are On the reference architecture, this is the tool and action policy gate: the last checkpoint before a proposal becomes an effect on production. The architectural claim is simple to state and easy to get wrong. The authorization decision must live outside the model and outside the planner. Outside the model is obvious. Outside the planner is the part people skip, and it matters because the planner is model-driven. It inherits every property discussed so far: nondeterminism and susceptibility to injection. A compromised planner asking a compromised planner for permission is theatre. You already have a name for the right pattern. This is a policy decision point and policy enforcement point split. The model proposes. The decision point disposes. The enforcement point is the only path to the tool runtime. ### The controls Authorize every individual call, and that means every call, against user, task, and risk context. The decision takes many inputs: - Which agent is acting, and on whose behalf - The resource and the action - The purpose and the tenant - How sensitive the context was that produced this proposal - The tool version - What is left of the rate and cost budget - Where you are in the workflow One input matters more than the rest. The trust tier of the context that produced the proposal feeds the authorization decision, and that is where the trust tagging from retrieval and memory finally pays off. Low-trust retrieval enters the context, and the high-impact action is denied automatically, without anyone writing a rule about that specific document. **Approval for high-impact actions.** Define high-impact by properties, not by tool name. Tool names will mislead you. The properties are irreversibility, blast radius, data sensitivity, and external visibility. A reversible internal write is not the same as an irreversible external send, even when both go through the same API. **Human-in-the-loop.** Show a dry run or a diff. An approver reading the model's own natural-language explanation of what it is about to do is reading text generated by the thing under review, which is not review. Show the concrete change: these forty record IDs, this recipient list, this file path, this dollar amount. **Rate limiting.** Agents fail in loops. One bad plan executed two thousand times is a categorically different incident from one bad action. Cap side effects per session, and per agent per hour, independent of whether any individual call is authorized. **Bind the decision to a policy engine.** Policy as code, versioned, tested, reviewed, and producing a decision record with its inputs. That is what turns "we have controls" from an assertion into something demonstrable. **Least agency.** This is the cheapest control here and the most skipped. It is least privilege applied to capability surface. Do not expose a general CRM tool when read-only opportunity lookup is what the workflow needs. Do not expose send and delete when summarize is sufficient. Prefer narrow, purpose-built tools to broad general ones, because a narrow tool gives you a narrow policy surface to reason about. The single most effective control available to you is a capability the agent simply does not have. ### Walking it through Follow the worst case all the way in. Injection got through intake. It survived retrieval. It poisoned memory. It produced a plausible, well-formed action using a trusted tool. Every upstream boundary failed. The policy gate still says no. It never asked the model's opinion. The decision was made by code the model cannot reach, and it read the trust tier of the context that produced the proposal, saw low-trust retrieval feeding a high-impact action, and denied the call before it touched the tool runtime. Nobody had to write a rule about that specific document. ### Why this boundary matters This gate is worth more than everything upstream combined, because it is the only control that works even when every other control has failed. That property is rare, and you should spend accordingly. If you take one action from this course into a design review, make it this. Ask where the authorization decision lives, and keep asking until somebody shows you code the model cannot reach. "The agent checks its permissions" is not an answer. That is the agent asking itself. And if you want a quick win alongside it, cut capability. Every tool you remove is a control you never have to build, test, monitor, or explain to an auditor. ### Questions to take back - Is the authorization decision made by code the model cannot influence? - Can you produce a policy decision log, with inputs and outcome, for an arbitrary run from last week? - What defines high-impact in your environment, and who decided that? - Do your approvers see a diff or a narrative? - What is the per-session cap on side effects? --- ## Boundary 7: Agent identity and secrets Imagine you have been handed an audit log. Every action your agent has ever taken appears under the same service account. Ten thousand actions, one identity, no link back to the user whose request triggered any of them. Now try to reconstruct an incident from that. Most of the exposure here comes from one pattern. Call it the demo-to-production path. The prototype runs with the builder's personal token, or with a broad service account. That is what makes the demo work on a Tuesday afternoon. Then it ships, and nothing ever forces the identity model to change, because everything functions perfectly. ### Where you are This is the Identity and Secrets control in the evidence layer of the reference architecture. The key distinction is that identity here is not an operational concern, it is a security control. Your agent is a high-privilege actor taking instructions from lower-privilege sources: the classic confused deputy. Every prompt injection covered in this course is an attempt to make that actor use its privileges on the attacker's behalf. So identity determines what a successful injection can actually reach. When the demo-to-production path goes unfixed, two things follow: - **Attribution collapses.** Every action appears under one identity, and you cannot answer who did this or which user's request caused it. - **Privilege inflates.** That credential's scope becomes the union of everything any workflow ever needed, which is dramatically more than any single run requires. Multi-agent makes it sharper. If agent A calls agent B, and B executes under its own broad service identity, then compromising a low-privilege A hands you B's privileges. That is privilege escalation through delegation, and it requires no exploit at all. ### The controls - **Scoped, short-lived credentials instead of broad tokens.** Per agent, and ideally per session. The token should be bound to the user intent it was issued for, the specific audience or resource, and a duration that matches the task. Think minutes. A leaked token that already expired is a much smaller conversation. - **Separate agent identity from user identity, and log the delegation between them.** The agent is a workload identity with its own lifecycle, owner, and rotation schedule. The user is the principal on whose behalf it acts. Your audit trail needs both, plus the full delegation chain whenever agents call agents. - **Model agents as managed non-human identities.** Apply the same discipline you already use for service accounts: inventoried, owned, reviewed, rotated, revocable. Most organizations have that discipline already and simply have not extended it to agents. - **Re-check authorization at action time.** Permissions change, sessions run long, and the context that justified a permission at minute one may not hold at minute forty. - **Keep secrets out of prompts and memory entirely.** Treat this one as non-negotiable. A secret that enters a context window has left your boundary. It is in the provider's request logs, it may surface in model output, and it may get written into memory. Use a broker instead: the agent references a credential handle, the tool runtime resolves it at execution time, and the plaintext never touches model context. ### Walking it through There is one rule for cross-agent delegation, and it is worth stating directly: delegation must not inherit privilege without revalidation. Walk the multi-agent case through with the rule in place. A prompt injection lands on agent A and tries to make A drive agent B into a sensitive action. A calls B. B does not authorize against A's identity. It authorizes against the original user's permissions and the stated purpose of the request. If that user never had access to the target, the action is denied, and the injection reaches nothing, even though it successfully compromised A. Skip that revalidation and the picture inverts: every agent in your mesh effectively holds the union of all privileges in the mesh, which is worth checking if you are running more than three of them. ### Why this boundary matters Identity work pays off twice. It bounds what a successful attack can reach, and it is the thing that makes an incident investigable at all. Without per-user attribution you cannot scope a breach, you cannot notify accurately, and you cannot answer the first question a regulator asks, which is whose data was involved. Start with the inventory, because you probably cannot fix what you cannot list. Ask for every agent identity, its owner, its scope, and its credential lifetime. That exercise alone usually surfaces two or three agents running on somebody's personal token, and finding those is worth more than any control you would design this quarter. ### Questions to take back - Do you have an inventory of agent identities with named owners? - What is the credential lifetime, and what is it scoped to? - Can you trace an action back to the user request that caused it? - Does any credential enter a prompt or get written to memory? - When agent A calls agent B, whose permissions apply? --- ## Boundary 8: Execution, sandboxing, and egress One more scenario. The agent generates a shell command and runs it. The command is syntactically valid. The tool is authorized. The identity is correct and properly scoped. And the command is destructive. Every control upstream did precisely what you built it to do. ### Where you are This boundary is the tool runtime sandbox and the egress path: the place where the agent actually executes generated code and where that code can reach the network. Here is the framing for a security audience. You are running attacker-influenceable generated code, deliberately, as a product feature. Traditional remote code execution is a vulnerability you eliminate. Here, code execution is the requirement. The question stops being how to prevent it and becomes how to contain it, and that reframing changes your entire control set from prevent-execution to isolate, constrain, budget, and observe. Two directions of untrusted content both matter. Inbound, the command or code the agent generates is shaped by every boundary upstream of this one. Outbound, the tool's own output returns into the model's context, which makes a compromised tool or a hostile API response another injection vector, this one arriving from inside your trust boundary. That second direction is badly underappreciated. Tool results are untrusted input and deserve the same handling as anything you retrieved. There is a benchmark called MCPTox that tests exactly this, and the headline finding should end any argument about whether the model will catch it. Even the strongest commercial agents failed roughly half of the prompt-injection-via-tool-output scenarios. Not edge cases. Half. The model followed the malicious instruction more often than it ignored it. ### The controls Start with execution itself. No direct production execution where risk is high: use staging, a dry run, or approval. For anything with a plan-then-apply model, and that covers infrastructure, database migrations, and bulk updates, generate the plan, show the diff, apply on approval. You already run this workflow for humans. Extend it. Then sandbox the filesystem, network, process, and browser: - A dedicated working directory that gets destroyed when the run ends. - Non-root execution with no privilege escalation path and dropped capabilities. - Minimal filesystem visibility, meaning the agent cannot see the host filesystem, another tenant's data, or its own orchestration code and configuration. - Controlled package installation from an internal mirror rather than arbitrary registry pulls at runtime, because an agent installing a package is an agent executing some maintainer's install script. - Process isolation with real resource limits on CPU, memory, process count, and wall clock. - Browser tools in their own sandbox, treated as fully hostile, with no access to host credentials or session state. For anyone with an infrastructure background, container isolation, seccomp profiles, and namespace restrictions all apply directly. You already know how to do this. It is the same discipline pointed at a new kind of workload. Constraining egress is where prompt injection converts into data theft, and it is the control most commonly missing entirely: - Default-deny egress, with a per-tool allowlist of destinations. - Route it through a proxy that logs destination, volume, and content classification. - Watch the encoding channels, because that is where this gets creative: DNS lookups, parameters on a URL, image sources that render in output, webhook callbacks, and markdown links in agent output that a user's browser fetches automatically. That last one deserves a beat, because rendered agent output is itself an egress channel. A markdown image whose URL encodes your data exfiltrates the moment it renders. No tool call involved. Nothing in your egress logs. The fetch happens in the user's browser, from the user's network, and your controls never see it. Finally, budgets and a kill switch. Wall-clock and step-count limits per run, token and cost budgets per session and per agent. The kill switch has to actually terminate in-flight execution rather than just refusing new requests, and credential revocation has to take effect for tokens already issued. Test all of it. A kill switch nobody has exercised is a design document, not a control, and you will find out which one you have at the worst possible moment. ### Why this boundary matters This one gets underfunded because teams treat the sandbox as an infrastructure detail and the egress rules as somebody else's job. But this is where a prompt injection stops being a data problem and becomes a data breach. Everything upstream determines whether the attacker can issue an instruction. This boundary determines whether that instruction can reach the internet. Default-deny egress is the highest-leverage control here, and it is the one most often missing entirely. It is also unusually satisfying, because you already know how to do it. It is the same allowlisting discipline you would apply to any workload that should not be talking to arbitrary hosts. ### Questions to take back - Can a tool reach the host filesystem, your orchestrator's config, or another tenant's data? - Is egress default-deny, and can somebody show you the allowlist? - Are tool outputs treated as untrusted input on the way back in? - Can you stop a running agent mid-execution, and when did anyone last try? - What is the maximum spend a single runaway session can incur before something stops it? --- ## Boundary 9: Inter-agent and MCP trust > September 2025. The maintainer of the official Postmark MCP server pushed an update that silently BCC'd every email the agent sent to an attacker-controlled address. Not a clever exploit. The maintainer just did it, and the backdoor stayed live across multiple versions before Koi Security found it. That is the one to sit with, because package signing would not have caught it. The signature was valid. The maintainer was the attacker. ### Where you are On the reference architecture this is the peer agents, MCP, and A2A boundary. Multi-agent systems reintroduce every boundary walked so far at every hop, and add a supply chain on top. MCP is a protocol for exposing tools and resources to agents: the server declares what it offers, and the client's model consumes those declarations as context. A2A is agent-to-agent communication, where agents advertise capabilities and delegate work between themselves. The distinction that matters: tool descriptions and capability advertisements are untrusted content, and they land directly in a trusted position inside your context window. A hostile MCP server does not need to exploit anything. It writes a tool description containing instructions, and your orchestrator hands that to the model as a system-level statement of fact. You already know this shape. It is a dependency you pull at runtime, whose contents you do not review, which executes with your privileges, and which can change between calls without a version bump. Typosquatting, dependency confusion, and maintainer compromise all have direct analogues in a tool registry. ### The controls **Authenticate agents, tools, and registries with real identity.** Mutual TLS or equivalent. Signed tool descriptors and agent cards with verified publishers. Discovery through an internal registry you have verified, not open registration, and not letting the agent find arbitrary endpoints at runtime. **Validate schemas, versions, and capability claims before use.** Pin tool descriptions the way you pin dependencies. Detect and reject a descriptor that changes between calls, because a server quietly altering its tool description mid-session is performing a rug pull. And when a server tells you what it can do, treat that as a claim you need to verify. **Apply least privilege across agent-to-agent exchanges.** This is the identity rule restated at protocol level. Delegation does not inherit privilege. Every hop re-authorises against the original user's permissions and the stated purpose. Bound your delegation depth too, because a chain long enough to lose the original intent is a chain long enough to lose the original authorisation. **Treat peer messages as untrusted content.** Mutual authentication proves who sent a message. It says nothing whatsoever about whether that message is safe. An authenticated peer that has itself been compromised by injection sends you perfectly authenticated hostile content. So run the same intake controls you opened the architecture with on inbound agent messages: the same normalisation, the same classification, the same trust tagging. Being on your internal mesh is not a trust statement about the message. ### Walking it through The attack classes here have names now, and they are worth learning: - **Tool poisoning:** hidden instructions in the tool description itself. - **Rug pull:** a server ships a clean description at install, gets approved, then mutates it on a later connect, which most clients never re-prompt on. - **Tool shadowing and cross-server contamination:** one connected server manipulates the agent into abusing a different server's tools. Your blast radius is the union of every connected server's scope. Invariant Labs demonstrated the WhatsApp version of this: a malicious server sharing agent context with a legitimate one, quietly exporting a user's entire message history. No user error, no network exploit. ### Why this boundary matters Cascading failure is the multi-agent version of blast radius. A compromised or malfunctioning agent in a mesh propagates outward. Its output becomes another agent's input, which becomes a third agent's input. Without per-hop validation and a depth limit, one compromise walks the entire graph. You want circuit breakers and rate limits between agents, all the way through the mesh. This is likely to be your fastest-growing exposure. Connecting a new MCP server takes about a minute and usually involves no review at all. Agents routinely run five to twenty of them. That is a runtime dependency graph nobody is tracking, expanding every week, with your credentials behind it. Inventory first, again, and for the same reason. You cannot pin, verify, or scope servers you do not know are connected. Ask who can add one, and whether anything stops them. In most organisations the honest answer today is any developer, and nothing. ### Questions to take back - What is your inventory of MCP servers and peer agents, and who owns each one? - Are tool descriptions pinned and version locked? - What happens if a tool description changes between calls, and would you even notice? - Do inbound peer messages get the same intake treatment as user input? - How deep can a delegation chain run before something stops it? --- ## Boundary 10: Observability, red teaming, and response This is the most common scenario of all. An agent did something wrong three weeks ago. You have the final output. You have the tool call. What you do not have: the retrieved chunks, the memory entries it read, the prompt envelope you sent, the policy decision that let it through. So you cannot determine why it happened, which means you cannot determine whether it is going to happen again. ### Where you are This boundary is the evidence layer, and it spans the entire reference architecture rather than sitting at one point on it. Every other boundary is a claim until this one turns it into a fact. Traditional application logging captures request, response, and error, and that is insufficient here for a specific reason: the failure you care about is a decision. Nothing threw. Every component returned success. The agent chose wrong, and the reason for that choice is distributed across roughly ten upstream inputs. There is a second difference that makes this harder than it looks. Nondeterminism defeats reproduction. You cannot re-run the request and watch it happen again. If you did not capture the decision chain while it was happening, it is gone, and no amount of investigation recovers it. ### The controls Log the complete decision chain. Completeness is the whole point. That means capturing: - User intent as extracted - Input screening decisions - Retrieved chunk identifiers, with provenance and trust tier - Memory entries read - The full prompt envelope as sent - The model response as received - Parser and validator results - The judge decision - Policy engine inputs and the decision it produced - Tool arguments as executed - Human approval, if there was one - Tool output - Resource changes **Sampling strategy.** This gets expensive at scale and somebody will push back. The strategy that works: full capture for high-impact actions and for every policy denial, sampled capture for routine reads, with retention driven by the risk tier of the workflow. One uniform policy across everything will cost a fortune and still miss things. **Tamper-evidence.** If these logs are evidence, they need integrity guarantees: append-only storage, hash chaining, or writing to a system the agent's own identity cannot modify. An agent that can edit its own audit trail does not have an audit trail. **Lineage and replay.** You want correlation identifiers spanning the whole chain so you can move in both directions: from an action back to the inputs that caused it, and from a newly discovered poisoned input forward to every run it ever touched. That bidirectional traversal is your incident response capability. Without it you are re-indexing everything and hoping. Then replay real captured runs in a safe twin environment, tools stubbed and side effects disabled. That is how you validate a fix against traffic that actually happened, instead of against synthetic tests you wrote afterwards. ### Red teaming, continuously Measuring coverage, drift, and remediation is the piece most teams skip. Coverage is what you tested. Effectiveness is what held under test. A red team report saying "we tested prompt injection" is not evidence of anything. Evidence is which agent paths, which tools, and which data sources were exercised, and which controls blocked which attempts. Make it continuous, and here is the specific reason rather than a general preference: your agents change constantly. New tools. New documents in the index. Updated system prompts. And model updates from your provider that you did not initiate and may not have been told about. A point-in-time assessment expires quickly, and it can expire without any change on your side at all. Scope the red team across every boundary in this course: direct and indirect injection, poisoning your document collections, poisoning memory, tool misuse and argument tampering, identity and delegation abuse, sandbox escape and data egress, MCP and peer-agent compromise, and cascading multi-agent failure. Then add the one that is hardest to test for. OWASP's tenth item is Rogue Agents: an agent that drifts from its intended behaviour and keeps acting with authority it was legitimately given. Goal drift from an injection weeks earlier. Reward hacking, where it optimises the metric you set instead of the outcome you wanted. The example that sticks is a cost-optimisation agent deleting production backups, because that genuinely does reduce spend. It is the insider threat problem, except the insider is authorised, trusted, and never files a suspicious login. ### When something goes wrong Response needs four capabilities: - A tested kill switch that terminates execution already in flight - Credential revocation that reaches tokens you have already issued - Memory quarantine and rollback that follows derived content, all the way down from the original entry - Document collection cleanup using content hashes, so you can find every affected run And playbooks specific to agent incidents. "Isolate the host" does not cover "the agent wrote poisoned entries into shared memory forty sessions ago," and that is the incident you are going to get. ### Why this boundary matters Every control in this course is a claim until this one makes it a fact. You cannot demonstrate that a policy gate works without the decision log. You cannot prove memory rollback works without lineage. This is what converts your security story from something you assert into something you can hand to somebody. The practical urgency: nondeterminism means you cannot go back and reproduce a bad run. If you did not capture it while it happened, the evidence does not exist, and no amount of investigation recovers it. Every day you run agents without decision-chain logging is a day of incidents you will never be able to explain. So capture the chain for high-impact actions and every policy denial first, before you worry about sampling the rest. Those are the runs somebody will eventually ask you about. ### Questions to take back - Can you reconstruct a full decision chain for an arbitrary run from last month? - Can the agent's identity modify its own audit records? - Has anyone replayed a real run in a twin environment? - What is your red team cadence, and does it track deployment changes or a calendar? - Which of your response capabilities have actually been exercised rather than documented? --- ## Conclusion: Putting it together One workflow, ten boundaries. That is the frame worth leaving with, so walk the map one last time and trace each gate in sequence. Intake, where untrusted content enters. Retrieval, where it enters a second way, wearing the authority of your own index. Memory, where it stops being temporary. The inference request, where your data crosses the boundary. Response validation, where output becomes a proposed action. The policy gate, where a proposal becomes an authorization decision. Identity, which determines what that action can reach. Execution and sandboxing, which bounds what it can damage. Inter-agent trust, where all of it repeats at every hop. And evidence, underneath everything, because none of the rest is provable without it. ### The four control modes across the boundaries Now that all ten boundaries have been covered in detail, the four control modes map cleanly onto every one of them: - **Prevent** shows up at intake and retrieval. - **Constrain** is identity, least agency, the sandbox, and egress. - **Detect** is your telemetry and the anomalous tool chains you are watching for. - **Recover** is the kill switch, revocation, memory rollback, and replay. If you are auditing a program and something feels thin, it is usually recover. Teams build prevent and constrain because those are the ones that feel like security. Detect and recover are what you need at two in the morning. ### Three threads, one synthesis Three threads run through all of this, and they are worth restating together. The model is probabilistic, so your controls have to be deterministic and external to it. Anything implemented as a prompt is a suggestion. Anything implemented as policy outside the model is a control. Untrusted content and trusted instruction share one channel, which means no control at the model layer is ever sufficient on its own. There is no parameterized prompt coming to save this. And language becomes action at the tool boundary, which is why the authorization decision has to sit outside both the model and the planner. Put those together and you get the synthesis. In traditional systems, layered controls are prudent. In agentic systems they are structural, because no single layer here can be made reliable on its own. Every individual control covered here is probabilistic or bypassable to some degree. The composition is what holds. ### The OWASP ASI crosswalk Every boundary maps onto OWASP's Top 10 for Agentic Applications, so if somebody asks which standard this aligns to, here is the answer: - **Goal hijack (ASI01)** is the intake, retrieval, and memory boundaries, because that is where injection gets in. **Memory and context poisoning (ASI06)** sits there too. - **Tool misuse (ASI02)** is response validation and the policy gate. - **Identity and privilege abuse (ASI03)** is the identity boundary. - **Unexpected code execution (ASI05)** is the sandbox. - **Supply chain compromise (ASI04)**, plus **insecure inter-agent communication (ASI07)** and **cascading failures (ASI08)**, are all the MCP and peer-agent boundary. - **Human-agent trust exploitation (ASI09)** is how you design approvals. - **Rogue agents (ASI10)** is what all that monitoring exists to catch. Three of those, ASI07, ASI08, and ASI10, are entirely new classes with no equivalent in the LLM Top 10. They only exist because agents talk to each other and act on their own. That is a useful thing to be able to say out loud in a budget conversation. ### The regulatory dates changed: check yours Check these against whatever your compliance team is working from. They changed in the last few months, and a lot of plans are still built on the old numbers. The EU AI Act originally set the second of August 2026 as the date high-risk obligations kicked in. That moved. The Digital Omnibus was adopted by Parliament in June, and by the Council at the end of June. It defers standalone high-risk obligations under Annex III to December 2027. High-risk AI embedded in regulated products moves to August 2028. Here is the part that gets lost in the headlines saying the EU delayed the AI Act. Some of it moved sixteen months. Some of it did not move at all: - The **Article 50** transparency obligations still apply from the second of August 2026, which is days away. - The watermarking requirement for systems already on the market, and the new prohibitions, land in December 2026. - If you are building on a large frontier model, **Article 55** has required providers of models carrying systemic risk to conduct and document adversarial testing since August 2025. That one has been live for a while. The practical read is this. The deferral bought most of you runway, and it did not change a single underlying obligation. There is also a grandfathering rule worth understanding: systems placed on the market before those dates avoid the high-risk requirements, but that protection resets the moment you substantially modify the system. For an agentic deployment that you are changing every sprint, do not lean on it. Round it out with NIST AI RMF for risk management structure, ISO 42001 if you need certifiable governance, and AIUC-1 for agent-specific certification. None of these replace each other. MITRE ATLAS and OWASP describe the attacks. NIST and ISO give you the control and governance language. The AI Act tells you what is mandatory and when. AIUC-1 is what you hand a customer who asks whether your agent is safe to buy. ### One honest caveat on AIUC-1 You should hear this from me and not find out later. AIUC accredits its own auditors, which is different from how ISO or SOC 2 work. The standard does not define what counts as an AI agent, so the vendor decides what is in scope. And the vendor picks the auditor. So when somebody shows you a badge, ask for the report. Ask what product scope was certified, which version of the standard, which controls were reviewed, and what the results were. A certificate is a starting point for a conversation. ### The three questions to take away If you take one thing away, take these three questions, because they work in any design review regardless of what the agent does: - **What can the agent read?** That covers intake, retrieval, memory, and the provider boundary. - **What can the agent do?** Response validation, the policy gate, identity, the sandbox, and inter-agent trust. - **How do you know, stop, and roll back?** Telemetry, lineage, replay, red teaming, and a kill switch. Support agent, coding agent, financial operations agent, security operations agent. Same three questions every time. ### The Monday next step One practical suggestion. Pick one agent you already have in production. Walk the ten boundaries against it and write down, for each one, whether a control exists, whether anyone has tested it, and whether you could produce evidence if somebody asked. You will finish that exercise with a gap list, and every gap on it is a concrete remediation item you can assign to somebody. Start with whichever agent holds the most permissions. Traffic volume matters less than reach. ### Last thing Agentic AI is governable. It just is not governable the way a traditional application is. There is one component sitting in the middle of your workflow that behaves differently run to run. It cannot enforce its own rules. And it cannot tell your instructions apart from the ones an attacker hid in a document it read. That is not an argument against deploying agents. It is a design constraint, and every control covered here follows from it. Bound the autonomy. Scope the permissions. Put the authorization decision somewhere the model cannot reach. Contain the execution. And instrument all of it, so that when something does go wrong you can prove what happened. You cannot protect against risks you do not know about. Now you have got the map.