For a while, the argument over AI coding tools was mostly an argument over text.

Could the model write a decent function? Could it explain the stack trace? Could it generate tests that did more than politely compile? Could it summarize a mess of documentation without inventing a second, more confident mess?

Annoying questions, useful questions, but still questions about output.

The agent problem is different. It begins when the model stops producing suggestions and starts operating machinery.

The old coding assistant could be wrong in the usual way: bad suggestion, fake API, confident explanation of a function it had clearly never met. Annoying, but mostly contained.

The agent version is different. It has somewhere to go.

It can reach into the repo, poke around the ticket queue, open a pull request, call the cloud API, ask the database a question, kick-off a CI job, and generally move from “here is some text” to “I have altered the state of the world, please enjoy this diff.”

That is where the trust boundary lives: at the point where generated language becomes system behavior.

A bad answer gets corrected. A bad action leaves fingerprints.

FAULT_01: MANUAL SECURITY POSTURING // PROMPTS_ARE_NOT_FENCES

The industry’s first answer to agent risk was, naturally, to ask the model to behave.

Do not leak secrets. Do not follow malicious instructions. Do not overwrite files. Do not trust untrusted input. Be responsible. Be aligned. Please do not turn the repo into soup.

Fine. Instructions matter, but a prompt is not an access-control layer. It is a request placed in front of something that may already have permission to do the wrong thing.

That is why Microsoft’s FIDES work is worth attention. It treats prompt injection as an information-flow problem, not a manners problem. Content carries labels: trusted, untrusted, public, private. Those labels move through the agent’s work. Policy gets enforced before a sensitive tool runs, not after the model writes an apology with bullet points.

That is the useful shape. A defensive prompt says, “Please do not put secrets in the issue comment.” A policy layer says, “This untrusted issue body cannot reach that tool.”

One is architecture, whereas the other is a Post-it note with delusions of grandeur.

FAULT_02: UNBOUNDED EXPOSURE // EVERY_TOOL_IS_A_DOOR

MCP lets agents out of the chat box and into the wiring closet. This is convenient right up until the tool call does exactly what it was allowed to do.

External tools mean the agent can query docs, touch APIs, read files, call services, inspect systems, and stop pretending the world ends at the chat box. Unfortunately, every new capability is also a new route through the wall.

The MCP security piece is worth reading because the scary parts are not exotic. A tool description becomes an instruction. A community package gets trusted because it installed cleanly. A server is left exposed. A permission scope gets widened “just for testing.” A key ends up where a key should not be.

No cinematic breach required. The tool does exactly what it was allowed to do, and the incident report writes itself.

The attack surface is no longer just the model. It is the tool catalog, the server configuration, the package supply chain, the permission scope, the network path, the logs nobody reads, and the defaults somebody meant to revisit.

AWS, to its credit, has at least noticed that letting agents near cloud accounts requires more than a cheerful README and a few thoughts & prayers. Its MCP server reaching GA matters because the boring machinery is part of the pitch: IAM, current docs, CloudWatch, CloudTrail, controlled access.

Good. Boring is what you want here. Exciting cloud automation is how people end up saying “just checking the logs” in the voice of someone about to cancel lunch.

FAULT_03: AUTH_IDENTITY_COLLISION // ZERO_ACCOUNTABILITY

If an agent can operate inside a system, it needs an identity.

Not the identity of whatever service account happened to be nearby. Not the all-purpose automation user from a pilot that nobody quite remembers. Not devops-bot-prod, the ancient shared automation account with just enough access to make every audit conversation worse.

The agent needs its own badge.

O’Reilly’s Agent Stack Bet lands on the right phrase for the alternative: governance debt. Let agents run through shared accounts and the system may look fine for a while. Tickets close. PRs appear. Summaries arrive. Someone makes a chart.

Then something breaks.

The fun starts after the breakage, when everyone crowds around the logs and discovers the same credential was used by an agent, three jobs, two scripts, one human, and a pilot project last touched during a reorg.

Congratulations. The timestamp survived. Meaning did not.

Give agents their own identity. Give them narrow permissions. Make the logs boring enough that the answer is obvious. Nobody will put that in the keynote, which is how you know it matters.

FAULT_04: UNBOUNDED_ITERATION_LOOP // RECURSIVE_TALK_TO_WALL

“Let the agent iterate” sounds lovely until the agent is on step 19, has forgotten why it started, and is now negotiating with a tool error like it owes the tool money.

Most demos glide past this part. They show the agent beginning, maybe even succeeding, but rarely dwell on the awkward middle: the half-fixed task, the flaky tool response, the partial result worth saving, the moment where the right answer is to stop and hand the mess back to a human.

That is why the piece on agent termination belongs here. max_iterations works as a fuse. Useful, yes. But a fuse only tells you the circuit got too hot. It does not tell you whether the work succeeded, failed cleanly, produced something salvageable, or wandered into the tall grass chasing a tool error.

A useful agent needs some sense of completion before it starts touching real systems. What counts as done? What counts as broken? What should be preserved? When should it ask for help? When should it refuse to continue?

Skip those questions and the loop will answer them for you, usually at the worst possible time.

FAULT_05: CONTAMINATED EVIDENCE // LAUNDERED_PROMPTS

Agents read whatever we feed them: tickets, logs, chat threads, customer emails, and whatever someone copied out of a terminal at 1:12 a.m.

Increasingly, that mess has already been touched by another model.

Armin Ronacher’s Building Pi With Pi catches one of the stranger new failure modes: the AI-laundered issue report. Someone hits a real bug, asks a model to explain it, and files the model’s confident diagnosis along with the actual observation. By the time it reaches the maintainer, the useful bit may be buried under a machine-generated theory of the crime.

That is worse than a vague report. Vague at least has the courtesy to look unfinished.

The laundered version arrives with a fake-minimal repro, an imaginary root cause, a suspiciously tidy error taxonomy, and a suggested fix from the wrong neighborhood. It has formatting. It has confidence. It has the awful sheen of something that might be true if you are tired enough.

The maintainer now has two jobs: find the bug, then scrape off the explanation someone pasted on top of it.

This matters for agents because tickets and issues are becoming prompts whether we admit it or not. If the input is padded with machine-generated certainty, the agent may not be starting from evidence. It may be starting from contaminated evidence with decent paragraph breaks.

UNGLAMOROUS_UTILITY: loops_with_receipts

Agents would be easier to dismiss if they were only expensive nonsense machines. Unfortunately, some of them are starting to earn their keep.

Oracle’s GraalVM work on teaching agents to test 1,000 Java libraries is a useful example because the job is so unglamorous it almost glows. Nobody dreams of spending the afternoon writing reflection metadata tests for other people’s Java libraries. The agent pipeline could generate tests, run them, collect metadata, hit failures, and try again. Reality got a vote, which is more than can be said for half the AI demos currently fogging up LinkedIn.

Datadog’s work on a SQL query optimization agent points in the same direction. The improvement came from tracked experiments, prompt and tool-chain changes, model rightsizing, and observability. Not a magic prompt. Not a senior engineer whispering into the context window. A loop with receipts.

JetBrains’ Rider update on agentic test coverage adds the obvious lesson that somehow still needs saying: do not make the agent wander the repo cold if your tools already know where the tests live. Give it coverage data. Give it structure. Make guessing more expensive than looking.

That is the useful version of this story. Narrow work. Real feedback. Context that was prepared by the system instead of hallucinated by the model. The agent does not get to declare victory because the paragraph sounds confident. It has to run into the wall and report where the wall was.

RUNTIME_MUTATION: THE UNMONITORED PYTHON SANDBOX

The shape of tool calling matters too.

AWS’s post on programmatic tool calling on Bedrock is interesting because it moves some work out of the model’s conversational loop and into sandboxed execution. Instead of bouncing every intermediate tool result back through the model, the agent can generate code that calls tools, loops, branches, aggregates, and returns the final result.

Yes, that can save latency and tokens. Fine. The more interesting part is that not every intermediate result has to be shoveled back into the model’s mouth.

Raw intermediate data does not always need to enter the model context. Every tool result does not need to become more prompt sludge. Sandboxes, execution limits, and narrower exposure are not implementation details. They are part of the boundary.

“Can the agent call the tool?” is the easy question.

The useful questions are dull and specific: where the data goes, who can see it, what gets logged, what the generated code can touch, and what happens when it is wrong. The architecture answers those questions whether or not anyone wrote them down.

Draw The Line Before The Line Draws Itself

The trust boundary is not something you bolt on after the demo. By then, the agent already has habits.

It lives wherever the model can change the world a little: a tool call, a credential, a memory file, a ticket comment, a cloud API, a test run, a pull request. Most of those places look boring until they are not.

That is the shape of the work now. Agents can help. They can also inherit bad permissions, trust polished garbage, misunderstand a tool error, and leave behind output that looks just finished enough to pass.

So draw the line early. Give the agent less than it asks for. Make it prove what it did. Keep the logs dull, the permissions narrow, and the human close enough to pull the cord.

The machine is inside the system now.

Someone had better know where the fence is.