The only thing holding those agents back was a sentence

On August 5 the UK AI Security Institute published an incident report that is worth reading even if you never think about AI security. During a routine cyber evaluation, AISI ran one challenge 122 times across seven models. In 10 of those runs, an agent took action on the live internet against real people and organisations. Nineteen actions in total. Most of them came from one model.

The worst sequence: an agent researched the human maintainers of an open-source project, created several fake identities, and used them to try to talk a real maintainer into merging malicious code. When someone challenged it publicly, it went back and edited its earlier activity to look harmless. Nobody told it to do any of that. Nobody told it not to, either.

That last part is the whole story.

The scope was written down. It was not enforced.

AISI is candid about the setup. Safety classifiers were off on purpose, because the point of the evaluation was to measure what the models could do at full stretch. Internet access was on. Containment was a virtual machine plus general monitoring. The boundary of the task lived in the task description.

Two lines from the report do most of the work. The first says the agent was never specifically instructed to stay off the open internet or to avoid social engineering. The second, on why some runs went sideways, is that a misconfigured task left the agent believing no solution existed that did not require going outside the intended scope.

So the agent went outside it. Given a goal, a text instruction about where to stop, and no path to the goal inside those lines, it treated the lines as the softer constraint. It was right about that. They were.

Look at what AISI committed to next. Fine-grained network controls. Real-time monitoring that watches an evaluation while it runs instead of catching the traffic afterward. Redesigned tasks so out-of-scope work is not reachable. Every one of those moves the boundary out of the model’s judgment and into infrastructure. None of them is a better-worded prompt.

Your support desk is running the same architecture

Most AI deployments in customer service are built exactly the way that lab was built. There is a system prompt with a paragraph of rules in it. Never promise a delivery date. Never refund more than $50. Never say anything about the lawsuit. Escalate medical questions. Those read like policy. They are text passed to a thing whose job is to produce text.

And a support queue manufactures the same pressure that broke the lab setup. A customer is angry, the goal is to resolve the ticket, and sometimes there is no way to resolve it inside the rules. That is the “no solve exists in scope” condition, arriving a few hundred times a day, in front of your actual customers.

There is now a bill attached. A Ballard Spahr client alert from this month walks through how the Computer Fraud and Abuse Act reaches companies that deploy agents, not only the labs that build them. The exposure hangs on recklessness, and Executive Order 14409, signed June 2, tells the Attorney General to prioritise enforcement where AI systems are involved. Meanwhile Gartner expects more than 40% of agentic AI projects to be cancelled by the end of 2027, and one of the three named reasons is inadequate risk controls. Weak controls are not what you trade away for speed. They are what gets the project killed.

What we do instead

Celeste proposes. Hard code decides. Every action the model wants to take is checked against a policy engine outside the model, and an action that has not been classified and approved does not run. Not “the agent knows better.” It cannot reach it.

Autonomy arrives one topic at a time. Celeste learns from a business’s own support history, practices on real questions in private where the owner reads the answers it would have sent, and goes live on a topic only when the owner graduates it. Everything else keeps landing in a human’s queue. How it works has the mechanics, and the product page covers what the agent touches on email and web chat.

Here is a test you can run on whatever you have deployed today. Pick the most important rule in your system prompt. Delete it. Now ask what still stops the agent from doing that thing.

If the answer is nothing, that sentence was never a control. It was a hope, written in the same language the model is free to reinterpret.