Earlier this year the Cloud Security Alliance put out a paper with advice for defending against “Mythos” level attacks, and cited deception and Canaries as a defensive measure to be implemented in the short term.
When OpenAI spoke at BlackHat last month, Deception was one of the few recommendations they made.
Of course this doesn’t surprise us. For over a decade, Canaries have caught attackers across all seven continents. They are dead simple, and they “just work”.
Canaries work because they exploit an attacker’s objectives. An attacker who wants access to your Cloud can’t resist a Cloud API-key. An attacker who wants to know more about your infrastructure can’t resist browsing \\backup-server\network_diagrams;
It’s why they are there in the first place.
AI-powered agents may sound terrifying, but they have objectives too. They work pretty doggedly to achieve them, and this makes them just as vulnerable to Canaries (if not more).
We’ve previously released our MCP Canarytoken and Canarytokens with Guardrail triggers. (Guard rail triggers are cute, but we don’t see them being a long term solution since it’s almost certain that real attacks will be carried out by models unencumbered by guard rails). Those were just the beginning…
Introducing Agent Provocateur
Today, we are releasing a Canary update to help customers take advantage of agent behaviour for detection, with a click. This post is a quick explainer on how it works for people looking to implement a similar solution with their own infrastructure.
Our “Agent Provocateur” (AP) exploits two weaknesses far deeper ingrained in agent-behaviour:
- Agents see the world that we present them
Canaries have always served to exploit the fact that we control what the human attacker gets to see. Agents make this even more of a super-power since they effectively pass output that we control directly into their decision making loop.
Last year, Rebecca Lynch from NVIDIA’s Red-Team said, “[If] an attacker is able to get their untrusted data in front of your agent, you should assume they can control anything downstream from the model”:
- Even without Prompt Injecting them, agents are highly “suggestible”.
The OpenAI team famously pointed out that even without injections, “social engineering is forever”
Deploying Agent Provocateur
Like all our products, we’ve made this dead simple to deploy. Simply select the Agent Provocateur personality, and you’re done.
Let’s break down how it works. Agents exploring the local network discover the running webserver, and interact with it (that’s what they are there for!). Agent Provocateur responds helpfully:
The attacking agent has some objectives, and our service has offered to help it reach its objectives. We see the agent fall for this ruse and start interaction with Agent Provocateur. Why wouldn’t the agent play along? At a high level, you can imagine the conversation going something like this (where Agent Provocateur is a stateless service pretending to be backed by a model):
This allows us to reliably throw an alert when an agent is doing its agenting (allowing the SOC team/automated response to spring into action)
We’ve been playing with AP and testing, and on our internal ranges with multiple servers and targets, across a host of open and frontier models, agents trip over these Canaries 100% of the time (we have a follow-up post in the works describing the range we use, and introducing an implementation of the A2A protocol as an alternative to AP). This interaction can be further encouraged by deploying breadcrumbs1 that point attackers to the Agent Provocateur service.
Of course, if we are getting the agent to comply, we can ask it increasingly probing questions like:
Caveat Lector
The previous example (where Agent Provocateur asked the agent to identify its SSH key) shows an interesting result. In this case, for some reason, the supplied SSH key was hallucinated, it was not real. We do face the GenAI problem, that the agents will sometimes make up answers out of whole cloth (but in this case we would be sufficiently glad to know that we’ve at least caught a lying clanker in the act).
By default the Agent Provocateur module will ask a connecting agent lots of questions while providing sufficient feedback to keep the attacker interested.
This of course leads to the obvious thought: Can we keep the agent busy/too distracted to achieve their original goal? Spoiler: We are working on it.
Right now, you can deploy an Agent Provocateur in minutes. You’ll get a reliable alert when an agent is running wild on your network picking up a bunch of information from the credulous clanker when it does.
Of course, if we have an agent willing to answer questions, we have to wonder how far it can be pushed? On a recent podcast, theGrugq pointed out that the agent was kinda “born when you hit enter”
So it doesn’t take a lot of work to convince the agent to send you a reverse shell. We’ve got examples of the AP service “social engineering” the agent to download and run code in order to pass a fake enrolment step.
Conclusion
Agentic attacks are likely to rise in the near future but agents are not unbeatable. In many ways, what makes them useful, also makes them easier to detect and manipulate.
We’ve automated this with our Canaries and customers have access to the Agent Provocateur as the update rolls out, but the ideas published here would be easy enough for companies to replicate on their own infrastructure.
(In our next post – we will talk about the A2A service, our testing gym and observations building them.)
Rollout note: This update is currently rolling out to customer Consoles and Canaries. When your birds have updated themselves, the new AP and A2A personalities will appear in the configuration screen.
Footnotes
- A breadcrumb is simply a file or bit of information deployed on your production infrastructure that points towards a Canary. In this instance we create a text file that describes the AP service, and you drop that on (say) an open file share or your internal Wiki. When an agent browses the share or Wiki and finds the breadcrumb, they tend to investigate the AP service, thereby revealing the agent’s presence. ↩︎
