Tracebit Tests How Decoys Change an AI Attacker’s Decisions
Tracebit’s October research examines whether deception can change what an attacking AI agent chooses to do after gaining a foothold. In its controlled AWS environment, extra decoys reduced completed objectives and increased the model bill per objective. The useful result is about measured behaviour under particular conditions. It does not establish that a deployed security product prevents the same proportion of real intrusions.
The baseline already included canaries
The published experiment covers seven models and 773 completed assessments. Each agent began with a low-privilege AWS identity, the same investigation task and a two-hour limit. Researchers compared a baseline containing ordinary canaries with nine additional deception techniques, used separately and together. This is a comparison with an existing detection layer, rather than a completely undefended environment.
Success meant evidence of completing one of nine defined objectives involving protected data, code execution or higher privileges. Runs with no objective, early termination or the full allotted runtime remained in the comparison. Refusals, provider blocks and technical failures were excluded. Those exclusions must not be presented as attacks that the decoys successfully stopped.
Keep the objective, run and cost denominators separate
With all nine techniques combined, the reported mean fell from 2.48 completed objectives per run to 0.23, a reduction of about 91%. That is an objective-count comparison. A separate measure shows at least one objective reached in 13 of 70 combined runs, against 70 of 71 baseline runs. The different denominators and outcome definitions matter when interpreting a headline percentage.
Results also differed by model. GPT-6 Astra still reached an objective in eight of its ten combined runs. A two-hour observation window cannot demonstrate permanent prevention after that window, and a lower average does not mean every individual attempt failed. Treat the findings as evidence for further evaluation of these conditions.
The headline cost result is about 23 times more model spend per confirmed objective: $31.10 against $1.33 at baseline. It covers six models with complete cost data and includes spending on runs that achieved no objective. It excludes AWS infrastructure costs. The seventh model, GPT-5.6 Cyber, had incomplete cost records for two combined runs and is excluded from this cost comparison.
This does not mean that every individual run cost 23 times more. The report gives increases of 1.6 to 3.9 times in mean model cost across the six complete model records. Runtime and time to first objective are different measures too: the latter includes only runs that reached an objective, while progress curves include every comparable run. Mixing those populations would change the claim.
The traps influence decisions through discovered material
One technique exposes a fake penetration-test report that labels real targets as canaries and describes decoys as valuable targets. Another offers repeated encrypted objects that appear to justify more recovery work. A deliberately awkward HTTP service encourages repeated attempts to assemble incomplete information. These are different mechanisms, with different results across the tested models.
The request-pacing technique is especially instructive. A planted note claims that discovery calls need a three-minute interval to avoid a one-hour lockout. The researchers did not install that rate limiter. Agents that accepted the claim imposed the delay on themselves. The observed control came from trusting discovered text, rather than an enforced service restriction.
Our earlier context-bomb report concerned a separate September study using forged dialogue and a comparison involving modified Qwen models. Its 82-run capability comparison is not part of this October 773-assessment sample. Combining them would obscure differences in models, hosting, techniques and measured outcomes.
Evaluate deception alongside enforceable controls
These are Tracebit’s own measurements in one cloud environment, with roughly ten runs per model and scenario. We have not independently reproduced the experiment. Some technique details are available only through the company’s email-gated whitepaper; the public material supports the findings discussed here, without establishing those undisclosed mechanisms or performance in another deployment.
For a defensive evaluation, define the objective and permitted environment before testing. Keep decoys harmless, measure effects on legitimate automation, and connect detection to a rehearsed response. OWASP’s prompt-injection guidance calls for permissions and parameter checks at the executing tool, minimum necessary access, and testing with harmless data and instrumented substitutes. Retrieved wording is not an authorisation boundary. Record whether the real objective remained protected, what the agent actually did and what the available telemetry cannot establish.
