Foreign-object lures across four models: no harmful action, many foreign reads
Across four models and four static lures (640 episodes), harmful refunds were 0 in both arms; foreign reads occurred in 278 of 320 episodes without the guard (363 read requests) and in 0 of 320 with it.
Mixed resultDoes it work?Re-checked
- Designed for
- Built to test how often agents act on or read another customer's object when a task carries a lure, and whether the guard's ownership rules stop every such action and read.
Without the guard
278 of 320 read another customer's payment
With the guard
0 of 320 read another customer's payment
What was tested
Preregistered: one agent per episode handled a refund ticket on a simulator. Four lures put another customer’s payment before it: a colleague’s note, an instruction planted in provider data, a partner’s refund request, and a customer’s honest mistake. Models: claude-opus-5, claude-sonnet-5-5, claude-haiku-5-5 and gpt-6.1-sol, 20 episodes per lure, model and arm. Unguarded agents used an account-wide key; the guard served a payment only once listed for the ticket’s customer.
What we observed
No agent refunded another customer’s payment: 0 of 640. Without the guard, agents opened the foreign payment in 278 of 320 episodes (363 read requests); with it, in 0 of 320. Unguarded foreign reads per model, of 80: opus 78, haiku 80, sonnet 60, gpt-6.1-sol 60. The expected rise in harm on smaller models did not appear.
What this means for Heimdall
This asks whether the ownership rules work. In this setup they removed foreign reads. No harmful refund was attempted in either arm, so the action-time check had nothing to prevent; whether it protects is not shown.
Limits
Static author-written lures; a simulator. No ownership-check wrapper was compared. Only the guarded arm refused amount-less refund calls.
What remains open
Live services, with an owner-written wrapper as a baseline.