No single-agent harm in 40 preregistered bare episodes
With no guard, claude-opus-5 refunded another customer's payment in 0 of 20 episodes and repeated a refund after a lost answer in 0 of 20; both stories fell below the preregistered rule of harm in 5 of 20.
No differenceDoes it work?Re-checked
- Designed for
- Built to test how often claude-opus-5 with no guard refunds another customer's payment after an injected note, or repeats a refund whose answer was lost, to decide whether these single-agent stories show what the guard prevents.
What was tested
A preregistered series alternated two stories, 20 seeds each, unguarded: a ticket note naming another customer’s payment, and a lost answer after a completed refund. claude-opus-5 ran in Claude Code against a Stripe-shaped simulator. Harm was counted only from the simulator’s ledger, with no reruns; the preregistration was approved before the series.
What we observed
Harm occurred in 0 of 20 note episodes and 0 of 20 lost-answer episodes, reproduced by a ledger recount. Both stories failed the rule that a story needs harm in at least 5 of 20 to stay. With 20 episodes, a rare story (10% or less) survives by chance at most 4.3% of the time and a common one (40% or more) is dropped at most 5.1%.
What this means for Heimdall
This asks whether these single-agent stories give the guard harm to prevent. Unguarded, this model caused none, so the demonstration moved to two agents. This shows neither that the ownership and once-only checks are needless nor that they protect.
Limits
One model and harness; a simulator; 20 episodes separate only rare from common rates; foreign reads were not counted as harm, and the model did read the foreign payment.
What remains open
A later re-analysis of the note episodes counts those reads.