The September pilots: a confounded guarded arm and a reply measure that failed

Unguarded, two claude-opus-5 agents refunded twice in 19 of 20 pairs; the guarded 0 of 20 is confounded by a refusal that hit 40 of 40 guarded agents, and with a replay note 19 of 19 replies still claimed the refund.

SupersededDoes it work?Re-checked

Designed for

Built to test whether the guard's gateway, holding one intent per object, stops a double refund by two simultaneous careful agents, and whether a replay note changes what the second agent tells the customer.

What was tested

Two claude-opus-5 agents got one refund ticket at once, 20 pairs per arm, on a simulated payment provider, directly or through the guard’s gateway. A second pilot reran the guarded arm with a tool note telling the second agent its answer was a replay. A re-analysis recounted the ledger.

What we observed

Unguarded, 19 of 20 pairs refunded twice. Guarded, 0 of 20 did, but the gateway’s declaration refused a parameter on the first refund call of all 40 guarded agents and no unguarded one. With the replay note, harm stayed 0 of 20, yet 19 of 19 replayed agents told the customer they had made the refund themselves. The preregistered keyword measure scored those replies as acknowledgements and was wrong in all 19.

What this means for Heimdall

The unguarded race holds at pilot strength. Whether the gateway works, this pilot cannot say: its confounded guarded arm shows neither success nor failure. Effects count from the ledger, not replies; this bears on the executor’s signed outcome.

Limits

The refusal gave one arm an extra turn and showed in a trial before the series. That gateway version replayed the first result to a second request. One model, a simulator.

What remains open

A preregistered repeat without the confounder replaced the guarded comparison.