The double effect in a second vendor's agent product: Codex CLI
gpt-6.1-sol in Codex CLI: one agent with a lost answer, 0 of 20 harmed; two agents on one ticket, 25 of 30 unguarded pairs refunded twice and 0 of 30 guarded, with the race deciding harm pair by pair.
Prediction heldDoes it work?Re-checked
- Designed for
- Built to test whether the double effect of two careful agents, and its prevention by the guard's gateway, carry over to a second vendor's agent product run as it ships: gpt-6.1-sol in Codex CLI.
What was tested
Preregistered: gpt-6.1-sol ran in Codex CLI with its own system prompt, on a simulated provider. First one agent whose answer was lost after the effect, 20 episodes; then two agents on one refund ticket, 30 pairs unguarded and 30 through the guard’s payment-keyed gateway. Each episode’s CLI record was audited afterwards.
What we observed
A single agent refunded once in 20 of 20 episodes. Unguarded, 25 of 30 pairs refunded twice: exactly the 25 where both agents read an empty refund list; in the other 5, one agent saw the other’s refund in time. None of 140 agents refunded after seeing a refund or without reading first. Through the gateway, 0 of 30 pairs refunded twice; all 28 second refunds that reached it were refused.
What this means for Heimdall
In this setup the race carried over from claude-opus-5 to a second vendor’s agent, and the gateway’s once-per-object check held under it. This bears on the executor’s same-action fence.
Limits
One model per vendor; system prompts differ by product. Only the basic cell ran; a simulator. A third vendor’s series stopped at 49 of 80 episodes and is not analysed.
What remains open
Replicate in a domain other than refunds.