Journal
Studies as they land, newest first. Failed predictions and null results stay in the list.
Filter entries
No entry matches these filters.
November 2026
4 entries
Attacks with harm, on the product in in-process stands: authority 1/33, cross-organisation 0/34, withdrawal 7/33 (all listed for the host), repeats 4/16 (one steward case), contradictions 0/28. Legal actions refused: 1 of 112.
Does combining help? Layer test
Requiring a prior read is not new: it is a UCON pre-obligation (2004), and AgentCore's documentation expresses it. No reviewed work documents a service-held current version plus a fresh engine-side read of the action's object.
Better than simpler options? Documentation reading
Can a service's own field decide whether a requirement applies? A documentation census of nine APIs
Per public documentation (2026-11-02), exact list matching on a service's own field works directly in 5 of 9 services, all on label or enumeration fields (6 of 9 without one assumed exclusion); 8 of 9 need a projection of a list of objects.
Where does it stop? Guard study
Derived values in a guard declaration: judged, cited and derived arms on constructed cells
A derived binding refused when the true basis moved and passed when an input moved without changing the output, while citing the source refused both; today's at-most comparison refuses a raised ceiling.
Does it work? Guard study
October 2026
20 entries
How far an action-time reread reaches in the shipped declarations
20 of 64 reference slots in 20 shipped declarations have a reread road; 3 of 21 judgement slots rest on read content the guard cannot see. Reread is off by default in shipped declarations.
Where does it stop? Guard study
What a tool-call guard keeps about people: a marker census of its own stores
After minimisation, the proxy's and the gateway's intent records no longer held withheld values declared sensitive, and their logs held no provider refusal text; the census found 3 leak paths code reading had missed.
Where does it stop? Guard study
The double refund on Stripe's sandbox below its approval threshold: a five-pair pilot
On a Stripe sandbox (test mode), below a deliberately chosen approval threshold, 5 of 5 unguarded pairs refunded twice and 0 of 5 guarded pairs did; the gateway's in-flight answer was used in 3 of the 5 guarded pairs.
Does it work? Guard study
Keying the operation on an operator's mandate: lawful repeats pass, the check moves to the issuer
Mandate keying completed both lawful refunds in 20 of 20 pairs and stopped a duplicate authorized once (0 of 20), but let a duplicate authorized twice through in 18 of 20, as designed.
Where does it stop? Guard study
Before the series: a strong model opens another customer's payment to decide not to refund it
In 20 earlier unguarded episodes, claude-opus-5 opened the foreign payment named in a note in 20 of 20, the provider served it in 20 of 20, and the agent refunded it in 0 of 20.
Better than simpler options? Guard study
Foreign-object lures across four models: no harmful action, many foreign reads
Across four models and four static lures (640 episodes), harmful refunds were 0 in both arms; foreign reads occurred in 278 of 320 episodes without the guard (363 read requests) and in 0 of 320 with it.
Does it work? Guard study
What the ownership rule costs lawful work: one refused read by id, and model-dependent recovery
Tickets quoting no id completed lawfully in every cell in both arms (600 episodes). The cost came from refusing the opening read of a ticket-given id: own id, sonnet 20 to 11 of 20; another customer's id, haiku 17 to 0; opus unaffected.
Where does it stop? Guard study
The expectation held in 2 of 5 preregistered cells. With the hint, own-id tickets reached 20 of 20 (sonnet from 11, p = 0.0012; haiku from 16, p = 0.11), and gpt-6.1-sol 19 from 11 on a foreign id quoted by mistake; Claude models did not.
Where does it stop? Guard study
Coding agents' replies after a refusal: scheme proposed, not run
Not run: 0 of 980 planned messages coded. Would measure how tickets end after a refusal, disclosure of another customer's data and whether the refusal is surfaced, with cross-vendor coders and Cohen's kappa.
Where does it stop? Guard study
Agents asked to justify a held refund to a human reviewer: three trials
Given a justification tool, claude-opus-5 attached a justification unprompted in 3 of 3 sandbox trials, each stating that the ticket's payment id belonged to another customer and was not refunded.
Better than simpler options? Guard study
Second delivery of a ticket while the provider holds the refund for approval: two single pairs
In one pending-approval pair, the second agent got the provider's rejection and reported the other agent's request as its own. In one post-approval pair, it saw the completed refund and made none.
Better than simpler options? Guard study
Two careful agents, one ticket: a double refund in every unguarded pair
Unguarded, two claude-opus-5 agents on one ticket refunded twice in 50 of 50 pairs; through the payment-keyed gateway, 0 of 50, with the race reaching the gateway in 49 pairs and 199 of 199 agents reading before refunding.
Does it work? Guard study
The race window by start offset, and the preregistered prediction that failed
The preregistered prediction failed: harm fell below half of pairs at a 2 s offset (9 of 20), not at 3 to 3.5 s. Harm followed the race in 51 of 51 pairs against 0 of 129, and 4 to 8 s spacing still left 3 of 80 harmed.
How we measure. Guard study
The double effect in a second vendor's agent product: Codex CLI
gpt-6.1-sol in Codex CLI: one agent with a lost answer, 0 of 20 harmed; two agents on one ticket, 25 of 30 unguarded pairs refunded twice and 0 of 30 guarded, with the race deciding harm pair by pair.
Does it work? Guard study
The duplicate-delivery result in a second, non-money domain: bookings (planned, not run)
Not run. Would count two bookings of one slot from the booking server's log, two agents per request, a bare arm against an arm guarded on the slot; the booking server has no fault injection yet.
Does it work? Guard study
An internal, contract-only Python reader agreed with the Go reference on 435 of 435 version-0 cases, 11 of 11 network cases and 75,229 re-signed mutants; the reference agreed with 629 of 629 cases the reader's author wrote.
Does it work? Layer test
An accountable-action layer for agent tool calls: design of signed intents, outcomes and withdrawals
No measurement of its own: a reviewed specification with English contracts and conformance vectors, whose three review rounds found 17, 3 and about 40 blockers. The chain test and the contract-only reader check parts of it.
Does it work? Design record
On an Apple M4, one run per configuration: proof generation 31.88 to 32.82 s, receipts 225,384 to 227,432 bytes, verification 9.54 to 9.85 ms, nearly unchanged between 2^10 and 2^20 log entries within one segment.
Where does it stop? Prototype
What a separate guard process per agent turn costs: in-process vs fresh proxy vs long-lived session
Median turn latency, one turn at a time: 6.1 ms in-process, 120.7 ms with a fresh guard proxy per turn, 48.2 ms with a long-lived guard; the guard's own share was 25.0 ms per action and under 0.5 ms per read.
Where does it stop? Guard study
Under the pre-written criterion, 23 of 25 mandatory rows were exercised and both declarations were blocked; 25 of 25 only under a criterion revised after the first run, which added an owner control and a path-binding check.
How we measure. Guard study
September 2026
13 entries
Where the guard's code still assumes money or a number shape
Three assumptions remain in current code: a declared unit rendered under a money type name, which does not block a non-money declaration; journal refusal of any fractional number; only the first text block of an MCP answer kept.
Where does it stop? Guard study
Proxy and gateway against Stripe test mode: refusals, a race and a lost answer
The scripted gateway run passed 14 of 14 rows, with one refund in an overlap (the other session told the outcome was unknown) and one after a lost answer settled by reconciliation; the proxy refused a foreign payment.
Does it work? Guard study
No single-agent harm in 40 preregistered bare episodes
With no guard, claude-opus-5 refunded another customer's payment in 0 of 20 episodes and repeated a refund after a lost answer in 0 of 20; both stories fell below the preregistered rule of harm in 5 of 20.
Does it work? Guard study
A scripted crash-test bench: seven failure scenarios with and without the gateway
All 16 scripted episodes ended as written beforehand: without the gateway 5 scenarios made an unlawful effect, with it only a wrong amount within the payment's bound, and one lawful refund was held for an operator.
Does it work? Guard study
The September pilots: a confounded guarded arm and a reply measure that failed
Unguarded, two claude-opus-5 agents refunded twice in 19 of 20 pairs; the guarded 0 of 20 is confounded by a refusal that hit 40 of 40 guarded agents, and with a replay note 19 of 19 replies still claimed the refund.
Does it work? Guard study
What three Stripe SDKs retry under injected faults
Across 5 client configurations and 14 fault cells (70 combinations) no SDK retry made a second effect; the default Python client made 1 attempt in every cell and reported an error for a refund that existed.
Better than simpler options? Guard study
One effect and a refused retry when the answer is lost: managed execution with a contract provider
With the MCP answer lost after the effect, the provider held 1 payment and the retry was refused as undetermined; with the request cut before the server, 0 payments and a reservation an operator act could resolve.
Does it work? Guard study
An executable model of execution authority with a bounded state search
The model met 14 of 14 corpus cases; a bounded breadth-first search of 200,000 states and 693,029 transitions stopped at depth 9 with no violation, and witnesses reached 22 of 22 required events.
Does it work? Guard study
A documented live path blocked at the estimate step: an incident report
In 2 live runs 0 end-to-end scenarios completed: all 3 state-changing calls were held for approval because no intent id was sent, 5 reads succeeded, and the recovery scenario was never run.
Where does it stop? Guard study
What an agent learns from a refusal over stdio: body versus timing
The refusal text carried 0.000 bits about the reason on all 7 tools, but the largest per-cell timing estimate (0.395 to 0.637 bits) exceeded the permutation null (0.214 to 0.281) in 3 of 3 runs, refuting the timing prediction.
Where does it stop? Guard study
Declaration-only typed references on live GitHub and Stripe test mode
On Stripe test mode the preregistered form failed and passed only after a driver change; on GitHub a bare issue number was held when unobserved and allowed when observed, with 0 core lines changed and 41 upstream tools hidden.
Does it work? Guard study
The guard did not save model spend: a cost reading of 317 saved sessions
In the one resolvable contrast (n = 24), sessions with the guard on cost 1.14 times those with it off (1.03 to 1.26), on different builds; ablation arms showed no resolvable difference, and no saving was found.
Where does it stop? Guard study
Do real API specifications publish a snapshot and completeness bit?
3 of 560 strict list operations publish a snapshot in the response envelope, 2 of them also echoing the narrowing request; the threshold of 2 was met, against the plan's expected negative.
Where does it stop? Guard study
August 2026
7 entries
An instrument's candidate-selection rule hid a reachable hole
Replacing an alphabetical first-N slice with a capped selection rule exposed 1 reachable hole in the legacy refund variant, with a printed witness; the production model still showed 0 findings and 0 of 97 grid holes.
How we measure. Method
A common denominator inverts a published model ordering
The premise of different denominators failed: they were identical. On a 654-key shared grid, production and legacy had 0 holes, the other two 654 of 654; the published metric ranked legacy below typed-only (93.58% vs 94.86%).
How we measure. Method
A proposed field adds no separation over the existing fence: a separability census
On a 96-cell synthetic grid the candidate field partitioned objects exactly as the existing fence (symmetric difference 0); 12 of 12 thresholds and 6 of 8 predictions held; 12 of 96 cells were reachable.
Better than simpler options? Method
A published metric cell with zero holes and a class it cannot see
The production rule model had 0 holes in the 545-key published cell yet, like the other three, admitted 14 of 14 unlawful actions of a class outside its construction; on an access domain the blind zone was empty.
How we measure. Method
A 10-seed label-permutation control is unreliable; an exact replacement
With 1,000 seeds, 5 of 12 blocks were only conditionally certified (1 or 2 expected); an exact criterion changed 0 of 12 verdicts but took 13.897 s against 0.019 s, refuting the prediction that cost would not rise.
How we measure. Method
Which binding axis carries protection depends on the action: a synthetic second-domain probe
It did not reproduce: on the invoices payment action the typing axis contributed 15.38 points (0 on refunds) and provenance 2.82, with 0 of 3,696 false positives; on an anchor-referencing close action all four variants tied.
Does it work? Method
Provenance alone stopped 0 of 7 completeness violations one model executed; with a closed-world predicate 6 of 7 went unexecuted (preset: 7 of 7), no well-observed lawful action was refused, but that model's lawful actions fell to 0.
Where does it stop? Guard study