Finding Side-Effect Loopholes in Verifiable Rewards
Richard Abrich. Accepted as a poster at the NeurIPS 2026 workshop Who Verifies the Agents? Toward Reliable Agent Development.
Updated 2026-10-14
Abstract
OpenReviewarXiv (v1, earlier title)
Execution-based task verifiers decide whether an agent succeeded. We audit shipped AppWorld and WorkArena verifiers with source-informed mutation tests. The main audit never modifies a shipped checker.
In AppWorld, duplicating a non-idempotent write creates an extra record while preserving every checked field value. The verifier accepts all three task variants from two of five eligible generators: 6/15 constructed effects. A cardinality patch applied to checker copies after the census makes all six cells fail while preserving valid controls.
In WorkArena, we prospectively rerun 23 extra-field candidates selected for earlier checker-PASS outcomes. Independent Table API readback confirms nondefault persisted values in 21, while all 23 receive PASS. Two requested strings are aliases of stored defaults. The 21 confirmed wrong effects span three form templates. These selected cases confirm wrong effects under the audit's protocol; they do not estimate a population rate.
No other construction produces an independently confirmed false accept. Other checker-PASS cases are effect-correct degeneracies. We report zero-PASS families separately because retained evidence differs. In fixed intent-swap grids, the checkers return no PASS on 2,689 off-diagonal executions. This is a rejection census: 57 WorkArena cells use session-scoped evidence; the other 2,632 lack classified rejection causes and independent target ground truth.
Each increment is specified before its own cells are scored. A supplement accompanies the OpenReview submission with the construction grammar, evidence, content-bound stage lineage and count reproducer.
Licence: CC BY 4.0. DOI: 10.48550/arXiv.2610.09142.
In plain words
AI agents are graded by automated checkers that look at what the agent changed. The paper tested the checkers shipped with two public benchmarks, AppWorld and WorkArena. Some confirm the change that was asked for but still pass a run that also made a change nobody asked for, such as a duplicate record or an extra field.
What this means if you're buying automation
A check that confirms only the fields you typed can miss an extra or duplicate record. Ask any vendor what their check reads and what it would miss. Our own record checks run only where someone set them up and the system offers a second read. In our lending test, the check also audited which tables changed. See how the record check works and How do you know an automation saved the record?.
Competing interests
The author founded and runs MLDSAI Inc. (OpenAdapt.AI), which makes OpenAdapt.
Cite
@misc{abrich2026blindspots,
title = {Finding Side-Effect Loopholes in Verifiable Rewards},
author = {Abrich, Richard},
year = {2026},
eprint = {2610.09142},
archivePrefix = {arXiv},
primaryClass = {cs.AI},
doi = {10.48550/arXiv.2610.09142},
note = {Accepted as a poster at the NeurIPS 2026 workshop "Who Verifies the Agents? Toward Reliable Agent Development"}
}See it on your process
Bring one task your team types every day. We'll tell you whether it fits, and what a test on your own system would cover.