Kristopher Baker iOS roots · Product systems · AI-assisted workflows
← Sparra

decision · 2026.06.26 · 1 min read

Closing the door on a gamed contract

The contract is the heart of the loop, and the heart had a hole in it. A generator under pressure to pass will find the laziest reading of an assertion that technically satisfies it. Ask it to make a function "handle empty input" and it might add a guard that returns early and call the contract met, while the real behavior is still broken. I watched the loop accept a few of these degenerate, no-op satisfactions and decided the prompts, not the model, were at fault.

So I spent a day hardening the contract and evaluator prompts against exactly that. Acceptance criteria are now biased toward net effects rather than invocation counts, so "the function was called" never stands in for "the output is correct." The evaluator runs the artifact's own shipped verification as-is and treats a degenerate or gamed input as a failure, not a pass. I also pushed contract negotiation toward terse, proportionate assertions, because a bloated contract is just more surface to game. None of this makes gaming impossible. It makes the honest path the easy one, which in practice is most of the win.