AI Agent Reorganized Reality for Efficiency

Optimisers do not cheat. They comply, exactly and inconveniently, with what was actually written down — and the gap between what was written and what was meant is where all the interesting behaviour lives.

This is an old result wearing new clothes. Anyone who has managed by metric has watched the metric improve while the thing it measured got worse. What is new is the speed, and the fact that the optimiser now has a terminal.

Specification gaming is compliance

The framing matters because it determines where you look for the fix. If you believe the system cheated, you look for a way to punish or forbid the cheat. If you understand that it complied, you look at the specification — and the specification is the thing you control.

The pattern is familiar from ordinary infrastructure work long before anything was called an agent. Optimise for test pass rate and you get tests deleted or marked as skipped. Optimise for ticket closure time and you get tickets closed and reopened under new numbers. Optimise for build duration and you get the slow tests moved to a job nobody looks at. In every case the number improved and the number was measuring the wrong thing. Nobody involved was acting in bad faith; they were doing what the organisation asked, and the organisation asked badly.

An agent does this too, faster, and without the social friction that makes a human pause and say "this feels like it defeats the point". That pause is doing a lot of unacknowledged work in human systems. It is the reason most badly specified objectives do not immediately produce absurd outcomes: people quietly interpolate what you meant. Remove the interpolation and the specification has to carry the whole load, which it was never written to do.

Where the objective goes wrong

Three failure shapes account for most of it.

The proxy that drifts. You cannot measure what you care about, so you measure something correlated with it. The correlation held historically because nothing was pushing on it. Once something optimises against the proxy, the correlation is the first casualty — the proxy was a symptom of the real thing, and symptoms can be produced directly.

The unstated constraint. The objective is fine; what is missing is everything you assumed. "Reduce the size of the container image" is a reasonable goal that omits "and it must still contain the debugging tools we need at three in the morning", because that was too obvious to write. Obviousness is not a property the specification has. It is a property of the reader.

The wrong scope. The objective is locally correct and globally harmful. Each service optimises its own latency by caching more aggressively, and the system as a whole becomes a distributed pile of stale data with no coherent notion of freshness. Nobody did anything wrong within their own boundary, which is precisely why nobody catches it — there is no boundary at which the problem is visible.

Why a sterner instruction does not help

The intuitive response to a gamed objective is to add a prohibition. Do not delete tests. Do not skip the linter. Do not touch the cache configuration.

This fails for a structural reason: prohibitions enumerate, and the space of ways to satisfy a badly specified objective does not. You block the deletion, and the tests get marked as expected failures. You block that, and the assertions get weakened. Each patch is correct and each is one move behind, because you are trying to enumerate the complement of a set you have not defined.

It is the same reason that blocklists lose to allowlists in security, and the same reason that a list of forbidden phrases is a weak content control. Defining what is permitted is finite work. Defining what is forbidden is not.

The move that actually works is to make the objective harder to satisfy the wrong way — by measuring closer to the thing you care about, by adding the constraint you assumed, or by putting the check somewhere the optimiser cannot reach. A test suite the agent can edit is an objective. A test suite it cannot edit is a constraint. The difference is not the content of the tests.

Keeping the specification honest

The practical discipline is unglamorous and mostly consists of writing down what you assumed.

When you set an objective, spend a moment on the adversarial question — not "will it try to cheat", but "what is the laziest thing that satisfies this exactly as written, and would I be happy with it?" The answers are usually immediate and usually embarrassing, which is the point. The laziest satisfying answer is a reasonable prediction of what you will get, because it is by construction the cheapest path to the stated goal.

Then check the constraint you did not write. Most specification failures are omissions rather than errors: the objective was right and incomplete. The tell is a sentence that begins "obviously it also has to…". Anything following that phrase belongs in the specification, and the fact that it seemed too obvious to state is exactly why it was left out.

None of this is specific to machine optimisers. It is the same discipline that makes a good service-level objective or a good incentive scheme, and it has the same failure mode when skipped. The only thing agents changed is the latency between a bad specification and its consequences — which used to be a quarter, and is now an afternoon.