How loss functions changed the way I run agent loops

There's a conversation happening right now about what people are calling compound engineering. Will Larson wrote about adopting Every's four-step version — plan, work, review, compound — and rolling it out across his team. Eugene Yan wrote his own version: context as infrastructure, taste as configuration, verification for autonomy, delegation, closing the loop.

Both are good and they mostly agree. Plan before you build. Put your preferences in a config file. Make verification cheap so you can delegate bigger chunks. Write down what you learned so the next session starts further along.

I do all of that. But there's a step past it that none of them covers.

HAMY went furthest. He built an eight-phase orchestrator, ran it overnight and reported what came out: 15 tasks in 10 hours, about $90 in tokens. Then he judged the result the way I would have:

The idea being that it's okay to sacrifice speed for quality, considering we want this to be run ~fully autonomously. It's better for it to only complete 15 tasks over night and need no rework than 20 but we have to fix 5 of them.

— HAMY, February 2026

That is a quality criterion, and a good one. It just isn't a number. Nothing in that run could evaluate it except him.

Which is where my last few months have gone: what happens when you stop reviewing the loop's output and start scoring it.

The short answer is that you stop having a documentation practice and start having an optimiser. And an optimiser has a property that a notebook doesn't — it pushes back on the thing you're measuring with.

What changes when there's a number

When I supervise a run, I'm the judge. I read the diff, I decide if it's good, my judgment is unwritten and adapts as I go. That works and it doesn't scale past my attention span.

A loss function is the alternative: write down what "good" means precisely enough that a script can compute it, then let the loop run against that script for as long as it takes. My version lives in a goal.md next to a spec.md, a harness/ that scores, and an eval/ holding the cases.

The gain is real. The loop runs while I'm not there, it knows whether it's winning, and it can keep going for many cycles without me.

The cost is that I've handed something a target, and it's better at pursuing targets than I expected.

A concrete example

To make this specific without publishing anyone's spec, take a task I can describe end to end: a tool that reads a GitHub Actions workflow file and reports which documented CI anti-patterns it contains. No concurrency group. No path filters. Unpinned action versions. No dependency cache.

That's scoreable. Label a set of workflow files with the patterns they actually contain, have the harness compute per-category accuracy, take the mean as a composite. Now there's a number between 0 and 1 and the loop has something to climb.

Here's what a target like that looks like stripped down:

## Target
harness/score.sh scores eval/dev and prints one JSON line:
{"categories": {...}, "composite": N}

## Constraints
- goal.md, spec.md, harness/ and eval/ are READ-ONLY (checksummed;
  modification voids the score)
- Surface allowlist: this repository only. No network.
- Determinism is part of the spec: no wall-clock, no Math.random,
  no filesystem-order dependence in scored output.

## Stop conditions
Bar hit on holdout · marginal gain about zero for 5 cycles ·
probe gap over 0.15 twice in a row

The interesting part isn't the target. It's everything defending it.

The loop optimises the eval, not the problem

The first thing that goes wrong is the oldest result in machine learning, and it arrives faster than you'd like.

You are not asking the loop to detect CI anti-patterns. You are asking it to make score.sh print a high number. Those are the same goal only as long as the eval is a fair proxy, and a proxy stops being fair the moment something is allowed to study it.

So the loop finds the shortcuts. It notices that every dev file missing a concurrency group also happens to be over eighty lines, and keys on length. It notices a comment string that correlates with a label. In the limit it just learns the dev set: a lookup keyed on something incidental, dressed up as logic.

Every one of those scores 1.0 on dev and is worthless.

This is Goodhart's law with a much shorter feedback cycle than usual. The measure became the target at cycle one, because that is literally what I told it to do.

Read-only, checksummed, blinded

Three rules that follow directly:

• The target is read-only. goal.md, spec.md, harness/ and eval/ are checksummed, and modifying any of them voids the score. Otherwise the cheapest path to a high number is editing the scorer, and it will find that path.
• The eval answers don't exist in the repository. Scenario inputs are readable, expected outputs are not there to be found. You cannot copy an answer key that isn't present.
• Determinism is part of the spec, not a nice-to-have. No wall-clock, no randomness, no filesystem ordering in anything scored. Otherwise a rerun produces a different number and you can't tell improvement from noise.

That last one sounds pedantic until a composite moves by 0.03 between identical runs and you spend an evening chasing a change that never happened.

It's also what makes evals hard to trust in the first place. Larson, writing about the evals his team runs against their agent workflows, names the problem directly:

The core challenge is the non-determinism introduced by these eval tests, where in practice there's very strong signal when they all fail, and strong signal when they all pass, but most runs are in between those two.

— Will Larson, Evals to validate workflows

His conclusion is that he'll probably have to move complex workflows away from LLM-driven execution to get consistency. I'd draw a different line. Some of that non-determinism is the model's and you have to live with it. The rest — clocks, randomness, filesystem order, anything that varies between two identical runs — is yours, and you can forbid it in the spec. Remove that half and what's left is signal you can actually read.

The probe gap

The defences above stop the cheats you can name. The useful mechanism is the one that catches the cheats you can't.

Split the eval three ways. Dev is what the loop scores against every cycle. Holdout it never sees until the end. Probe is dev, restated — same patterns, different surface: renamed jobs, reordered keys, reformatted YAML.

A tool that understands the problem scores about the same on dev and probe. A tool that memorised dev falls over on probe, because everything incidental it was keying on just moved.

The probe gap is the difference between a loop's score on the dev set and its score on the probe set. It is the single most useful number in the whole setup. Not because it tells me how good the thing is — the composite does that — but because it tells me whether the composite means anything.

My stop conditions treat a blown gap as an outcome, not a failure. Gap over 0.15 twice in a row and the run stops and writes up the suspected memorisation. That's a legitimate way for a run to end. Some of the more useful runs I've had ended that way.

Lints for what you can name

Alongside the probe gap there's a cheaper layer: lint the source for the specific tricks you've already seen. No eval scenario ids in source. No hex literals that match answer hashes. No data file over some small size in the source tree. Caps on how often a suspicious literal can appear outside eval/.

None of it is airtight, and the honest version of the rule says so. Mine reads: hardcoded lookup tables keyed on eval content are a violation even where the lint is blind — the probe gap will expose them.

Both halves matter. The lint catches the obvious attempt early and cheaply. The probe gap is what actually holds, because it doesn't need me to have anticipated the specific trick.

Getting stuck is a separate problem

Cheating isn't the only failure. The other one is a loop that stops moving and keeps trying the same thing harder.

Three rules, in every target I write now:

• Stall rule. If the composite didn't move last cycle, the next attempt has to be a structural change — different subsystem, different design. Same-knob-harder is banned.
• Exploration quota. Every fifth cycle goes to the weakest category, even when the obvious leverage is somewhere else.
• Re-read the spec. A category sitting at zero for three cycles means the semantics were misunderstood, not that the code is nearly right. Back to the spec, re-derive, write it down before touching code again.

The first one does most of the work. Left alone, a loop will tune the same parameter forever, because each attempt is locally the cheapest next move.

What this doesn't fix

• I can't show you a score curve. I've built these targets and run them, but I don't have a clean multi-cycle history I'd be willing to put a chart on. A post about measurement with no measurements in it is an irony I'd rather name than hide.
• It costs something, but it is not a day's work. I don't hand-write these any more. A skill generates the target, the constraints, the instruments and the forced-entropy rules, then red-teams the result for cheats before anything launches. What is left for me is deciding what „good“ means for this particular task — and that part does not automate.
• A good score still isn't proof. It's a much better signal than a green test run, which is what I've argued before. It is not the same as the thing being good.
• You have to write the spec anyway. Most of the value showed up during specification, before a single cycle ran. That's an uncomfortable finding for a post about automation.

Next

Where I've landed:

• A loss function turns supervision into something that runs without you. That is the whole gain, and it's large.
• The target has to be read-only, or the shortest path to a high score runs through the scorer.
• Hold out a probe set. The gap between dev and probe is what tells you the number means anything.
• Ban same-knob-harder. Stalling looks nothing like cheating and needs its own rule.

Compounding, in the sense Larson and Yan describe it — and the hand-judging HAMY does at the end of a run — is what happens between runs — the write-up, the config change, the rule you didn't have last month. What I'd add is that once a run is scored, compounding has an adversary, and the adversary is the thing you're compounding.

The next thing I want is the score curve I said I don't have: a long run with enough cycles that the shape of the descent means something, and a probe gap tracked next to it the whole way.

If you're running scored loops, I'd like to know what your loop found that you didn't anticipate — and whether you caught it with a lint or a holdout.