Agent fleets in production: a field report

The pitch for delegated coding agents is parallelism: run five tasks at once, get five results, multiply throughput.

Having run this for a while at meaningful volume, here is what actually happens.

what works#

Mechanical, well-specified, verifiable work. This is not a hedge, it is the finding. The tasks where fleets genuinely deliver:

• Dependency upgrades across many services.
• Migrating a deprecated API call across a large codebase.
• Adding tests to modules with poor coverage.
• Converting between formats or frameworks with a mechanical mapping.
• Fixing a class of lint or type error across a repository.

What these share: a clear acceptance criterion the agent can check itself, a bounded scope, and no design decisions.

For this category the leverage is real and large. Work that would have been a week of tedium becomes an afternoon of review.

Investigation in parallel. Spawning several agents to independently investigate a bug from different angles — read the logs, bisect the history, read the related code, reproduce it — and reading all four reports is genuinely faster than doing them serially. The agents are cheap; your attention is not; parallelizing the cheap thing is correct.

what does not work#

Anything requiring shared context. Five agents working on related parts of a system produce five changes that each make sense and collectively do not. They duplicate helper functions. They pick different names for the same concept. They each add a slightly different retry wrapper.

You end up with a merge problem that is worse than the original work, because the conflicts are semantic rather than textual — the diffs apply cleanly and the result is incoherent.

Anything with genuine design decisions. An agent given an ambiguous task will resolve the ambiguity confidently and silently, and it will pick a reasonable option that may not be the one you wanted. Five agents will each pick differently.

Anything where the acceptance criterion is subjective. "Improve the error messages" produces five different notions of improved.

the actual bottleneck#

Review, exactly as predicted, and worse than predicted.

Five pull requests per hour is not a throughput improvement if you can meaningfully review two. What you get is a queue, and queue pressure degrades review quality in a way that is invisible in the metrics and visible in the defect rate a quarter later.

The honest arithmetic: your throughput is min(generation rate, review rate), and review rate did not change.

what actually helps#

Make the agent produce a reviewable artifact, not just a diff. A summary of what it did, what it decided, and what it was unsure about. Reviewing "I chose to use the existing retry helper rather than adding a new one, and I left the timeout at 30s because the surrounding code does" is dramatically faster than inferring that from a diff.

Batch related work into one agent, not many. If five tasks touch the same module, one agent doing all five produces a coherent change. Five agents produce five incoherent ones. Parallelism across independent work only.

Invest heavily in verification. A strong test suite is what makes review cheaper, because you are reviewing design rather than correctness. Teams with weak tests get no benefit from agent fleets — they just get more code to verify by reading.

Cap concurrency at your review capacity. Running more agents than you can review does not help. It produces a backlog that goes stale and gets abandoned, which is worse than not starting.

Reject on size. A machine can generate a 2,000-line diff effortlessly. Say no. Ask it to split.

the number#

For our work, the useful concurrency turned out to be two to three agents on independent tasks, with one person reviewing. Beyond that, quality degraded and the extra output was not landing.

That is a real multiplier and it is much less than the demos suggest, and the constraint is entirely on the human side.

the thing I would tell someone starting#

Do not start with a fleet. Start with one agent, on the mechanical work you have been putting off, and measure whether the output actually lands.

If your review process cannot absorb one agent's output, adding four more is solving the wrong problem.