Refactoring under an agent

Large mechanical refactors — rename this concept across four hundred files, migrate every call site to a new API, convert a pattern used everywhere — are the single clearest win available from coding agents.

They are also where an unattended agent can do the most damage quietly. Here is the process that has worked.

why this is the sweet spot#

Mechanical refactors have exactly the properties agents are good at:

• Verifiable. The tests either pass or they do not.
• Repetitive. The same transformation, many times, which is where humans make mistakes from fatigue.
• Well-specified. You can state the transformation precisely.
• Boring. Nobody enjoys this work and nobody does it carefully after hour two.

And they have the property that makes human refactoring risky: the scale defeats attention. A person converting four hundred call sites will be careful for the first fifty.

the process#

1. Do ten by hand first.

Before writing any instruction, do a representative sample yourself. You will discover:

• The cases where the mechanical transformation is wrong.
• The variations you did not know existed.
• What the actual rule is, as opposed to what you thought it was.

This is the step people skip and it is the one that determines whether the whole thing works. You cannot specify a transformation you have not performed.

2. Write the transformation down precisely.

Not "modernize the error handling." The exact before and after, with the exceptions named:

Replace every call of the form:
    result, err := doThing(x)
    if err != nil { return nil, err }
with:
    result, err := doThing(x)
    if err != nil { return nil, fmt.Errorf("doing thing for %s: %w", x.ID, err) }

Exceptions:
- Do not change anything in internal/legacy/ (frozen).
- Do not change error handling inside deferred functions.
- If the error is already wrapped, leave it.

3. Establish the safety net before you start.

• Clean working tree, dedicated branch.
• Full test suite passing, with a recorded baseline.
• A way to check the transformation was applied correctly beyond the tests — a grep, a linter rule, an AST query.

4. Batch it.

Not four hundred files in one change. Twenty to fifty files per commit, grouped by module.

This matters for two reasons: a reviewable diff size, and the ability to bisect. If something is wrong, you want to know which batch introduced it.

5. Review the first batch line by line.

Every line. This is where you catch the systematic error, and a systematic error caught in batch one costs twenty minutes while the same error caught in batch twenty costs a day.

6. Spot-check subsequent batches, review the anomalies.

Once the pattern is verified, review by sampling — but read every diff that looks different from the pattern. Agent output that deviates from the established shape is where the interesting failures are.

7. Verify mechanically at the end.

# nothing left in the old form
rg 'return nil, err$' --type go | rg -v 'internal/legacy'

If the transformation is complete, the old pattern should not exist outside the exceptions. This catches the files that were silently skipped, which is a real failure mode.

where it goes wrong#

The agent "improves" things you did not ask about. It renames a variable while fixing the error handling. Individually reasonable, collectively it makes the diff unreviewable because you can no longer scan for the pattern.

Instruct explicitly: change only what was specified, nothing else.

Semantic drift across batches. Batch one wraps errors one way, batch fifteen does it slightly differently, because the context is different and the model made a different reasonable choice.

Fix: put the exact target form in the instruction file, with examples, and check for consistency at the end with a grep.

Tests pass and behavior changed. The most dangerous case. Your tests did not cover the path that broke.

This is why the mechanical verification in step seven matters. It checks the transformation, not the behavior, and it catches things tests do not.

Silent skips. The agent processes 380 of 400 files and reports success. The twenty it skipped are the ones with unusual structure, which are the ones most likely to matter.

Always count. Always verify the remainder is empty.

the honest assessment#

For this class of work the leverage is real and large. A refactor that would have been a week of tedium — and would therefore never have happened, which is why the codebase has the problem — becomes an afternoon.

That last part is the underrated benefit. The refactors that get done are the ones that are cheap enough to do. Lowering the cost means more of them happen, and a codebase where the cross-cutting cleanup actually gets done is a meaningfully better codebase.

Just count the files at the end.