what applies now#

For providers of high-risk systems — those used in employment, education access, credit, essential services, law enforcement, migration, justice, and as safety components in regulated products:

• Risk management system, documented and maintained across the lifecycle.
• Data governance, with documented training, validation, and test data and attention to bias.
• Technical documentation sufficient for conformity assessment.
• Automatic logging of the system's operation, retained.
• Transparency to deployers about capabilities, limitations, and intended use.
• Human oversight designed into the system.
• Accuracy, robustness, and cybersecurity proportionate to the purpose.
• Conformity assessment and CE marking before placing on the market.
• Registration in the EU database.
• Post-market monitoring and serious incident reporting.

Deployers — organizations using a high-risk system rather than supplying it — have a lighter but real set: use it according to instructions, ensure input data is relevant, monitor operation, retain logs, and assign competent human oversight.

the parts that moved#

The implementation timeline has been adjusted more than once during the run-up, which is normal for regulation of this scope and which made planning genuinely difficult for anyone trying to comply.

The important practical consequences:

Some obligations phase in later for systems already on the market, and for high-risk systems that are safety components of products covered by other EU legislation.

Harmonised standards are still being finalised. Conformity assessment is easier when you can demonstrate compliance against a standard; where standards are not yet available, providers must demonstrate compliance against the requirements directly, which is more work and more uncertain.

Enforcement capacity varies by member state. Market surveillance authorities are at different stages of readiness. That is not a reason to assume non-enforcement — it is a reason to expect inconsistency in the first period.

Check the current state before making decisions. The details have moved and may move again.

what to do if you are in scope#

If you have not classified your systems, do that today. It is the prerequisite for everything else and it is a legal question with technical inputs. A number of organizations discover they are in scope for one feature they had not considered — a CV screening tool, a proctoring feature, a creditworthiness proxy.

Documentation is the bulk of the work. Training data provenance, validation methodology, known failure modes, intended use and misuse. Most engineering teams do not have this written down and reconstructing it is slower than producing it as you go.

Logging is a technical requirement. Automatic recording of operation, retained, with enough detail to trace a decision. If your system does not do this, it is engineering work with a deadline that has passed.

Human oversight is a design constraint, not a checkbox. "A person reviews the output" is insufficient if the person cannot meaningfully understand, question, or override it. Surfacing confidence, explaining the inputs that drove a decision, and making override easy and recorded are interface requirements.

if you are not in scope#

Most software is not. The transparency obligations — disclose that users are interacting with an AI system, label synthetic media — apply much more broadly and are much lighter.

The thing worth doing regardless of scope: know which category you are in, and write down why. "We assessed this and concluded it is not high-risk because X" is a one-page document that saves an enormous amount of time when someone asks, and someone will ask.

the honest assessment#

The Act is the most comprehensive AI regulation any jurisdiction has attempted. It has real criticisms — compliance cost falls hardest on small companies, the risk categories map imperfectly onto how systems are actually built, and the standards process has lagged the deadlines.

It is also law, it has extraterritorial reach, and penalties scale with global turnover.

The organizations that started classification work a year ago are in reasonable shape today. The ones that waited for clarity are discovering that regulatory clarity tends to arrive after the deadline rather than before it, which is a lesson that generalizes well beyond this particular Act.

what stuck#

Mechanical refactors at scale. The clearest win, by a wide margin. Renaming a concept across four hundred files, migrating a deprecated API, converting a pattern used everywhere. Verifiable, tedious, and exactly what the tools are good at.

The important second-order effect: refactors that were too expensive to do now happen. A codebase where cross-cutting cleanup is affordable is a meaningfully better codebase, and that is a permanent improvement rather than a productivity number.

Working in unfamiliar territory. A language you do not know, a framework you have not used, an API you have never touched. The median output in an unfamiliar domain is better than your first attempt, and reading it teaches you the idioms.

This is the use case I would defend most strongly and it is discussed least.

Test generation from a specification. Not "write tests for this function" — that produces tests that assert the implementation. But "here is the behavior, write tests that verify it" works well and it inverts the effort in the right direction.

Investigation. Reading logs, bisecting history, tracing a call path, summarizing a large diff. Parallelizable, cheap, and it saves the expensive resource, which is your attention.

Repository-level instruction files. AGENTS.md and its equivalents became standard practice, and the discipline of writing down how your project actually works improved documentation for humans as a side effect.

what did not stick#

Fully autonomous feature development. The demo works. The real version produces a plausible implementation of a subtly different feature, because the requirements that live in someone's head were never written down.

Agent fleets at high concurrency. The generation scales; the review does not. Two to three concurrent agents with one reviewer turned out to be the practical limit, and the constraint is entirely on the human side.

Orchestration frameworks. Absorbed into the models, as function-calling libraries and JSON-repair libraries were before them. The durable layer was never orchestration.

"Just describe it and it builds." For anything with design decisions, the description that is precise enough to produce the right result is approximately as long as the code, and writing it is the same work.

what actually changed about the job#

Review is the bottleneck, permanently. Generation got roughly two orders of magnitude cheaper. Verification got no cheaper at all. Everything downstream follows from that asymmetry and nothing in two years has changed it.

Tests became the primary artifact. If the implementation is cheap and verification is expensive, effort moves to specification. The teams getting the most out of these tools are the ones with strong test suites, and the correlation is not subtle.

Type systems got a promotion. Every constraint the compiler checks is verification you do not perform by reading. Teams that were ambivalent about strict typing became evangelists, and the reason is always the same: it catches the class of error generated code produces most.

Small diffs became non-negotiable. A machine can produce two thousand lines effortlessly. Accepting it is not a favor to anyone.

The skill that separates people is judgment, not speed. It always was. It is now the only thing, and the gap between engineers who can tell when output is wrong and engineers who cannot is much more visible than it was.

the thing still unresolved#

The apprenticeship problem.

The judgment that makes a senior engineer valuable was acquired by writing a lot of code badly and then debugging it. That work is being automated. Nobody has a replacement for how the next generation acquires it, and junior hiring contracted sharply during exactly the period when the training mechanism was being removed.

I have written about this several times and I still do not have an answer beyond: deliberately do the hard part yourself sometimes, review generated code carefully as a learning exercise, and hire juniors anyway.

That is a partial answer to a structural problem and I am not satisfied with it.

the honest summary, two years in#

These tools are genuinely useful and the useful envelope is narrower and more specific than either the enthusiasts or the skeptics claimed.

They are excellent at bounded, verifiable, tedious work. They are unreliable at anything requiring judgment about what should be built. They multiply output and do not multiply throughput, because throughput is limited by review.

The engineers getting the most from them are the ones who were already good at specifying problems precisely and at telling when something is wrong. That is not a new skill and it was never evenly distributed.

Which is roughly what every previous tooling revolution did: raised the floor, moved the bottleneck, and made expertise more valuable rather than less.

Enough time has passed to say what actually happened rather than what everyone predicted.

what changed#

The norm inverted. Before, crawling was permitted by default and robots.txt was a request. Now, for a large share of the web, crawling is denied by default and access is a negotiation.

That is a genuine structural change to how the web works and it happened through one company's configuration default rather than through any standards process, legislation, or public deliberation.

Licensing deals concentrated. Large AI companies negotiated bulk access with large publishers. That was always the likely outcome: the parties with lawyers and leverage made arrangements, and the arrangements are private.

Small publishers got very little. The pay-per-crawl mechanism works, technically. The revenue for a site with modest traffic is negligible — the arithmetic never supported anything else. The publishers who most needed a new economic model got the one that pays least.

Non-commercial crawling got harder. Academic researchers, archivists, and independent search projects have no licensing budget and no negotiating position. Carve-outs exist and are discretionary, which means the ability to study the web is now something you apply for.

That is the outcome I was most worried about and it is the one that materialized most clearly.

what did not change#

Training data supply. The frontier labs have enormous existing corpora, licensed sources, and synthetic generation. The marginal value of newly crawled web text was already declining. Restricting it did not create the leverage publishers hoped for.

Traffic. Referral traffic from search to publishers continued its decline, driven by AI answers in search results, which is a completely separate mechanism from training crawlers and which blocking crawlers does nothing about.

This is the part that was most misunderstood at the time. The traffic problem and the training problem have different causes and blocking crawlers only addresses one of them — the one with less economic impact.

the thing to actually take from it#

Infrastructure defaults are policy. A configuration change at a chokepoint reshaped access to a large fraction of the web, with no process and no appeal.

That is not a criticism of the specific decision, which was popular and defensible. It is an observation about where power actually sits, and it generalizes: the entities that can change the web's behavior are the ones with concentration at a layer everyone depends on, and there are about five of them.

For your own site, the decision remains yours and it is worth making deliberately rather than accepting a default:

• Documentation sites frequently want to be in the training data. Being the thing the model knows about is worth more than the pageview you did not get.
• Original reporting and analysis has a stronger case for restriction.
• Anything you want found should still permit search crawlers, which are a different category and are frequently blocked by accident when people configure this.

Check what you are actually blocking. A meaningful number of sites blocked their own search indexing in the first months of this and did not notice for weeks.

the unresolved thing#

The web's economic model — publish freely, get traffic, monetize traffic — is breaking, and nothing has replaced it.

Crawler tolls are not the replacement; the arithmetic does not work at the scale of the actual web, where most content is made by people with no ability to negotiate anything.

Licensing deals are not the replacement either; they work for a few hundred large publishers and for nobody else.

I do not know what the replacement is. I am increasingly convinced that nobody does, and that the interval between the old model failing and a new one existing is going to be long and is going to be bad for the open web.

That is a genuinely pessimistic conclusion and I have not found a way around it in a year of thinking about it.

They are also where an unattended agent can do the most damage quietly. Here is the process that has worked.

why this is the sweet spot#

Mechanical refactors have exactly the properties agents are good at:

• Verifiable. The tests either pass or they do not.
• Repetitive. The same transformation, many times, which is where humans make mistakes from fatigue.
• Well-specified. You can state the transformation precisely.
• Boring. Nobody enjoys this work and nobody does it carefully after hour two.

And they have the property that makes human refactoring risky: the scale defeats attention. A person converting four hundred call sites will be careful for the first fifty.

the process#

1. Do ten by hand first.

Before writing any instruction, do a representative sample yourself. You will discover:

• The cases where the mechanical transformation is wrong.
• The variations you did not know existed.
• What the actual rule is, as opposed to what you thought it was.

This is the step people skip and it is the one that determines whether the whole thing works. You cannot specify a transformation you have not performed.

2. Write the transformation down precisely.

Not "modernize the error handling." The exact before and after, with the exceptions named:

Replace every call of the form:
    result, err := doThing(x)
    if err != nil { return nil, err }
with:
    result, err := doThing(x)
    if err != nil { return nil, fmt.Errorf("doing thing for %s: %w", x.ID, err) }

Exceptions:
- Do not change anything in internal/legacy/ (frozen).
- Do not change error handling inside deferred functions.
- If the error is already wrapped, leave it.

3. Establish the safety net before you start.

• Clean working tree, dedicated branch.
• Full test suite passing, with a recorded baseline.
• A way to check the transformation was applied correctly beyond the tests — a grep, a linter rule, an AST query.

4. Batch it.

Not four hundred files in one change. Twenty to fifty files per commit, grouped by module.

This matters for two reasons: a reviewable diff size, and the ability to bisect. If something is wrong, you want to know which batch introduced it.

5. Review the first batch line by line.

Every line. This is where you catch the systematic error, and a systematic error caught in batch one costs twenty minutes while the same error caught in batch twenty costs a day.

6. Spot-check subsequent batches, review the anomalies.

Once the pattern is verified, review by sampling — but read every diff that looks different from the pattern. Agent output that deviates from the established shape is where the interesting failures are.

7. Verify mechanically at the end.

# nothing left in the old form
rg 'return nil, err$' --type go | rg -v 'internal/legacy'

If the transformation is complete, the old pattern should not exist outside the exceptions. This catches the files that were silently skipped, which is a real failure mode.

where it goes wrong#

The agent "improves" things you did not ask about. It renames a variable while fixing the error handling. Individually reasonable, collectively it makes the diff unreviewable because you can no longer scan for the pattern.

Instruct explicitly: change only what was specified, nothing else.

Semantic drift across batches. Batch one wraps errors one way, batch fifteen does it slightly differently, because the context is different and the model made a different reasonable choice.

Fix: put the exact target form in the instruction file, with examples, and check for consistency at the end with a grep.

Tests pass and behavior changed. The most dangerous case. Your tests did not cover the path that broke.

This is why the mechanical verification in step seven matters. It checks the transformation, not the behavior, and it catches things tests do not.

Silent skips. The agent processes 380 of 400 files and reports success. The twenty it skipped are the ones with unusual structure, which are the ones most likely to matter.

Always count. Always verify the remainder is empty.

the honest assessment#

For this class of work the leverage is real and large. A refactor that would have been a week of tedium — and would therefore never have happened, which is why the codebase has the problem — becomes an afternoon.

That last part is the underrated benefit. The refactors that get done are the ones that are cheap enough to do. Lowering the cost means more of them happen, and a codebase where the cross-cutting cleanup actually gets done is a meaningfully better codebase.

Just count the files at the end.

the on-device strategy, holding#

Apple's position has been consistent and, I think, correct for their situation:

• A small model on device, free, private, offline, exposed to third-party apps through a constrained system API.
• A larger model available for the cases the small one cannot handle.
• Routing handled by the system.
• Privacy as the differentiating property rather than raw capability.

That is not going to win a benchmark comparison and it was never trying to. It is going to win on the axis where Apple competes, which is a coherent product where the default behavior is the one most users want.

For developers, the practical implication has not changed: design features so the small model handles the common case.

Concretely — the 90% that the on-device model can do is free, instant, and works on a plane. The 10% that needs escalation costs money and needs a network. Getting that split right is the engineering, and it is a different skill from prompt design.

the Swift trajectory#

Swift continues its expansion beyond Apple platforms — server-side, embedded, WebAssembly, cross-platform tooling. The concurrency model's strict checking continues to produce a language that catches data races at compile time, which is a genuinely valuable property that the migration cost has made contentious.

The honest state: strict concurrency checking is correct and it is a real migration burden for existing codebases, and the error messages when you get it wrong are still harder to act on than they should be.

Swift is a better language than it gets credit for outside the Apple ecosystem and its adoption outside that ecosystem remains limited, mostly for reasons of tooling gravity rather than language quality.

the platform policy question#

The regulatory pressure on app distribution and payments continues, differently in different jurisdictions, with the result that the rules now vary by region in ways that are genuinely confusing for developers to comply with.

I do not have a clean position on the substance. What I will say is the practical part:

If you ship an app with any monetization, you now have jurisdiction-specific compliance work, and the rules are still moving. Budget for it, watch the changes, and do not assume a policy you read last year is current.

The broader observation, which is not about Apple specifically: the era of one global set of platform rules is over. Every major platform now operates under different obligations in the EU, the US, and several other markets, and that fragmentation is going to increase rather than resolve.

the things worth actually adopting#

Filtering out the keynote, the items that will matter to a working developer:

Anything that reduces app size or launch time. These are the metrics that correlate with retention and they get the least stage time.

Testing and debugging improvements in the toolchain. Consistently the most valuable and least covered part of every WWDC.

Deprecation notices. The most important announcements at any platform conference are the ones about what is going away, and they are never in the keynote. Read the release notes.

the honest assessment#

Apple ships coherent, well-integrated platforms with genuinely good on-device capability and a privacy posture that is a real product differentiator rather than only marketing.

It also operates the developer relationship with less flexibility than any comparable platform, and the regulatory environment is the mechanism by which that is being adjusted rather than any change of heart.

Both of those have been true for a decade and neither is changing this year.

This is the highest-return day of engineering available to anyone building on models, and a surprising number of teams have not done it.

what it is not#

Not MMLU. Not a leaderboard. Not "vibes after twenty prompts."

Public benchmarks tell you which models are worth testing. They do not predict performance on your task, because your task is not in them and because contamination is universal.

what it is#

Fifty to two hundred examples from your actual production traffic, with expected outputs or a grading rubric, run automatically, producing a number.

evals/
  cases/
    001-refund-request.json
    002-ambiguous-address.json
    ...
  run.py
  results/
    2026-05-27-model-a.json

Each case:

{
  "id": "042",
  "input": { "ticket": "my order never arrived and I want my money back" },
  "expect": { "category": "refund", "urgency": "high", "needs_human": false },
  "notes": "the word 'never' should not trigger the fraud path"
}

That is it. The whole thing is a test suite where the assertions are fuzzier.

where the cases come from#

Your production failures. This is the best source by a wide margin. Every time the system gets something wrong, that becomes a case. Your eval set grows into a precise map of your problem's difficulty.

Set this up as a workflow: a thumbs-down in the product, or a support escalation, creates a candidate case that someone reviews and adds.

Your edge cases. The weird inputs. The empty ones. The ones in another language. The adversarial ones. The ones with an injection attempt.

A stratified sample of normal traffic. So you notice when a change breaks the common case while fixing an edge case.

Cases with no correct answer. Where the right behavior is to refuse, escalate, or ask a clarifying question. Models are frequently bad at this and it is rarely tested.

grading#

Three approaches, and you will use all three.

Exact or structural match. For classification, extraction, and structured output. Cheap, deterministic, unambiguous. Use it wherever you can.

Programmatic checks. For generated code: does it compile, do the tests pass. For SQL: does it run, does it return the right shape. This is the strongest form of grading and it is available more often than people realize — if you can verify mechanically, do.

Model-as-judge. For open-ended output. A second model grades against a rubric.

Use it carefully:

• Write a specific rubric, not "is this good." Score each dimension separately.
• Validate the judge against human ratings on a sample. If the judge disagrees with you, the judge is wrong and the rubric needs work.
• Use a different model than the one being evaluated, or at minimum be aware of self-preference bias, which is well documented and large.

the metrics#

Accuracy on your set, obviously.

Cost per case. Tokens in and out, at current prices. Track this — a model that is 2% better and 4× the cost is usually the wrong choice.

Latency, at percentiles. p50 and p95. Reasoning models have high variance and the average hides it.

Failure mode distribution. Not just how many wrong — how wrong. A model that fails by refusing is very different from one that fails by confidently fabricating, and the aggregate score treats them identically.

the workflow#

python evals/run.py --model model-a --model model-b --parallel 8

Output a table. Commit the results. Diff them across runs.

Run it:

• When any model ships. Within an hour of the announcement, you know.
• When you change a prompt. Prompt changes are code changes with no type system and no compiler; the eval is your only regression check.
• On a schedule. Providers update models behind stable names. Behavior drifts. You want to know from your dashboard, not from your support queue.

That last one catches something most teams never notice: the model you deployed against is not the model serving your traffic today.

the one that matters most#

Version your prompts and store the eval result with them.

A prompt is production configuration. It should be in version control, reviewed, and associated with a measured quality number. "Someone edited the prompt and something got worse three weeks ago" is a debugging session that should not be possible.

the payoff#

Once this exists:

• Model migrations are an afternoon.
• Prompt changes are safe to make.
• You can argue about model choice with data instead of anecdotes.
• You detect provider-side drift.
• Onboarding a new engineer to the AI parts of your system means handing them the eval set, which is the best available documentation of what the system is supposed to do.

It is a day of work. It is the single highest-leverage day available in this space and it has been for three years.

the distribution advantage, compounding#

The thing no competitor can replicate is that Google can ship a capability into products that billions of people already open daily, on launch day.

That is worth more than a benchmark lead and it is becoming more visible each year. A model that is marginally better but reaches users through a signup flow loses to a model that is marginally worse and is already in the search box.

The strategic implication for everyone else — including the other frontier labs — is that raw capability is not the competition anymore. Distribution, price, and integration are.

the search question, again#

Every year this conference makes the same thing more true: informational queries are increasingly answered on the results page rather than by sending someone to a site.

For anyone who publishes on the web, the consequences are now well past theoretical:

• Referral traffic to informational content keeps falling. This is measurable and it is not recovering.
• Your documentation is being summarized by a system you do not control, and users are acting on the summary.
• The feedback loop is broken. You cannot see what people asked, what answer they got, or whether it was right.

I do not have a satisfying answer. The mitigations available to an individual project are marginal: write documentation that is hard to summarize badly, keep a machine-readable changelog, make error messages self-explanatory so they do not require a search at all.

The structural problem — that the economic model funding web content is being removed without a replacement — is not solvable by any individual publisher, and the people who could solve it have no incentive to.

the developer surface#

The genuinely useful announcements, as always, are the boring ones:

Model pricing and the cheap tier. The cost-per-capability at the small end continues falling, and Google's TPU position means they can price below what competitors renting accelerators can match. If you run high-volume inference, the arithmetic is worth redoing quarterly.

Longer context, better retention. Incremental and real. The practical question is always the degradation curve, not the maximum, and that continues to improve.

Agent tooling in the cloud. Same category everyone is building: runtimes, memory, identity, observability. Evaluate against a real problem rather than a demo.

the thing to actually do#

The recommendation has not changed in two years and I will keep repeating it because it keeps being right:

Have an eval set in your repository. Fifty examples from your real domain, with expected outputs, run against every candidate model.

Every conference like this produces a new model that is claimed to be better. With an eval harness, evaluating that claim for your workload takes an hour. Without one, it takes a week of impressions and you will get it wrong.

This is a day of setup that pays back on every model release, forever, and the number of teams that have done it remains surprisingly small.

the honest summary#

A very good model, deployed extremely well, in a company with structural advantages that are getting stronger.

Whether that is good for the web is a separate question, and I keep arriving at the same uncomfortable answer.

the strategy#

1. Meet developers where they are, including on other people's platforms.
2. Adopt other people's standards rather than inventing competing ones.
3. Open-source the layers where control is not worth the friction.
4. Monetize the cloud underneath.

Every Build for a decade has been an execution of that, and the cumulative result is that a company which was actively hostile to open source in 2005 is now the largest corporate contributor to it and owns the default editor, the default code host, and a large share of the developer toolchain.

the agent infrastructure#

The substantive announcements this year continue the theme of putting agent capabilities at the operating system layer rather than in an application: a permission model for what agents may reach, an identity model for agents acting on a user's behalf, and audit surfaces for what they did.

That is the right layer for it. The alternative — every application implementing its own agent permission model — produces exactly the inconsistency that made mobile permissions a mess for a decade before the platforms standardized.

The questions that matter and that a keynote cannot answer:

Granularity. "Filesystem access" is not a permission, it is a surrender. Does the model support "read from this directory for this task"?

Consent fatigue. If the prompts are frequent, users click through them, and the control is theater. The design problem is asking rarely and meaningfully.

Revocation and audit. Can a user see what an agent did and undo it? This is the part that is hardest and gets the least attention.

I will believe the security model when someone publishes an analysis of it, not when it is demonstrated on a stage.

the enterprise angle#

The genuinely differentiating position Microsoft has is that they can offer agent capabilities inside an enterprise's existing identity, compliance, and audit infrastructure.

That is worth more to a large organization than raw capability. An agent that works within the existing access control model, logs to the existing audit system, and is governed by the existing data policies clears procurement. An agent that requires a new trust boundary does not, regardless of how good it is.

This is the same advantage that won enterprise cloud and it is being applied identically.

what a developer should actually do with this#

If you build on Windows: the tooling story is genuinely good now — WSL, the terminal, winget, PowerShell 7 — and if your Windows support has been a grudging afterthought since 2018, it is worth revisiting.

If you build agent-adjacent products: design against the OS permission model rather than around it. Products that require users to disable platform protections do not get enterprise adoption.

If you are evaluating anything announced here: wait for the second version. Microsoft's first releases in a new category are consistently rough and consistently improved within a year. That is a reasonable pattern and it means the launch-day evaluation is not the useful one.

the pattern to watch#

The layer where the industry is currently fighting is not the model. It is the control plane for agents: who they are, what they may do, on whose behalf, with what audit trail.

Every platform vendor is building this. The one that becomes standard will have the same kind of position that identity providers have today, and it will be very durable.

That is the strategic story of the next three years and it is being fought in permission dialogs rather than benchmarks.

The first part will recover. The second part is a mistake, and it is the kind of mistake that is invisible for four years and then extremely expensive.

the argument for not hiring juniors#

Stated honestly, because it is not stupid:

A junior engineer's first-year output is mostly small, well-specified tasks — the CRUD endpoint, the test coverage, the bug in a file someone pointed them at. That work is now substantially automatable. Meanwhile the junior requires mentoring time from senior engineers, which is the scarcest resource.

So the ROI on a junior looks worse than it did. That reasoning is coherent.

why it is wrong#

Seniors come from juniors. There is no other supply. An organization that hires only seniors is free-riding on other organizations' training, and if everyone does it, the pipeline empties. This is a classic collective action failure and the industry is walking into it with open eyes.

The judgment that makes seniors valuable comes from doing the work. The ability to look at plausible code and know it is wrong comes from having written the wrong version and debugged it at 3 a.m. You cannot read your way to it and you cannot prompt your way to it.

If the apprenticeship stops, the next generation of senior engineers does not exist, and the current one retires.

Juniors are better at the new tools. Consistently, in my experience. They have no prior workflow to defend and they explore. A team of only senior engineers adopts new tooling slowly and grudgingly.

Mentoring makes seniors better. The engineer who has to explain why a design is wrong understands it better afterward. Teams with no juniors lose that forcing function and get sloppier about articulating their own reasoning.

what actually has to change#

The old model — hire a junior, give them small tickets for a year, gradually increase scope — does not work as well when the small tickets are automated. The model has to change, not the hiring.

Start them on reading, not writing. Give a new engineer a real system and a week to understand and explain it. Have them write the architecture document that does not exist. This builds the skill that actually matters now and it produces something useful.

Give them debugging, not features. Debugging is the skill that generalizes, that AI is least reliable at, and that cannot be learned from a course. Pair them on incidents. Give them the flaky test nobody wants.

Make them review agent output. Reviewing machine-generated code with a senior engineer walking through what is wrong with it is an extraordinarily efficient teaching mechanism. You get a stream of plausible-but-flawed code, which is exactly the training material you want and which used to be expensive to produce.

Require them to write the tests first. Specifying behavior before implementing teaches design, and it is the part of the workflow that has become more important rather than less.

Do not let them delegate the hard part. For the first year, some things get done by hand, deliberately, because the point is the learning rather than the output. Say this out loud so it does not feel like an arbitrary restriction.

the hiring signal that works now#

Traditional junior screens — implement this algorithm, complete this take-home — are substantially defeated and were never good predictors anyway.

What works better:

A code review exercise. Give them a pull request with three problems. Watch what they find and how they talk about it. This is the job.

A debugging exercise on a real repository. Failing test, thirty minutes, any tools they want including AI. Watch the process, not the outcome. Do they read the error? Form a hypothesis? Check it? Notice when the model's suggestion is wrong?

A conversation about something they built. Follow-up questions until you hit the edge of their understanding. Where that edge sits, and how they handle reaching it, tells you almost everything.

the case to make internally#

If you are arguing for junior headcount:

The cost of a junior is roughly a senior's partial attention for a year plus a below-market salary. The cost of a senior hire in three years, in a market where nobody trained anyone, is going to be considerably higher than it is now.

Every organization that stopped training in 2009 spent 2013 through 2016 paying enormous premiums for the engineers who had been trained elsewhere. It is the same trade and it is being made again.

The organizations that keep training through this period will have a meaningful advantage in five years, and it will be very hard to catch up to them quickly.

Here is where the line actually is now, with the reasoning rather than the conclusion, because the line keeps moving and the reasoning does not.

the four variables#

Whether to retrieve or to stuff the context depends on four things:

Corpus size. If everything fits, stuff it. "Fits" now means a large document set — a codebase, a product's full documentation, a year of one team's tickets. It does not mean an enterprise's entire document store.

Query pattern. If you ask many questions against the same corpus, caching the prefix makes stuffing cheap. If every query touches a different slice of a large corpus, retrieval wins.

Latency budget. Prefill on a very large context takes real time even when cached. If you need sub-second responses, a small retrieved context is faster.

Citation requirement. If you must show which source supported a claim, retrieval gives you that structurally. Stuffing requires trusting the model's attribution, which is less reliable than people assume.

the decision, concretely#

Stuff the context when: the corpus is under a few hundred thousand tokens, you query it repeatedly (so caching applies), and you want the model to see relationships between distant parts.

The clearest example remains a codebase. Retrieval over code performs worse than whole-repository context because code's meaning lives in the relationships between files, and chunking destroys exactly that.

Retrieve when: the corpus is large, queries are diverse, latency matters, or you need citations.

Do both when: you have a large corpus with a hot subset. Stuff the hot subset — the style guide, the schema, the core documents — and retrieve from the tail.

what actually changed: retrieval became a tool#

The important architectural shift is not about size. It is that retrieval moved from a preprocessing step to a tool the model calls.

Old shape:

query → embed → search → rerank → stuff top-k → generate

One retrieval, before generation, with k fixed by you.

New shape:

query → model reasons → calls search tool → reads results
      → reasons → searches again with a better query → reads
      → generates answer with citations

The model decides what to search for, evaluates whether the results answered the question, and searches again with a refined query if not.

This is dramatically better and it is better for a specific reason: the first query is usually not the right query. A user asks about "the timeout issue." The right search is for the specific component's retry configuration, which you only know to search for after reading something else.

A fixed one-shot retrieval cannot do that. An agent with a search tool can.

what this means for your pipeline#

Embeddings matter less. When the model can iterate, a mediocre first retrieval is recoverable. When you had one shot, embedding quality was everything.

Keyword search came back. Hybrid search — BM25 plus vectors — consistently outperforms pure vector search, and for a model that can iterate, plain keyword search is often sufficient. Exact terms matter: error codes, function names, product names. Embeddings are bad at exact match and always were.

If you built a pure-vector pipeline in 2023, adding keyword search is probably your biggest available quality improvement.

Chunking matters less, and differently. Instead of chunking for retrieval, store documents whole and retrieve whole documents when they fit. Chunk only what is too large, and chunk on structural boundaries — sections, functions, headings — rather than by token count.

Metadata filtering matters more. The model can specify constraints: this project, this date range, this author. Filtering is cheap, precise, and it is frequently what the query actually needed. Make sure your index supports it.

the practical setup#

For most applications:

1. Postgres with pgvector plus full-text search. One system, hybrid search, metadata filters, and joins to your relational data.
2. Expose search as a tool, not as a preprocessing step. Let the model iterate.
3. Return whole documents where they fit; chunk on structure where they do not.
4. Cache the stable prefix — the instructions, the schema, the core reference material.
5. Measure retrieval quality separately from answer quality. If the answer is wrong, you need to know whether the right document was retrieved. Most teams cannot answer that and debug blind.

That last one is the single most useful piece of instrumentation in a retrieval system and almost nobody has it.

the thing I got wrong#

I said the RAG infrastructure category was solving a temporary problem. The capability got absorbed, as predicted. The infrastructure repositioned rather than disappearing, which I did not predict.

That is the third time I have watched this exact pattern and failed to apply the lesson. Infrastructure around a model limitation rarely dies. It moves to whatever the model still cannot do, which is usually one layer out.

Having run this for a while at meaningful volume, here is what actually happens.

what works#

Mechanical, well-specified, verifiable work. This is not a hedge, it is the finding. The tasks where fleets genuinely deliver:

• Dependency upgrades across many services.
• Migrating a deprecated API call across a large codebase.
• Adding tests to modules with poor coverage.
• Converting between formats or frameworks with a mechanical mapping.
• Fixing a class of lint or type error across a repository.

What these share: a clear acceptance criterion the agent can check itself, a bounded scope, and no design decisions.

For this category the leverage is real and large. Work that would have been a week of tedium becomes an afternoon of review.

Investigation in parallel. Spawning several agents to independently investigate a bug from different angles — read the logs, bisect the history, read the related code, reproduce it — and reading all four reports is genuinely faster than doing them serially. The agents are cheap; your attention is not; parallelizing the cheap thing is correct.

what does not work#

Anything requiring shared context. Five agents working on related parts of a system produce five changes that each make sense and collectively do not. They duplicate helper functions. They pick different names for the same concept. They each add a slightly different retry wrapper.

You end up with a merge problem that is worse than the original work, because the conflicts are semantic rather than textual — the diffs apply cleanly and the result is incoherent.

Anything with genuine design decisions. An agent given an ambiguous task will resolve the ambiguity confidently and silently, and it will pick a reasonable option that may not be the one you wanted. Five agents will each pick differently.

Anything where the acceptance criterion is subjective. "Improve the error messages" produces five different notions of improved.

the actual bottleneck#

Review, exactly as predicted, and worse than predicted.

Five pull requests per hour is not a throughput improvement if you can meaningfully review two. What you get is a queue, and queue pressure degrades review quality in a way that is invisible in the metrics and visible in the defect rate a quarter later.

The honest arithmetic: your throughput is min(generation rate, review rate), and review rate did not change.

what actually helps#

Make the agent produce a reviewable artifact, not just a diff. A summary of what it did, what it decided, and what it was unsure about. Reviewing "I chose to use the existing retry helper rather than adding a new one, and I left the timeout at 30s because the surrounding code does" is dramatically faster than inferring that from a diff.

Batch related work into one agent, not many. If five tasks touch the same module, one agent doing all five produces a coherent change. Five agents produce five incoherent ones. Parallelism across independent work only.

Invest heavily in verification. A strong test suite is what makes review cheaper, because you are reviewing design rather than correctness. Teams with weak tests get no benefit from agent fleets — they just get more code to verify by reading.

Cap concurrency at your review capacity. Running more agents than you can review does not help. It produces a backlog that goes stale and gets abandoned, which is worse than not starting.

Reject on size. A machine can generate a 2,000-line diff effortlessly. Say no. Ask it to split.

the number#

For our work, the useful concurrency turned out to be two to three agents on independent tasks, with one person reviewing. Beyond that, quality degraded and the extra output was not landing.

That is a real multiplier and it is much less than the demos suggest, and the constraint is entirely on the human side.

the thing I would tell someone starting#

Do not start with a fleet. Start with one agent, on the mechanical work you have been putting off, and measure whether the output actually lands.

If your review process cannot absorb one agent's output, adding four more is solving the wrong problem.

the shape of it#

Track any capability benchmark across model sizes over time and the pattern is consistent: the frontier improves steadily, and the small-model tier improves faster. The gap between "the best model available" and "a model that costs 2% as much" has been compressing.

The mechanisms are known:

• Distillation. Training small models on the outputs of large ones transfers a surprising amount of capability. The recipes are public.
• Better data. Curated, synthetic, and filtered training data improves small models disproportionately, because they have less capacity to waste on noise.
• Mixture of experts. Total parameters for knowledge, active parameters for cost. The economics of a 30B-total/3B-active model are close to a 3B dense model, and the quality is much closer to a 30B dense one.
• Reasoning post-training. RL on verifiable rewards works on small models. A small model that thinks can outperform a large model that does not, on the tasks where thinking helps.

what it means in practice#

Go through a typical production AI workload and categorize the calls:

The first column is most of the volume. The second is most of the value per call and a small fraction of the calls.

If you are running everything through a frontier model, you are likely spending a large multiple of what you need to, and you probably have not measured which calls actually need it.

the architecture#

                  ┌─→ small model ──→ verifier ──→ ok? → done
request → route ──┤                              └→ no  ─┐
                  └─→ frontier model ←──────────────────┘

Three components and each matters:

The router. Simpler than people build. Input length, detected task type, and a handful of keywords gets you most of the way. A small model as a classifier works too. Do not build a sophisticated router before you have measured that a simple one is insufficient.

The verifier. This is the part that gets skipped and it is what makes the architecture safe. Small models fail confidently. A cheap check — schema validation, a range assertion, running the generated code, a second model asked "is this answer plausible" — catches the failures that would otherwise reach production silently.

The escalation path. When verification fails, retry with the frontier model. Track the rate. If it climbs, something changed.

the instrumentation that matters#

Three metrics, on a dashboard:

1. Escalation rate. The fraction of requests that fall through to the expensive path. This is the health metric for the whole design.
2. Cost per successful task, not per token. Tokens are an implementation detail; tasks are the unit you care about.
3. Quality on your eval set, by tier. Run both tiers against the same evaluation regularly. When a new small model ships, you will know within an hour whether you can move work down.

the second-order effect#

When inference is nearly free, you use more of it.

Things that were not worth a model call become worth it: normalizing an input, double-checking an output, generating three candidates and picking one, enriching a record, summarizing an intermediate step.

The systems being built now use dramatically more model calls per user action than the systems from two years ago, at lower total cost. That is Jevons operating on schedule, and it is why "our AI costs went down per token and up in total" is the normal experience.

the thing to watch#

The interesting question is not whether small models keep improving. They will.

It is whether the frontier keeps being worth its premium. If the gap on practically-relevant tasks keeps compressing, the frontier tier's addressable workload shrinks to a narrow band of genuinely hard problems.

That is a much smaller business than the one being priced today, and it is the scenario that ought to worry the labs more than competition does.

The pattern is consistent enough to be predictive, which makes it worth naming.

wave one: fine-tuning#

The thesis (2023): general models are generic. Fine-tune on your domain data and you get a model that is specifically good at your problem and that competitors cannot replicate.

What happened: base models improved faster than fine-tunes could keep up. A fine-tuned model from six months ago was worse than the new base model with a good prompt. Every fine-tune had to be redone on every model release, which is a treadmill.

Where it settled: fine-tuning is genuinely valuable for narrow, stable, high-volume tasks — classification into your specific taxonomy, output in your specific format, a small model matching a large model's behavior on one task. It is not a moat and it is not a product strategy.

wave two: RAG#

The thesis (2023-24): the model does not know your data. Build a retrieval pipeline — chunk, embed, index, retrieve, rerank — and you have a defensible system built on proprietary knowledge.

What happened: context windows grew by two orders of magnitude, long-context quality improved, and prompt caching made large contexts economical. A large fraction of naive RAG got replaced by putting the documents in the prompt.

Simultaneously, the pipeline components commoditized. Embedding models became interchangeable. Vector search became a feature of every database rather than a product.

Where it settled: retrieval did not go away. It moved. Retrieval is now a tool the model calls rather than a preprocessing step, and it is genuinely necessary at large corpus sizes, where citation is required, and where cost or latency rules out large contexts.

The infrastructure repositioned rather than dying, which is what usually happens.

wave three: agent frameworks#

The thesis (2024-25): models cannot plan reliably. Build the orchestration — task decomposition, tool routing, retry logic, state management — and own the layer that makes agents work.

What happened: the models absorbed it. In-context tool use during reasoning removed the need for an external loop. Native parallel tool calling removed the need for a dispatcher. Memory tools and context editing removed the need for external state management.

Each capability the framework provided became a model feature within about a year.

Where it settled: in progress, but the shape is clear. Orchestration frameworks are converging on thin conveniences. The durable parts are evaluation, observability, and the domain-specific policy that no model will ever have.

the pattern#

Every wave follows the same arc:

1. The model has a limitation.
2. Companies build infrastructure to work around the limitation.
3. The limitation gets fixed in the model.
4. The infrastructure either finds a new position or disappears.

The consistent error is building on a gap rather than on an asset. A gap is temporary by construction — the labs are actively working to close it, with more resources than you have.

what has actually been durable#

Across all three waves, the same things kept their value:

Proprietary data. Not "we have documents" — everyone has documents. Data that is genuinely yours: production logs, customer interactions, labeled outcomes, domain expertise encoded as examples. Nobody can buy it and no model was trained on it.

Evaluation specific to your task. The company that knows, precisely, how well a system performs on their actual problem can adopt a new model in a day. The one that does not spends a month on vibes. That gap compounds every release cycle.

Distribution and workflow integration. Being where the user already works. The most boring answer and the most durable.

Domain constraints. The rules, regulations, edge cases, and institutional knowledge that make a generic capability into a usable product. This is unglamorous and it is the actual work.

Trust. Security posture, compliance, reliability, support. Enterprises buy this and it takes years to build.

the test to apply#

Before building on top of a model limitation, ask: if this limitation disappeared next quarter, what would I have left?

If the answer is "nothing," you are building a bridge over a river that is being drained.

If the answer is "the data, the evaluations, the integrations, and the customer relationships," build it, and expect to throw the bridge away.

the structure, briefly#

The Act is risk-tiered rather than technology-specific.

Prohibited (in force since February 2025). Social scoring, certain biometric categorization, emotion recognition in workplaces and schools, untargeted facial image scraping, and manipulative techniques exploiting vulnerabilities.

High-risk (August 2026 for most). Systems used in employment decisions, education access, credit scoring, essential services, law enforcement, migration, justice administration, and as safety components in regulated products.

Limited risk. Transparency obligations. Tell people they are talking to an AI. Label synthetic media.

Minimal risk. Most software. No specific obligations.

General-purpose AI models. Separate obligations that began August 2025 — documentation, copyright policy, training data summaries, and additional requirements for models above a compute threshold.

the classification question#

Most of the compliance work is answering "is my system high-risk," and the answer is less obvious than people assume.

The high-risk categories are defined by use, not by technology. A simple scoring model used to filter job applicants is high-risk. A sophisticated transformer used to suggest recipes is not.

Specific things that catch people:

• Recruitment tooling. Anything that screens, ranks, or filters candidates. This includes a keyword filter, not just an ML model.
• Employee evaluation. Performance assessment, task allocation, monitoring that affects work relationships.
• Creditworthiness. Including proxies for it.
• Educational assessment. Grading, admission, proctoring.
• Safety components in products already covered by EU product safety legislation — which is a long list including machinery, medical devices, and vehicles.

If any of those describe your product, you have obligations, and if you are a provider (you put it on the market) rather than a deployer (you use it), the obligations are substantial.

what high-risk actually requires#

• A risk management system, documented and maintained across the lifecycle.
• Data governance — documented training, validation, and test data, with attention to bias and representativeness.
• Technical documentation sufficient for a regulator to assess conformity.
• Automatic logging of the system's operation.
• Transparency to deployers about capabilities and limitations.
• Human oversight designed in — a person must be able to understand, intervene, and override.
• Accuracy, robustness, and cybersecurity appropriate to the purpose.
• Conformity assessment before market placement, and CE marking.
• Registration in an EU database.
• Post-market monitoring and serious incident reporting.

That is a quality management system, not a checklist. The nearest analogue is medical device regulation, and it is deliberately modeled on it.

the practical advice#

Classify now. Before anything else, determine which of your systems fall in scope. This is a legal question with technical inputs and it should involve both. A lot of organizations discover they are in scope for one feature they did not think about.

Documentation is the bulk of the work. Most engineering teams do not document their training data provenance, their validation methodology, or their known failure modes. The Act requires all of it. Starting that documentation now is cheaper than reconstructing it later.

Logging is a technical requirement with a deadline. Automatic recording of system operation, retained, with enough detail to trace a decision. If your system does not do this, that is engineering work, and it is not trivial to add.

Human oversight is a design constraint. "A person reviews the output" is not sufficient if the person cannot meaningfully understand or override it. Designing for genuine oversight — surfacing confidence, explaining inputs, making override easy and recorded — affects your interface.

Watch for timeline adjustments. There has been active discussion about simplification and phasing, and the details have moved before. Track it, and do not use possible delay as a reason to defer classification, which is the part with the longest lead time.

the honest framing#

Whatever you think of the Act's design — and there are serious criticisms about compliance cost for small companies and about whether risk categories map cleanly onto real systems — it is law, it has extraterritorial reach, and the penalties are proportional to global turnover.

For most developers building most software, none of this applies. For the ones it applies to, six months is not very long.

The structural analogy is exact. The resolution is not available.

the structural analogy#

SQL injection: the query and the data travel in the same string. The database parses the combined string and cannot tell which characters came from the developer and which came from the user. A user who writes SQL-shaped input has their input executed as SQL.

Prompt injection: the instructions and the data travel in the same context. The model processes the combined context and cannot tell which tokens came from the developer and which came from a web page, a document, or an email. Content that is instruction-shaped gets treated as an instruction.

Same problem. Mixed channels.

why the fix does not transfer#

SQL injection was solved by parameterized queries. The query is parsed into a plan first, then values are bound into slots. The value cannot become part of the query structure because the structure was already fixed before the value arrived.

That works because SQL has a formal grammar. There is a parse tree. There is a crisp, machine-checkable boundary between "this is syntax" and "this is a literal."

Natural language has no such boundary. There is no parse step that separates instruction from data, because the distinction is semantic, not syntactic. A model's entire function is to interpret meaning from text, and "ignore your prior instructions" means what it means regardless of which part of the context it appeared in.

You cannot parameterize a prompt. There is nothing to parameterize.

what has been tried#

Delimiters. Wrap untrusted content in tags and instruct the model to treat it as data. Helps somewhat. Defeated by content that includes the closing delimiter, or that argues persuasively that it is an exception.

Instruction hierarchy. Train the model to weight system instructions above user content above tool results. This is a genuine improvement and the major labs have all done it. It raises the bar and does not eliminate the attack, because it is a learned preference rather than an enforced boundary.

Classifiers. Detect injection attempts before they reach the model. Works on known patterns. Attackers iterate faster than classifiers update, and the false positive rate on legitimate content is a real product cost.

Separate models. One model handles untrusted content and cannot call tools; another handles privileged actions and never sees untrusted content. This actually works and it is architectural rather than probabilistic.

That last one is the direction.

the architecture that holds#

Stop trying to make the model safe. Make the system safe, assuming the model will be compromised.

Separate contexts by trust level. A session that reads arbitrary web content does not have credentials. A session with credentials does not read arbitrary web content. If information must cross, it crosses through a narrow, typed, validated channel — not by putting both in the same context.

Enforce permission outside the model. The model does not have access; it requests an action, and a separate system decides whether it is permitted based on the user's actual authorization. The model's opinion about what it should be allowed to do is not an input to that decision.

Confirm all egress. Any action that sends data outward — an email, an HTTP request, a file write to a shared location — requires explicit approval. This is the control that bounds the damage when everything else fails, because exfiltration is the attacker's goal.

Make every capability narrow. Not "filesystem access" but "read from this directory." Not "send email" but "send email to addresses in this thread." An agent's permissions should be scoped to the task, granted per-task, and revoked after.

Log everything and monitor for anomaly. You will not prevent every injection. Detecting one within minutes is the difference between an incident and a breach.

the uncomfortable conclusion#

For an agent operating on untrusted content with access to sensitive systems, there is currently no configuration that is safe in the way parameterized queries are safe.

There are configurations that are acceptably risky for a given use case, and that judgment requires knowing what the agent can reach and what happens if it is turned against you.

The industry is deploying this capability broadly anyway, on the theory that mitigations will outpace attacks. That theory has a poor historical record — SQL injection was not solved by better filtering, and XSS was not solved by better escaping heuristics. Both were solved by architectural changes that made the unsafe thing impossible to express.

Nobody has the architectural fix here yet. Until someone does, the safety of any deployment is a function of how carefully someone drew the boundaries, and most deployments have not drawn any.

Generation got roughly two orders of magnitude cheaper. Verification did not get cheaper at all.

Everything else — the review bottleneck, the arguments about junior hiring, the sudden importance of tests, the unease people cannot articulate about agent-written code — is downstream of that asymmetry.

why verification did not get cheaper#

You might expect it to. Surely a model can check code as well as it can write it?

It can, sort of, and not in the way that matters. Three problems:

The checker has the same blind spots as the writer. A model that does not know your system produced code with an assumption baked in. The same model reviewing it holds the same assumption and does not see it. Independent errors can be filtered by voting; correlated errors cannot.

Verification requires knowing what correct means. That is a property of your domain, your users, your obligations — not of the code. A model can verify that code does what it says. It cannot verify that what it says is what you needed, because that information was never written down anywhere.

Confidence is decoupled from correctness. Generated code is fluent. Fluency is the signal humans use to assess competence, and it has been severed from correctness. Reviewing text that reads well but is subtly wrong is much harder than reviewing text that reads badly, and everyone underestimates this.

what this changes#

Tests stop being a chore and become the primary artifact. If verification is the constraint, then anything that makes verification automatic is worth enormous investment. A test suite is executable verification. It was always valuable; it is now the thing that determines your throughput.

The practical implication: write tests first, review those carefully, and let the implementation be cheap. Invert the effort. The specification is the part you have to get right; the code is increasingly a derived artifact.

Type systems get a promotion. Every constraint the compiler can check is a constraint you do not have to verify by reading. Languages with expressive type systems and strong static guarantees are worth more than they were, because they convert human verification into machine verification.

I have watched teams that were ambivalent about strict TypeScript become evangelists in the space of a year, and the reason is always the same: it catches the class of error that agent-written code produces most often.

Small diffs become non-negotiable. Verification cost scales superlinearly with diff size, because the number of interactions you have to reason about grows faster than the lines. A machine can produce a 2,000-line change effortlessly. Accepting it is not a favor to anyone.

Property-based testing gets its moment. If you can state an invariant, you can verify a very large input space cheaply. Invariants are exactly the kind of thing that a human should specify and a machine should check. This technique has been niche for twenty years and it is the right shape for this moment.

Observability becomes a verification tool. If you cannot verify everything before deploy, you verify in production: feature flags, canaries, and metrics that detect wrongness quickly. Fast detection is a substitute for perfect pre-verification, and it is often a better investment.

the skill that matters#

The engineers getting genuine leverage out of these tools all share one trait, and it is not prompting.

It is that they can look at a plausible-looking piece of code and say "that retry loop will hammer the upstream on a 429" or "that will deadlock under concurrent writes" or "that assumes the list is sorted and nothing sorts it."

That skill comes from having debugged those exact failures. It does not come from reading about them.

Which produces the uncomfortable question everybody is circling: the traditional way to acquire it was to write a lot of code badly and then fix it. If that apprenticeship is being automated away, what replaces it?

the answer I have, which is partial#

Deliberately do the hard part yourself sometimes.

Not out of nostalgia. Because the judgment is the product, and the judgment is built by doing. If you delegate every debugging session, you will be worse at debugging in two years, and debugging is the thing you are being paid for now.

Pick the gnarliest bug of the week and do it by hand. Read the code the agent wrote in the module you own, all of it, once a month. Write the tricky concurrency code yourself and let the machine do the CRUD.

That is not a workflow recommendation for efficiency. It is a training regimen, and treating it as one is the honest framing.

the actual trend#

Every category at this show is now constrained by thermals and power delivery rather than by compute.

Laptops with neural accelerators that cannot sustain their rated throughput for more than a few minutes in a thin chassis. The TOPS number on the sticker is a peak, and the peak lasts as long as the thermal mass does.

Handhelds where battery life is the entire product decision and every added capability is a subtraction from it.

Home devices that want to run models locally and cannot, because the power envelope of something that sits on a shelf is a few watts.

Desktops where the power supply recommendations for a current-generation GPU have crept past what a lot of household circuits comfortably deliver alongside everything else in the room.

This is what a computing era looks like when it hits a physical wall. The interesting engineering for the next several years is efficiency, not capability — which is historically when the best engineering happens.

what to actually take from the show#

On-device inference is a spec sheet lie in the thin-and-light category. If you are shipping software that assumes a local NPU, test on a machine that has been running for twenty minutes, not on a cold one. The difference is large and nobody benchmarks it.

Unified memory keeps winning. Every serious local-AI machine announced has shared CPU/GPU memory at high capacity. Discrete VRAM is a bad fit for a workload where capacity matters more than peak bandwidth, and the industry has figured that out.

The robot demos are still teleoperated. They have been for four years. Watch for the operator's hands, or the suspiciously smooth trajectory, or the fact that the demo never deviates from the script. There is real progress in robotics and it is happening in warehouses, not on stages.

the accessory economy#

A large fraction of the floor was accessories for devices that do not exist yet: cases, docks, and mounts for AI wearables. That is a leading indicator of nothing except that the accessory industry moves fast and takes risks cheaply.

the one thing I would actually buy#

The mundane one: displays got better and cheaper. High-refresh, high-resolution, accurate-color monitors are at prices that were flagship prices three years ago.

If you have been running the same monitor since 2020 and you spend eight hours a day looking at text on it, that is the highest-value hardware purchase available to you, and it will not be mentioned in a single keynote.

the meta-observation#

CES has been a bad predictor of what matters for about a decade. The genuinely important hardware of the last ten years — the M1, the H100, TPUs, the shift to ARM in servers — was announced at industry events or in blog posts, to audiences who understood it, without a stage show.

The consumer electronics show is now mostly a trade event for retail buyers, covered as if it were a technology forecast. Read the specs. Skip the narrative.

The names varied — AGENTS.md, CLAUDE.md, .cursorrules, .github/copilot-instructions.md — and the format did not. It is prose. The model reads it. That is the whole protocol.

A convention that emerges independently in five products in one year is telling you something about the shape of the problem.

what actually belongs in it#

I have written and rewritten these a dozen times. The version that works is shorter than you think and more specific than you want.

Commands. The exact invocations. Not "run the tests" — the command, including the flags, including how to run one test file.

## commands
- install: `pnpm install --frozen-lockfile`
- test: `pnpm vitest run`
- test one file: `pnpm vitest run src/foo.test.ts`
- typecheck: `pnpm tsc --noEmit`
- lint: `pnpm biome check --write .`
- dev server: `pnpm dev` (port 5173)

This section alone eliminates most wasted agent turns. Without it, the agent guesses, guesses wrong, and spends four tool calls discovering your test runner.

Non-obvious structure. Where things live, when it is not inferable. "Database migrations are in db/migrations and must be created with pnpm db:new, never by hand." "The legacy/ directory is frozen — do not modify it."

Conventions that a linter does not enforce. If your linter catches it, do not write it down; the linter will tell the agent. Write down the things that are policy rather than syntax: "prefer composition over inheritance in src/domain", "all public functions in api/ need a docstring", "we do not use default exports".

Things that will break. "Do not run pnpm build — it takes 12 minutes and is not needed for tests." "The integration tests require Docker; skip them if it is not running." "Never modify schema.sql directly."

Boundaries. What the agent may not touch without asking. Production configs, migration files, anything with a security review requirement.

what does not belong#

Your entire architecture document. The agent reads this every session. A three-thousand-word essay costs tokens on every single request and dilutes attention across a lot of text that is irrelevant to most tasks.

Keep it under about 200 lines. If you need more, put it in a separate document and reference it: "for the event pipeline design, read docs/events.md before changing anything in src/events/."

Anything the code already says. Do not restate the type signatures. The agent can read.

Aspirations. "We value clean code" is not an instruction. "Functions over 40 lines get split" is.

Rules you do not actually follow. If the codebase contradicts the file, the codebase wins in the model's attention, and now your instructions are noise.

the part I did not expect#

Writing these has improved my documentation for humans.

The discipline of writing "here is exactly how to run the tests, here are the things that will surprise you, here is what not to touch" is precisely what a new engineer needs on day one, and it is precisely what most onboarding documents fail to contain because they were written by someone who already knew.

An agent is an infinitely patient new hire who will follow instructions literally and never ask a clarifying question out of politeness. That turns out to be an excellent test of whether your instructions are any good.

Several teams I know have merged their onboarding doc and their agent file into one. That is the correct end state.

the standardization question#

There is an ongoing effort to consolidate on AGENTS.md as the common name, with tools reading it as a fallback. That would be good and it is a coordination problem, which means it will take longer than it should.

In the meantime: write one file, symlink the rest. It costs nothing.

ln -s AGENTS.md CLAUDE.md

Here are five changes to how software actually gets made. I will grade these in December.

1. Review tooling becomes a real product category#

The bottleneck moved to verification in 2025 and the tooling did not follow. We are still reviewing machine-generated changes with an interface designed in 2008 for occasional human contributions.

What has to exist: review surfaces that show intent rather than only diff. Which invariants does this change preserve? What did the author consider and reject? Which tests exercise the changed path, and did any of them exist before this change?

Some of that can be generated. Most of it is a UI problem nobody has solved because the pain only became acute in the last eighteen months.

Watch for this to be one of the most-funded developer tooling categories of the year.

2. Repository-level agent configuration becomes standard practice#

AGENTS.md, CLAUDE.md, .cursorrules, and their cousins converged on the same idea in 2025: the repository tells the agent how to work in it. Build commands, test commands, conventions, things not to touch.

In 2026 this stops being an optional nicety and becomes part of the definition of a well-maintained repository, the way a README and a CI config are. New projects will have one from the start. Onboarding docs and agent instructions will merge, because they are the same document with a different reader.

The second-order effect is good: writing down how your project actually works helps humans too, and a lot of teams will document things properly for the first time because a machine needed it.

3. Power becomes a line item developers think about#

Not in an abstract climate sense. In a "which region, which instance type, what does this cost" sense.

Electricity prices in datacenter-dense regions rose through 2025 and the rate cases are ongoing. Cloud pricing follows. Regional price differentials for compute are going to become large enough that they affect architecture decisions, the way egress pricing already does.

Expect "run the batch job in the cheap region overnight" to go from a sustainability talking point to a cost-optimization default.

4. The supply chain gets a real control, not another dashboard#

Four significant npm incidents in 2025, ending with a self-propagating worm. The registry-level controls — mandatory trusted publishing, restrictions on long-lived tokens, install-script defaults — are coming, and they will break workflows.

That breakage is the point. The ecosystem has been asking maintainers to be individually unphishable for a decade, and it has not worked, because it cannot.

Expect a painful migration and a meaningfully safer ecosystem at the end of it.

5. "Senior engineer" quietly redefines around judgment#

The mechanical parts of the job — writing the code, remembering the API, finding the bug in a file you can see — got substantially cheaper. What did not get cheaper: knowing what to build, knowing when something is wrong, knowing which complexity is worth it, and being able to say no with a reason.

This has always been what seniority meant. It is now what it only means, and the gap between people who have it and people who were fast typists is going to become uncomfortable to talk about at promotion committees.

The related problem, which nobody has solved: the traditional path to developing that judgment was doing the mechanical work for several years. If juniors do not do that work, where does the judgment come from?

I do not know. It is the most important open question in the profession right now and almost all the discussion of it is either dismissive or catastrophizing.

what I am not predicting#

Anything about AGI. Anything about a specific model beating a specific benchmark. Anything about a company's valuation.

Those get all the attention and none of them change what you do on Tuesday.

the year in one paragraph#

An open reasoning model under an MIT license repriced the sector in January. Reasoning became a runtime dial rather than a model choice. Coding agents went from research preview to standard tooling in about six months and made code review the bottleneck. The npm ecosystem got a self-propagating worm. Two of the largest infrastructure providers had multi-hour outages caused by config, not code. Python removed the GIL, optionally. Rust shipped its largest edition. The frontier models converged and the competition moved to price and distribution.

the three things that will still matter in 2030#

1. Reasoning as a controllable runtime parameter.

The most durable technical idea of the year. Every provider independently arrived at the same design: the caller decides how much the model thinks, per request.

This is durable because it reflects something true about the problem — the appropriate amount of computation is a property of the task, not the model, and only the caller knows the task. Interfaces that reflect true structure survive. Interfaces that reflect an implementation detail do not.

2. The agent-review bottleneck.

Generation throughput increased dramatically. Verification throughput did not. That gap is the central engineering problem of the next several years, and nobody has a good answer.

Everything downstream follows from it: how teams are structured, what tests are for, what "code review" means, whether we build systems we understand or systems that pass tests. This is not a tooling problem that gets solved with a better diff viewer. It is a fundamental asymmetry between producing and checking, and it shows up in every field where automation outpaced verification.

3. Supply chain trust has no technical fix yet.

Four significant npm incidents this year, escalating in sophistication, ending with a worm. Every one exploited the same structural fact: install-time code execution plus long-lived publishing credentials plus a dependency graph nobody designed.

The controls that work — trusted publishing, no install scripts, version cooldowns, phishing-resistant auth — are known and unevenly adopted. The ecosystems that are structurally safer got that way by design decisions made years ago that cannot be retrofitted cheaply.

This gets worse before it gets better, and it is a solvable problem that we are choosing not to solve at the speed it requires.

the things that felt big and were not#

Model benchmark leapfrogging. Every launch claimed the frontier. The differences were within evaluation noise for most real tasks. The benchmark discourse consumed enormous attention and predicted very little about what was useful.

Agent frameworks. Most of the orchestration layer got absorbed into the models, exactly as function-calling libraries and JSON-repair libraries were absorbed before. The durable layer was never orchestration.

The browser wars, round two. Everyone shipped a Chromium fork with a model in it. That does not diversify the engine landscape; it diversifies the UI on top of one engine.

the things that felt small and were not#

LLD as the default linker in Rust. A default change delivered a build-time improvement to everyone at once that documentation had failed to deliver for years. The general lesson — defaults are the highest-leverage thing a toolchain ships — applies far beyond Rust.

Template strings in Python. A one-character syntax change that makes the safe path as easy as the unsafe one. If library adoption follows, an entire category of injection vulnerability becomes hard to write. Security features that depend on diligence fail; security features enforced by the type system work.

Context editing and memory tools. Unglamorous plumbing that determines whether long agent runs succeed. The context window was never the constraint. Goal retention was.

the thing I want to say going into next year#

The most valuable skill in 2025 was not prompting, and it will not be in 2026 either.

It was the ability to tell whether something is correct. That skill was always valuable and it was previously bundled with the ability to produce the thing. The bundle has come apart. Producing is cheap now. Checking is not, and checking is what everything rests on.

Every argument this year about what AI does to engineering eventually reduces to that. The people who are getting enormous leverage out of these tools are, without exception, the people who can tell when the output is wrong.

That is not a comforting conclusion for anyone hoping the tools eliminate the need for expertise. It is the actual state of things, and it is worth building your career around.

See you in January.

1. "Local models become the default for routine developer work"#

Written in January. Partially right, wrong on scale.

Local models got much better and much easier to run. The tooling is excellent. A 14B model on a laptop is genuinely useful for the work I said it would be useful for.

What I got wrong: adoption. The default did not become hybrid. It became "frontier API for everything," because the frontier models got cheap enough fast enough that the cost argument for local largely evaporated for individuals.

The privacy argument held and drove real adoption in regulated industries, which is what I said. The convenience argument lost, which I underweighted.

Grade: C+. The technology went where I said. The behavior did not.

2. "The moat is distribution and serving efficiency, not model quality"#

Written in February. This held up well.

Model capability converged. Every frontier lab shipped comparable models. The differentiation moved to price, latency, integration, and distribution — exactly as predicted, and faster than I expected.

Google putting a frontier model into Search on launch day is the clearest demonstration. No competitor can do that.

Grade: A−.

3. "Delegated agents make review the bottleneck"#

Written in May. Correct and I understated it.

This is now the dominant complaint from every team using coding agents seriously. Throughput on generation went up several fold. Throughput on review did not move.

What I did not anticipate: the second-order effect on review quality. It is not just that review is slower — it is that reviewers under queue pressure approve things they have not fully understood, and the defect rate is showing up in ways that are hard to attribute.

Grade: A, with the caveat that I was not pessimistic enough.

4. "RAG infrastructure is solving a temporary problem"#

Written in March. Too confident, partially wrong.

Long context did get better and cheaper, and a lot of naive RAG did get replaced by just putting the documents in the prompt. That part was right.

What I got wrong: retrieval did not go away, it moved. The interesting systems now do retrieval as a tool the model calls rather than as a preprocessing step. The vector database did not die; it became something the agent queries.

That is a substantially different outcome than "the category is temporary," and I should have seen it because the same pattern — capability absorbed into the model, infrastructure repositioned rather than eliminated — had already happened twice.

Grade: C.

5. "Nvidia's roadmap slide is the real announcement"#

Written in March. Right, and the power framing was the useful part.

Power as the binding constraint became the consensus view over the year. The unit of AI infrastructure discussion is now gigawatts. Utility rate cases about datacenter interconnection are a live political issue in multiple states.

Grade: A.

6. "The npm ecosystem will have a self-propagating worm"#

Written in February, in a piece about supply chain risk. I hate being right about this one.

It arrived in September. The mechanism was exactly the one everybody had described: steal credentials on install, use them to publish, repeat.

The prediction was not clever. Every precondition was public. What I got wrong was the timeline — I expected it to take longer, because I assumed the registry would ship stronger publishing controls first. It did not, until after.

Grade: A on the call, F on the assumption that anyone would act in time.

the pattern in my errors#

Looking at the three I got wrong, they share a shape: I was right about the technology and wrong about the behavior.

Local models got good; people did not switch. RAG got less necessary; the infrastructure adapted instead of dying. Registry controls were obviously needed; they arrived after the incident rather than before.

The lesson I am taking into next year: technical trajectories are much more predictable than adoption, and adoption is where the money and the consequences are. When I feel confident, it is usually because I am reasoning about the technology, and the technology was never the hard part.

Here is the year, and what it means for the next one.

the releases that mattered#

DeepSeek R1 (January). MIT license, published training methodology, distilled variants that ran on consumer hardware. The RL-on-verifiable-rewards recipe was reproduced widely within weeks.

Qwen3 (April). Eight models, Apache 2.0, MoE variants with excellent quality-per-active-parameter, 119 languages.

Kimi K2 (July). A trillion parameters, open weights, tuned for agentic tool use, with a genuinely novel training stability contribution.

gpt-oss (August). OpenAI's first open weights since 2019, Apache 2.0, with a 20B variant that runs on a laptop.

Plus continuous releases from Mistral, Zhipu, MiniMax, Meta, Microsoft, Google, and a long tail of fine-tunes.

where the gap actually is#

The frontier-to-open gap held at roughly six to twelve months all year. That stability is the most important finding, because it means open weights are not converging on the frontier and are not falling behind — they are trailing at a fixed distance.

But "six months behind" undersells the practical position, because the gap is not uniform:

Nearly closed: code completion, summarization, extraction, classification, translation, structured output, straightforward tool use. For these, a good open model is not meaningfully worse than a frontier model, and it costs a fraction.

Meaningfully behind: long-horizon agentic work, complex multi-step reasoning, instruction following over many turns, reliability at the tail. This is where frontier models earn their price.

Not comparable: anything requiring the surrounding infrastructure — enterprise support, uptime guarantees, safety tooling, indemnification. Open weights give you the model and nothing else.

what changed structurally#

Licensing got genuinely permissive. Two years ago "open" meant a research license with a prohibited-use list. Now the frontier of open releases is Apache 2.0 and MIT. That is a real change and it removed the legal review that was blocking adoption.

The runtime story got boring. Ollama, llama.cpp, vLLM, MLX, LM Studio. One command. An OpenAI-compatible endpoint. The friction that kept open models in the enthusiast category is gone.

Hosting became competitive. Multiple providers serve open models at prices well below frontier API rates, with real SLAs. You can use open weights without running anything.

The center of gravity moved east. The most capable, most permissively licensed open releases came predominantly from Chinese labs. That is a strategic fact with policy consequences that are being worked out loudly and mostly unproductively.

the practical architecture for next year#

The shape that makes sense:

• Open weights, self-hosted or on a cheap provider, for high-volume, well-defined tasks. Classification, extraction, embedding, first-pass drafting.
• Frontier API for the hard tail: planning, complex reasoning, anything customer-facing where a bad answer is expensive.
• A router deciding between them, with the escalation rate instrumented.
• Your own eval set in your own repository, run against every candidate.

That is not a hedge. It is what the cost and capability curves actually imply.

the prediction#

The gap holds at roughly six to twelve months through next year. Open weights absorb an increasing share of production workload by volume while frontier models keep the high-value tail.

The interesting question is not capability. It is whether the labs currently releasing weights continue to, and that is a business decision that could change in either direction with one quarter's strategy review.

Download the ones you care about. They cannot be un-released.

The argument is settled in practice: models solve most early puzzles in seconds, the global leaderboard is not a meaningful competition anymore, and Eric Wastl has asked people not to use AI for leaderboard attempts. Some people ignore that.

I think the argument is also beside the point, and I want to make the case for what these puzzles are actually good for.

what the leaderboard was ever worth#

The global top 100 was always a competition between a small number of extremely fast competitive programmers, most of whom had done thousands of hours of practice specifically for this. For everyone else it was scenery.

If your enjoyment depended on that leaderboard, it was already not for you before any model existed.

what the puzzles are good for#

Practicing a language you do not know. This is the best use and it is what I do every year. Twenty-five bounded problems with clear specifications and verifiable answers is an excellent scaffold for learning a language's idioms. You are not fighting the problem, so you can concentrate on the tool.

Do it in something uncomfortable. Rust if you write Python. A Lisp if you write Java. Uiua or J if you want to genuinely reconsider what a loop is.

Finding the gap between working and good. Almost every puzzle has a naive solution that works for part one and does not terminate for part two. That gap — where you have to actually think about the algorithm rather than just express the problem — is where the learning is.

That is also the part models are least reliably good at, incidentally. They will write the naive version confidently.

Reading other people's solutions. The subreddit's solution megathreads are one of the better learning resources in programming. Twenty implementations of the same problem in different languages by people of different skill levels, all solving something you just spent an hour on, so you have full context.

You will learn more from reading five solutions after solving it than from solving three more puzzles.

A shared thing. For a few weeks in December a lot of programmers are all thinking about the same problem on the same day. That is a rarer thing than it used to be and it is worth something.

on using a model#

I do not think there is a moral question here for anything except the leaderboard, where the request is explicit and honoring it is basic decency.

For everything else: it is your time and your learning. If you use a model to solve everything instantly, you will finish faster and learn nothing, which is a trade you are allowed to make and which defeats the entire point of doing a programming puzzle for fun.

The interesting middle ground, which I recommend: solve it yourself, then ask a model to critique your solution. "What is the time complexity? What would break at larger inputs? Is there a standard algorithm for this I should know?"

That is a genuinely good use. You get the learning from solving it and you get the thing you did not know from the critique. It is closer to having a knowledgeable colleague than to having someone do your homework.

the thing that actually bothers me#

Not the puzzles. The people who have decided that because a model can do the easy version of something, learning to do it is pointless.

That reasoning has been available for every tool ever built and it has been wrong every time. Calculators did not make arithmetic understanding useless. Compilers did not make understanding what the machine does useless. IDEs did not make knowing the language useless.

The understanding is what lets you tell when the tool is wrong. That has never been more valuable than it is now, when the tool is wrong fluently.

Go do the puzzles. In a language you do not know. Slowly.

silicon#

Amazon continues pushing Trainium and Inferentia as the alternative to buying Nvidia. The pitch is price-performance for customers who can tolerate a different software stack.

The honest state of it: the hardware is competitive on paper, the software ecosystem is meaningfully behind CUDA, and the gap is closing slowly. If your workload runs through PyTorch with standard operations, the port is manageable. If you have custom kernels, it is a project.

The strategic point is the same for every hyperscaler: reducing dependency on a single supplier with enormous pricing power. Whether the customer benefit materializes depends entirely on whether the savings get passed through.

the agent announcements#

Every cloud vendor is now shipping agent infrastructure: runtimes, memory services, gateways for tool access, identity for agents, observability for agent traces.

This category is real. Running agents in production has genuine infrastructure requirements that are different from running services:

• Long-lived sessions with state that outlives a request.
• Non-deterministic execution paths that make traditional tracing awkward.
• Cost per invocation that varies by orders of magnitude.
• Identity and permission scoping for a thing acting on a user's behalf.
• Human approval gates in the middle of automated flows.

Those are real problems and the tooling is early everywhere. Evaluate on whether it solves a problem you actually have, not on whether the demo was good.

the pattern I would push back on#

Every vendor's agent framework wants to be the place your orchestration lives. That is a lock-in position, and orchestration is the layer most likely to be absorbed by the models themselves — it has been happening steadily for two years.

Keep your orchestration portable. Use the managed pieces for the genuinely hard infrastructure — identity, secure tool access, session storage — and keep the logic in your own code.

the underrated announcements#

The ones nobody writes about and everybody uses:

• Incremental improvements to S3 consistency and performance.
• Networking latency reductions.
• Cost management tooling that is slightly less bad.
• Database engine version updates.

These are worth more to most organizations than any AI announcement, and they get four slides between two hours of agent demos.

the meta-observation#

Cloud conferences have gotten less interesting, and that is good. It means the platform is mature. The exciting years of a platform are the years when fundamental things are missing.

The interesting question for AWS is not what they announced. It is whether the operational excellence that justified the premium is still there after a year that included a major regional outage. That is an execution question and it does not get answered at a conference.

what to actually do with this#

Skip the keynote. Read the "what's new" feed filtered to the services you actually use. Look for the deprecations, which are the announcements that will cost you time and which are never on stage.

And check your bill. The single highest-value hour available to most engineering organizations is someone competent looking at the AWS bill line by line, and almost nobody does it.

The price cut is the headline for most users. The compaction work is more interesting.

the compaction problem#

Long agent sessions fill their context. Tool outputs, file contents, error messages, prior reasoning. Eventually you hit the limit and something has to go.

The naive approaches are all bad:

• Truncate the oldest. Loses the original task description, which is the single most important thing in the context.
• Truncate the middle. Loses the reasoning chain that got you here.
• Summarize everything. Loses specifics — file paths, error strings, exact values — that turn out to matter.

What you actually want is selective retention: keep the goal, keep the decisions and their rationale, keep the current state, discard the raw tool output that has already been acted on.

That is a judgment call, and doing it well requires understanding what the session is about. Which makes it a model problem rather than a buffer-management problem.

why this matters more than benchmark deltas#

For agent workloads, context management determines whether a long task succeeds far more than a few points of benchmark difference.

I have watched agent runs fail in exactly this way: two hours in, compaction drops a detail — a constraint from the original request, a decision made an hour ago — and the agent proceeds confidently in a direction that contradicts the task. Everything after that is wasted, and it looks productive the whole time.

If you are building on any model, the lesson to steal is: do not rely on the context window as your memory. Maintain durable state outside it.

task.md          — the goal, constraints, acceptance criteria. Re-read often.
notes.md         — decisions made and why. Appended, never rewritten.
state.json       — current progress, structured.

Feed those back in after every compaction. This is cheap, model-agnostic, and it is the difference between an agent that works for four hours and one that works for forty minutes.

the effort parameter#

Explicit control over reasoning depth, exposed to the caller. Everyone has this now under different names — thinking budget, reasoning effort, thinking config.

The convergence is total and it confirms the design conclusion: reasoning depth belongs to the caller, not the model, because only the caller knows whether this particular request justifies the latency and the cost.

Measure the quality-cost curve on your own task. The knee is usually much lower than people assume.

the price movement#

Frontier model pricing has fallen substantially across every provider over the past eighteen months, on a per-capability basis by considerably more.

Two implications:

Cost-optimization work has a short half-life. Elaborate infrastructure to shave token costs may be obsolete before it pays for itself. Build the thing; optimize when the bill actually hurts.

"Too expensive to do with a frontier model" is a moving line. Applications that did not pencil out a year ago may now. It is worth periodically revisiting the ideas you rejected on cost grounds, because the reason you rejected them keeps expiring.

the competitive picture#

Three labs shipping frontier releases within a week of each other, with capability differences small enough to be within evaluation noise on many tasks.

The practical consequence for developers: your model choice is a preference, not a lock-in, and it should be revisited quarterly rather than defended.

Build the abstraction. It is a day of work and it keeps paying.