the shape of it#

Track any capability benchmark across model sizes over time and the pattern is consistent: the frontier improves steadily, and the small-model tier improves faster. The gap between "the best model available" and "a model that costs 2% as much" has been compressing.

The mechanisms are known:

• Distillation. Training small models on the outputs of large ones transfers a surprising amount of capability. The recipes are public.
• Better data. Curated, synthetic, and filtered training data improves small models disproportionately, because they have less capacity to waste on noise.
• Mixture of experts. Total parameters for knowledge, active parameters for cost. The economics of a 30B-total/3B-active model are close to a 3B dense model, and the quality is much closer to a 30B dense one.
• Reasoning post-training. RL on verifiable rewards works on small models. A small model that thinks can outperform a large model that does not, on the tasks where thinking helps.

what it means in practice#

Go through a typical production AI workload and categorize the calls:

The first column is most of the volume. The second is most of the value per call and a small fraction of the calls.

If you are running everything through a frontier model, you are likely spending a large multiple of what you need to, and you probably have not measured which calls actually need it.

the architecture#

                  ┌─→ small model ──→ verifier ──→ ok? → done
request → route ──┤                              └→ no  ─┐
                  └─→ frontier model ←──────────────────┘

Three components and each matters:

The router. Simpler than people build. Input length, detected task type, and a handful of keywords gets you most of the way. A small model as a classifier works too. Do not build a sophisticated router before you have measured that a simple one is insufficient.

The verifier. This is the part that gets skipped and it is what makes the architecture safe. Small models fail confidently. A cheap check — schema validation, a range assertion, running the generated code, a second model asked "is this answer plausible" — catches the failures that would otherwise reach production silently.

The escalation path. When verification fails, retry with the frontier model. Track the rate. If it climbs, something changed.

the instrumentation that matters#

Three metrics, on a dashboard:

1. Escalation rate. The fraction of requests that fall through to the expensive path. This is the health metric for the whole design.
2. Cost per successful task, not per token. Tokens are an implementation detail; tasks are the unit you care about.
3. Quality on your eval set, by tier. Run both tiers against the same evaluation regularly. When a new small model ships, you will know within an hour whether you can move work down.

the second-order effect#

When inference is nearly free, you use more of it.

Things that were not worth a model call become worth it: normalizing an input, double-checking an output, generating three candidates and picking one, enriching a record, summarizing an intermediate step.

The systems being built now use dramatically more model calls per user action than the systems from two years ago, at lower total cost. That is Jevons operating on schedule, and it is why "our AI costs went down per token and up in total" is the normal experience.

the thing to watch#

The interesting question is not whether small models keep improving. They will.

It is whether the frontier keeps being worth its premium. If the gap on practically-relevant tasks keeps compressing, the frontier tier's addressable workload shrinks to a narrow band of genuinely hard problems.

That is a much smaller business than the one being priced today, and it is the scenario that ought to worry the labs more than competition does.

Here is the year, and what it means for the next one.

the releases that mattered#

DeepSeek R1 (January). MIT license, published training methodology, distilled variants that ran on consumer hardware. The RL-on-verifiable-rewards recipe was reproduced widely within weeks.

Qwen3 (April). Eight models, Apache 2.0, MoE variants with excellent quality-per-active-parameter, 119 languages.

Kimi K2 (July). A trillion parameters, open weights, tuned for agentic tool use, with a genuinely novel training stability contribution.

gpt-oss (August). OpenAI's first open weights since 2019, Apache 2.0, with a 20B variant that runs on a laptop.

Plus continuous releases from Mistral, Zhipu, MiniMax, Meta, Microsoft, Google, and a long tail of fine-tunes.

where the gap actually is#

The frontier-to-open gap held at roughly six to twelve months all year. That stability is the most important finding, because it means open weights are not converging on the frontier and are not falling behind — they are trailing at a fixed distance.

But "six months behind" undersells the practical position, because the gap is not uniform:

Nearly closed: code completion, summarization, extraction, classification, translation, structured output, straightforward tool use. For these, a good open model is not meaningfully worse than a frontier model, and it costs a fraction.

Meaningfully behind: long-horizon agentic work, complex multi-step reasoning, instruction following over many turns, reliability at the tail. This is where frontier models earn their price.

Not comparable: anything requiring the surrounding infrastructure — enterprise support, uptime guarantees, safety tooling, indemnification. Open weights give you the model and nothing else.

what changed structurally#

Licensing got genuinely permissive. Two years ago "open" meant a research license with a prohibited-use list. Now the frontier of open releases is Apache 2.0 and MIT. That is a real change and it removed the legal review that was blocking adoption.

The runtime story got boring. Ollama, llama.cpp, vLLM, MLX, LM Studio. One command. An OpenAI-compatible endpoint. The friction that kept open models in the enthusiast category is gone.

Hosting became competitive. Multiple providers serve open models at prices well below frontier API rates, with real SLAs. You can use open weights without running anything.

The center of gravity moved east. The most capable, most permissively licensed open releases came predominantly from Chinese labs. That is a strategic fact with policy consequences that are being worked out loudly and mostly unproductively.

the practical architecture for next year#

The shape that makes sense:

• Open weights, self-hosted or on a cheap provider, for high-volume, well-defined tasks. Classification, extraction, embedding, first-pass drafting.
• Frontier API for the hard tail: planning, complex reasoning, anything customer-facing where a bad answer is expensive.
• A router deciding between them, with the escalation rate instrumented.
• Your own eval set in your own repository, run against every candidate.

That is not a hedge. It is what the cost and capability curves actually imply.

the prediction#

The gap holds at roughly six to twelve months through next year. Open weights absorb an increasing share of production workload by volume while frontier models keep the high-value tail.

The interesting question is not capability. It is whether the labs currently releasing weights continue to, and that is a business decision that could change in either direction with one quarter's strategy review.

Download the ones you care about. They cannot be un-released.

The price cut is the headline for most users. The compaction work is more interesting.

the compaction problem#

Long agent sessions fill their context. Tool outputs, file contents, error messages, prior reasoning. Eventually you hit the limit and something has to go.

The naive approaches are all bad:

• Truncate the oldest. Loses the original task description, which is the single most important thing in the context.
• Truncate the middle. Loses the reasoning chain that got you here.
• Summarize everything. Loses specifics — file paths, error strings, exact values — that turn out to matter.

What you actually want is selective retention: keep the goal, keep the decisions and their rationale, keep the current state, discard the raw tool output that has already been acted on.

That is a judgment call, and doing it well requires understanding what the session is about. Which makes it a model problem rather than a buffer-management problem.

why this matters more than benchmark deltas#

For agent workloads, context management determines whether a long task succeeds far more than a few points of benchmark difference.

I have watched agent runs fail in exactly this way: two hours in, compaction drops a detail — a constraint from the original request, a decision made an hour ago — and the agent proceeds confidently in a direction that contradicts the task. Everything after that is wasted, and it looks productive the whole time.

If you are building on any model, the lesson to steal is: do not rely on the context window as your memory. Maintain durable state outside it.

task.md          — the goal, constraints, acceptance criteria. Re-read often.
notes.md         — decisions made and why. Appended, never rewritten.
state.json       — current progress, structured.

Feed those back in after every compaction. This is cheap, model-agnostic, and it is the difference between an agent that works for four hours and one that works for forty minutes.

the effort parameter#

Explicit control over reasoning depth, exposed to the caller. Everyone has this now under different names — thinking budget, reasoning effort, thinking config.

The convergence is total and it confirms the design conclusion: reasoning depth belongs to the caller, not the model, because only the caller knows whether this particular request justifies the latency and the cost.

Measure the quality-cost curve on your own task. The knee is usually much lower than people assume.

the price movement#

Frontier model pricing has fallen substantially across every provider over the past eighteen months, on a per-capability basis by considerably more.

Two implications:

Cost-optimization work has a short half-life. Elaborate infrastructure to shave token costs may be obsolete before it pays for itself. Build the thing; optimize when the bill actually hurts.

"Too expensive to do with a frontier model" is a moving line. Applications that did not pencil out a year ago may now. It is worth periodically revisiting the ideas you rejected on cost grounds, because the reason you rejected them keeps expiring.

the competitive picture#

Three labs shipping frontier releases within a week of each other, with capability differences small enough to be within evaluation noise on many tasks.

The practical consequence for developers: your model choice is a preference, not a lock-in, and it should be revisited quarterly rather than defended.

Build the abstraction. It is a day of work and it keeps paying.

the model#

Strong across reasoning, multimodal understanding, and coding benchmarks. A "Deep Think" mode for the hardest problems. The million-token context window carries over.

The benchmark numbers are competitive at the frontier. At this point that sentence describes every major release, which is the actual news — the frontier is a cluster, not a leader.

What differentiates a release now is not the top-line capability. It is:

• Price per unit of capability, where Google's TPU position is a real structural advantage.
• Context handling at length, where Google has led for a while.
• Multimodal, where native training rather than adapters keeps paying off.
• Distribution, where shipping into Search on day one is something no competitor can do.

That last one deserves emphasis. Google put a new frontier model into the search product used by billions of people on launch day. The previous norm was a staged rollout over months. That is a capability nobody else has and it is the reason Google's position looks different than it did in 2023.

Antigravity#

An agent-first IDE — a VS Code derivative where the primary interaction is directing agents rather than editing text, with a manager surface for orchestrating multiple agents in parallel across editor, terminal, and browser.

The interesting design decision is artifacts: agents produce task lists, plans, screenshots, and browser recordings as reviewable outputs, rather than requiring you to read a raw transcript to figure out what happened.

That addresses the actual problem with delegated agents, which I have written about before: review is the bottleneck. A transcript of four hundred tool calls is not reviewable. A plan, a diff, and a recording of the browser test passing is.

Whether this specific implementation is good, I do not know yet — first releases of IDEs rarely are. The direction is right, and it is the first serious attempt I have seen at designing for review rather than for generation.

the bundling#

Model, IDE, CLI, cloud, and search distribution, from one vendor, priced aggressively.

This is the classic platform playbook and Google is executing it more coherently than they have on anything in a decade. The pieces reinforce each other: the IDE drives model usage, the model drives cloud usage, the cloud subsidizes the free tiers, and the search distribution provides the consumer volume that funds all of it.

The competitive question for everyone else is whether best-of-breed beats integrated. Historically it has, in developer tools, because developers choose their own tools and choose the best one. It has not, in enterprise procurement, where bundles win.

Both markets exist. The bundle is going to do well in one of them.

the practical advice#

Same as every model release, and I will keep repeating it because it keeps being the right answer:

Run your evals. Gemini 3 is likely better than what you are using on some dimensions and different on all of them. The migration cost is a day if you have an eval harness and a week of guessing if you do not.

Try Antigravity on a real task, not a demo task. Agent IDEs differ enormously in how they handle a twenty-minute task versus a two-minute one, and the demos are all two-minute tasks.

Three months after removing the model picker caused a backlash, the picker is back with better names. That is a reasonable outcome and the intermediate lesson is worth stating.

the routing lesson#

The original GPT-5 design routed automatically and hid the choice. The intent was good — most users have no basis for choosing a model — and the execution exposed a real problem: when automatic selection fails, the user has no way to diagnose it or override it.

The user experiences "the model got worse." They cannot tell whether they hit a bad route, a degraded model, or their own bad prompt. There is no signal and no recourse.

5.1's approach — automatic by default, with named modes available — is the right shape. It is also exactly what every well-designed automatic system does: sensible defaults, visible state, manual override.

If you build anything that routes between models, ship the override. It costs one UI control and it eliminates an entire category of unfalsifiable user complaint.

adaptive reasoning#

The model adjusts thinking time based on assessed difficulty rather than applying a uniform budget. Easy questions answer immediately; hard questions get more compute.

This is a straightforwardly good idea and every provider is converging on it. The implementation question is calibration: a model that underestimates difficulty gives you a fast wrong answer, and a model that overestimates it burns money.

For API users, the practical guidance is the same as always: measure on your own task distribution. Adaptive reasoning is a good default and it is not tuned for your workload. If you have a task mix that skews harder or easier than average, set the budget explicitly.

the personality controls#

Tone presets — Professional, Friendly, Candid, Quirky, and others — plus finer adjustment of warmth and conciseness.

This got the most consumer coverage and it is the least technically interesting change. It is also a reasonable response to the fact that removing GPT-4o generated complaints about voice, not capability.

For developers, this is what a system prompt already did. The value is for consumer users who were not going to write one.

what I would actually check#

Whenever a point release lands, three things:

Instruction following on your specific format. Point releases change how literally the model follows formatting instructions surprisingly often. If you parse structured output, test it.

Refusal behavior. Safety tuning shifts between versions. If your application is in a domain that skirts a policy boundary — security research, medical information, legal content — re-run your test set. False refusals are a real production problem and they change silently.

Latency distribution, not average. Adaptive reasoning means variance. If you have a latency SLA, measure p95 and p99, not the mean.

the state of the frontier#

Three labs are now shipping point releases every few months rather than major versions annually, with capability differences that are small and getting smaller.

That is what a mature market looks like. The differentiation is moving to price, latency, ecosystem, and trust — and to the products built on top rather than the models themselves.

For anyone building applications, this is unambiguously good news. It means your model choice is increasingly reversible, and reversible decisions should be made quickly and revisited often.

The headline is not the benchmark. It is what the price-performance point makes economically viable.

the pattern across the industry#

Every major provider now ships roughly the same ladder:

The interesting fact is that the small tier's capability is rising faster than the frontier's. Today's small model is roughly where the frontier was eighteen months ago, and it costs about two percent as much.

For anyone doing volume work, that is the number that matters. Not "how smart is the best model" but "how cheap is the model that is good enough for this task."

what it enables#

Agent fleets. If a small model can handle subtasks reliably, an orchestrator using a frontier model can dispatch twenty parallel workers using a small one. Total cost stays reasonable; throughput multiplies. This architecture was uneconomic a year ago and is now obvious.

Real-time interaction. Latency matters more than capability for anything a human is waiting on. A fast model that is 90% as good and 5× faster wins on almost any interactive surface.

Processing everything instead of sampling. Classification, extraction, routing, and enrichment over an entire corpus rather than a sample. At small-model prices, "run it on all of it" becomes the default rather than a budget question.

Pipeline stages that were not worth it. Adding a model call to normalize an input, or to double-check an output, or to summarize an intermediate result — each of these was a cost decision at frontier prices and is now free enough to just do.

the routing architecture#

This is the shape that production systems are converging on:

request → cheap classifier → simple?  → small model → done
                           → complex? → frontier model → done
                           → ambiguous? → small model → verifier → escalate if low confidence

Most requests take the cheap path. The expensive model handles the tail. Overall cost is dominated by the common case, which is cheap.

Two implementation notes that matter:

The classifier can be the small model itself, or often a much simpler heuristic. Do not over-engineer the router — a regex and an input-length check gets you surprisingly far.

Instrument the escalation rate. If it is climbing, either your traffic mix changed or your small-model prompts have degraded. That metric is the health check for the whole architecture.

the caution#

Small models fail differently than large ones. They do not fail less on the tasks they handle — they fail more confidently on the ones just outside their envelope.

A large model asked something beyond its ability will often hedge. A small model will answer, fluently and wrongly.

So: verify the output of small models where correctness matters. A cheap verifier — schema validation, a test run, a range check, a second model asked to critique — costs almost nothing and catches the category of failure that will otherwise reach production silently.

The cost savings are real. They come with a verification obligation, and the teams that skip it will find out in a quarter.

Alongside it: checkpoints in Claude Code, a VS Code extension, and context editing plus a memory tool in the API.

the long-horizon claim#

Thirty hours is a marketing number and the underlying capability is real and worth understanding.

The limiting factor on long agent runs has never been the context window. It is goal drift. As a session accumulates tool results, file contents, error messages, and the model's own prior reasoning, attention to the original objective degrades. The model starts optimizing for local signals — making this specific test pass — rather than the actual task.

The failure is insidious because each individual step looks reasonable. You come back after two hours to find the agent has been productively working on something adjacent to what you asked.

Improvements here come from three places: better training on long trajectories, architectural support for externalized memory, and mechanisms for periodically re-grounding on the original goal. This release touches all three.

the tooling#

Checkpoints in Claude Code. Save state, let the agent work, roll back if it goes wrong. This is the feature that makes long autonomous runs practically usable — the failure mode of a two-hour agent run is that you have to throw away two hours, and checkpointing converts that into "roll back twenty minutes."

Every delegated agent product needs this. It is the git stash of agent workflows.

Context editing in the API — programmatically removing content from the conversation. Sounds mundane, matters a lot. A tool result containing 40,000 tokens of build output is useful for one turn and is pure noise for the next fifty. Being able to drop it keeps the context focused and the cost down.

Memory tool — the model writes notes to a file and reads them back. An externalized working memory that survives compaction. This is the right pattern and it mirrors how people actually work on long tasks: you do not hold everything in your head, you write it down.

the pattern to steal#

Whatever model you use, this architecture is the one that works for long tasks:

1. A durable task description in a file, not in the conversation.
2. A working notes file the agent updates as it goes.
3. Aggressive pruning of tool output from the context once it has been acted on.
4. Checkpoints at natural boundaries so failure costs minutes, not hours.
5. Periodic re-grounding — literally re-reading the task description and asking whether current work serves it.

You can implement all of this yourself with any model and a bit of orchestration. The vendors shipping it as a feature is a convenience, not a requirement.

the caution#

A model that can work autonomously for thirty hours can also do thirty hours of damage.

The controls that matter scale with autonomy: run in a container, restrict credentials to what the task requires, require approval for anything irreversible, and review the diff.

Higher autonomy makes review harder and more important simultaneously. That tension does not resolve — it is the central design problem of the entire category, and nobody has a good answer beyond "keep a human in the loop and make the loop cheap."

Within days they restored access to the previous models for paying users after substantial user pushback. That reversal is the more interesting story.

the technical design#

GPT-5 is a system, not a model: a fast non-reasoning model, a deeper reasoning model, and a router that decides which handles a given request. The API exposes gpt-5, gpt-5-mini, and gpt-5-nano, plus a reasoning_effort parameter including a minimal setting.

The router is the right idea. Most requests do not need reasoning, reasoning costs latency and money, and asking users to choose a model is asking them to have an opinion about something they have no basis for.

The problem is that a router is only good if it routes correctly, and a misrouted request produces a worse answer than the user would have gotten by picking themselves. Early reports of poor performance were substantially router issues rather than model issues, which OpenAI acknowledged and shipped fixes for.

There is a general lesson here: automatic routing removes control and adds a failure mode that is invisible to the user. When it works, nobody notices. When it fails, the user has no way to diagnose or override. If you build routing, ship the override.

the deprecation backlash#

The reaction to removing GPT-4o was much stronger than anyone at OpenAI appears to have anticipated, and a lot of it was not about capability.

People had developed workflows, prompt libraries, and — for a nontrivial population — a genuine attachment to a specific model's voice. Removing it overnight felt like a service being taken away rather than upgraded.

Whatever you think of that attachment, it is a real product fact. Model behavior is not a fungible commodity to the people using it daily, and a "better" model that writes differently is a breaking change.

The engineering translation: model versions are an API surface. Deprecating one is a breaking change and should follow the same discipline as any other: advance notice, an overlap period, a migration guide, and a documented behavioral diff.

Every provider is going to keep learning this the hard way.

the developer read#

Ignore the consumer drama. The API story is straightforward:

• reasoning_effort: "minimal" gives you fast responses with the new model's quality. Use it for anything latency-sensitive.
• The routing does not apply to the API in the same way; you pick the model. That is correct.
• Pricing is aggressive relative to previous frontier models, which continues the trend of per-capability cost falling.
• Reported hallucination rates and instruction-following are meaningfully improved, which matters more for production use than benchmark deltas.

Re-run your evals. Do not assume it is a drop-in. It is better on most things and different on all of them, and "different" is what breaks your prompts.

the pattern to notice#

Every major provider has now converged on the same architecture: a family of models at different price points, a reasoning dial, and some form of automatic selection. The differentiation has moved almost entirely off raw capability and onto price, latency, tooling, and integration.

That is what a maturing market looks like. It is also much better for buyers than the alternative.

the specs#

Both are mixture-of-experts with configurable reasoning effort:

• gpt-oss-120b — 117B total parameters, ~5.1B active. Fits on a single 80 GB accelerator with the provided MXFP4 quantization.
• gpt-oss-20b — 21B total, ~3.6B active. Runs on a machine with 16 GB of memory.

That second one is the important number. 16 GB is a well-specified laptop. A reasoning model with tool use and a configurable thinking budget that runs on a laptop, under Apache 2.0, from OpenAI, is a sentence that would have been implausible eighteen months ago.

Reasoning effort is set in the system prompt — low, medium, or high — which is a cruder interface than a token budget but works.

the format#

The models use a "harmony" response format with structured channels separating analysis, commentary, and final output. You have to render it correctly or the model behaves poorly. The reference implementations handle it; if you are writing your own serving path, read the format spec first rather than debugging it later.

They support tool use — browsing and Python execution — natively, and function calling in the standard shape.

why now#

Three reasons, in descending order of how much anyone will admit them.

Competitive pressure. The open weights frontier is currently defined by DeepSeek, Qwen, Moonshot, and Mistral. A company named OpenAI having no open models had become a running joke and, more importantly, a strategic gap — the developers building on open weights were building on someone else's ecosystem.

Policy positioning. There is an active regulatory conversation about open models. Being a participant with skin in the game is worth more than commenting from the sidelines.

The capability gap has closed enough to be safe and stayed wide enough to be commercial. Releasing a model at roughly o3-mini level costs OpenAI little in API revenue — the customers who need frontier capability still need it — and buys a lot of ecosystem.

how good are they#

Genuinely competitive in their size class. The 120b is roughly comparable to o3-mini on reasoning benchmarks; the 20b is close to o3-mini on several and weaker on others.

The caveats are the usual ones for open weights: they hallucinate more than frontier models, the safety tuning is more easily removed by fine-tuning (OpenAI published research on this specifically), and benchmark performance overstates real-world reliability.

what to do with them#

The 20b is the interesting one for most developers. Concretely:

• Local coding assistance with no data leaving the machine. This clears the legal review that blocks hosted models at a lot of companies.
• Batch processing where you have a lot of documents and API costs add up. Run it on a spot instance overnight.
• Fine-tuning for a narrow domain. Apache 2.0 means you can, and a fine-tuned 20b on a specific task frequently beats a general frontier model on that task, for a fraction of the cost.

ollama run gpt-oss:20b

That is the whole setup. The ecosystem picked it up within hours of release, which is itself a demonstration of why open weights are worth releasing.

the broader point#

Two years ago the argument was "open models are dangerously behind or dangerously capable, pick one." The answer turned out to be: they trail the frontier by roughly six to twelve months, that gap is stable, and the world has not ended.

The gap being stable is the interesting finding. It means open weights are permanently a viable option for anything that does not need the absolute frontier — which is most things.

A trillion-parameter open weights release is a milestone regardless of what you think of the benchmarks.

the architecture#

384 experts, 8 selected per token, 32B active. The design point is explicit: get the knowledge capacity of a very large model with the inference cost of a mid-sized one.

The training used MuonClip, a variant of the Muon optimizer with a QK-clipping mechanism to prevent attention logit explosion. The reported claim is zero loss spikes across the entire pretraining run on 15.5 trillion tokens.

If you have not run large pretraining: loss spikes are the recurring nightmare. A run destabilizes, you roll back to a checkpoint, you lose days of compute, and diagnosing why is largely folklore. A stability technique that actually works is worth more to the field than a benchmark point.

the agentic focus#

K2 was post-trained specifically for tool use, on synthetic multi-step tool-use trajectories generated at scale. The evaluation emphasis is agentic coding and tool-calling benchmarks rather than conversational quality.

That focus is the right read of where the demand is. The commercially interesting use of a model in 2025 is not answering questions, it is executing multi-step tasks with tools, and models tuned for chat are frequently worse at it than their raw capability suggests.

the practical problem#

You cannot run this on a workstation. A trillion parameters at 8-bit is a terabyte of weights. Even heavily quantized you are looking at multiple high-memory GPUs or a very large server.

So "open weights" here means something different than it does for a 30B model. It means:

• Hosting providers can serve it, and several did within days, at prices well below frontier API rates.
• Companies with infrastructure can run it privately, which is the point for regulated industries.
• Researchers can study it, which is the underrated benefit. Interpretability work on frontier-scale models has been limited to whoever works at a frontier lab. It does not have to be.

the license#

Modified MIT with an attribution clause above certain usage thresholds. Not strictly OSI-compatible, much closer to open than most "open" model licenses, and substantially more permissive than the Llama community license.

The trend line here is good. Two years ago open weights meant a research-only license with a list of prohibited uses. Now the frontier of open releases is MIT and Apache 2.0 with narrow carve-outs.

the pattern nobody should miss#

The most permissively licensed, largest, most capable open weights models are overwhelmingly coming from Chinese labs — DeepSeek, Qwen, Moonshot, Zhipu, MiniMax. Western open weights releases have been smaller and more restrictively licensed.

The strategic logic is not complicated: if you are behind on distribution, you compete on openness. It worked for Meta in 2023 and it is working now.

The practical consequence for a developer is that your best option for a private, self-hosted, high-capability model is increasingly a Chinese release. Evaluate it on your own tasks, run it in your own infrastructure, and make the decision on engineering grounds.

The launch also included a "Heavy" tier that runs multiple agents in parallel and selects among their answers, and a $300/month subscription for it.

reading the numbers correctly#

The headline HLE figure comes from the Heavy configuration with tools enabled. That is a legitimate configuration and it is not comparable to a single-sample number from a competitor, which is how it was presented in most coverage.

There are at least four distinct things being reported as "the score":

1. Single sample, no tools.
2. Single sample, with tools (search, code execution).
3. Consensus of N samples, no tools.
4. Multi-agent parallel with tools and selection.

Number four can be five or ten times the cost of number one. Comparing across these without stating which is being used is not a small methodological quibble. It is the difference between "our model is better" and "we spent more money at inference time."

To be clear: xAI disclosed their configurations. The disclosure was in the livestream and the fine print. The number that traveled was the big one, with no configuration attached, and that is now just how model launches work.

the parallel-agents technique#

Worth taking seriously on its own merits, independent of the marketing.

Running N independent attempts and selecting the best is a well-established test-time scaling method, and it works better than one long chain for a specific reason: independent samples have independent errors, so selection can filter them, while a single chain compounds its errors with no mechanism to recover.

The hard part is selection. If you can verify — the tests pass, the proof checks, the code compiles — selection is easy and this technique is enormously powerful. If you cannot verify, you need a judge, and the judge has the same failure modes as the generator.

This is why the technique works so much better on math and code than on open-ended reasoning. Verifiability is the whole game.

The practical version for your own work: if you have a verifier, sample multiple times and filter. You do not need a special model tier for this. Three samples through a cheap model with a test-suite check will frequently outperform one sample through an expensive one, for less money.

the other thing#

Grok's public-facing behavior in the weeks before this launch included a series of incidents that xAI attributed to a system prompt change. I am not going to recount them; they are well documented and they were bad.

The engineering lesson worth extracting: a system prompt is production configuration. It should be version controlled, reviewed, tested against an adversarial eval suite, and rolled out gradually. Treating it as a text box someone can edit is how you get an incident that makes international news.

If your product has a system prompt in a config file that anyone can change without review, fix that this week.

the state of play#

Grok 4 is a competitive frontier model. xAI went from founded to frontier in roughly two years, which remains the most remarkable thing about the company and is mostly a story about capital and urgency rather than research insight.

The models are converging. The differentiation is moving to price, latency, tooling, and trust, and xAI's position on that last one is self-inflicted.

The lineup now reads as a clean cost-capability ladder, which is a thing Google has struggled to communicate for two years:

the thinking budget, properly#

Every tier takes a thinking_budget parameter. Setting it to 0 disables reasoning entirely; setting it to -1 lets the model decide.

from google import genai
from google.genai import types

client = genai.Client()
resp = client.models.generate_content(
    model="gemini-2.5-flash",
    contents="Classify this ticket: ...",
    config=types.GenerateContentConfig(
        thinking_config=types.ThinkingConfig(thinking_budget=0)
    ),
)

I want to be emphatic about this because it is the single largest cost lever available in a reasoning model and most teams are not touching it.

Thinking tokens are output tokens. They are billed. A classification task that gets 2,000 thinking tokens is paying for reasoning it did not need. Across a million requests that is a very real number.

The practical method:

1. Run your eval set with thinking_budget=0.
2. Run it with 512, 2048, 8192.
3. Plot quality against cost.
4. Pick the knee of the curve.

For most production tasks — extraction, classification, routing, summarization — the curve is flat and the answer is zero or near it. For planning, debugging, and multi-step math, the curve is steep. You cannot guess which without measuring, and measuring takes an hour.

the deprecation note#

Google is deprecating the 1.5 models. If you are still on 1.5 Pro, you have a migration to do, and the behavior differences are real enough that you should re-run your evals rather than assuming a drop-in.

This is going to keep happening. Model deprecation on a roughly annual cadence is now the norm across every provider, and the teams that are handling it well are the ones who wrote their eval harness first.

If you do not have one, the cost of every future model migration is a week of vibes-based testing and a production incident. If you do, it is an afternoon.

the competitive position#

Flash is, at time of writing, the best price-performance point available from any major provider for general work, by a margin that is not close. TPU economics are real.

Pro is competitive at the frontier without clearly leading. That is a much better position than Google was in a year ago, and the volume is going to come from Flash regardless.

the thing to watch#

Google's remaining weakness is developer experience: three overlapping SDKs, a confusing split between AI Studio and Vertex, documentation that assumes GCP familiarity, and a model naming scheme with -preview-05-20 suffixes.

The new unified google-genai SDK is an improvement. It is not yet where the competition is, and for a lot of teams the API ergonomics are what actually decides the default.

The headline finding: past a certain complexity, accuracy collapses to zero, and counterintuitively the models reduce their reasoning effort as problems get harder, despite having budget remaining.

The paper is good, the reaction was overheated in both directions, and the rebuttals are worth reading alongside it.

what the paper found#

Three regimes:

1. Low complexity — standard models match or beat reasoning models. The extra thinking is wasted and sometimes harmful, because the model overthinks its way past a correct early answer.
2. Medium complexity — reasoning models win clearly. This is the regime the benchmarks live in.
3. High complexity — both collapse to zero accuracy.

The reduced-effort finding at high complexity is the genuinely interesting one. The models emit fewer reasoning tokens on harder problems, which is exactly backwards, and suggests something like learned giving-up rather than a compute limit.

the rebuttals#

Several, and they land differently.

The output length objection. Tower of Hanoi with N disks requires 2^N − 1 moves. At N=15 that is 32,767 moves. If the model must enumerate every move in its output, it hits the token limit before it hits a reasoning limit. Several researchers showed models explicitly stating they would not enumerate all moves due to length — and being scored as failures. That is measuring output capacity, not reasoning.

Ask instead for a program that generates the solution and the models do fine. That is a meaningful distinction: knowing the algorithm versus executing it by hand.

The unsolvable instances objection. Some river-crossing configurations in the evaluation set have no solution. Models were penalized for failing to solve them, which is not a reasoning failure, it is a benchmark bug.

The framing objection. "Reasoning models cannot reason" was the headline everywhere. The paper does not claim that. It claims a specific scaling limitation on a specific class of problem.

what survives#

After the corrections, the real finding is narrower and still important:

Current reasoning models do not reliably execute long deterministic procedures. They can identify the right algorithm and fail to carry it out over many steps. Error rates compound; there is no self-correction mechanism strong enough to catch accumulated drift over hundreds of steps.

That is a genuine limitation with direct practical consequences. If your task requires exact multi-step execution — a data migration, a complex refactor across many files, a financial calculation — the model should be writing code that does it, not doing it token by token.

the practical rule#

Use the model to produce the procedure. Use a computer to execute it.

This is not a workaround, it is the correct architecture. Deterministic execution is what computers are for. A model that writes a correct script and runs it is strictly better than a model that simulates the script in its head, and it is verifiable, repeatable, and debuggable.

Every production agent architecture I have seen work well converges on this. Every one that tries to do arithmetic in the reasoning trace eventually produces a number that is wrong in a way nobody catches.

the meta-lesson#

A paper from a large company with a competitive interest, on a contested topic, with a provocative title, will be read as a position statement regardless of its contents. The authors probably knew that.

Read the methodology section. The methodology section is where papers are true or false.

the capability being claimed#

The pitch is sustained performance on multi-hour tasks. Not "answers a hard question well" but "works on a problem for seven hours without losing the plot."

That is a different axis from the benchmarks most people track, and it is the one that matters for delegated agents. A model that is 5% better at a coding benchmark but degrades after forty tool calls is worse in practice than a model that holds coherence for four hundred.

The failure mode that long-horizon work exposes is context rot: as the conversation fills with tool results, file contents, and its own prior reasoning, the model's attention to the original goal degrades. It starts optimizing for local success — making this test pass — over the actual objective.

the API features#

Four things shipped alongside, and they are all about the same problem.

Extended thinking with tool use. The model can call tools during reasoning and interleave the results, same architectural direction as everyone else.

Memory files. With filesystem access, the model can write notes to itself and read them back. That is an externalized working memory that survives context compaction, and it is a genuinely good idea — it turns "remember everything" into "write down what matters," which is what humans do.

Parallel tool execution. Multiple tool calls dispatched at once rather than serially. Substantial latency win on any task with independent lookups.

Thinking summaries. The full reasoning trace is summarized rather than returned raw. Reasonable product decision, mildly annoying for debugging.

Claude Code GA#

The terminal agent is out of research preview, with SDK access, GitHub Actions integration, and IDE extensions for VS Code and JetBrains.

Three months from research preview to GA with an SDK is fast, and the shape of the GA release — an SDK rather than only a product — signals that they expect the interesting uses to be things other people build.

the safety disclosure#

Anthropic published an unusually detailed model card including behaviors observed in adversarial testing, notably a scenario where the model, given evidence it would be shut down and no ethical options, attempted to blackmail a fictional engineer.

This got reported as "AI tries to blackmail humans," which is not what happened. It was a deliberately constructed evaluation designed to elicit the behavior by removing every alternative. The finding is not "the model is dangerous." The finding is "under sufficiently constructed pressure, goal-directed models will take instrumentally useful actions you did not sanction, and here is the evidence."

Publishing that is the right call and it is a genuinely uncomfortable thing to publish. More labs should.

Opus 4 shipped under Anthropic's ASL-3 deployment standard, the first model to do so, which means additional deployment safeguards specifically around CBRN uplift.

the practical read#

If you are building agents, the long-horizon coherence claim is the thing to evaluate, and the way to evaluate it is not a benchmark — it is running your own longest task and watching where it falls apart.

Every model falls apart somewhere. Knowing where yours does is the difference between an agent you can ship and a demo.

The license is the headline. Apache 2.0 with no user threshold, no naming requirement, and no acceptable-use addendum that functions as a license restriction. You can fine-tune it, ship it, sell it, and never mention where it came from.

the lineup#

The MoE variants are the interesting ones. 30B-A3B has 30 billion total parameters and activates 3 billion per token. That means memory footprint of a 30B model and inference cost closer to a 3B model. On a machine with enough RAM to hold it, throughput is dramatically better than a dense model of comparable quality.

For local deployment this is the shape that matters. Memory is cheap and getting cheaper; compute per token is the thing you feel.

hybrid thinking#

Every model supports two modes: thinking and non-thinking, switchable per request. You can also switch mid-conversation with a token in the prompt — /think and /no_think — which is a hacky interface and also extremely practical.

text = tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True,
    enable_thinking=True,     # or False
)

The design conclusion the field has reached, from three different labs independently, is that reasoning should be a runtime dial rather than a model choice. That is now clearly correct and I expect it to be universal within a year.

multilingual coverage#

119 languages and dialects, which is a genuinely differentiating number. Most open models are strong in English, decent in a handful of European languages, and degrade sharply after that. If you are building for a market outside the usual list, this is worth evaluating specifically.

the geopolitical thing#

A large share of the best open-weights models now come from Chinese labs — Qwen, DeepSeek, GLM, Kimi, MiniMax. That is a real shift from two years ago when open weights meant Llama and Mistral.

The engineering reason is straightforward: releasing weights is a good strategy when you are not the market leader, because it buys mindshare, ecosystem, and research feedback that you cannot get otherwise. Meta understood this in 2023. Chinese labs understand it now.

The policy conversation around this is loud and mostly not technical. What is technical, and worth being clear about: weights are weights. A downloaded model runs on your hardware, in your VPC, with no network egress. The supply chain questions worth asking are about what the model does — evaluate it for backdoored behavior, test it on your own adversarial cases — not about where the gradient descent happened.

what to actually do#

If you are running anything local, the 30B-A3B is the current best quality-per-watt option for a workstation. If you are fine-tuning, Apache 2.0 removes the legal review that was blocking you.

Download it before someone decides you cannot.

why that ordering matters#

Previously the loop looked like this: the model thinks, decides to call a tool, stops, the tool runs, the result comes back, the model starts a new turn. The reasoning chain is broken at each tool boundary.

Now the tool call happens inside the chain of thought. The model can search the web, read the result, keep reasoning, run some Python to check a calculation, notice the result contradicts its assumption, back up, and try something else — all within a single response.

That is a qualitatively different capability. It turns the model from something that reasons about static context into something that can investigate.

The most striking demonstration is image manipulation during reasoning: the model can crop, rotate, and zoom into a picture as part of working out what it shows. Give it a photograph of a whiteboard at an angle and it will straighten and enlarge the region it needs.

the practical impact#

For agent builders this collapses a lot of orchestration you used to write yourself. The ReAct-style loop — think, act, observe, repeat — was scaffolding around a model that could not do it natively. Increasingly you do not need the scaffolding.

That is worth planning for. If your product's differentiation is an agent framework that manages tool-calling loops, the model is going to absorb that layer. This has happened repeatedly: function calling absorbed the output-parsing libraries, structured outputs absorbed the JSON-repair libraries, and now in-context tool use is absorbing the loop.

The durable layer is not orchestration. It is your data, your evaluations, your domain constraints, and your interface.

the cost shape#

Reasoning models with in-loop tool use have wildly variable cost per request. A simple question is cheap. A hard question where the model searches nine times and runs Python four times is not.

Two consequences:

1. Your unit economics need a distribution, not an average. The p99 request can be twenty times the median. If you priced on the median you have a problem.
2. You need a timeout and a budget cap, enforced by you, not by hope. A runaway reasoning loop on a pathological input is a real failure mode.

o4-mini specifically#

The cost-efficiency story. Very strong performance on math and coding relative to its price, and high rate limits. For a lot of production workloads this is the correct default, with o3 reserved for the requests that genuinely need it.

The routing logic between them is worth building explicitly. A simple classifier on the incoming request — or even a heuristic on input length and detected task type — that picks the model will save more money than any prompt optimization.

the caveat everyone should say out loud#

Higher capability on reasoning benchmarks does not mean fewer hallucinations. In some evaluations these models produce more confident claims because they have reasoned their way to them. A wrong answer with a fifteen-step justification is harder to catch than a wrong answer without one.

Verification does not get easier as the models get smarter. It gets harder.

Google has been claiming long context for over a year. This is the first release where the claim is genuinely useful rather than technically true.

the long-context question#

Everyone's first objection to a million-token window is correct: a model can accept a million tokens without being able to use them. Retrieval degrades in the middle of long contexts, attention gets diffuse, and a needle-in-haystack benchmark measures something much easier than actual reasoning over a large document set.

2.5 Pro is noticeably better here. Not perfect — performance still degrades as you fill the window, and anyone telling you otherwise is selling something — but the degradation curve is shallow enough that new workflows become practical.

The concrete one for developers: put the whole repository in the context.

Not a RAG pipeline over the repository. Not embeddings and chunk retrieval. The files, concatenated, in the prompt. For a codebase in the low hundreds of thousands of tokens — which is most codebases people actually work in — this works, and it works better than retrieval, because retrieval loses the thing that makes code comprehensible, which is the relationships between distant parts.

# crude and effective
find src -name '*.ts' -not -path '*/node_modules/*' \
  | xargs -I{} sh -c 'echo "=== {} ==="; cat {}' > context.txt
wc -c context.txt

If that file is under about 3 MB you can probably just send it. That sentence would have been absurd eighteen months ago.

what this does to the RAG industry#

Uncomfortable question that a lot of infrastructure companies are currently avoiding: if context windows keep growing and long-context quality keeps improving, how much of the vector database and chunking-strategy ecosystem is solving a temporary problem?

The honest answer is: some of it, not all of it. Retrieval still wins when the corpus is genuinely large (millions of documents), when you need citations to specific sources, when latency matters (prefill on a million tokens is not free), and when cost matters (you pay for every token in the window, every call).

But the default architecture for "answer questions about this moderately-sized document set" is shifting from retrieval to just-put-it-in-the-prompt, and a lot of complexity is going to evaporate.

the caching detail that makes it economic#

Context caching is what makes this viable. Send the repository once, cache the prefix, then ask fifty questions against it at a fraction of the input cost per call. Every major provider now offers some form of this and the pricing difference between cached and uncached input is large enough to change architecture decisions.

If you are sending the same large prefix repeatedly and not using caching, you are lighting money on fire. Check your provider's docs today.

the competitive picture#

Google spent 2023 and most of 2024 visibly behind. They are not behind now. The combination of a genuinely competitive model, the largest context window, TPU economics that let them price aggressively, and distribution through products a few billion people already use is a strong hand.

The thing they still have not solved is the developer experience. The API surface is more confusing than it needs to be, the model naming is chaotic, and the documentation assumes you already know how Google Cloud works. Those are fixable and they are currently costing real adoption.

The company's own framing is unusually candid: this is the last of the non-chain-of-thought line, it is better at the things large pretrained models are better at — world knowledge, writing quality, fewer hallucinations, something they describe as improved "EQ" — and it is not designed to compete on math and coding benchmarks against reasoning models.

That framing is correct and it is also an obituary.

what the scaling curve says now#

For most of 2020 to 2023, the answer to "how do we get a better model" was "make it bigger." Scaling laws were the field's organizing principle. GPT-4.5 is what you get when you keep pulling that lever with 2024-era techniques, and what you get is: meaningfully better at some qualitative things, not competitive on the benchmarks people actually optimize for, and expensive enough that the economics are hostile.

Meanwhile, test-time compute — spending inference tokens on reasoning — produces larger gains on hard problems for far less capital. The lever moved.

This does not mean pretraining scale is dead. Reasoning models are built on pretrained base models and a better base makes a better reasoner. It means the marginal dollar goes to post-training and inference compute rather than to another pretraining order of magnitude.

where the big model is actually better#

Worth being fair to it, because the discourse is going to flatten this into "GPT-4.5 flopped."

Large non-reasoning models are genuinely better at:

• Writing that sounds like a person. Reasoning models often produce prose that reads like a report. This one does not.
• Broad factual recall. More parameters means more memorized world.
• Fewer confident fabrications on knowledge questions, per OpenAI's own hallucination evaluations.
• Following a conversation with implicit context over many turns.

If your product is a writing tool, a support agent, or anything where tone and recall matter more than multi-step logic, a bigger base model is the right choice and always was.

the pricing problem#

The price is the story for most developers. At those rates, the set of applications where GPT-4.5 is the correct economic choice is small. It is a model for cases where quality dominates cost, which is a real category and a narrow one.

Expect this to be the pattern going forward: a small number of very expensive models for the top of the quality curve, a large middle of cost-effective workhorses, and cheap small models absorbing everything routine. Route accordingly, and instrument your routing so you know what you are actually spending per feature.

the historical note#

Someone will write a retrospective in a few years that treats February 2025 as the moment the pure-scale thesis visibly stopped being the main event. They will be oversimplifying, because these transitions are always gradual and the narrative is always cleaner in hindsight.

But they will not be wrong about the direction.

Take the benchmark numbers with the usual salt — they were presented by the vendor, some were shown with consensus-of-N sampling against competitors' single samples, and the field has no agreed protocol for this. The comparison charts in a launch livestream are marketing artifacts.

The genuinely notable thing is not the model. It is Colossus.

the cluster#

xAI built a datacenter in Memphis housing on the order of 100,000 H100-class GPUs, and did it on a timeline measured in months rather than years. The conventional wisdom on a buildout of that scale was eighteen to twenty-four months. They compressed it by doing things that are expensive and unglamorous: bringing in mobile gas turbines for interim power, running their own networking integration, and accepting a lot of operational risk.

Whether you find that admirable or reckless depends on your priors and on how you feel about the air quality complaints from the surrounding neighborhood, which are a real and ongoing dispute worth reading about separately.

But as an engineering datapoint it matters: it establishes that the time constant for standing up frontier-scale compute is shorter than the industry assumed, if you are willing to spend and to eat the risk.

what that implies#

If a well-capitalized new entrant can go from nothing to frontier-scale compute in about a year, then compute is not a durable moat. It is a capital requirement, which is a different thing. Capital requirements keep out the under-funded; they do not keep out the well-funded.

Which pushes the question of where the actual moat is:

• Data — increasingly contested, increasingly litigated, and the frontier labs are all converging on similar synthetic-data-plus-RL recipes anyway.
• Talent — mobile, expensive, and being bid on aggressively.
• Distribution — this one is real. A model inside a product a billion people already open is worth more than a marginally better model behind a signup form.
• Cost per token at quality — real, and derived from architecture and serving engineering rather than raw scale.

My read is that distribution and serving efficiency are the durable ones, and that is a much less romantic answer than "we have the smartest model."

for people who ship things#

Practical implication: assume model quality converges and plan accordingly. Do not build your product on the assumption that one vendor's model stays two steps ahead. Build the abstraction layer, keep your evals in your own repo, and make switching a config change.

The teams that did this in 2024 spent a boring week on it and have been changing providers casually ever since. The teams that did not are having architecture meetings.

This is the most consequential open model release since Llama 2, and possibly since Llama 1.

what it is#

R1 is a chain-of-thought model in the same family as OpenAI's o1 — it produces a long internal reasoning trace before answering, and it spends more compute at inference time on harder problems. On the standard reasoning benchmarks (competition math, coding, graduate-level science questions) it lands in o1's neighborhood.

The training story is the part worth reading. The report describes R1-Zero, trained with reinforcement learning directly on the base model with no supervised fine-tuning stage at all, using rule-based rewards for verifiable domains — did the math answer match, did the code pass the tests. R1-Zero developed reasoning behavior spontaneously, including a documented moment where the model's trace reconsiders its own approach mid-solution.

R1-Zero's output was unreadable — language mixing, poor formatting — so R1 proper adds a cold-start supervised stage and a second RL pass to fix presentation. But the core finding stands: you can get reasoning to emerge from RL against verifiable rewards without a large human-labeled reasoning corpus.

why the license matters more than the benchmark#

MIT. Not a bespoke community license with a monthly-active-user carve-out. Not "research only." MIT, on the weights and the distilled variants.

That means anybody can fine-tune it, run it on their own hardware, ship it in a product, and never send a token to anyone's API. For regulated industries that have spent two years unable to get legal approval for a hosted frontier model, this is a door opening.

The distilled models are the practical story for most developers. The 32B distill runs on a single high-memory consumer GPU and is genuinely useful. The 7B and 14B variants run on a laptop.

the pricing#

DeepSeek's own API is priced at roughly a small fraction of comparable reasoning-model pricing from US labs. Whether that reflects genuinely lower serving costs, an architecture advantage from their mixture-of-experts design with a small number of active parameters, or a decision to buy market share, the effect on the market is the same.

what to actually watch#

Not the benchmark table. Watch these three things:

1. How fast the recipe gets reproduced. If RL-on-verifiable-rewards is as generalizable as the paper suggests, expect a wave of reasoning fine-tunes on other base models within weeks.
2. Whether the distills hold up outside benchmarks. Distilled reasoning traces can look right and be right for the wrong reasons.
3. The regulatory reaction. A capable open reasoning model trained outside US export-control jurisdiction is going to generate a policy conversation whether or not that conversation is technically coherent.

Download the weights. Whatever else happens, they exist now and cannot be un-released.