tech, developers, and the code underneath

issue 007· news·

The day the market priced in efficiency

Nvidia lost roughly $600 billion of market value in a session. The trigger was a paper about training costs.

Nvidia closed down about 17% today. Broadcom, Vertiv, Constellation Energy and most of the power-adjacent complex fell with it. The proximate cause was a week-old model release from a Chinese lab and a number in its technical report.

the number#

DeepSeek's V3 paper stated a final training run cost of roughly $5.6 million in GPU-hours. That figure travelled around the world in about four days, usually stripped of every qualifier attached to it.

The qualifiers matter enormously:

  • It is the cost of the final run only. It explicitly excludes research, failed runs, ablations, and data pipeline work — which in any frontier lab is the overwhelming majority of total spend.
  • It excludes the capital cost of the cluster itself.
  • It says nothing about R1's RL training, which came later and separately.

So "they trained a frontier model for $5.6M" is not what the paper says. It is what the internet decided the paper says.

why the market reacted anyway#

Because the directionally correct read survives the correction, and the market is a machine for reacting to directionally correct reads badly.

The directionally correct read: architectural efficiency gains are real and large. DeepSeek's mixture-of-experts design activates a small fraction of total parameters per token. Their multi-head latent attention cuts KV cache size substantially. Their FP8 training pipeline halves memory traffic against BF16. These are engineering wins, they are published, and they are reproducible.

If capability-per-FLOP is improving that fast, then the number of FLOPs you need to buy to reach a given capability is falling. That is the thesis that repriced today.

the counter-thesis#

Jevons. If compute gets cheaper per unit of capability, you do not buy less of it — you find more things to do with it. Reasoning models in particular consume enormous inference compute; a model that thinks for thirty seconds before answering is a very different demand curve than one that answers immediately. Efficiency gains in training get spent on inference.

Both theses are defensible. The honest answer is that nobody knows the shape of the demand curve, and a 17% single-day move in the largest company on earth is not a considered judgement about that. It is a positioning unwind.

for engineers specifically#

The useful lesson has nothing to do with stock prices. It is this: the performance-per-dollar frontier is moving fast enough that any architecture decision you make today assuming current inference costs will be wrong within a year, in your favor.

Do not build elaborate caching and routing infrastructure to shave token costs that are going to fall by an order of magnitude anyway. Build the thing. Measure it. Optimize when the bill actually hurts.

That advice would have been wrong in most previous computing eras. It is right now, and it will stop being right at some point, and watching for that moment is most of the job.

Dom, January 27, 2025

get README in your inbox

One dispatch, no noise. Tech and developer news, plus the occasional long piece on the craft.

subscribe →