Here is the tedious work, in a manageable order.

the audit, in one afternoon#

For each service, answer four questions. Write the answers down.

1. What identity does it run as? Not "a service account" — which one. A surprising number of services run as something shared, or as a human's credentials that were used for the initial deploy and never replaced.

2. What can that identity do? Enumerate the actual permissions. In most cloud platforms this is one API call. The answer is usually much broader than anyone expects, because permissions accumulate and nothing removes them.

3. What does it actually use? This is the gap. Cloud providers log every authorised call — CloudTrail, audit logs, equivalents elsewhere. Ninety days of logs tell you precisely which permissions were exercised.

4. What is the delta? Everything granted and never used is a candidate for removal, and that set is typically most of the grant.

That fourth number is the finding. In every audit I have seen, a service uses somewhere between 5% and 20% of what it is permitted to do.

the specific over-grants to look for#

Wildcards. s3:* on *. Almost always the result of "it wasn't working so I broadened it until it did," which is a debugging technique that becomes a permanent security posture.

Write access where only read is used. Extremely common for anything reading configuration or reference data.

Access to everything of a type. A service that needs one bucket having access to all buckets. Scope to the resource, and where possible to a prefix within it.

Permissions for a feature that was removed. The code went; the grant stayed.

Human roles used by machines. A deploy pipeline running as a role designed for an engineer's console access — which usually includes the ability to change permissions, and that is the one that turns a compromise into a takeover.

iam:* or its equivalent. The permission to grant permissions. Anything with this is effectively an administrator regardless of what else it has. Treat it as its own category and audit it separately.

the direction to move#

Short-lived over long-lived. Workload identity — the container proves what it is and receives a token that expires in minutes — removes the stealable credential entirely. This is the single highest-value change on this list, and it is architectural rather than a matter of tightening a policy.

Scoped over broad. One bucket and one prefix, not the account.

Read over write. Split the identity if the same service does both, so the read path cannot write.

Deny at the boundary. A service in a network that cannot reach the internet cannot exfiltrate, whatever its IAM permissions say. Network policy and identity policy are independent layers and both matter.

the thing that makes it stick#

An audit is a snapshot. Permissions creep back the next time something does not work at 6 p.m. on a Friday.

Two mechanisms hold the line:

Permissions in code, reviewed like code. If a grant requires a pull request, the broad one gets a comment. If it is a console click, it does not.

A scheduled review. Quarterly, using the same used-versus-granted delta. Twenty minutes per service, and it catches both the emergency grant nobody reverted and the feature that was deleted.

the honest framing for a sceptical audience#

Least privilege does not prevent compromise. It bounds what a compromise costs.

That is the argument to make when someone asks why it is worth the effort: the question is not whether a credential will leak — a dependency, a laptop, a log file, a misconfigured bucket, eventually one will. The question is whether the answer to "what could they do with it" is "read one bucket" or "anything."

Those two answers are the difference between an incident report and a breach notification, and the work that separates them is an afternoon per service.

1. stop long-lived credentials existing#

The single highest-value change, and it is architectural rather than a tool.

A static credential can be stolen, leaked, committed, logged, or exfiltrated from a developer laptop. A credential that lives for fifteen minutes and is scoped to one operation cannot be usefully stolen.

Workload identity. Your CI job, your container, your function proves its identity to the cloud provider and receives a short-lived token. No stored secret at all.

# GitHub Actions with OIDC — no stored cloud credentials
permissions:
  id-token: write
steps:
  - uses: aws-actions/configure-aws-credentials@v4
    with:
      role-to-assume: arn:aws:iam::123456789012:role/deploy
      aws-region: us-east-1

That eliminates the most commonly stolen class of credential entirely. If you do one thing from this article, do this one.

Database credentials can work the same way — IAM authentication, or dynamically generated credentials from a secret manager with a short lease.

2. keep them out of the places they end up#

Not in version control. Obvious, universally violated. Run a scanner in pre-commit and in CI. Both — pre-commit catches it before it happens, CI catches it when someone skipped the hook.

Once committed, assume compromised. Rewriting history does not help; the object is in every clone and in every fork. Rotate, then clean up.

Not in the container image. Layers are inspectable. docker history and a layer extraction tool will find it.

Not in environment variables, ideally. This is more controversial. Environment variables leak: into crash dumps, into child processes, into /proc, into logs when someone prints the environment for debugging, into error tracking services that capture context.

Files with restrictive permissions, mounted at runtime, are better. Environment variables are convenient and are the pragmatic choice for many systems — just know what you are accepting.

Not in logs. Redact at the logger, with a deny-list of key names, not at each call site. Somebody will forget at a call site. The logger never forgets.

REDACT = {"password", "token", "secret", "api_key", "authorization", "cookie"}

def scrub(d):
    return {k: ("***" if k.lower() in REDACT else v) for k, v in d.items()}

Not in error tracking. Most error trackers capture local variables and request headers by default. Configure the redaction, then verify it by triggering a test error and reading what arrived.

3. rotate, and test the rotation#

Rotation is only real if it has been executed. A rotation procedure that has never run is a document, not a control.

The property that makes rotation painless: support two valid credentials at once. Add the new one, deploy, verify, remove the old one. Without overlap, rotation requires downtime, so it does not happen.

Design for this when you create the credential, not when you need to rotate it under incident pressure.

4. scope narrowly#

A credential should permit exactly what its holder needs.

• Read-only where writes are not needed.
• One bucket, one prefix, not the account.
• One database, one schema, one set of tables.
• Time-bounded where possible.

The test: if this credential leaked, what could an attacker do? If the answer is "anything," the scope is wrong regardless of how well you protect it.

5. know what you have#

An inventory of every secret: what it is, where it lives, what it grants, who owns it, when it was last rotated.

Most organizations cannot produce this, which means they cannot answer "what do we rotate" during an incident, which turns a two-hour response into a two-day one.

the incident procedure#

When a secret is exposed, in this order:

1. Rotate first. Before investigating, before cleanup, before the postmortem. Every minute the credential is valid is a minute of exposure.
2. Then assess what it could reach, and check logs for use.
3. Then clean up the exposure.
4. Then work out how it happened.

The common mistake is investigating first. While you investigate, the credential is live.

the tooling note#

Every major cloud has a secret manager. They are all adequate. The dedicated tools add dynamic credential generation, fine-grained policy, and audit — genuinely useful at scale and not where to start.

Start with: OIDC workload identity for machine-to-machine, a cloud secret manager for what remains, scanning in CI, redaction in the logger, and a tested rotation procedure.

That covers the overwhelming majority of real-world credential compromise, and it is a week of work rather than a platform migration.

Enough time has passed to say what actually happened rather than what everyone predicted.

what changed#

The norm inverted. Before, crawling was permitted by default and robots.txt was a request. Now, for a large share of the web, crawling is denied by default and access is a negotiation.

That is a genuine structural change to how the web works and it happened through one company's configuration default rather than through any standards process, legislation, or public deliberation.

Licensing deals concentrated. Large AI companies negotiated bulk access with large publishers. That was always the likely outcome: the parties with lawyers and leverage made arrangements, and the arrangements are private.

Small publishers got very little. The pay-per-crawl mechanism works, technically. The revenue for a site with modest traffic is negligible — the arithmetic never supported anything else. The publishers who most needed a new economic model got the one that pays least.

Non-commercial crawling got harder. Academic researchers, archivists, and independent search projects have no licensing budget and no negotiating position. Carve-outs exist and are discretionary, which means the ability to study the web is now something you apply for.

That is the outcome I was most worried about and it is the one that materialized most clearly.

what did not change#

Training data supply. The frontier labs have enormous existing corpora, licensed sources, and synthetic generation. The marginal value of newly crawled web text was already declining. Restricting it did not create the leverage publishers hoped for.

Traffic. Referral traffic from search to publishers continued its decline, driven by AI answers in search results, which is a completely separate mechanism from training crawlers and which blocking crawlers does nothing about.

This is the part that was most misunderstood at the time. The traffic problem and the training problem have different causes and blocking crawlers only addresses one of them — the one with less economic impact.

the thing to actually take from it#

Infrastructure defaults are policy. A configuration change at a chokepoint reshaped access to a large fraction of the web, with no process and no appeal.

That is not a criticism of the specific decision, which was popular and defensible. It is an observation about where power actually sits, and it generalizes: the entities that can change the web's behavior are the ones with concentration at a layer everyone depends on, and there are about five of them.

For your own site, the decision remains yours and it is worth making deliberately rather than accepting a default:

• Documentation sites frequently want to be in the training data. Being the thing the model knows about is worth more than the pageview you did not get.
• Original reporting and analysis has a stronger case for restriction.
• Anything you want found should still permit search crawlers, which are a different category and are frequently blocked by accident when people configure this.

Check what you are actually blocking. A meaningful number of sites blocked their own search indexing in the first months of this and did not notice for weeks.

the unresolved thing#

The web's economic model — publish freely, get traffic, monetize traffic — is breaking, and nothing has replaced it.

Crawler tolls are not the replacement; the arithmetic does not work at the scale of the actual web, where most content is made by people with no ability to negotiate anything.

Licensing deals are not the replacement either; they work for a few hundred large publishers and for nobody else.

I do not know what the replacement is. I am increasingly convinced that nobody does, and that the interval between the old model failing and a new one existing is going to be long and is going to be bad for the open web.

That is a genuinely pessimistic conclusion and I have not found a way around it in a year of thinking about it.

From an engineering perspective this is the most demanding recurring event in consumer computing, and almost nothing about how it works gets written up.

the shape of the demand#

Synchronized, not smooth. Streaming traffic for a scheduled match is a step function. Millions of concurrent sessions establish within a two-minute window around kickoff, and the ones that fail to establish are a product failure with no recovery — the user missed the start.

Multi-peak within a session. A goal produces a spike in social traffic, in betting platforms, in messaging, in news sites, and in the streams of people switching from another match. These arrive within seconds of each other and are correlated across completely unrelated companies.

Correlated across the industry. This is the part that is genuinely unusual. Every CDN, every mobile network, every payment processor, and every messaging platform experiences the peak simultaneously. There is no borrowing capacity from a quiet neighbor, because there is no quiet neighbor.

Multi-region with different profiles. Matches in three countries across several time zones means the traffic profile shifts across the tournament in ways that capacity planning must anticipate.

what breaks, historically#

Payment processing at scale. Betting and merchandise platforms see enormous transaction spikes at specific moments. Payment processors are a shared dependency and their capacity is a hard constraint nobody downstream controls.

Mobile networks in venue areas. Sixty thousand people in one place, all trying to upload video. This is a well-understood problem with an expensive solution (temporary cell capacity) and it still degrades.

Authentication systems. Everyone logs in at once. Login is frequently the least scaled part of a streaming stack because it is not on the hot path during normal operation.

The thundering herd on recovery. Something fails, comes back, and every client reconnects simultaneously — causing a second failure. Retry jitter is the single most important line of code in a system like this and it is routinely absent.

Ad insertion. Server-side ad insertion personalizes streams per viewer while keeping segments cacheable. It is a hard constraint and it is a disproportionate source of live-stream incidents.

what the good operators do#

Pre-scale, do not autoscale. Autoscaling responds to load after it arrives. Instance startup is measured in tens of seconds; the demand arrives in one. For a known event at a known time, capacity is provisioned in advance and the autoscaler is a safety net, not the mechanism.

Load shed by feature, not by user. When capacity is short, disable the recommendations, the comments, the statistics overlay — keep the video. A degraded experience for everyone beats a perfect experience for 80% and nothing for the rest.

Multi-CDN with active steering. Not for capacity alone — for the fact that any single CDN will have a bad region on a given day. A client-side or DNS-level steering layer that measures real performance and shifts traffic is the difference between a degraded minute and an outage.

Rehearse. Group stage matches are the rehearsal for the knockout rounds. The teams that treat early matches as production load tests, with instrumentation and a retrospective after each one, are the ones that survive the final.

A war room with authority. Not a monitoring dashboard — a room with the people who can make decisions, including the decision to turn features off, without an approval chain.

the generalizable lesson#

The interesting property here is demand you cannot smooth, shed, or refuse.

Most systems get to spread load over time, queue it, or degrade gracefully by making users wait. None of those work when the product is a live event — a queue means the user misses the goal.

What is left is: provision for peak, make every layer redundant, degrade by feature rather than by user, and rehearse.

That is expensive, unglamorous, and the only thing that works. It is worth remembering the next time someone proposes autoscaling as the answer to a spike that arrives faster than a machine can boot.

Some capacity you have to already own.

The physics is real. The applicability is narrower than the marketing, and the reason is data.

the physics#

Light in fiber travels roughly 200,000 km/s. New York to London and back is about 55 ms of pure propagation, before any processing. Add TLS handshakes, TCP setup, and real-world routing that is not a great circle, and a cross-Atlantic round trip is frequently 100 ms or more.

Running code 20 ms from the user instead of 120 ms is a genuine improvement and it is not achievable any other way.

the problem#

Your data is not at the edge. It is in a database, in one region, and if your edge function needs it, you have moved the compute closer to the user and left the round trip in place — plus added a hop.

user → edge (5ms) → origin database (120ms) → edge → user

That is slower than the user talking to the origin directly, because you added a hop to a path that was always dominated by the database call.

This is the single most common edge computing mistake and it is easy to make, because the architecture diagram looks right.

what edge is genuinely good for#

Anything that needs no origin data:

• Redirects and rewrites. URL normalization, locale routing, legacy path mapping.
• Authentication token validation. A signed JWT can be verified with a public key at the edge, and an invalid request never reaches your origin. This is a real win — you reject bad traffic at the perimeter.
• A/B test assignment. Deterministic hash of a cookie into a bucket. No state required.
• Header manipulation. Security headers, CORS, feature policy.
• Bot filtering and rate limiting. Reject at the edge, before the request costs you anything.
• Personalization of cached content. Fetch the cached page, inject the user's name from a cookie, return. The expensive part stays cached.

Anything where the data is genuinely replicated to the edge:

Several platforms now offer edge-replicated key-value and SQL storage. If your data is small, read-heavy, and tolerant of replication lag — configuration, feature flags, product catalogs, translations — this works well and the latency win is real.

The constraints are real too: writes go to a primary, replication is eventual, and storage per location is limited.

what edge is bad for#

Anything write-heavy. Writes need coordination. Coordination needs a primary. The primary is in one place.

Anything requiring strong consistency. By definition, this needs coordination, which needs round trips, which is what you were trying to avoid.

Anything with a large working set. You cannot replicate a terabyte to three hundred locations.

Anything computationally heavy. Edge runtimes have tight CPU and memory limits. They are designed for milliseconds of work per request.

Anything that needs a specific runtime. Most edge platforms run a constrained JavaScript or WebAssembly environment. Your Python dependency with a C extension is not going there.

the architecture that works#

Layered, with each layer doing what it is good at:

edge      → auth check, rate limit, routing, cached content, header work
regional  → application logic, caching, session state
origin    → the database, the writes, the truth

Most requests are answered at the edge from cache. Some go to a regional application tier. Few reach the origin.

That is a CDN with programmability, which is what edge computing actually is, and framing it that way produces much better decisions than framing it as "serverless everywhere."

the thing to measure first#

Before adopting any of this: where does your latency actually go?

Break down a typical request:

• DNS
• TLS handshake
• Network round trip
• Time to first byte at origin
• Origin processing
• Database time within that
• Response transfer

If origin processing is 400 ms and network is 40 ms, moving compute to the edge addresses 40 ms of a 440 ms problem. Fix the 400 first.

This is the most common reason edge adoption disappoints: it was applied to a latency problem that was not a network problem.

the honest summary#

Edge is a very good CDN with programmability, and that is genuinely valuable — it lets you do real work at the perimeter that used to require an origin request.

It is not a general application platform, and the platforms selling it as one are selling the constraint as a feature.

Use it for the perimeter. Keep your data where it can be consistent.

the layers where it pays#

HTTP responses. Everyone does this and most do it badly. Check yours:

curl -sI -H 'Accept-Encoding: br, gzip' https://example.com/api/data | grep -i content-encoding

If that returns nothing, you are shipping uncompressed JSON, and JSON compresses extraordinarily well — commonly 80–90% for typical API responses, because it is mostly repeated key names.

Brotli beats gzip by roughly 15–20% on text at comparable CPU cost, and is supported everywhere. Use it for static assets at maximum level (precomputed, so the CPU cost is paid once) and at a moderate level for dynamic responses.

Database storage. Most modern databases support per-table or per-column compression. On a table of text or JSON, it frequently halves the storage — which also halves the I/O, which means more of the working set fits in memory, which is where the real win is.

The CPU cost of decompression is almost always smaller than the I/O cost you avoided.

Logs and telemetry. Log shipping is often a meaningful fraction of internal network traffic and vendor cost. Compressed batching typically reduces it by an order of magnitude, and the batching itself reduces request overhead.

Backups and object storage. Storage is cheap and it is not free, and egress definitely is not. Compression at rest is a direct cost reduction with no downside for cold data.

Container images. Zstandard-compressed layers pull faster than gzip, which matters for cold starts and for anything that scales by launching new instances.

Inter-service traffic. gRPC and Protocol Buffers are already compact. If you are sending JSON between services — and most people are — compressing it is a large win that costs one configuration line.

which algorithm#

The default answer is zstd. It has a level parameter spanning from faster-than-lz4 to nearly-as-small-as-xz, decompression is fast at every level, and it has dictionary support.

the dictionary trick#

The most underused feature in compression, and the one with the biggest payoff for small messages.

Compression works by finding repetition. A 200-byte JSON message has almost no internal repetition, so compression barely helps — sometimes it makes it bigger.

But across many messages, there is enormous repetition: the same field names, the same enum values, the same URL prefixes.

A shared dictionary trained on representative samples gives the compressor that repetition up front:

zstd --train samples/*.json -o dict.zst

Then compress each message against the dictionary. Small-message ratios that were 1.1× become 3× or better. For any system moving many small similar messages — event streams, queue payloads, cache values — this is a large and nearly free win.

when not to compress#

Already-compressed data. Images, video, audio, archives. You will spend CPU to make it slightly larger.

Very small payloads without a dictionary. Under a few hundred bytes, compression overhead can exceed the savings.

When you are CPU-bound and not bandwidth-bound. Measure. This is rarer than people assume but it does happen.

Encrypted data, compressed after encryption. Pointless — ciphertext is incompressible. And compressing before encryption can leak information about the plaintext through the ciphertext length, which is the class of attack that includes CRIME and BREACH. If you compress and encrypt, know why it is safe in your context.

the ten-minute audit#

1. Check that HTTP responses are compressed, including API responses, not just HTML.
2. Check that Brotli is enabled, not just gzip.
3. Check your log shipping compresses and batches.
4. Check whether your largest database tables support compression and whether it is on.
5. If you move many small similar messages, train a dictionary.

That is an afternoon and it routinely produces a larger improvement than a month of application-level optimization, at a fraction of the risk.

The reason it does not happen is that nobody gets promoted for enabling Brotli.

It is also the move that introduces the most subtle failure modes, and the gap between "we added a queue" and "we added a queue correctly" is large.

what it buys#

Latency. The user gets a response when the work is accepted, not when it is done. A checkout that returns in 80 ms and sends the confirmation email asynchronously is a much better product than one that returns in 900 ms.

Absorbing spikes. A queue is a buffer. Traffic that would overwhelm a synchronous system accumulates and drains. This is the difference between a slow period and an outage.

Isolation. If the email provider is down, checkout still works. The messages accumulate and send later.

Retry for free. A failed message goes back on the queue. A failed synchronous call is a user-visible error.

what it costs#

Eventual consistency, everywhere. The user completed checkout and the confirmation has not arrived. The record exists and the search index does not have it. Every asynchronous boundary introduces a window where the system is inconsistent, and your UI has to be honest about it.

Debugging across the boundary. A synchronous stack trace tells you the whole story. An asynchronous failure requires correlating a producer, a broker, and a consumer, possibly hours apart.

Ordering. Most queues do not guarantee it, or guarantee it only within a partition. If your consumer must process events in order, that is a design constraint that reaches back into how you partition.

Duplicate delivery. Almost all queues are at-least-once. Your consumer will receive the same message twice. If that is not safe, you have a bug that appears under load, weeks after launch.

the rules#

1. Consumers must be idempotent. Non-negotiable.

At-least-once delivery means duplicates. The consumer must produce the same result whether it processes a message once or five times.

The usual implementation: a natural idempotency key, and a record of processed keys.

def handle(msg):
    key = msg.idempotency_key
    with tx():
        if already_processed(key):
            return
        do_the_work(msg)
        mark_processed(key)

The mark_processed must be in the same transaction as the work, or you have moved the race rather than eliminated it.

2. Every queue needs a dead letter queue, and someone must watch it.

A message that fails repeatedly must go somewhere. A DLQ nobody monitors is a place where data goes to be silently lost, which is worse than an error, because errors are visible.

Alert on DLQ depth. Not on it being non-zero — on it growing.

3. Retry with backoff and jitter, and cap the attempts.

Immediate retry on a failing downstream is a denial of service you are performing against yourself. Exponential backoff with jitter, a maximum attempt count, then the DLQ.

4. Monitor queue depth and age, not just throughput.

Throughput looks healthy right up until it does not. The metrics that tell you something is wrong:

• Depth — how many messages are waiting.
• Oldest message age — the most useful single metric. If it is growing, your consumers cannot keep up, and you know how far behind you are in time rather than in count.

5. Decide what happens when the queue is full.

It will be. Reject the producer, drop messages, or block? Each is right in different cases and the default is usually wrong for you. An unbounded queue is not a solution; it is a memory leak with extra steps.

6. Keep the payload small and the reference stable.

Put an ID in the message, not the whole object. The consumer fetches current state. This avoids stale data in the message and keeps the broker fast.

The exception: if you need the state as it was when the event occurred, put it in the message deliberately, and say so.

7. Version your message schema from day one.

Producers and consumers deploy independently. Old consumers will see new messages. Include a version field. Make additive changes only, or handle both shapes.

the thing that surprises people#

Queues do not reduce load. They defer it.

If your consumer processes 100 messages per second and you produce 150, you do not have a working system with a buffer. You have a system that is failing slowly, and the queue depth graph is a countdown.

A queue absorbs bursts. It does not fix a sustained capacity deficit, and the failure mode when you use it that way is a queue that grows for six hours and then an incident where you are simultaneously behind and unable to catch up.

Alert on the age, watch the trend, and size the consumers for the sustained rate.

It is also, for a majority of the teams running it, a substantial amount of complexity in exchange for benefits they do not receive, and saying so is still mildly heretical.

the problem it actually solves#

Kubernetes was built for: many services, many teams, heterogeneous workloads, on a fleet of machines, where you want bin-packing efficiency and declarative self-healing, and where the platform is operated by people whose job that is.

If you have all of those, it is the right answer and there is no close second.

what it costs#

A permanent learning tax. Pods, deployments, services, ingresses, configmaps, secrets, persistent volume claims, storage classes, service accounts, roles, network policies, resource quotas, and a YAML dialect for each. Every engineer who deploys anything must learn a meaningful fraction of it.

Operational surface. Control plane upgrades, node upgrades, CNI plugin, CSI driver, ingress controller, cert manager, metrics server, log shipper. Each is a component that can break and that must be upgraded on someone else's schedule.

Debugging distance. "Why is my service not reachable" has a dozen possible answers across five layers, and diagnosing it requires understanding all of them.

Cost, frequently. A managed control plane plus nodes sized for the platform's own overhead plus the observability stack it needs is often more than the equivalent capacity on simpler infrastructure.

Resume-driven adoption. This is real and worth naming. Kubernetes on your CV is worth money. That is a genuine incentive pointed away from the simplest solution that works.

the honest threshold#

You probably want Kubernetes if:

• More than roughly fifteen to twenty distinct services, deployed independently.
• More than a handful of teams that need to deploy without coordinating.
• You have someone whose job includes operating the platform, not as a side task.
• Genuinely heterogeneous workloads with different scaling characteristics.
• Multi-tenancy requirements with real isolation needs.

You probably do not if:

• Under ten services.
• One or two teams.
• Nobody owns the platform.
• Traffic is predictable.
• You are running one application with a database.

what to do below the threshold#

The options are better than they were, and all of them are boring:

A platform-as-a-service. Push code, it runs. This is the correct answer for a very large number of applications and the reason people avoid it is usually aesthetic.

Containers on a managed container service without the orchestrator — the various "run this container, scale it, load balance it" products every cloud offers. You get containers, autoscaling, and rolling deploys without the platform.

A couple of servers and a process manager. Systemd units, a reverse proxy with automatic certificates, and a deploy script. This runs an enormous amount of traffic, is trivially debuggable, and every engineer already understands it.

Docker Compose on one machine. For staging, for internal tools, for anything where a single host is enough. Unfashionable, works.

the migration-path argument#

"We will need Kubernetes eventually, so we should start now."

This is the most common justification and it is usually wrong, for two reasons.

First, the complexity cost is paid every day from now until then, and "eventually" frequently never arrives.

Second, containerizing your application is the actual hard part of a future migration, and you can do that without an orchestrator. A containerized application running under a simple process manager can move to Kubernetes later in a couple of weeks.

Build the container. Skip the platform until you have the problem it solves.

the position I will defend#

The teams I have seen most successfully run Kubernetes are large organizations with dedicated platform teams, where it is genuinely the right tool.

The teams I have seen most damaged by it are small ones where a single engineer set it up, that engineer left, and the remaining team is operating a system nobody understands and is afraid to touch.

That second failure is common, expensive, and entirely predictable from the staffing at adoption time. If nobody's job is going to be operating the platform, you should not have a platform.

What did not get settled is how to do it without producing something everyone hates. Here is what a decade of watching this has produced.

the state file is the whole problem#

Declarative infrastructure tools work by comparing three things: your configuration, a recorded state, and reality. Every hard problem comes from those three disagreeing.

State drift. Someone changed something by hand. Now state says one thing and reality says another, and the next apply will either revert their change or fail confusingly.

The fix is process, not tooling: nobody has write access to production infrastructure except the pipeline. Break-glass access exists, is audited, and triggers a drift check afterward. Without this, drift is continuous and IaC becomes theater.

State locking. Two applies at once corrupt state. Every backend supports locking. Verify yours is actually enabled — this is a default that people leave off and discover during an incident.

State as a blast radius. One giant state file means every apply touches everything and a corrupted state loses everything. Split by lifecycle and blast radius: networking, data stores, and applications should not share state.

Rule of thumb: if a plan takes more than a couple of minutes, your state is too big.

modules that became frameworks#

The pattern is universal. A team writes a module to wrap a resource with sensible defaults. It grows parameters. It grows conditionals. Three years later it has forty inputs, nested conditional logic, and nobody can tell what it creates without running a plan.

At that point the abstraction is costing more than the duplication it prevented.

The guidance that holds:

• Wrap only when there is real shared policy — tagging, naming, security baselines. Not to save typing.
• Modules should be shallow. A module that takes one resource and adds organizational defaults is good. A module that composes twelve resources with conditional branches is a framework, and frameworks need maintainers.
• Prefer duplication over the wrong abstraction, more so here than in application code, because infrastructure changes less and the cost of a bad abstraction is higher.
• No conditionals that change what resources exist. count = var.enabled ? 1 : 0 is fine once. Nested versions of it produce configurations nobody can reason about.

the practice that separates teams#

Plan output is reviewed as part of the pull request.

Not "run apply and see." The plan — the exact set of creates, changes, and destroys — posted to the pull request, read by a human, before merge.

Teams that do this catch the accidental destroy, the security group opened to the world, the database replacement that would have caused an outage. Teams that do not find out during apply.

This is a one-day CI setup and it is the single highest-value practice in the whole category.

The corollary: pay attention to every destroy in a plan. Most infrastructure incidents caused by IaC are a resource being replaced when the author expected it to be updated in place. Some attribute changes force replacement. The plan tells you. Nobody reads it.

the things to keep out#

Secrets. Never in the configuration, never in state. State files contain resource attributes in plaintext, including some you would not expect. Use a secret manager and reference it, and encrypt the state backend regardless.

Application deployment. Infrastructure tools are bad at deploying applications — they are declarative and convergent, and deployment is imperative and sequenced. Use them to provision the platform; use something else to deploy onto it.

Anything with a fast change cadence. If a value changes weekly, it should be configuration read at runtime, not infrastructure applied through a pipeline.

the fork question#

The OpenTofu fork means there is now a genuinely open-source implementation with a foundation behind it, alongside the commercial one. Both work. The configuration language is largely compatible.

The practical guidance: this is a lower-stakes decision than it feels. The lock-in is in your modules and your provider usage, not in the binary. Pick based on your licensing requirements and your appetite for the respective governance models, and know that migrating between them is a smaller project than it sounds.

the thing I would tell a team starting today#

Start smaller than you think. One state file per environment per major system. Plain resources before modules. Plan review in CI from day one. No manual changes, ever, enforced by permissions.

The teams that hate their infrastructure code all made the same mistake: they built an abstraction layer before they understood the domain, and then they were maintaining two things.

Several things changed and the answer is now more nuanced than the reflex suggests.

what changed in favor of self-hosting#

Machines got enormous. A single server you can rent for a few hundred dollars a month has more cores, more memory, and dramatically more I/O than a rack of hardware from 2012. A very large number of applications fit on one machine with room to spare.

The tooling got good. Configuration management, container runtimes, reverse proxies with automatic certificates, backup tooling. What required a team a decade ago requires a competent person and a weekend.

Cloud egress pricing did not fall. Compute prices came down. Bandwidth pricing at the major clouds is still a large multiple of what it costs, and for bandwidth-heavy applications it dominates the bill.

Managed service prices are high relative to the alternative. A managed database costs several times what the equivalent instance costs, for operational convenience that is real and is not always worth the multiple.

what changed against it#

Security expectations rose. Patching, hardening, monitoring, incident response. Self-hosting means you own all of it, and the threat environment is worse than it was.

Compliance frameworks assume cloud controls. SOC 2, ISO 27001, and their relatives are achievable self-hosted and the evidence collection is more work.

The talent assumption inverted. A decade ago every team had someone who knew Linux systems administration. Now a lot of teams do not, and hiring for it is harder than hiring for cloud skills.

the honest decision framework#

Self-host when:

• Your workload is steady rather than spiky. Cloud's core value proposition is elasticity, and you are paying for elasticity you do not use.
• Bandwidth is a large share of your bill.
• You have or can hire operational competence.
• Data locality or sovereignty is a requirement.
• You are at a scale where the cloud premium is a meaningful number — which starts lower than most people assume.

Use the cloud when:

• Traffic is spiky or unpredictable.
• You are early and optimizing for speed of iteration over unit economics.
• You need global presence and do not want to operate it.
• Your team's time is better spent on the product, which for most early-stage companies it is.
• Compliance requirements are easier to satisfy with a provider's attestations.

The hybrid that most people should consider: run the steady baseline on owned or rented hardware, burst to cloud for peaks, keep object storage and CDN with a provider. This captures most of the cost advantage without giving up elasticity where it matters.

the middle option nobody talks about#

Between "hyperscaler" and "rack in a colo" there is a large market of dedicated server providers and mid-size clouds: a real machine, in a real datacenter, with network and power handled, for a monthly fee.

You get root, predictable performance without noisy neighbors, and bandwidth allowances that are not priced as a profit center. You do not get managed databases, autoscaling, or a hundred adjacent services.

For a very large number of applications this is the correct answer and it is under-considered because the discourse is binary.

the operational minimum#

If you self-host, these are non-negotiable:

• Automated, tested restores. Not backups — restores. A backup you have never restored is a hypothesis. Test it quarterly, on a schedule, with a timer running.
• Unattended security updates, at least for the OS.
• Monitoring with alerting that reaches a human. Disk full is the most common self-hosted outage and it is entirely preventable.
• Infrastructure as code. The machine must be reproducible. If rebuilding it requires someone's memory, you have a single point of failure that is a person.
• A documented runbook for the failures you expect: disk, certificate expiration, service crash, host failure.

That is a weekend of setup and a few hours a month. If nobody on the team will own those hours, use the cloud — that is a legitimate reason and it is the actual deciding factor more often than cost is.

the thing that changed my mind#

I used to treat "we run our own servers" as a red flag. I now treat "we are on the cloud and have never modeled the alternative" as an equal one.

Both are defaults applied without analysis. The analysis takes an afternoon and the answer is frequently not what the reflex says.

It is still worth understanding what it actually changed, because a few of the consequences affect how you build.

the problem it solved#

HTTP/2 introduced multiplexing: many logical streams over one TCP connection. That fixed head-of-line blocking at the HTTP layer.

It did not fix it at the TCP layer. TCP delivers bytes in order. If one packet is lost, everything behind it waits, including data for streams that were completely unaffected. On a lossy connection — mobile, congested wifi — HTTP/2 could be worse than HTTP/1.1 with six connections, because one loss stalled everything instead of one sixth of everything.

QUIC moves the transport to UDP and implements reliability, ordering, and congestion control per stream. A lost packet stalls only the stream it belonged to.

what else came with it#

Encryption is mandatory and integrated. TLS 1.3 is part of the protocol rather than a layer on top. The handshake is one round trip, or zero for a resumed connection.

Connection migration. A QUIC connection is identified by a connection ID, not by the four-tuple of IP addresses and ports. Change networks — wifi to cellular — and the connection survives. Your download does not restart.

This is the feature users notice without knowing why. Walking out of a building while a video plays used to stall it.

Better loss recovery. QUIC distinguishes between packet loss and reordering more accurately than TCP, and its acknowledgment format carries more information. Recovery is faster.

Evolvability. TCP is implemented in kernels and middleboxes and cannot change, because the internet is full of devices that will drop anything unfamiliar. QUIC is in userspace and encrypted, so its internals are invisible to middleboxes and can actually be updated.

That last point is arguably the most important long-term consequence. Transport protocol ossification was a genuine crisis and QUIC is the escape hatch.

the practical consequences for you#

Domain sharding is now actively harmful. Splitting assets across static1.example.com and static2.example.com was a workaround for HTTP/1.1's connection limit. Under HTTP/2 it was pointless. Under HTTP/3 it is worse than pointless, because each domain requires a separate connection with a separate handshake and separate congestion state.

One origin. If you still have sharding from a 2014 optimization guide, remove it.

Concatenating and spriting are counterproductive. Same reasoning. Many small files multiplex fine and cache better individually.

Priority matters and is under-configured. HTTP/3 has an extensible priority scheme. Most servers use defaults. If you have a page where certain resources are critical, priority hints (fetchpriority) are worth setting and are widely supported.

UDP blocking is real but small. Some corporate networks block UDP/443. Clients fall back to HTTP/2 automatically, so this is a performance issue rather than a correctness one. Do not build anything that requires HTTP/3.

Your observability may not see it. A lot of network monitoring tooling was built for TCP. Check whether your tools actually understand QUIC or are silently reporting nothing.

the parts that were harder than expected#

CPU cost. QUIC's userspace implementation and per-packet encryption use more CPU than kernel TCP. This has improved substantially with offload support and better implementations, and it is a real cost at high volume.

Middlebox hostility. Some networks throttle or block UDP because it looks like something they should throttle. This is improving as QUIC becomes normal traffic.

Debugging is harder. You cannot read a QUIC connection with tcpdump the way you could read HTTP/1.1. qlog and browser devtools help. The tooling is younger.

the assessment#

For a user on a good connection, HTTP/3 is roughly a wash. For a user on a bad connection — mobile, congested, high latency, lossy — it is a substantial improvement, and those users are a large fraction of the world.

That is exactly the right kind of improvement: invisible to the people who were already fine, meaningful to the people who were not.

Enable it, remove your HTTP/1.1-era workarounds, and go back to not thinking about the transport layer.

the unit of sale#

The thing being sold is a rack-scale system — dozens of accelerators, a co-designed interconnect, integrated liquid cooling, and a power distribution architecture — that arrives as a unit and is installed by people who specialize in installing it.

That is a fundamentally different business from selling PCIe cards, and it is the moat. Competing on a single accelerator's FLOPs per watt is a fight several companies can have. Competing on an integrated rack with a proprietary high-bandwidth interconnect between every device in it is a fight almost nobody can have, because the interconnect is where a decade of engineering lives.

the power numbers#

The per-rack power figures continue climbing. The industry has moved from "how many kilowatts per rack" to "how many racks per megawatt," which is a different mental model.

The consequences, which are the actual story:

Existing datacenter shells are mostly unusable. A facility designed for 10 kW racks cannot host these regardless of floor space. The buildout is new construction, near new substations, on a permitting timeline.

Liquid cooling is not optional and not new. Direct-to-chip liquid cooling is now the baseline, and the interesting engineering has moved to facility-level heat rejection — where does the heat go, and can it be sold to someone.

Higher-voltage DC distribution is being adopted to reduce conversion losses and copper mass at these current levels. This is a genuine architectural change to how datacenters distribute power and it is being driven entirely by this workload.

the software announcements#

The inference serving stack continues to be where the practical developer value is. Disaggregated serving — running prefill and decode on separate hardware pools because they have completely different compute and memory characteristics — is now mature enough to be the default recommendation rather than an optimization.

If you operate inference at any scale and have not looked at disaggregation, that remains the largest single efficiency win available. Prefill is compute-bound and parallelizable. Decode is memory-bandwidth-bound and sequential. Running them on one homogeneous pool wastes a substantial fraction of your silicon on whichever phase is not currently limiting.

the competitive picture#

Every hyperscaler now has its own accelerator in volume production. The software gap versus CUDA remains the deciding factor, and it is narrowing slowly rather than quickly, because CUDA's advantage is fifteen years of accumulated libraries, kernels, tooling, and institutional knowledge rather than any single technical feature.

The realistic near-term outcome is not displacement. It is a market where hyperscalers run their own silicon for their own high-volume internal workloads and buy Nvidia for everything else, which is enough to constrain pricing without threatening the position.

what a developer should take from this#

Almost none of it directly. You will not buy one of these. You will rent time on one.

What matters to you:

• Capacity comes in steps, tied to construction schedules. Plan around availability, not just price.
• Efficiency work compounds. Every optimization that reduces tokens, batches better, or caches more is worth more in an environment where the underlying resource is physically constrained.
• The abstraction layer is your friend. The hardware underneath your API calls will change several times. If your code cares, that is a design problem you created.

the sentence that summarizes the era#

The bottleneck on artificial intelligence is currently electrical engineering and civil construction, and it has been for about two years.

Every discussion about model capability that ignores this is discussing a hypothetical.

Live streaming a global sporting event has a combination of constraints that almost nothing else has.

why it is hard#

The demand is synchronized. Web traffic is normally a smooth curve. A live event is a step function — millions of people join within the same sixty seconds, at a moment you know in advance and cannot move.

You cannot shed load. The standard answer to overload is to degrade: serve cached content, drop non-essential features, queue people. None of those work when the product is a live video feed. A queue means the user misses the thing.

Latency is a correctness constraint. If your stream is forty seconds behind broadcast, viewers learn the result from their phone before they see it. That is a product failure with no technical symptom — every metric is green and the experience is ruined.

Peak is unpredictable within a predictable window. You know the event starts at 8 p.m. You do not know that a particular race will be close and that traffic will spike 3× in the final ninety seconds.

You cannot test at scale. There is no staging environment with fifty million concurrent viewers. Load tests approximate. The real event is the test.

the architecture, roughly#

Ingest — camera feeds encoded at the venue, sent over dedicated circuits, with redundant paths because a fiber cut during a final is a career event.

Transcode — one source becomes a ladder of bitrates and resolutions, times several codecs, times audio tracks and languages. That is a large multiplication and it happens in real time with no room to fall behind.

Packaging — segmented into chunks, typically 2 to 6 seconds, in HLS and DASH. Low-latency variants use much smaller chunks or chunked transfer encoding to cut the delay, at the cost of more requests and worse cache behavior.

Distribution — multiple CDNs, always. Not for capacity alone but because CDNs have bad days, and a client-side switching layer that measures performance and moves traffic is the difference between a degraded minute and an outage.

The client — adaptive bitrate logic that decides which quality to request based on measured bandwidth and buffer level. This is where a surprising amount of the perceived quality difference between services actually lives.

the parts that are counterintuitive#

Cache hit ratio is everything, and low-latency streaming ruins it. A 6-second segment requested by a million people is one origin fetch and a million edge hits. Cut to 1-second segments for lower latency and you have six times the requests, each with a shorter window to accumulate hits. Latency and efficiency are in direct tension and every service picks a point on that curve.

The last mile is not yours and dominates the experience. Home wifi, congested cell towers, and oversubscribed ISP links cause most of the buffering users blame on you. The only lever you have is graceful adaptation — dropping quality smoothly rather than stalling.

Advertising insertion is a distributed systems problem. Server-side ad insertion means personalizing a stream per viewer while keeping segments cacheable, which is a genuinely hard constraint and is where a lot of live-stream failures actually originate.

The failure mode that matters is the thundering herd on recovery. When something breaks and comes back, every client reconnects at once. Without jitter on the retry, recovery causes a second outage. This is the single most common way these events go badly.

the lesson that generalizes#

Almost every hard part here is about synchronized demand you cannot smooth and cannot refuse.

Most systems get to spread load over time, shed it, or queue it. When you cannot do any of those, you are left with provisioning for peak, redundancy at every layer, and graceful degradation that preserves the core experience.

That is expensive and it is the only thing that works. Which is worth remembering when someone proposes autoscaling as the answer to a spike that arrives faster than an instance can boot.

Some problems you solve with capacity you already own.

the mechanism#

A datacenter campus needs a lot of power at one point on the grid. Delivering it requires transmission upgrades, substation construction, and often new generation. Those cost billions and take years.

Who pays is decided by state public utility commissions in rate cases. The options:

• The datacenter pays, through special contracts with minimum-take provisions and upfront infrastructure contributions.
• All ratepayers pay, spread across the customer base, which is how transmission has traditionally been socialized.
• Some blend, which is what actually happens.

Utilities generally prefer the blend, because socialized costs are easier to recover and because they earn a regulated return on capital investment — which means building more infrastructure is directly profitable for them regardless of who uses it.

Consumer advocates object. Datacenter operators object to bearing the full cost. Commissions split the difference, differently in every state.

why this is now the binding constraint#

Chips are available. Capital is available. What is not available is a grid interconnection on a useful timeline.

Interconnection queues in most US markets run years. Transformer lead times remain long. Transmission line construction requires siting approval, which requires public process, which is where projects die.

So the sequence for new capacity is: secure a site with grid access, sign a power agreement, wait. The waiting is the schedule. Everything else can be compressed with money; this cannot.

what it means for compute prices#

Two effects pulling in opposite directions.

Regional differentiation increases. Regions with available power and cooperative regulators get capacity. Regions without do not. That produces real price differences between cloud regions for the same instance type, larger than the historical spread.

Off-peak becomes genuinely cheaper. Grid economics are about peak demand. Workloads that can shift to off-peak hours are worth real money to operators, and that value will get passed through as pricing. Batch inference, training runs, and anything asynchronous is a candidate.

what to actually do#

Check regional pricing before you pick a region by habit. The default region in your organization was chosen years ago for reasons that may no longer hold. For a large workload the difference is material.

Make batch work time-flexible. If your nightly job can run in a four-hour window instead of at a fixed time, you can take spot capacity and off-peak pricing. This is a small engineering change with a large cost effect and almost nobody does it.

Measure your actual utilization. The cheapest watt is the one you do not use. A very large fraction of provisioned cloud compute runs at low utilization, and in an environment where power is the constraint, that waste is now expensive rather than merely inelegant.

Watch your provider's regional capacity announcements if you are planning anything large. Capacity comes online in steps tied to substation energization dates, and knowing the schedule is worth something.

the part that is not about your bill#

There is a legitimate public policy question here that the industry mostly does not want to engage with: whether the cost of connecting datacenters should be borne by households whose electricity bills are rising.

The industry's answer is usually that datacenters bring jobs and tax revenue, which is true and is a smaller number than the capital involved. The honest version is that the buildout is happening faster than the institutions that allocate its costs can deliberate, and the allocations being made now will be argued about for a decade.

Engineers do not decide this. Engineers do decide how much power their systems consume, and that has stopped being an abstract concern.

harvest now, decrypt later#

The threat model that makes this urgent has nothing to do with when a cryptographically relevant quantum computer arrives.

An adversary records your encrypted traffic today and stores it. When they can break the key exchange — in five years, in fifteen — they decrypt the archive.

So the question is not "when will quantum computers work." It is: how long does your data need to stay confidential?

• Session tokens: minutes. Do not care.
• Customer PII: years to decades. Care a lot.
• Medical records: a lifetime. Care enormously.
• Government and defense: generational.
• Source code and trade secrets: depends, usually longer than you think.

If anything you transmit has a confidentiality horizon past roughly 2035, traffic you send today is already at risk. That is the whole argument and it does not depend on any prediction about quantum hardware.

what is standardized#

NIST finalized the core standards:

• ML-KEM (FIPS 203), formerly Kyber — key encapsulation. This is the one that matters for TLS.
• ML-DSA (FIPS 204), formerly Dilithium — digital signatures.
• SLH-DSA (FIPS 205), formerly SPHINCS+ — hash-based signatures, conservative fallback with larger signatures.

The guidance across national security agencies is consistent: begin migration now, complete it well before 2035, prioritize by confidentiality horizon.

what is already shipping#

More than most people realize.

TLS key exchange. Hybrid X25519 plus ML-KEM is deployed by default in major browsers and supported by major CDNs and load balancers. A meaningful fraction of web traffic is already post-quantum protected for key exchange, and most people running those services did not do anything to enable it.

Check yours:

openssl s_client -connect example.com:443 -groups X25519MLKEM768 </dev/null 2>&1 \
  | grep -i "negotiated\|group"

SSH. OpenSSH has shipped post-quantum key exchange by default for several releases. If your servers are current, your SSH sessions are already hybrid.

Signatures are lagging, and that is the harder half. Certificate chains, code signing, and firmware verification all involve long-lived trust anchors and ecosystem-wide coordination. ML-DSA signatures and public keys are substantially larger than ECDSA, which has real consequences for handshake size, embedded devices, and anything with a fixed-size signature field.

what to actually do this year#

1. Inventory your cryptography. Where do you use asymmetric crypto, with what algorithm, and what is the confidentiality or integrity horizon? Most organizations cannot answer this and the exercise is more valuable than any individual migration step.

2. Get to hybrid key exchange. For most people this means: update TLS libraries, update the load balancer, verify the negotiated group. It is largely free and it addresses the harvest-now threat, which is the urgent one.

3. Build crypto-agility. The lasting fix is not "migrate to ML-KEM." It is "be able to change algorithms without a rewrite." Hard-coded algorithm choices scattered through a codebase are the actual problem, and they will be the problem again for whatever comes after this.

Centralize crypto in one module. Make the algorithm a configuration value. Version your protocol so you can negotiate.

4. Ask your vendors. Every SaaS provider, every library, every hardware security module. The answers will be uneven and asking creates the pressure that fixes it.

5. Do not roll your own. Use the vetted implementations. The failure mode for post-quantum crypto is a side-channel in a hand-written implementation of a lattice operation, and that failure is silent.

the thing that makes this hard#

It is a migration with no visible benefit. Nothing gets faster. No feature ships. The success condition is that in fifteen years, nothing bad happens.

That is the hardest kind of project to fund, and it is why the organizations that do it well will be the ones that started when it was still early enough to be cheap.

the actual trend#

Every category at this show is now constrained by thermals and power delivery rather than by compute.

Laptops with neural accelerators that cannot sustain their rated throughput for more than a few minutes in a thin chassis. The TOPS number on the sticker is a peak, and the peak lasts as long as the thermal mass does.

Handhelds where battery life is the entire product decision and every added capability is a subtraction from it.

Home devices that want to run models locally and cannot, because the power envelope of something that sits on a shelf is a few watts.

Desktops where the power supply recommendations for a current-generation GPU have crept past what a lot of household circuits comfortably deliver alongside everything else in the room.

This is what a computing era looks like when it hits a physical wall. The interesting engineering for the next several years is efficiency, not capability — which is historically when the best engineering happens.

what to actually take from the show#

On-device inference is a spec sheet lie in the thin-and-light category. If you are shipping software that assumes a local NPU, test on a machine that has been running for twenty minutes, not on a cold one. The difference is large and nobody benchmarks it.

Unified memory keeps winning. Every serious local-AI machine announced has shared CPU/GPU memory at high capacity. Discrete VRAM is a bad fit for a workload where capacity matters more than peak bandwidth, and the industry has figured that out.

The robot demos are still teleoperated. They have been for four years. Watch for the operator's hands, or the suspiciously smooth trajectory, or the fact that the demo never deviates from the script. There is real progress in robotics and it is happening in warehouses, not on stages.

the accessory economy#

A large fraction of the floor was accessories for devices that do not exist yet: cases, docks, and mounts for AI wearables. That is a leading indicator of nothing except that the accessory industry moves fast and takes risks cheaply.

the one thing I would actually buy#

The mundane one: displays got better and cheaper. High-refresh, high-resolution, accurate-color monitors are at prices that were flagship prices three years ago.

If you have been running the same monitor since 2020 and you spend eight hours a day looking at text on it, that is the highest-value hardware purchase available to you, and it will not be mentioned in a single keynote.

the meta-observation#

CES has been a bad predictor of what matters for about a decade. The genuinely important hardware of the last ten years — the M1, the H100, TPUs, the shift to ARM in servers — was announced at industry events or in blog posts, to audiences who understood it, without a stage show.

The consumer electronics show is now mostly a trade event for retail buyers, covered as if it were a technology forecast. Read the specs. Skip the narrative.

silicon#

Amazon continues pushing Trainium and Inferentia as the alternative to buying Nvidia. The pitch is price-performance for customers who can tolerate a different software stack.

The honest state of it: the hardware is competitive on paper, the software ecosystem is meaningfully behind CUDA, and the gap is closing slowly. If your workload runs through PyTorch with standard operations, the port is manageable. If you have custom kernels, it is a project.

The strategic point is the same for every hyperscaler: reducing dependency on a single supplier with enormous pricing power. Whether the customer benefit materializes depends entirely on whether the savings get passed through.

the agent announcements#

Every cloud vendor is now shipping agent infrastructure: runtimes, memory services, gateways for tool access, identity for agents, observability for agent traces.

This category is real. Running agents in production has genuine infrastructure requirements that are different from running services:

• Long-lived sessions with state that outlives a request.
• Non-deterministic execution paths that make traditional tracing awkward.
• Cost per invocation that varies by orders of magnitude.
• Identity and permission scoping for a thing acting on a user's behalf.
• Human approval gates in the middle of automated flows.

Those are real problems and the tooling is early everywhere. Evaluate on whether it solves a problem you actually have, not on whether the demo was good.

the pattern I would push back on#

Every vendor's agent framework wants to be the place your orchestration lives. That is a lock-in position, and orchestration is the layer most likely to be absorbed by the models themselves — it has been happening steadily for two years.

Keep your orchestration portable. Use the managed pieces for the genuinely hard infrastructure — identity, secure tool access, session storage — and keep the logic in your own code.

the underrated announcements#

The ones nobody writes about and everybody uses:

• Incremental improvements to S3 consistency and performance.
• Networking latency reductions.
• Cost management tooling that is slightly less bad.
• Database engine version updates.

These are worth more to most organizations than any AI announcement, and they get four slides between two hours of agent demos.

the meta-observation#

Cloud conferences have gotten less interesting, and that is good. It means the platform is mature. The exciting years of a platform are the years when fundamental things are missing.

The interesting question for AWS is not what they announced. It is whether the operational excellence that justified the premium is still there after a year that included a major regional outage. That is an execution question and it does not get answered at a conference.

what to actually do with this#

Skip the keynote. Read the "what's new" feed filtered to the services you actually use. Look for the deprecations, which are the announcements that will cost you time and which are never on stage.

And check your bill. The single highest-value hour available to most engineering organizations is someone competent looking at the AWS bill line by line, and almost nobody does it.

The stock's reaction was muted relative to the beat, which tells you the debate has moved from "is demand real" to "is demand sustainable, and how much of it is funded by Nvidia."

the bull case#

Straightforward and well supported:

• Every hyperscaler raised capital expenditure guidance again.
• Inference demand is growing faster than training demand, and inference is the recurring workload rather than the one-time one.
• Reasoning models consume dramatically more inference compute per request than their predecessors, and adoption is rising.
• Supply remains the constraint. Lead times are long. Customers are queueing.
• The rack-scale systems business has a moat that individual chip competition does not touch — the interconnect is the product.

the bear case#

Also straightforward:

• A meaningful share of revenue traces to customers Nvidia has invested in or financed, which makes the demand signal less independent.
• Depreciation schedules on AI hardware are assumed at five to six years. If the useful life is closer to three — which some operators argue, given the pace of generational improvement — reported profitability across the sector is overstated.
• Hyperscalers are building their own silicon. Google's TPUs are mature, Amazon's Trainium is shipping in volume, and every one of those deployments is a substituted Nvidia sale.
• Model efficiency improvements keep arriving. If capability-per-FLOP keeps improving as fast as it has, required FLOPs for a given capability fall.
• The financing environment for the buildout depends on continued access to cheap debt.

what nobody knows#

Whether AI application revenue will eventually justify the infrastructure spend.

Current annualized revenue across the AI application layer is a fraction of annual AI capital expenditure. That gap can close — infrastructure is built ahead of demand in every capital cycle, and railroads, fiber, and cloud all looked insane at the equivalent stage.

It can also not close. Fiber overbuild in 2000 was followed by a decade of dark fiber and a lot of bankruptcies, and the eventual users of that fiber were not the companies that laid it.

Both patterns are real. The people confidently predicting which one applies here are pattern-matching, not analyzing, and that includes the ones I agree with.

why an engineer should care#

Not for investing advice. For planning.

Compute pricing is not going to fall smoothly. If the capex cycle continues, capacity comes online in steps and prices drift down. If it contracts, capacity tightens and prices firm. Do not build a business model that requires a specific trajectory.

Efficiency work has enduring value. Whatever happens to the capex cycle, using less compute for the same result is good. Prompt caching, model routing, smaller models for routine tasks, batch processing where latency permits — all of it pays regardless of the macro environment.

Multi-provider capability is cheap insurance. The cost of abstracting your model calls is a day. The cost of being locked to a provider whose pricing or availability changes is much larger.

the sentence I keep coming back to#

The infrastructure being built is real, the demand today is real, and whether they match at the scale being assumed is genuinely unknown to everyone including the people spending the money.

That is an uncomfortable place to be, and pretending otherwise — in either direction — is the main thing to avoid.

The chain of events is a small masterpiece of the genre.

what happened#

A database permissions change caused a query that generates a Bot Management feature configuration file to return duplicate rows. The query had been returning one row per feature; after the permissions change it returned rows from multiple underlying schemas.

The generated file therefore roughly doubled in size.

The proxy that consumes this file preallocates a fixed-size buffer sized against a limit of 200 features — comfortably above the ~60 actually in use. The doubled file exceeded that limit.

The Rust code handling this hit an unrecoverable error path and the proxy panicked rather than degrading. Because the configuration file propagates network-wide every few minutes, the failure propagated network-wide within minutes.

Recovery was complicated by the fact that the bad file kept regenerating and redeploying.

the lessons, which are old#

Configuration is code and needs the same rigor. This was not a code deploy. It was a data change that propagated to production automatically with no staging, no canary, and no validation. Config deployment pipelines are consistently held to a lower standard than code deployment pipelines, and config causes a large share of major outages.

Fixed-size limits need to fail soft. The 200-feature limit was reasonable. Panicking when exceeded was not. The correct behavior for a proxy encountering an oversized config is to log loudly, alert, and continue with the previous known-good version.

Validate generated artifacts before propagation. A size check, a schema check, a sanity check on row count against the previous version — any of these would have caught it. Generated files should be validated as rigorously as user input, because the generator can be wrong.

Blast radius follows deployment speed. Config that propagates globally in minutes is a feature until it propagates a bad config globally in minutes. Staged rollout applies to configuration too.

Have a kill switch for automated pipelines. Much of the recovery time went to stopping the thing that kept redeploying the bad file. Every automated deployment path needs a way to stop it that does not require fixing the underlying problem first.

the Rust note#

This will be used as an argument about Rust, and it should not be.

The panic was a deliberate choice at that call site — the code used a construct that terminates on error rather than propagating it. That is a design decision about error handling, available in any language. In C the equivalent code would have written past the buffer, which is worse.

The actual lesson is about where you choose to make errors fatal. In a proxy handling live traffic, almost nothing should be fatal. Degrade, alert, continue. "Fail fast" is good advice for a batch job and bad advice for a load balancer.

the credit due#

Cloudflare published a detailed technical postmortem within a day, named the specific code path, and did not hide behind "an issue with a third-party provider."

That is how it should be done and it is rarer than it should be. A company that publishes real postmortems earns more trust than one that never has visible incidents, because the second one is not telling you about them.

AWS's public summary attributes the trigger to a latent race condition in the automation that manages DynamoDB's DNS records, which resulted in an empty record set for a regional endpoint and no automatic recovery path.

the cascade#

The failure sequence is the interesting part.

DynamoDB's endpoint became unresolvable. That alone would be bad. What made it a regional event is that an enormous number of AWS's own services use DynamoDB internally. The EC2 instance launch path, IAM's control plane, Lambda's invocation machinery, and dozens of others depend on it.

So the failure propagated: DynamoDB down means new EC2 instances cannot launch, which means autoscaling cannot replace failing capacity, which means load shedding, which means more failures. Network Load Balancer health checks destabilized. The recovery itself was slowed by the backlog of queued work that had accumulated.

This is textbook metastable failure: a system that is stable under normal load and stable under no load, but which, once pushed past a threshold, sustains its own failure through retry amplification and queue buildup even after the original trigger is fixed.

why us-east-1#

It is the oldest region, the largest, and the default in approximately every tutorial ever written. Several global AWS control planes are homed there — IAM, CloudFront configuration, Route 53's control plane, and others. That means a us-east-1 event has global blast radius even for customers with no resources in the region.

That architecture is a historical artifact. It is also extremely hard to change now, which is a lesson about early decisions in systems that grow.

the honest customer takeaway#

The reflexive response is "multi-region." Before you spend a year on that, do the arithmetic.

Multi-region active-active is genuinely hard. Data consistency across regions, failover testing that actually works, doubled infrastructure cost, and a substantially more complex system that fails in new ways. Many organizations that attempt it end up with a system that is less reliable overall because the complexity introduces more failure modes than the regional risk it removes.

The dependency you cannot escape. If your multi-region architecture depends on a global control plane that lives in us-east-1, you did not achieve independence. Check this specifically. A lot of people discovered it yesterday.

What is actually worth doing, in order:

1. Know your dependencies. Most teams cannot enumerate what their service requires to start. Write it down. The exercise is revealing.
2. Static stability. Design so existing capacity keeps serving when the control plane is unavailable. If your service needs to call an API to keep running, it will stop when that API stops. Cache aggressively, fail open where safe, and do not require a control plane call on the request path.
3. Graceful degradation. Decide in advance which features can be turned off. A checkout that works without recommendations is much better than a site that is down.
4. Exponential backoff with jitter, and circuit breakers. Retry storms are what turns an incident into an outage. This is the single highest-leverage code change available.
5. Multi-region for the tier that genuinely warrants it. Which is usually not everything.

the industry-level observation#

A meaningful fraction of the world's software depends on a small number of regions operated by a small number of companies. That concentration produces excellent reliability most of the time and correlated failure occasionally.

There is no individual fix. Every company independently choosing the most reliable provider produces exactly this concentration. It is a collective action problem, and the only actors who can address it are regulators thinking about systemic risk, who are — belatedly — starting to.

Read that structure carefully, because it is the interesting part.

the circularity#

Nvidia invests in OpenAI. OpenAI uses the money to buy Nvidia systems. The purchase is recognized as Nvidia revenue. The investment is staged against deployment milestones.

This is not fraud and it is not unusual in capital-intensive industries — vendor financing has been standard in telecom, aviation, and semiconductor equipment for decades. A supplier finances a customer's purchase because the supplier has the balance sheet and wants the volume.

It does deserve scrutiny for a specific reason: it makes the demand signal less informative. When a supplier funds its customer's purchases, revenue growth no longer cleanly indicates independent market demand. Some portion of it is the supplier's own capital cycling through.

Analysts have been tracking a widening set of these arrangements across the AI sector — investments in customers, prepayments, equity stakes in companies that are also large purchasers. Individually each is defensible. Collectively they make the sector's growth figures harder to interpret.

the gigawatt as a unit#

Note what is being measured. Not chips, not dollars, not FLOPs. Gigawatts.

Ten gigawatts is on the order of the electricity consumption of a large metropolitan area. It is roughly ten large nuclear reactors' worth of continuous generation.

The industry has converged on power as the natural unit because power is the binding constraint. You can order chips. You cannot order a substation and have it next quarter.

The consequences flow outward: electricity prices in datacenter-heavy regions, grid interconnection queues, transmission buildout, and the political economy of who pays for it. Several US utility regulators are now handling rate cases that are effectively about whether residential customers subsidize datacenter connections.

That fight is going to define a lot of the next five years and it is being had in public utility commission hearings that nobody in tech reads.

what it means for you#

If you are building on AI APIs, the practical questions are:

Is my provider's capacity growing? Rate limits and availability during demand spikes are the observable symptom. Announcements like this are a positive signal for capacity.

Am I exposed to a single provider's economics? If the financing environment tightens, pricing changes. Multi-provider capability is cheap insurance and you should have built it anyway for reliability reasons.

Are my costs actually falling? Per-token prices have fallen consistently. Per task costs have not fallen as much, because reasoning models consume more tokens. Measure the thing you pay for.

the honest uncertainty#

Nobody knows whether the capex cycle is correctly sized. The bull case is that inference demand compounds and every gigawatt gets used. The bear case is that efficiency improvements outrun demand and a lot of concrete is stranded.

Both are held sincerely by smart people with access to the same information. That is what genuine uncertainty looks like, and anyone expressing confidence in either direction is telling you about their position, not about the world.

Cloudflare sits in front of roughly a fifth of the web. A default change at that position is not a product launch, it is a policy change for the internet.

the mechanism#

Two pieces.

Default blocking. New zones get AI crawler blocking on unless the operator opts out. Cloudflare maintains the bot classification — separating search crawlers, which drive traffic back, from training crawlers, which do not.

Pay-per-crawl. A site sets a price. A crawler that wants the content gets an HTTP 402 Payment Required with terms. Cloudflare handles settlement.

HTTP 402 has been "reserved for future use" since 1997. It is genuinely funny that this is what activated it.

why the old system failed#

robots.txt is a request, not a control. It works because well-behaved crawlers choose to honor it, and that consensus held for thirty years because search engines had an incentive to be well-behaved — they needed publishers to not block them.

AI training crawlers have no such incentive. The content is valuable to them and the traffic they return is zero or nearly so. Multiple studies found training crawlers ignoring robots.txt, rotating user agents, and using residential proxy pools. Once a norm has no enforcement and no incentive, it stops being a norm.

Cloudflare's move replaces a request with a control. That is the actual innovation and it required no new technology at all — just someone at a chokepoint deciding to enforce.

the case against#

Concentrating the ability to gate the web at one CDN is not obviously good, even if this specific use of the power is popular.

The precedent is: an infrastructure company can unilaterally change how content is accessed for a large fraction of the internet, and the mechanism generalizes to things other than AI crawlers. Cloudflare has been thoughtful and has taken public positions on not being an arbiter, and the concentration is still real.

There is also a smaller-player problem. Large AI companies can negotiate licensing deals directly. Researchers, startups, the Internet Archive, and academic crawlers cannot. A tollbooth is regressive: it is a rounding error for the incumbents and a barrier for everyone else. Cloudflare has carve-outs for some of these and the carve-outs are discretionary, which is the point.

for developers#

Two practical items.

If you run a site, decide deliberately. Blocking training crawlers is now the default; that may not be what you want. Documentation sites in particular may prefer to be in the training data, because being the thing the model knows about is worth more than the pageview you lost.

If you build anything that crawls, expect 402s and expect your user agent to matter. Identify honestly, respect the directives, and set up billing if you need paid access. The era of scraping quietly is ending, and the enforcement is technical now rather than legal.

the bigger shift#

The web's economic model was: publish freely, get traffic, monetize traffic. AI answers break the second step, which breaks the third, which will eventually break the first.

Pay-per-crawl is one proposed replacement. Licensing deals are another. Neither is obviously going to work at the scale of the actual web, where most content is made by people with no ability to negotiate anything.

What replaces it is genuinely unresolved, and 2025 is the year everybody stopped pretending otherwise.

Ironwood#

The seventh-generation TPU, and the first Google has explicitly positioned for inference rather than training. Large HBM capacity per chip, high interconnect bandwidth, deployable in pods of thousands.

The strategic point is not the specs. It is that Google is the only hyperscaler with a mature, decade-old, production-proven alternative to buying Nvidia. AWS has Trainium and Inferentia, which are real but younger. Microsoft has Maia, which is very young. Google has been running TPUs in production since 2015 and has an entire compiler stack (XLA) and framework story (JAX) built around them.

That translates directly into pricing freedom. When Google prices Gemini aggressively, they are not eating a margin on someone else's silicon.

A2A#

Agent2Agent: a protocol for agents from different vendors to discover each other and collaborate on tasks. Announced with a long list of partner companies.

The mental model is that MCP connects an agent to tools and data, while A2A connects an agent to other agents. An agent publishes an "agent card" at a well-known URL describing its capabilities; other agents discover it and delegate tasks over a JSON-RPC-ish protocol with support for long-running work and streaming updates.

My honest read: the problem is real, the timing is early, and the number of partner logos on the announcement slide is inversely correlated with how much production usage a protocol has on day one. Multi-agent systems in 2025 mostly do not work well enough to need a standard for interoperating — the hard part is that agents fail in the middle of tasks, not that they cannot find each other.

But standards need to exist before they are needed, and having the conversation now beats having it in 2027 with four incompatible implementations.

Gemini 2.5 Flash#

The cost-efficient reasoning model, with a controllable thinking budget. You set how many tokens the model may spend reasoning, including zero.

config = types.GenerateContentConfig(
    thinking_config=types.ThinkingConfig(thinking_budget=1024)
)

This is the right API shape and I expect everyone to converge on it. The decision of how much to think is workload-specific, it is the primary cost lever in a reasoning model, and hiding it inside the model's own judgment takes control away from the person paying the bill.

Set it to zero for classification. Set it high for planning. Measure the quality difference on your own task, because the curve is steeper for some tasks than others and there is no general answer.

the rest#

A great deal of "agentic" branding applied to existing products, an agent development kit, and a marketplace. Cloud vendor conferences have a genre and this one was firmly in it.

The signal-to-slide ratio was better than most.

• Blackwell Ultra (GB300) — second half of 2025, more HBM per package.
• Vera Rubin — 2026, new CPU (Vera) and new GPU (Rubin), HBM4.
• Rubin Ultra — 2027, a much larger rack-scale configuration.
• Feynman — 2028, named, not detailed.

Publishing a multi-year roadmap at this granularity is a foundry move, not a product-company move. The audience is not developers. It is the utilities, the construction firms, the HBM suppliers, and the CFOs who need to plan capital around it.

the number that should worry you#

The power figures for the rack-scale systems are the part of this keynote that will still matter in five years. Current NVL72 racks draw on the order of 120 kW. The Rubin Ultra generation is being discussed in the hundreds of kilowatts per rack.

A traditional enterprise datacenter rack is provisioned for 5 to 15 kW. Air cooling tops out somewhere around 40 kW with heroic effort. Everything past that is liquid, and everything past about 150 kW is liquid plus a fundamentally different power distribution architecture — which is why the roadmap includes 800 VDC distribution.

Translation: existing datacenter shells are largely unusable for this. The buildout is not "install new servers," it is "build new buildings near new substations." That is a multi-year, capital-intensive, permit-bound process, and it is the actual rate limiter on AI capacity through the rest of the decade.

the software announcements#

Dynamo, an open-source inference serving framework, is the developer-relevant release. It handles disaggregated serving — running the prefill phase and the decode phase on different hardware pools, because they have completely different compute and memory characteristics. Prefill is compute-bound and parallel; decode is memory-bandwidth-bound and sequential. Running them on the same homogeneous pool wastes a lot of silicon.

If you operate inference at any scale, disaggregation is probably the largest single efficiency win available to you right now, and having a maintained open implementation lowers the bar considerably.

NIM microservices continue to be Nvidia's attempt to own the deployment layer. Reasonable, containerized, and a lock-in vector you should evaluate with clear eyes.

the strategic read#

Nvidia's actual product is no longer a chip. It is a rack, plus the networking between racks, plus the software stack on top. Selling a GPU means competing with AMD and with every hyperscaler's internal silicon team. Selling an integrated rack-scale system with a co-designed interconnect means competing with nobody, because nobody else has the interconnect.

That is the moat, it is being widened deliberately, and the roadmap slide is the announcement of that strategy rather than a product list.

Two things are true at once and you need to hold both.

the number is not a number#

$500 billion is not committed capital. It is an aspiration with a press release attached. The initial equity is a fraction of it, the rest is contingent on debt markets, vendor financing, offtake agreements, and continued demand that nobody can underwrite four years out. Announcements at this scale are partly a coordination device — you say the number so suppliers, utilities and lenders plan around it.

Elon Musk publicly said the money isn't there. He is a hostile witness with obvious motives, and he is also not obviously wrong about the funding gap.

the constraint is not GPUs#

Here is the part developers should actually internalize, because it changes how you should think about compute pricing for the next several years.

The binding constraint on AI datacenter buildout is no longer semiconductor supply. It is:

• Power. A gigawatt-class campus needs an interconnection agreement, and US interconnection queues are measured in years. This is why the sites are going where they are going — West Texas has wind, gas, and a grid operator that can move faster than most.
• Transformers and switchgear. Lead times for high-voltage transformers ran past two years. You cannot software your way around a transformer.
• Cooling and water rights. Liquid cooling at rack densities north of 100 kW is a plumbing problem, and plumbing has a supply chain.
• Electricians. There is a genuine national shortage of people qualified to terminate high-voltage cable, and you cannot fine-tune one.

The Abilene site is the tell. They started building before the financing was finalized because the long pole is not money, it is queue position.

what it means for your bill#

If you are trying to model API costs, the useful frame is: capacity comes online in step functions, tied to substation energization dates, not to Nvidia's quarterly shipments. Prices per token will keep falling on a per-capability basis because of model efficiency gains, not because compute is getting cheap. Compute is not getting cheap. Compute is getting more available, in lumps, eighteen months after somebody signs a power purchase agreement.

the second-order thing#

Every one of these campuses is a bet that inference demand keeps compounding. If model efficiency improves faster than demand — if a 2027 model gets today's quality at a tenth of the FLOPs — a lot of this concrete is stranded.

That is not a reason to think the buildout is stupid. It is a reason to notice that "AI capex" and "AI capability" are two different bets, and only one of them is being made by the people writing these checks.