Operator, and the long road to an agent that can click

OpenAI released Operator as a research preview last week: an agent that operates a browser in a virtual machine, looking at screenshots and issuing mouse and keyboard actions to accomplish a task. Book a table. Fill out a form. Order groceries.

It works, sometimes. The parts where it does not work are more instructive than the parts where it does.

the architecture#

Underneath is a model trained specifically for computer use: it takes a screenshot, reasons about what is on screen, and emits an action — click at coordinates, type this string, scroll. Then it takes another screenshot. The loop runs until the task is done or the model gives up or asks for help.

This is the "act like a person" approach, as opposed to the "call the API" approach. It is inefficient and fragile by construction. It is also the only approach that works against the ninety-eight percent of the web that has no API.

where it breaks#

Latency compounds. Every step is a screenshot, a model call, an action, a page render. Twenty steps at three seconds each is a minute of watching a robot slowly use a website you could have used in fifteen seconds.

State is invisible. The model sees pixels. It does not know that clicking submit started a background job, or that the page it is on is a stale cache, or that a modal is about to appear. Humans use an enormous amount of context that is not on screen.

Recovery is hard. When a person hits an unexpected state, they back up and try something else with a mental model of what went wrong. The agent's version of this is much weaker, and long tasks fail in the middle with the world in a partially-mutated state. That is worse than failing at the start.

Sites do not want it. Cloudflare, hCaptcha, and every anti-bot vendor on earth have an obvious incentive here. Several major sites are already blocking it. The agent-vs-anti-bot arms race is going to be one of the defining web infrastructure fights of the next few years, and it will make life worse for accessibility tooling as collateral damage.

the safety design worth copying#

Operator requires human takeover for logins, payments, and CAPTCHAs. It does not handle credentials. That is not a limitation, that is the correct product decision, and anyone building in this space should copy it.

The general principle: an agent should be able to prepare an irreversible action and never commit one. Fill the cart, do not buy. Draft the email, do not send. The value is in the ninety percent of tedium before the decision, and the decision is where the liability lives.

the actual near-term winner#

Not general web agents. Domain-specific agents with real APIs underneath and a browser only as a fallback. The company that wins here will be the one that quietly negotiated integrations while everyone else was demoing screenshots.

Operator is a research preview and OpenAI is calling it one. Treat it as a capability probe, not a product. The capability is real, it is early, and the direction is clearly correct even if this specific implementation gets replaced twice before it is useful.