Why computer use alone is not enough

Once models learned to read screenshots and move a pointer, browser agents started to feel almost obvious. A person can use a website by looking at it and clicking things. Let the model do the same.

It is a remarkably general interface. It is also a strange interface to use inside a browser.

The browser already knows which tab is active. It knows the URL, whether a navigation is still running, which controls are interactive, and whether an action opened another page. Then we hide all of that, send the model a picture, and ask it to discover the same facts again.

I kept running into this while building Narada. A visual agent could complete a step, but the next screenshot did not tell it everything that had changed. A coordinate that pointed to a button could now point to a menu. Two controls could have the same label. A page could look ready while the browser was still navigating.

The problem was not that computer use did not work. The problem was asking it to be the only language.

Narada uses structured browser state for execution. An agent can inspect the granted tab, receive references to interactive elements, act on one of them, wait for a specific result, and inspect the page again. The person can see the plan, the milestones, and the tool results. The action produces something the next step can check, not just another image and a hopeful description of what happened.

This becomes important as a task gets longer. If every step starts with a fresh screenshot and another visual search, the agent has to keep finding its place. If the browser preserves tab identity and reports what changed, the agent can continue from known state.

But this is not an argument for removing computer use.

Some interfaces are visual by nature. A canvas, a PDF, an unfamiliar layout, or a page with poor structure may still need the same kind of inspection a person would use. Visual control is also useful for discovery. It can show us behavior before we have decided which parts deserve a stable tool.

So I have ended up with a simpler distinction: use visual interaction when seeing the page adds information. Use browser state when the browser already knows the answer.

That leaves a more difficult question. If an agent is no longer only a remote hand on the user’s mouse, what should the browser itself be responsible for?

That is the next part of Narada.