A browser agent, or computer use agent, is an AI model that operates a website or a desktop application the way a person does: it looks at the screen or reads the page structure, clicks, types, checks the result and repeats. It is the tool for systems that have no API, such as a supplier portal, a courier's back office or an older business application. It is also slow, costs real money per task, and finishes tasks more often than it gets them right. The rule of thumb: use an API whenever one exists. If none does, have the agent read the page's accessibility tree, the structured description of buttons and fields that screen readers use, and fall back to screenshots only where the page has no structure, such as maps, charts and drawing areas.
What is a computer use AI agent?
A computer use agent is a model given tools to control a screen: take a screenshot, click at coordinates, type, scroll and press keys. A browser agent is the same idea limited to a web browser, usually with extra tools that read the page structure and fill in forms directly. Anthropic, OpenAI and Google all offer such tools, and open-source projects such as Playwright MCP let any agent drive a browser (Anthropic, computer use). The agent runs in a loop, as described in agentic AI explained: observe the screen, decide, act, and observe again to check what the action actually did.
When should you use a browser agent instead of an API?
You should use a browser agent only when nothing better exists. The order of preference:
- An official API or webhook. Faster by orders of magnitude, cheaper, predictable, and it does not break when someone redesigns a page.
- A stable endpoint that the page itself calls, if the site's terms allow it.
- A fixed Playwright or Puppeteer script, with a model only for the parts that genuinely vary. It is cheap, fast and testable, and it fails loudly.
- A browser agent, for tasks that vary too much to script.
In Greece the typical candidates are supplier and courier portals, marketplace seller back offices, and older business software with a web screen but no API. Read each site's terms first, because many address automated access explicitly. Public services that offer an official interface should be used through it, for example myDATA for tax documents, as covered in myDATA and e-invoicing.
Screenshots or page structure: how should the agent see the page?
The agent should read the page structure wherever it exists and use screenshots only where it does not. The accessibility tree names every button, field and link, so the agent clicks an exact element instead of estimating coordinates, and a filtered read of an ordinary page costs far fewer tokens than an image. Screenshots are needed where the tree says nothing useful: charts, maps, seat pickers, PDF viewers and drag handles. The two approaches fail on different events. An agent working from the page structure breaks when a redesign renames or restructures elements. A screenshot agent survives that, but breaks on a new cookie banner, a layout shift or a different window size. That is why the providers' tools let one agent mix both, choosing per step. For scale: a screenshot of 1280 by 720 pixels costs about 1,200 input tokens on Claude, by Anthropic's published formula of one token per tile of 28 by 28 pixels.
What does a browser agent cost?
A browser agent costs more than the price per step suggests, because every step resends the whole conversation, screenshots included, so the cost grows faster than the number of steps. A worked example at Claude Opus 5's prices in September 2026, 5 dollars per million input tokens and 25 per million output: a 20-step task with a screenshot at every step uses about 415,000 input tokens, which is about 2.07 dollars without prompt caching and about 0.40 with it, plus about 0.20 for output. Run it 500 times a day and that comes to roughly 300 dollars a day with caching against more than 1,100 without. Real tasks are longer: one 2026 benchmark of 350 real-web tasks averaged 37.9 steps (arXiv 2608.17319). Failed runs cost the same as successful ones, so budget on the cost per completed task, as explained in what an AI agent costs to run. Time counts too: at 38 steps, even two seconds per model call adds up to more than a minute per task, so these agents suit background work, not a customer who is waiting.
How reliable are browser agents?
Browser agents are reliable enough for supervised work and not for unsupervised work. On that benchmark of long real-web tasks, a strong specialist model completed 65.1% of tasks, against 80.6% on an older benchmark of shorter ones, so most of the gap is task length. Finishing is also not the same as succeeding: a 2026 audit across 42 public websites found an agent that completed 99.2% of its tasks but respected every stated constraint in only 38.8% of them. Read benchmark scores of 60% to 80% on real sites as "roughly one run in three needs a person". The failure to design against is rarely a wrong click. It is a right click on a page that had not finished loading, followed by a dozen steps built on it. The fix is to make the agent look at the page again after every batch of actions, and to stop the run when three observations in a row show no change.
How do you keep a browser agent safe?
You keep a browser agent safe by limiting what it can reach and what it can confirm on its own:
- Never give it a password. A person logs in once, the session is saved, and the agent starts from that logged-in browser profile. It gets access without ever seeing a secret, and revoking it means ending the session.
- Allow only the domains the task needs, checked by your code on every navigation and again after redirects.
- Treat everything on the page as data, because a page can contain instructions aimed at the agent, as described in prompt injection.
- Require a person's confirmation for purchases, messages, account changes and accepting terms, as in human in the loop.
- Stop at a CAPTCHA or a login page and hand the run to a person. A CAPTCHA is a site saying no to automation, not an obstacle to solve.
- Set a step limit and a spending limit in your own code, because not every provider enforces one for you.
Isolation matters as much as the rules, so run the browser in its own container or virtual machine. Playwright MCP's documentation is blunt that the tool "is not a security boundary" (Playwright MCP). The wider permission model is in AI agent permissions. Moving data between systems that have no API is typical work for AI in operations and supply chain.
Frequently asked questions
Can a browser agent use my accounts on marketplaces or supplier portals?
Technically yes, through a saved session. Whether it may is up to each site's terms, and many restrict automated access, so read them first and prefer an official API where one exists.
Is a browser agent the same as web scraping?
The two overlap. A scraper reads pages with fixed code, while a browser agent decides what to do on each page. For reading public pages, a plain request for the page is cheaper than either.
Where does a browser agent pay off first?
In repetitive back-office work across systems that will never get an API: checking order statuses on a courier portal, copying data between two old applications, downloading monthly statements. The work should tolerate a person reviewing the result.
If your team spends hours copying data between web systems that have no API, send me the list of systems and what moves between them. Whether that needs an agent, a script or an API you did not know existed usually shows from that list.