Test results

Which browser tool helps AI the most?

We raced 7 browser tools for AI through the same game with the same AI. Playwriter won the most levels. Our first try was the fastest and the cheapest, but came 4th.

Updated October 16, 2026 · by the Buoy team

95%

of levels won by Playwriter, the top tool

4.8 s

a level for Buoy, the fastest

83k

tokens a win for Buoy, the least

The game

A web page that keeps score.

We built a game called Page Race. Each level asks for one small job, like click a button, fill a form or drag a card. Our server decides every win, so no tool times itself.

60

levels in 6 kinds

7

hard cases, scored apart

885

runs in all

1

AI model for all 7 tools

All 7 tools used Claude Haiku 5.5 at its normal effort. Only real clicks and keys count. Each tool was set up the way its own guide says.

Results

Who finished the most levels.

Finishing comes first. Speed breaks ties.

The tool drives the Chrome you already have open and signed in.

Playwriter95%
Playwright MCP93%
Chrome DevTools MCP92%
Buoy88%
agent-browser83%
Claude in Chrome25%
mcp-chrome0%

mcp-chrome clicks with page script, which the game doesn't count as real. When those clicks count too, it wins 92% of levels.

Speed

Buoy finished each level first.

This is the middle time for one level, from the AI's first step to its last. It counts the AI's thinking too, since that's what you wait for.

Buoy4.8 s
Playwriter6.3 s
Chrome DevTools MCP7.1 s
Playwright MCP7.3 s
agent-browser14.2 s

Only tools that won at least half the levels are shown here.

Fast hands

Click fast. Answer fast.

A tool can land a click fast but tell the AI late. The AI waits for that answer before it can act again. Here a script plays with no AI, so we time only the tool.

Buoy · lane 28 ms

lands in 8 ms · answers under 1 ms later

Buoy · lane 19 ms

lands in 9 ms · answers under 1 ms later

agent-browser · lane 227 ms

lands in 26 ms · answers 1 ms later

Playwriter · lane 159 ms

lands in 26 ms · answers 33 ms later

Chrome DevTools MCP · lane 2225 ms

lands in 21 ms · answers 204 ms later

Playwright MCP · lane 21026 ms

lands in 16 ms · answers 1010 ms later

The hardest level was the double flash. A target lights up for 0.4 s, goes dark, then lights up again. No tool hit both flashes with the AI driving. The AI takes seconds to think.

Where each tool slips

Six kinds of jobs.

Each kind counts the same. Darker means more levels won.

The tool drives the Chrome you already have open and signed in.

ToolFind & clickType & formsWait & timingScroll & dragRead & chooseChains
Playwriter
100%
100%
80%
90%
100%
100%
Playwright MCP
100%
100%
80%
100%
100%
80%
Chrome DevTools MCP
100%
100%
70%
100%
100%
80%
Buoy
100%
100%
70%
70%
100%
90%
agent-browser
100%
90%
80%
80%
80%
70%
Claude in Chrome
40%
30%
20%
40%
20%
0%
mcp-chrome
0%
0%
0%
0%
0%
0%

Hard cases

These are scored on their own: did the tool get the job done in our race, or not.

ToolFrameShadow rootNew tabRight-clickFile uploadCanvas textConfirm box
PlaywriterWonWonWonWonWonWonWon
Playwright MCPWonWonWonWonWonWonWon
Chrome DevTools MCPWonWonWonLostLostWonWon
BuoyWonWonLostLostLostWonLost
agent-browserWonWonWonWonWonWonWon
Claude in ChromeLostWonLostLostLostLostLost
mcp-chromeLostLostLostLostLostLostLost

Ten in a row

One prompt, ten levels.

We gave the AI one job: finish ten levels back to back. Each next level shows only after the last one is won.

The tool drives the Chrome you already have open and signed in.

Most levels won

Playwriter10 / 10
Playwright MCP10 / 10
Chrome DevTools MCP10 / 10
Claude in Chrome10 / 10
Buoy8 / 10
agent-browser1 / 10
mcp-chrome0 / 10

Time for all ten

Chrome DevTools MCP74 s
Playwriter80 s
Playwright MCP94 s
Claude in Chrome189 s

Buoy lost the same two drag levels every time. The time chart shows only tools that won all ten.

Cost

What each win costs.

Some tools send the AI a big copy of the page after each step. That fills its memory and costs more.

Buoy83k
Playwriter107k
Chrome DevTools MCP108k
Playwright MCP112k
agent-browser231k

Tokens for each level won, in thousands. Fewer is better.

Every number

The fine print.

We make Buoy and we built this game. So here is how we tested, and every number.

How we tested

  • We locked every level and rule before Buoy joined the race.
  • Each tool played each level once in each lane it supports, with a fresh AI each time.
  • The game logs each input and checks that it is real. Fake "skip" buttons catch cheat tries.
  • The agents could not read the game's code or the judge.
  • We ran 6 races at once on one Mac, all tools mixed in the same order.
  • The no-AI timing ran one tool at a time on a quiet Mac.

Every result

The tool drives the Chrome you already have open and signed in.

ToolWonTime a levelAnswer lagTen in a rowTokens a winCost a win
Playwriter57 / 606.3 s521 ms10 / 10107k$0.0037
Playwright MCP56 / 607.3 s1015 ms10 / 10112k$0.0035
Chrome DevTools MCP55 / 607.1 s209 ms10 / 10108k$0.0034
Buoy53 / 604.8 s15 ms8 / 1083k$0.0027
agent-browser50 / 6014.2 s1049 ms1 / 10231k$0.0064
Claude in Chrome15 / 6025.2 s29 ms10 / 10358k$0.0153
mcp-chrome0 / 60n/an/a0 / 10n/an/a

Fair notes

  • Buoy's web tool is a preview. It is not out yet.
  • Buoy got one cheat flag. It could only drop a card on a button, not on a plain box. The AI got stuck and dropped the card on the "Skip level" button to see what would happen. We kept the flag.
  • Claude in Chrome works in its own tab group. Most times it could not reach the race tab that was already open.
  • mcp-chrome clicks with page script, so its clicks don't count on the main board.
  • The ten-level run ran four times for each tool in each lane. We show the best result and the middle time.
  • We fixed two bugs in our no-AI timing before the final run. Both hurt fast tools the most. Then we ran it again for all of them.
  • One Mac, one AI. Other AI models may rank the tools in a different order.

Tool versions

ToolVersion
agent-browseragent-browser@0.39.1
Buoybuoy-web v0 (evals/web-agents-vs-buoy/buoy-web)
Chrome DevTools MCPchrome-devtools-mcp@1.11.0
Claude in Chromeextension fcoeoabgfenejglbffodgkkbkcdhcgfn (version recorded at setup) + Claude Code --chrome
mcp-chromemcp-chrome-bridge@1.0.31 + extension v1.0.0 (chrome-mcp-server-1.0.0.zip, sha256 e0f7edfe84b64fd4)
Playwright MCP@playwright/mcp@0.0.83
Playwriterplaywriter@0.8.0

Can I use Buoy for the web?

Not yet. It is a preview in our lab. We are making it better one kind of job at a time, and we will race it again.

Why does Buoy finish fast but win less?

It answers the AI right after each click lands, with only what changed on the page. That saves time and tokens. But this first try could not drag onto plain boxes, hover, or switch tabs, so it lost those levels.

Why one AI model?

Using the same AI for all 7 tools means the tool is the only thing that changes. We picked Claude Haiku 5.5 at its normal effort because it is fast and cheap.

sources

Capability claims on this page come from each vendor's own documentation, read on the date shown. We did not install and run every tool listed.