The most token-efficient iOS Simulator tool for agents.

Leaderboard

All on one Mac, one tool at a time. Ranked by what the agent reads back over the whole sign-in (read the screen, type the email, type the password, tap Sign In, read the screen to see "Welcome"), since every token of it goes through the model; ties by time. Median of 10 runs, each on a freshly started app. Sign-in times step by step and batched are in The whole sign-in.

The runs went in this order: simulator-use, AXe, MobileBuildMCP, mobile-mcp, agent-device, Argent, Xcode MCP, then the second runs under Not ranked here, then Maestro MCP and AXe's last run: their first tries stopped before any call, with the Simulator found shut down after its reboot. Tokens are estimates: characters / 4 for text (no Anthropic API key was set for this run), width * height / 750 for an image.

simulator-use against each tool, one page each: compare.

Read the screen

The tool's own screen read, 20 times on the unchanged sign-in screen, after one warm-up read.

Tap a button

Sign In with empty fields, 20 times, each after a fresh start of the app and an untimed read. The app's own log confirms each tap. "Shows the result" counts answers that hold the new status text, "Invalid email or password", in the answer itself or in the file it names for the agent to open.

Type into a field

gary@example.com into Email, 10 times, each on a fresh app: the tool's fill call, or a tap on the field and a type call where that is its way. The tool's next read must show the text.

The whole sign-in

Step by step, one call per action; and batched, the three actions in the tool's own batch or sequence call where it has one. Signed in means the final read showed "Welcome, gary@example.com" and the app logged the sign-in.

Not ranked here

The second runs. AXe and MobileBuildMCP ran again with Device Hub open, because AXe's README says: "Xcode 27 simulator automation uses Device Hub; Simulator.app is not required." With Device Hub open, their taps mostly did not reach the app. Run once more at the end, with Device Hub closed, both passed again. So on this Mac the headless runs, which every tool had, are the ones ranked.

• AXe with Device Hub open. Each axe tap answered that it tapped the right point, and 6 of the 20 reached the app. In axe batch the taps landed, but the first letters typed into the email were lost in 9 of 10 runs ("ary@example.com", "ry@example.com"), so the screen never showed "Welcome, gary@example.com".
• MobileBuildMCP with Device Hub open, which taps through a bundled AXe: 9 of 20 taps reached the app, each answered as done.
• AXe's two ways of reading. In MobileBuildMCP's ranked run and in AXe's last run, the first read timed out creating the Simulator's remote automation session ("Timed out creating the simulator remote automation session"). Every later call still worked, and faster: AXe read the screen in 0.62 s instead of 0.86 s and tapped in 1.09 s instead of 1.72 s, with a longer tree (55,106 tokens instead of 44,869). MobileBuildMCP's last run, which made the session, signed in in 17.97 s, against 12.26 s in its ranked run.
• simulator-use's own engine is built on AXe's frameworks; these rows are AXe's command line and MobileBuildMCP, as their users run them.

Before the first call

What the server's tool list and instructions cost an agent's context in every session, before it does anything.

Setup is what a tool's instructions say to do before the first action (open a session, pick or boot the device); its time is not in any number above.

What each tool is

From each tool's own README, license and docs, read on October 31, 2026; the sources are linked in each row.

What each tool needs

Every tool here needs a Mac with Xcode and an iOS Simulator runtime for the iOS Simulator. Quoted needs come from the sources above.

How each tool was driven

Each tool's own calls, chosen from its README, its tool descriptions or the hints in its answers. The code is in bench/tools.

• simulator-use. Read with ui_look; ui_act with tap #signIn or fill #email .... Batched: one ui_act with the three steps.
• AXe. No MCP server, so one axe process per step: describe-ui, tap --id, and tap --id then type. Batched: one axe batch with the five steps, as in its help. It ran twice: with the Simulator headless, like every tool, and again with Device Hub open, since its README says "Xcode 27 simulator automation uses Device Hub".
• MobileBuildMCP. Run twice, like AXe, which it taps and reads through: headless, and with Device Hub open. Its ui-automation workflow turned on. snapshot_ui, then tap and type_text by elementRef, and snapshot_ui again after each type_text (its descriptions say to refresh after layout changes, and the keyboard is one). Its read lists only tap targets, so the last step checks "Welcome" with its wait_for_ui textContains, as its tips say. No batched run: its batch takes taps, not typing.
• mobile-mcp. mobile_list_elements_on_screen, then mobile_click_on_screen_at_coordinates by @ref, and a click then mobile_type_keys. Batched: mobile_batch_commands. Its first read starts its on-device agent, which left the Home Screen in front; the setup starts the agent and the app is then started again, as before every step.
• agent-device. Setup: open with foreground: true, as its instructions say. snapshot with interactiveOnly, press and fill by id="..." selector with settle: true. Batched: its batch.
• Argent. Setup: the Simulator booted by Argent's boot-device (its first describe said it needs that for its accessibility settings), then launch-app. describe, then gesture-tap at the centre of the element's frame, and a tap then keyboard. Batched: run-sequence.
• Maestro MCP. inspect_screen, then run with inline YAML: tapOn by id, and tapOn then inputText in one call. Batched: one run with every step.
• Xcode MCP. Headless (sudo xcrun mcp-server enable). DeviceInteractionStartSession, then DeviceInteractionSynthesize: with no command to read (the agent opens the hierarchy file it names), t x y at the element's hitPoint, and t x y sender keyboard kbd ... to tap a field and type, from Xcode's own device-interaction skill. No batched run: typing must come last in a command.

Method

The harness and every call it made: bench, with the raw results in bench/results/2026-11-01 (each file records its Mac, Simulator, versions and whether Device Hub was open). bash bench/setup.sh, then bash bench/run-all.sh, runs it again on any Mac with Xcode; node bench/summarize.mjs and node bench/page.mjs turn the results into these tables.