Vercel agent-desktop Gives Coding Agents Free Computer Use
Most "computer use" AI demos work by taking screenshots and guessing where to click. Vercel Labs' agent-desktop, announced on October 30-31, 2026, takes a cheaper and sturdier route: it reads a desktop app through the operating system's accessibility tree — the same structure screen readers use — and hands your coding agent a list of named buttons and fields to act on. It is free and open source under the Apache 2.0 license, it runs on macOS, Windows and Linux, and it contains no AI model of its own. This is not a hands-on test yet — it is a careful read of the project docs and the October 31 coverage, with the limits stated plainly.
What agent-desktop actually is, and what it is not
agent-desktop is a tool that agents call, not an agent itself. It has no model inside, no graphical interface, and no interactive prompt, according to its docs. A coding agent asks it for a snapshot of an app, gets back JSON with a reference for every element it can act on, acts on one reference, then takes a new snapshot to check the result. That snapshot-act-verify loop is the whole product — the judgment still comes from whatever agent you connect to it. If you do not already use a coding agent, this tool alone gives you nothing to talk to, which is the first thing to understand before installing it.
It is also not a web-automation tool: Vercel's agent-browser remains the tool for web pages, while agent-desktop covers native desktop apps. Within that lane, though, it is genuinely model-agnostic — any shell-capable coding agent can call it, so you are not locked to one provider's agent or one model. That is what earns it a place alongside our no-sign-up agents directory: the free, open path is the product, not a trial mode.
Why the accessibility tree beats screenshots
Screenshot-based computer use burns tokens on every image and still guesses at pixel coordinates. Reading the accessibility tree means the agent clicks a named element — "Save button", "search field" — instead of a coordinate that shifts when a window moves. Actions run headless by default, so they do not move your pointer or steal focus while you keep working. In multi-agent mode, each agent gets its own drawn cursor, although physical clicks and key presses still run one agent at a time.
The token savings claimed by the builder are large and should be read as the builder's own figures, not an independent benchmark: skeleton drilling is reported to cut an Apple Maps view to 273 tokens from 3,811, and a simulated 52-step task to about 31,000 tokens from about 198,000. Treat those numbers as a reason to try it on your own workflow, not as a guarantee for yours.
What it runs on, and what it costs
The tool itself is free: install with npm i -g agent-desktop, with a Rust CLI and a C library underneath, on x64 and ARM64 machines. The stated requirements are macOS 13 or later (with Accessibility permission for the app running it), Windows 10 version 1809 or later or Windows Server 2019 or later, and Ubuntu 24.04 or later on Linux (in a graphical session with the at-spi2-core package). Official Homebrew and Scoop packages were still being submitted at launch, so npm is the install route to use for now.
One cost warning: free here means the tool, not the intelligence. agent-desktop adds no model bill of its own, but whatever coding agent and model you pair it with still bills the way it always did. If you pair it with a free-tier model from our free API guides, that is where the free story holds together; pair it with a paid model and you pay that model's price, not agent-desktop's.
Who it suits, and who should skip it
It suits you if: you already run a coding agent from the terminal, you are comfortable with npm, and you want that agent to drive real desktop apps — filing, formatting, or clicking through a GUI — without paying for a computer-use service or burning tokens on screenshots.
Skip it for now if: you want a chatbot you can talk to, you only need web automation, or the apps you care about expose no accessibility tree — those stay out of reach, and no tool in this category fixes a badly built app interface. Also skip it if you expect a polished installer: at version 1.0.2, released October 30, 2026 after a 1.0.0 release on October 28, this is a young command-line tool, transferred to Vercel Labs on October 30 from its original developer.
How to try it safely
1. Start with a throwaway task. Point it at a harmless app and a reversible action first — open a document, read a field — before letting any agent change files or send anything.
2. Grant one permission at a time. On macOS, give Accessibility permission only to the app that runs agent-desktop, and only when you are ready to test.
3. Keep the agent supervised. Headless does not mean unsupervised. Watch the first runs, and verify every file change, message, or submission before it leaves your machine.
4. Verify before you trust it. As with any agent, check names, numbers, dates, and links in its output before sending or publishing anything. A cheaper way to click does not make the clicks correct.
FAQ
Is agent-desktop an AI agent?
No. It is a tool that coding agents call. It embeds no language model and runs no reasoning loop — your agent supplies the judgment.
Is it really free?
The tool is free and open source under Apache 2.0. Your agent and model may still cost money; agent-desktop itself adds no model bill.
Does it work on Windows and Linux?
Yes — macOS 13 or later, Windows 10 version 1809 or later (or Windows Server 2019 or later), and Ubuntu 24.04 or later on Linux, on x64 and ARM64.
Does it take screenshots?
No. It reads the operating system's accessibility tree and returns a JSON snapshot with a reference for each actionable element.
Can it automate websites?
Not its job. Vercel's agent-browser remains the tool for web pages; agent-desktop covers native desktop apps.