Peekaboo: macOS AI Automation CLI
/Peekaboo
Peekaboo is a macOS CLI and menu-bar app that lets you, or an AI agent through its MCP server, see the screen, read a window's UI as structured JSON, and click or type through the Accessibility API instead of raw coordinates. Reach for it if you're scripting or agentifying real macOS UI work; skip it for a one-off screenshot, since that needs no Accessibility permissions at all.
What is Peekaboo?
Peekaboo is a macOS CLI and optional menu-bar application that captures screens, inspects accessibility, and automates native UI, with an MCP server that hands the same toolset to AI clients like Codex, Claude Code, or Cursor. It observes the screen or a running app through `peekaboo see`, which returns UI elements with opaque IDs, then lets you or an agent act on them with commands like `click`, `type`, and `press`.
Core Capabilities
- ✓Screen and app capture via `peekaboo see`, which can target the whole screen or a single app and writes a PNG plus, unless you pass `--no-elements`, a structured UI map.
- ✓Structured UI inspection: `peekaboo see --app Finder --json` returns elements with opaque IDs you reference in later commands instead of guessing pixel coordinates.
- ✓Native UI automation commands — `click`, `type`, `press`, `scroll`, `drag`, `set-value`, `action` — that target an element by its accessible label.
- ✓Background input delivery: targeted commands can reach an app that isn't frontmost when Peekaboo can resolve its process, with a `--foreground` flag for apps that only accept input in a focused window.
- ✓An `agent` command that takes a natural-language instruction, such as opening Safari and searching a site, and chains the observation/action tools to carry it out using a configured model provider.
- ✓An MCP server (`peekaboo mcp`) that exposes the same capture/inspect/act toolset to MCP clients such as Codex, Claude Code, or Cursor.
- ✓System-level control beyond a single app's UI — `app`, `window`, `menu`, `menubar`, `dock`, `dialog`, `space` commands.
- ✓As of version 4.0, an exact 33-command root surface with consistent duration and coordinate grammar, plus native `verify` polling to confirm an action's effect.
Getting Started with Peekaboo
The released CLI and app require macOS 15 or later. Install the CLI with Homebrew: `brew install steipete/tap/peekaboo`. For the MCP package, use npm/npx with Node.js 22 or later: `npx -y @steipete/peekaboo --version`, which bundles the CLI plus its MCP launcher — see the MCP setup doc to connect it to Codex, Claude Code, Cursor, or another MCP client. The Mac app itself is a signed DMG from the latest GitHub release; it handles permission onboarding, visual feedback, and agent sessions, but you still install the CLI separately if you want `peekaboo` on your PATH. Source builds additionally need Swift 6.2 or later, pnpm, and the repo's submodules, built with `pnpm install --frozen-lockfile` then `pnpm run build:cli`.
Automating macOS Tasks
Start with `peekaboo permissions status` to confirm Screen Recording and Accessibility are granted, then take a screenshot with `peekaboo see --no-elements --mode screen --path /tmp/peekaboo-screen.png`. To inspect a running app instead, `peekaboo see --app Finder --json` returns a structured UI map with opaque element IDs — that observe-then-act loop is the core of how Peekaboo works. To automate an app, target an element by its accessible label: `peekaboo click "Address and search bar" --app Safari`, then `peekaboo type "github.com/openclaw/Peekaboo" --app Safari && peekaboo press Return --app Safari`. For natural-language runs, `peekaboo agent "Open Safari, go to github.com, and search for Peekaboo"` chains the same tools, but it needs a configured model provider first.
Practical Automation Scenarios
- •Wiring an AI coding agent, such as Claude Code, Cursor, or Codex, to actually see and drive a macOS app during a task, through Peekaboo's MCP server instead of raw AppleScript.
- •Scripting a repeatable UI workflow, like opening Safari, typing a URL into the address bar, and pressing Return, using `click`/`type`/`press` against accessible labels.
- •Running `peekaboo agent "<instruction>"` when the task is easier to describe in natural language than to hand-code as a fixed script.
- •Automating a background app without stealing window focus, since targeted input uses background delivery whenever Peekaboo can resolve the target process.
- •Feeding screenshots to a local or remote AI model for visual question answering, per the project's own GitHub description.
Strengths
- ✓The `see` then element-ID then `click`/`type` workflow replaces blind pixel-coordinate scripting with references to actual accessible UI elements.
- ✓Background input delivery means you can automate an app without it stealing window focus, useful when a script or agent is driving more than one app.
- ✓One toolset covers three surfaces — direct CLI use, an `agent` command, and an MCP server for external AI clients — instead of separate tools for each.
- ✓MIT-licensed, with a documented command reference, a migration guide, and per-topic docs for permissions, automation, and MCP setup rather than just a single README.
- ✓Version 4.0's fixed 33-command root surface and consistent duration/coordinate grammar make the CLI's behavior easier to predict across commands.
Considerations and Requirements
- △Automation needs three separate macOS permission grants: Screen Recording for capture, Accessibility for UI control, and, per the permissions guide, an additional permission for synthetic input.
- △The `agent` command needs a configured model provider before it runs anything, so natural-language automation isn't available out of the box the way direct CLI commands are.
- △Version 4.0 changed the command and flag grammar enough that the README points existing scripts or agent instructions to a dedicated migration guide rather than promising backward compatibility.
- △Pre-dispatch argument parse or bind failures can omit the `effect` field from the JSON response, per the README's own 4.0 notes, so response-parsing code can't assume a fully consistent JSON shape.
- △Official releases are macOS-only; Windows and Linux users are pointed to separate third-party rewrites (PeekabooWin, PeekabooX) rather than a first-party port.
- △The npm package and MCP launcher need Node.js 22 or later on top of the macOS 15 CLI requirement, so the dependency chain is longer than a single Homebrew install.
Other macOS Automation Tools
Common Questions
Peekaboo's released CLI and app require macOS 15 or later, and source builds share that same macOS 15 minimum, per the README.
Peekaboo needs Screen Recording permission for screen capture and Accessibility permission for UI inspection and control, plus an additional permission for synthetic input covered in its permissions guide.
Peekaboo ships an MCP server; its documentation specifies that it can be connected to Codex, Claude Code, Cursor, or another MCP client, allowing an external AI agent to drive its tools directly.
Peekaboo is released under the MIT license, per its GitHub repository.
Peekaboo's `see` command returns a JSON map of an app's UI elements with opaque IDs; you target one by its accessible label using commands like `click`, `type`, `press`, `scroll`, `drag`, or `set-value`.
Peekaboo ships a menu-bar Mac app, distributed as a signed DMG on its GitHub releases page, that handles permission onboarding, visual feedback, and agent sessions; the CLI is installed separately if you want `peekaboo` on PATH.
The problem it solves
AI coding agents that only get a text description of a macOS app have to guess at pixel coordinates or lean on hand-written AppleScript for every UI action. Peekaboo's README frames its purpose around that gap: give an agent, or a script, a `see` command that returns real, labeled UI elements with opaque IDs, then act on those elements through the Accessibility API instead of hardcoded coordinates or app-specific scripting glue.
Who should try it — and who should skip
Try Peekaboo if you're building or scripting macOS automation and want structured UI element access instead of blind coordinates, especially if you're wiring an AI agent through MCP to actually see and act on a Mac screen. Skip it if you need cross-platform automation, since Peekaboo itself is macOS-only and points elsewhere for Windows or Linux, or if you just want a quick screenshot without granting Screen Recording and Accessibility permissions.
Related repositories
Want a second opinion on Peekaboo?
Ask an AI that can read this page — one click and you get its take on Peekaboo.