Peekaboo

What It Does

Commands share a snapshot cache so a single `see` call can power multiple subsequent interactions without re-capturing the screen.

Peekaboo is a comprehensive macOS UI automation CLI that gives AI agents and scripts full control over the Mac desktop. It covers screen capture and annotation, element targeting, mouse/keyboard input, app and window management, menus, the Dock, clipboard, and more — all with a consistent `--json` output flag for easy scripting.

Key Features

• Annotated UI Maps (`see`) — The `see` command captures a snapshot and overlays labeled element IDs onto the screen or window image. These IDs (e.g., `B1`, `T2`) can then be passed directly to `click`, `drag`, `type`, and other interaction commands, making targeting reliable without hardcoding pixel coordinates.
• Full Interaction Suite — Peekaboo covers every input primitive: `click`, `drag`, `swipe`, `scroll`, `move`, `hotkey`, `press`, `type`, and `paste`. Commands accept element IDs, window targeting parameters, or raw coordinates, and support human-like timing profiles and delays.
• App, Window & System Management — Launch, quit, hide, and switch apps; close, minimize, maximize, move, and resize windows; switch Spaces; click application menus by path (e.g., `Format > Font > Show Fonts`); interact with the Dock and status-bar menu extras — all from one tool.
• Screen Capture & Video Ingest — Capture full screens, specific windows, or menu bar regions in PNG/JPG at retina resolution. The `capture live` command records motion-aware video with configurable active/idle frame rates, change highlighting, and region targeting.
• JSON Output & Script Execution — Every command supports `--json`/`-j` for machine-readable output, making it straightforward to pipe results into other tools or agents. The `run` command executes `.peekaboo.json` script files for repeatable multi-step workflows.
• Snapshot Cache & Remote Bridge — Commands share a snapshot cache so multiple interaction steps reuse the same captured state without redundant screen grabs. The `bridge` command and `--bridge-socket` flag enable connectivity inspection for remote/agent setups.

Requirements

Grant via *System Settings > Privacy & Security > Accessibility*. - **`peekaboo` binary** — install via Homebrew: `brew install steipete/tap/peekaboo`.

- **Screen Recording permission** — required by macOS to capture screen content. Grant via *System Settings > Privacy & Security > Screen Recording*. - **Accessibility permission** — required for UI element inspection and input automation.

Use Cases

• AI agent controlling the macOS desktop — An agent calls `peekaboo see --annotate` to get a labeled screenshot, reads the JSON element map, then issues `click`, `type`, and `hotkey` commands to fill in forms, navigate dialogs, and confirm actions — all without a human at the keyboard.
• Automated login and multi-step form flows — Use `peekaboo see` to locate input fields by annotated ID, then `click` and `type` to enter credentials, `press tab` to advance, and `hotkey cmd,return` to submit — reproducible across app restarts.
• Dashboard screenshot and visual analysis — Schedule `peekaboo image --app Safari --window-title "Dashboard" --analyze "Summarize KPIs"` to capture a window and send it to an LLM for a written summary, producing regular automated reports without any manual screenshots.
• Scriptable app and window orchestration — Write a `.peekaboo.json` script that launches multiple apps, arranges their windows to specific bounds and positions, switches Spaces, and opens URLs — then run it with `peekaboo run` to reproduce a full workspace layout on demand.