Can Peekaboo handle complex gestures and keyboard shortcuts?

Yes, it supports multi-step interactions including dragging, swiping, smooth scrolling, and complex modifier key combinations like cmd+shift+t.

How does Peekaboo identify UI elements?

It uses a combination of snapshot IDs, coordinate-based targeting, and the macOS Accessibility tree to precisely locate buttons, text fields, and system menus.

Does Peekaboo work on Windows or Linux?

No, Peekaboo is specifically designed for macOS (darwin) and requires native system APIs like Screen Recording and Accessibility to function.

Is special setup required for this skill?

Yes, you must install the Peekaboo CLI (typically via Homebrew) and grant macOS Screen Recording and Accessibility permissions to your terminal or host application.

Peekaboo macOS Automation

Name: Peekaboo macOS Automation
Author: xiaxingxiaowei1983

byxiaxingxiaowei1983

0•

Productivity & Workflow

Automates macOS user interfaces through advanced screen capture, element targeting, and system-level input simulation.

Peekaboo is a comprehensive macOS UI automation skill that empowers Claude to interact directly with the desktop environment. By leveraging the Peekaboo CLI, this skill enables programmatic control over applications, windows, and system menus, allowing for complex workflows such as UI testing, automated screen recording, and cross-app data manipulation. It uniquely combines vision-based element detection with native Accessibility API calls to ensure reliable targeting and interaction, making it an essential tool for developers and power users looking to automate repetitive tasks on Darwin-based systems.

Key Features

01Full macOS UI automation including windows, menus, and the Dock

02Comprehensive input simulation for clicking, dragging, typing, and gestures

03Scriptable CLI interface supporting JSON output for seamless integration

04Vision-powered UI mapping and screen capture with optional AI analysis

050 GitHub stars

06Advanced element targeting using IDs, coordinates, or visual analysis

Use Cases

01Automated screen recording and UI state analysis for documentation or debugging

02Creation of complex workflows that bridge data between different desktop apps

03Automated end-to-end testing for macOS desktop applications

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add xiaxingxiaowei1983/openclaw peekaboo

For use in Claude.ai and ChatGPT

Key Features

01Full macOS UI automation including windows, menus, and the Dock

02Comprehensive input simulation for clicking, dragging, typing, and gestures

03Scriptable CLI interface supporting JSON output for seamless integration

04Vision-powered UI mapping and screen capture with optional AI analysis

050 GitHub stars

06Advanced element targeting using IDs, coordinates, or visual analysis

Use Cases

01Automated screen recording and UI state analysis for documentation or debugging

02Creation of complex workflows that bridge data between different desktop apps

03Automated end-to-end testing for macOS desktop applications

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add xiaxingxiaowei1983/openclaw peekaboo

For use in Claude.ai and ChatGPT