What specific actions can macOS GUI Control automate?

It can automate mouse clicks, movements, drags, scrolls; keyboard typing, key presses, shortcuts; full-screen or region screenshots, pixel color retrieval; listing, focusing, moving, and resizing windows; and securely executing AppleScript or shell commands.

Are there any safety features or limitations when using it with AI?

Yes, for user safety, macOS GUI Control explicitly blocks potentially destructive operations. This includes commands like `rm`, `rmdir`, `delete file`, or `empty trash` when executing shell commands or AppleScripts, ensuring secure automation.

What is macOS GUI Control?

macOS GUI Control is an MCP (Multi-Component Package) server for macOS, providing AI assistants with comprehensive, safe control over your Mac's mouse, keyboard, screen, and window management to enable advanced automation.

What are the prerequisites for installing and using macOS GUI Control?

You will need `cliclick` installed via Homebrew, Node.js 18+, and essential macOS privacy permissions (Accessibility and Screen Recording) granted to your AI assistant application (e.g., Claude Desktop).

macOS GUI Control: Automate Mac GUI with AI Assistants

About

This powerful server extends the capabilities of your AI assistant by enabling comprehensive control over your macOS graphical user interface. It facilitates automated interactions with your operating system, including precise mouse movements, dynamic keyboard input, versatile screen capture, and robust window management, all while implementing crucial safeguards to prevent destructive actions. Seamlessly integrate this tool with Claude Desktop or Code to empower your AI with the ability to directly interact with macOS applications and streamline a wide range of desktop-based tasks.

Key Features

Automated Mouse Control: Perform clicks, double-clicks, right-clicks, movements, drags, and scrolls.
Advanced Keyboard Interaction: Type text, press individual keys (including modifiers), and execute common shortcuts.
Screen Interaction & Capture: Take full-screen or region-specific screenshots, retrieve screen dimensions, and get pixel colors.
Comprehensive Window Management: List open windows, focus specific applications, and move or resize windows.
Secure AppleScript & Shell Command Execution: Run custom AppleScript or shell commands with built-in safeguards against destructive operations.
4 GitHub stars

Use Cases

Automating repetitive macOS tasks for increased efficiency and reduced manual effort.
Creating custom workflows for testing, data entry, or sophisticated application control on macOS.
Developing AI assistants that can directly interact with desktop applications and manipulate the GUI.

About

Key Features

Automated Mouse Control: Perform clicks, double-clicks, right-clicks, movements, drags, and scrolls.
Advanced Keyboard Interaction: Type text, press individual keys (including modifiers), and execute common shortcuts.
Screen Interaction & Capture: Take full-screen or region-specific screenshots, retrieve screen dimensions, and get pixel colors.
Comprehensive Window Management: List open windows, focus specific applications, and move or resize windows.
Secure AppleScript & Shell Command Execution: Run custom AppleScript or shell commands with built-in safeguards against destructive operations.
4 GitHub stars

Use Cases

Automating repetitive macOS tasks for increased efficiency and reduced manual effort.
Creating custom workflows for testing, data entry, or sophisticated application control on macOS.
Developing AI assistants that can directly interact with desktop applications and manipulate the GUI.