> ## Documentation Index
> Fetch the complete documentation index at: https://flowdeck.studio/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# macOS Automation

> Overview of FlowDeck's macOS app UI automation

FlowDeck ships built-in UI automation for macOS apps so you can drive native interfaces, validate UI state, and capture accessibility trees from the CLI. Use `flowdeck ui mac` to run automation commands.

## What It Does

Use macOS automation to:

* Capture screenshots and accessibility trees from any running macOS app.
* Click elements, type text, scroll, drag, and navigate menus.
* Wait for UI state changes and assert UI conditions.
* Manage windows, launch/quit apps, and send keyboard shortcuts.

## When to Use It

* Smoke tests for macOS app flows.
* Scripted demos or QA checks on native Mac apps.
* AI-driven interaction loops that need consistent UI state.
* Automating menu-driven workflows or window management.

## Prerequisites

macOS automation requires two system permissions:

1. **Accessibility** — allows FlowDeck to read the accessibility tree and inject input.
2. **Screen Recording** — allows FlowDeck to capture window screenshots (macOS 14+).

Check and request permissions:

```bash theme={null}
flowdeck ui mac check-permissions
flowdeck ui mac request-permissions
```

After granting permissions in System Settings > Privacy & Security, restart your terminal for changes to take effect.

## Quick Start

```bash theme={null}
# 1) Check permissions
flowdeck ui mac check-permissions

# 2) List running apps
flowdeck ui mac list apps

# 3) Start background session capture (trees + screenshots)
flowdeck ui mac session start --app "Safari" --json

# 4) Drive one interaction
flowdeck ui mac click "Downloads" --app "Safari"

# 5) Capture visual proof
flowdeck ui mac screen --app "Safari" --output /tmp/flowdeck-mac-ui-proof.png

# 6) End session
flowdeck ui mac session stop
```

<Note>
  The `--app` flag accepts an app name (e.g., `"Safari"`), bundle ID (e.g., `"com.apple.Safari"`), or PID (e.g., `"12345"`). It is required on most commands.
</Note>

## Tangible Workflows

### 1) Automate a macOS app form fill

Use this to fill out a form in a native macOS app and verify the result.

```bash theme={null}
flowdeck ui mac session start --app "MyApp"
flowdeck ui mac click "nameField" --app "MyApp" --by-id
flowdeck ui mac type "John Doe" --app "MyApp"
flowdeck ui mac click "emailField" --app "MyApp" --by-id
flowdeck ui mac type "john@example.com" --app "MyApp"
flowdeck ui mac click "submitButton" --app "MyApp" --by-id
flowdeck ui mac wait "Success" --app "MyApp" --timeout 10
flowdeck ui mac screen --app "MyApp" --output /tmp/form-result.png
flowdeck ui mac session stop
```

### 2) Drive a menu-based workflow

```bash theme={null}
flowdeck ui mac session start --app "TextEdit"
flowdeck ui mac menu list --app "TextEdit"
flowdeck ui mac menu click "File > Export as PDF" --app "TextEdit"
flowdeck ui mac wait "Save" --app "TextEdit" --timeout 5
flowdeck ui mac session stop
```

### 3) Window management

```bash theme={null}
flowdeck ui mac session start --app "Safari"
flowdeck ui mac window list --app "Safari"
flowdeck ui mac window move --app "Safari" --to 100,100
flowdeck ui mac window resize --app "Safari" --size 1200,800
flowdeck ui mac window focus --app "Safari" --index 1
flowdeck ui mac session stop
```

### 4) Verify a build output visually

Build and run your macOS app, then capture its UI:

```bash theme={null}
flowdeck run -D "My Mac"
flowdeck ui mac session start --app "MyApp"
flowdeck ui mac screen --app "MyApp" --output /tmp/current-ui.png
flowdeck ui mac session stop
```

Compare `/tmp/current-ui.png` with the expected design and iterate.

## App Targeting

The `--app` flag resolves targets in this order:

1. **Numeric** — treated as a PID (e.g., `--app "12345"`)
2. **Contains a dot** — treated as a bundle ID (e.g., `--app "com.apple.Safari"`)
3. **Otherwise** — fuzzy-matched against running app names (e.g., `--app "Safari"`)

To see all running GUI apps:

```bash theme={null}
flowdeck ui mac list apps
flowdeck ui mac list apps --include-agents --include-system
```

## Differences from iOS Simulator Automation

| Feature            | iOS (`ui simulator`)                          | macOS (`ui mac`)                              |
| ------------------ | --------------------------------------------- | --------------------------------------------- |
| Target flag        | `-S` (simulator name/UDID)                    | `--app` (name/bundle ID/PID)                  |
| Click/tap          | `tap`                                         | `click` (`tap` is a hidden alias)             |
| Right-click        | N/A                                           | `right-click`                                 |
| Drag               | N/A                                           | `drag --from x,y --to x,y`                    |
| Cursor movement    | N/A                                           | `move --point x,y`                            |
| Keyboard shortcuts | N/A                                           | `hotkey "cmd+s"`                              |
| Key input          | HID keycodes only                             | `--name` (e.g., `return`) or `--keycode`      |
| Menu interaction   | N/A                                           | `menu list` / `menu click`                    |
| Window management  | N/A                                           | `window list/move/resize/focus`               |
| App lifecycle      | N/A                                           | `launch` / `activate` / `quit`                |
| Sessions           | Background capture sessions                   | Background capture sessions                   |
| Assert             | `assert visible/hidden/enabled/disabled/text` | `assert visible/hidden/enabled/disabled/text` |
| Scroll             | Fraction-based `--distance`                   | Tick-based `--amount`                         |
| Scroll until       | `scroll --until`                              | `scroll --until`                              |
| Swipe              | Direction argument                            | `--direction` flag                            |
| Multi-touch        | `pinch`, `rotate`                             | N/A                                           |
| Hardware buttons   | `button home/lock/...`                        | N/A                                           |
| Deep links         | `open-url`                                    | N/A                                           |
| Appearance         | `set-appearance light/dark`                   | N/A                                           |
| Coordinates        | Points (normalized)                           | Screen-absolute points                        |
| Permissions        | None needed                                   | Accessibility + Screen Recording              |

## Performance and Reliability Tips

* Prefer accessibility identifiers and use `--by-id` for clicks, finds, and assertions (fastest and most reliable).
* For automation loops, start a session and read `latest-tree.json`/`latest.jpg` from disk instead of calling `screen` every step.
* For agent loops, run `flowdeck ui mac session start --app "MyApp" --json` to capture tree + screenshots in the background and read from `./.flowdeck/automation/mac-sessions/<session-short-id>/` (use `latest.json`, `latest.jpg`, and `latest-tree.json` to find the newest capture). Starting a session stops any active session first.
* `session start` returns the session directory plus `latest_screenshot` and `latest_tree` paths in JSON.
* Use `flowdeck ui mac screen --tree --json --app "MyApp"` when you only need the accessibility tree (no screenshot).
* Coordinates are screen-absolute points matching the values returned by `find`. Do not scale by Retina factors.
* Use `flowdeck ui mac find` to discover element labels and IDs before interacting.
* For smooth scrolling, use `--smooth` on the `scroll` command.
* Tune input timing with environment variables:
  * `FLOWDECK_HID_STABILIZATION_MS` (default 25) for click/gesture stability
  * `FLOWDECK_TYPE_DELAY_MS` (default 1) for typing speed

<Tip>
  For the complete command list and flags, see the [macOS UI Automation command reference](/cli/commands/ui/mac-automation).
</Tip>