Before you start

This guide assumes:

• FlowDeck is installed and the trial or license is active. If not, follow the getting-started guide first.

• A macOS app project that builds with flowdeck build -D "My Mac". Native macOS or Mac Catalyst both work.

• Willingness to grant Accessibility and Screen Recording permissions to FlowDeck. macOS UI automation can't function without them.

By the end you'll have built and run a macOS app from the terminal, streamed its logs, captured its UI state as JSON, and driven its interface with clicks, keystrokes, and hotkeys. The patterns apply to any macOS app, including ones whose source code you don't have.

Grant macOS permissions to FlowDeck

macOS automation requires two TCC permissions: Accessibility (so FlowDeck can read the accessibility tree of other apps and post synthetic events) and Screen Recording (so it can capture screenshots). Both are one-time grants.

flowdeck ui mac request-permissions

flowdeck ui mac request-permissions

flowdeck ui mac request-permissions

This pops the system prompts. Approve both. If Screen Recording prompts after the first grant, you may need to quit and restart your terminal, macOS caches the permission state per process. After the restart, verify:

flowdeck ui mac list permissions --json

flowdeck ui mac list permissions --json

flowdeck ui mac list permissions --json

Both should report as granted. If anything stays denied, open System Settings → Privacy & Security → Accessibility (and the matching Screen Recording panel) and confirm your terminal application is checked.

Prefer the keyboard? Everything below works from FlowDeck's interactive TUI too, run flowdeck -i in your project and use single-letter shortcuts (R run, L logs, T test, D target). The CLI snippets in the steps below are what the TUI runs under the hood; copy them when you want to script, hand them to an agent, or run one-off commands outside the TUI. See the TUI guide for the full keyboard reference.

Build and run for macOS

Build, install, and launch in one command. The target is "My Mac"; FlowDeck knows that means your local macOS host.

flowdeck build -D "My Mac"
flowdeck run   -D "My Mac"

flowdeck build -D "My Mac"
flowdeck run   -D "My Mac"

flowdeck build -D "My Mac"
flowdeck run   -D "My Mac"

If your project supports Mac Catalyst alongside iOS, the same -D "My Mac" flag works, FlowDeck resolves the destination based on the scheme's product type. For schemes that build both iOS and macOS, save the macOS target to your project config so you don't have to repeat the flag:

flowdeck config set -D "My Mac"

flowdeck config set -D "My Mac"

flowdeck config set -D "My Mac"

The output ends with the app's short ID and bundle identifier. Save the ID, you'll use it for log streaming.

Stream the app's logs

In a second terminal, stream the running app's logs:

flowdeck apps
flowdeck logs <app-id

flowdeck apps
flowdeck logs <app-id

flowdeck apps
flowdeck logs <app-id

FlowDeck registered the launch and routes the log read through the macOS log pathway automatically. Output is your app's print() calls, OSLog messages, and any runtime errors. No WindowServer or system daemon noise.

For a one-shot launch-and-stream pattern, use flowdeck run --log -D "My Mac".

macOS automation has no sandbox

This is the single most important thing to understand before letting an agent drive a macOS app, and the biggest difference from iOS Simulator automation.

On iOS, the agent's input lands inside a sandboxed simulator. You can keep typing in your editor while it runs; the two streams don't touch. On macOS, there's no sandbox. The agent uses your real keyboard, your real mouse, your real focus. While it runs:

• Your cursor moves on its own.

• Your keystrokes get diverted into whatever app the agent is driving, including, on slip-ups, into the wrong app if focus jumps.

• You can't use your computer for anything else.

So the operating mode is: kick the agent off, walk away, come back when it pings. This is closer to "headless run" than "live pairing." Plan tasks accordingly, automation that takes 30 minutes is a coffee break, not a thing you watch.

The upside of having no sandbox: the agent can drive any macOS app, not just yours. It can launch Safari for an OAuth handoff in your app, drive Slack to verify a notification was sent, or open Xcode to inspect a generated project. Things that simply aren't possible in the iOS Simulator.

Prompt with that scope in mind. Example:

Run MyApp, sign in via the Google OAuth flow (which
opens in Safari), come back to MyApp once Safari closes,
and confirm the user's name renders in the header.

Run MyApp, sign in via the Google OAuth flow (which
opens in Safari), come back to MyApp once Safari closes,
and confirm the user's name renders in the header.

Run MyApp, sign in via the Google OAuth flow (which
opens in Safari), come back to MyApp once Safari closes,
and confirm the user's name renders in the header.

The agent handles the cross-app handoff automatically, activate MyApp, click Sign in, switch focus to Safari, complete the OAuth dance, come back to MyApp once it regains focus. None of that is possible on iOS.

Focus is global, the activation rule

The flip side of no sandbox is that input goes to whichever app is frontmost at the moment the event fires. If the agent clicks while another app has focus, the click lands in the wrong place. The fix is mechanical: before every interaction sequence, activate the target app first.

The FlowDeck skill pack documents this rule for the agent, once installed (see the AI agents setup guide), the agent automatically calls flowdeck ui mac activate before any click / type / hotkey sequence. The pattern it runs under the hood:

flowdeck ui mac activate --app "MyApp"
flowdeck ui mac click "Save" --app "MyApp"
flowdeck ui mac type "Hello" --app "MyApp"

flowdeck ui mac activate --app "MyApp"
flowdeck ui mac click "Save" --app "MyApp"
flowdeck ui mac type "Hello" --app "MyApp"

flowdeck ui mac activate --app "MyApp"
flowdeck ui mac click "Save" --app "MyApp"
flowdeck ui mac type "Hello" --app "MyApp"

If you cmd-tab to a different app while the agent is working, the next activate call yanks focus back. Once you understand that's happening, the "walk away" mental model in Step 04 makes more sense, competing with an active automation for focus produces neither good work nor a usable computer.

Between interactions, the agent reads what's on the window the same way it does on iOS:

flowdeck ui mac screen --app "MyApp" --json

flowdeck ui mac screen --app "MyApp" --json

flowdeck ui mac screen --app "MyApp" --json

That returns a JPEG of the app's frontmost window and the accessibility tree as structured JSON, the same shape flowdeck ui simulator screen --json returns. The agent uses the tree to find elements by label or identifier, and the screenshot when it needs pixel-level context (custom-drawn UI, comparing against a design). For long sessions, flowdeck ui mac session start --app "MyApp" writes a fresh capture every ~500ms so the agent doesn't pay the capture cost on every action.

What the agent can and can't do

The macOS automation surface is wide. The agent reads the same accessibility tree macOS itself uses, and drives input through the standard accessibility APIs. That means a lot is possible, and a few things genuinely aren't.

What works

• Click, double-click, right-click. "Click Save," "right-click the first row." Right-click and context menus are first-class on macOS (no iOS equivalent).

• Type, with masking. "Type my email." Use --mask for passwords so credentials don't appear in logs.

• Hotkeys. "Cmd+S to save," "cmd+shift+P to open the command palette." A first-class macOS primitive that you reach for constantly.

• Menu navigation. "Pick File > Save As." The agent walks the menu bar by label path, faster and more reliable than driving menus with clicks and arrow keys.

• Window management. "Move the main window to the secondary display," "close every window except the frontmost." Multi-window choreography is possible because macOS apps actually have multiple windows.

• Cross-app workflows. Drive any installed app, not just yours. OAuth handoffs to Safari, deep-link tests through Mail, Xcode automation, all in scope.

• Read accessibility trees for native apps. SwiftUI and AppKit apps expose clean, labelled trees. The agent matches elements by label, role, or accessibility identifier.

Where accessibility quality varies

• Electron and Catalyst apps. Some expose decent AX trees, some are very flat with generic labels. The agent's first attempt may fail to find a labelled element; tell it to fall back to find for inspection, then to coordinates from a screenshot if labels really don't exist.

• Web content inside native apps. A WKWebView shows up in the tree as a single opaque element. The agent can't reliably read the HTML inside. For these, prompt it to use screenshot+coordinates instead of element matching.

• Custom-drawn UI. Apps that draw their own controls (audio software, design tools) often have minimal accessibility. The agent falls back to --point "x,y" coordinates pulled from screen --json.

What it can't do

• Bypass system security prompts. TCC prompts (camera, microphone, location, file access), the FileVault unlock screen, the lock screen, and the password-required Keychain dialog all require a human. The agent can't click through these on your behalf.

• Use Touch ID or Watch unlock. Biometric prompts pause the flow until you authenticate. The agent will wait, you'll authenticate, then it continues.

• Drive sandboxed system-level UI. The dock, Spotlight, Notification Center, and Mission Control are special. Some clicks work, many don't. Prefer hotkeys (cmd+space for Spotlight) over chasing those surfaces with the AX tree.

• Read your screen across desktop spaces simultaneously. The agent sees the space it's currently on. If the target app is on a different desktop space, instruct the agent to activate it (which switches to that space) before reading state.

Patterns that come up most often

Drive a third-party macOS app.
FlowDeck doesn't care whether you built the app. Launch any installed app with flowdeck ui mac launch --bundle-id com.example.app, activate it, drive it. Useful for end-to-end tests that span your app and another (e.g., your app handing off to Safari for OAuth).

Use find before clicking ambiguous labels.
"Settings" might appear in three places on a complex screen. flowdeck ui mac find "Settings" --app "MyApp" --json shows you exactly what elements match, with frames and identifiers. Pick the right one, then click it by --by-id instead of label.

Use coordinates from FlowDeck screenshots, not raw screen coords.
If you need to click a custom-drawn region that has no accessibility identity, read the screenshot from screen --json, find the pixel coordinates, and use --point "x,y". The coordinates in FlowDeck's JSON are in points (not @2x pixels), so no scaling math.

Capture before each action in long sessions.
Or use session start. Either way, never chain more than 2-3 actions without re-reading state. macOS apps are full of asynchronous behavior (loading spinners, modals, accessibility-tree refreshes) that breaks blind action chains.

Keep an eye on focus.
Even with activate, macOS apps can spontaneously yield focus (notifications, system modals, login items launching). If a sequence flakes, the first thing to check is whether the app was actually frontmost when the action fired.

How macOS automation differs from iOS Simulator automation

For reference, the differences between flowdeck ui mac and flowdeck ui simulator.

The shared primitives (click, type, scroll, wait, assert, screen capture) behave the same way. The macOS-only commands cover the things that don't have iOS equivalents.