Learn · UI automation

iOS UI automation has been stuck for a decade.

A practical look at why testing the screen is harder than testing the model, what changes when an AI agent can see and act on iOS UI, and the criteria that matter when you're evaluating an automation tool.

XCUITest shipped in 2015. Since then, the iOS app ecosystem has changed shape: SwiftUI, more complex navigation graphs, more screens per app, more accessibility surface, and a new category of work that didn't exist a decade ago, autonomous agents writing UI code. The testing layer hasn't kept up. Not because XCUITest is bad, but because it was built for one job, and that job is no longer the only one teams need a UI tool for.

If you've ever:

  • Spent a week getting an XCUITest target to compile against your app's signing configuration.
  • Watched XCUIElement.tap() fail because the element wasn't hittable, even though it was rendered.
  • Wanted to write a one-off automation script for a single bug, and decided manual was faster than the test scaffolding.
  • Tried to give an AI agent the ability to verify that the screen its code rendered actually looks right.
  • Sent a screenshot to a teammate because that was easier than describing what the app was doing.

…you've hit the friction this page is about. The gap isn't in XCUITest. The gap is everywhere XCUITest doesn't reach.

XCUITest is for engineers, not for everyone

XCUITest is a great tool for one specific job: writing repeatable, in-target regression tests that run on every CI commit. It's well-integrated with Xcode, produces XCResult bundles, plays with code coverage and Test Plans, and is supported by Apple. If you're writing a "verify the login flow still works after every change" suite, XCUITest is the right tool.

But XCUITest carries assumptions that don't fit the rest of the work iOS teams do:

  • Tests live inside a test bundle in your project. That means a compile step every time something changes. It also means an engineer has to write the test, and the test has to be in the language of the project.
  • The test target needs to be signed and provisioned like the app target. Onboarding the test bundle to a new app is a non-trivial config exercise.
  • The test bundle is paired with your app at signing time. XCUITest can launch and drive other apps via XCUIApplication(bundleIdentifier:), but the runner has to exist inside an Xcode project you compiled. You can't run an XCUITest script against a third-party binary you didn't build.
  • Output is structured for Xcode, not for streams. XCResult bundles are parsed after the fact; you don't get a stream of structured events while the test runs.
  • Discovery requires a build. Listing tests means compiling the test target, which is slow.

None of these are bugs. They're choices that make sense for codified regression. They make less sense for exploratory automation, one-off debugging scripts, third-party app interaction, AI agents driving the app, or shipping a UI tool to an engineer who is not on the iOS team.

What changes when the agent can see the screen

The thing that's been quietly transforming the testing category isn't a better assertion DSL. It's the rise of agents that can act on what's on screen, given two things they didn't reliably have before: a screenshot and a machine-readable description of every element.

The accessibility tree was always there. iOS has carried it since 2009 for VoiceOver. What changed is that until recently, the only ways to read it from a CLI were either XCUITest (in-process, in-target) or hand-written FBSimulatorControl plumbing. Both are non-starters for most teams. Once a tool exposes the tree as JSON, available to anything that can run a shell command, the category opens up:

  • An agent can see what shipped, not infer it. The agent writes a SwiftUI view, the build runs, the agent reads the screenshot and the accessibility tree, and verifies the view appears as intended. No human in the loop, no XCUITest target, no compile-and-rerun cycle.
  • A junior engineer can scripts an exploratory flow in 15 minutes, log in, navigate, tap, screenshot, assert, without writing a test class.
  • A QA engineer can capture state at every step of a release smoke test, store the artifacts, and run the same flow on the next release for comparison.
  • A bug report can include not just "a screenshot" but the underlying tree, so the receiving developer knows exactly which elements were rendered, where, and in what state.

The shift is conceptual: UI automation goes from being a thing engineers do once a quarter to a thing the team uses every day, because the friction is gone.

What this unlocks

Out-of-process, structured UI automation enables four things XCUITest was never trying to solve:

The agent loop closes.
When the agent can write code, build, launch, screenshot, and assert in one continuous flow, the time between "I think this works" and "I've verified this works" drops from hours to seconds. The cost of being wrong drops. Agents become useful for UI work where they were previously dangerous.
Exploratory testing becomes routine.
Most iOS bugs don't get caught by codified tests, they're found during a 10-minute click-around before a release. A scriptable, repeatable click-around layer turns that 10 minutes into an artifact a team can run nightly.
Reproducible bug reports.
"Tap login, then Forgot Password, then Back, the navigation is broken" plus a screenshot and the accessibility tree at each step. The receiving developer doesn't need to ask follow-up questions.
Cross-app workflows.
The agent can drive the app, take a deep link to a system app or third-party app, and continue. XCUITest can't follow the handoff; out-of-process automation can.

For teams that already have a healthy XCUITest suite, none of this replaces what they have. It adds a second layer for everything XCUITest was never asked to do.

Evaluating a UI automation tool

If you're considering adding a UI automation layer alongside (or instead of) XCUITest, the criteria that matter aren't features. They're properties of how the tool behaves under real conditions.

Out-of-process operation.
The tool should drive the simulator from outside, with no test target, no compilation step, and no dependency on the app's source code. If you can't run the tool against a third-party app you didn't write, it's not solving the bigger problem.
Accessibility tree as structured data.
The tool should expose the full accessibility tree as JSON: labels, IDs, roles, frames, enabled/visible state, hierarchy. Not just a screenshot. Without the tree, the agent is doing OCR on pixels; with it, the agent is reading the same information VoiceOver does.
Label, ID, role, and coordinate targeting.
Targeting by accessibility label is the default; by accessibility identifier is the fallback for ambiguity; by role and coordinates are the escape hatches for elements that don't expose either. A good tool supports all four with clear precedence rules.
Continuous capture mode.
For long-running agent sessions, the agent should be able to subscribe to a stream of fresh screenshots and trees instead of calling capture every turn. Saves cycles, simplifies the agent's logic.
Structured event output.
Every action, tap, type, assert, should return a structured result describing what happened, with non-zero exit codes on failure for CI use. XCResult bundles are not a substitute.
Composition with XCUITest.
The right tool doesn't replace XCUITest. It complements it. Codified regression stays in XCUITest. Exploratory, agent-driven, and out-of-target work goes through the new layer.

When XCUITest is still right

Out-of-process automation isn't a replacement for everything XCUITest does. The cases where XCUITest is the right answer:

  • Codified regression tests that run on every CI commit, with stable test IDs, code coverage integration, and Xcode Test Reports.
  • Internal-state assertions. Accessing the app's view-model state, private properties, or model layer requires in-process code. XCUITest can reach in; out-of-process tooling can't.
  • Performance assertions. XCUITest has dedicated metrics APIs for measuring app launch, memory, hangs, and other performance characteristics across runs.
  • First-party support guarantees. Apple maintains XCUITest. For teams that need to depend on a vendor-supported testing stack, that matters.

The healthy pattern for most teams is to keep XCUITest for regression and add an out-of-process layer for the long tail. Each tool gets the job it was built for.

Questions teams actually ask

If our team already has a full XCUITest suite, why add another layer?

For the work XCUITest was never asked to do: exploratory testing during release prep, third-party app interaction, bug reports with state captured at each step, and most commonly now, autonomous agent loops. If your team's pain is regression coverage, you don't need a second layer. If your team's pain is everything around the regression suite, you might.

Will this work on apps without accessibility identifiers?

Partially. The tool can still target by label and by role, and it can tap raw coordinates. But apps that don't expose meaningful accessibility data are also apps that are hard to use for users who rely on VoiceOver. Setting accessibility identifiers is good for UI automation. It's also good for accessibility, which is the actual reason to do it.

How does this work with SwiftUI vs UIKit?

Both. SwiftUI views feed into the same accessibility system UIKit views do; the tool reads the resulting tree regardless of which framework rendered it. SwiftUI's .accessibilityLabel() and .accessibilityIdentifier() modifiers are the equivalent of UIKit's properties, and both produce identical results for the tool.

Can we run this in CI alongside our XCUITest suite?

Yes. They don't conflict. The pattern most teams adopt: XCUITest runs as a CI gate on every PR; out-of-process automation runs as a nightly smoke or as a release-day exploration. The two don't share state, and the simulator is happy to host both.

Will this work on physical devices?

Not for full UI automation. Apple does not expose the out-of-process accessibility hooks on physical devices for security reasons. You can still build, run, and stream logs from a physical device with FlowDeck; just not drive the UI. For physical-device UI testing, XCUITest is the available option.

What about flaky tests?

The biggest source of flakiness in UI automation is timing: tapping an element before it's ready, asserting state during an animation. Out-of-process tools tend to be more deliberate here because they don't share a process with the app's run loop; explicit wait commands with --enabled, --stable, and --gone flags reduce flake at the cost of forcing you to be explicit about what "ready" means. The trade-off is honest: less magic, fewer false positives.

How does this compare to Maestro, Detox, or Appium?

Maestro, Detox, and Appium are cross-platform automation frameworks designed to write tests once and run them on iOS and Android. FlowDeck is iOS-only and isn't trying to replace them. The split: if you need cross-platform tests, the cross-platform tools are the right answer; if you need a tool tuned for iOS workflows and AI agents specifically, native is faster and gives you the full accessibility tree without translation.

How fast is this in practice?

Capture is typically 200-400ms per call on an M-series Mac for a standard iPhone simulator (tree + screenshot). A continuous capture session writes a fresh snapshot every ~500ms in the background; agents read the file instead of paying the capture cost each turn. Tap and type are essentially instantaneous; the cost is the implicit wait for the next render, which the tool handles via the same wait primitives.

Going deeper