Allowed runtime deps
xcrunxcrun simctlxcodebuildXCTest/XCUITest- Swift code owned by this repository
Tales V1 supports iOS automation through Apple’s official tooling and a repository-owned Swift/XCUITest HTTP driver. There is no Appium server, no Maestro runtime, no IDB requirement, and no external WebDriverAgent dependency.
.tales scenario → step "mobile" → internal/runtime/mobile.go → internal/provider/mobile (Go) → xcrun simctl + xcodebuild → embedded XCUITest driver (extracted from the binary on first use) → XCUIApplication(bundleIdentifier: <SUT>)The Go provider owns simulator lifecycle, app installation (and clear_state uninstall/install) via simctl, step serialization per mobile target, implicit waits, and artifact collection. App launch and termination are driven through the Swift driver (XCUIApplication.launch() / terminate()) so XCTest keeps a live handle on the app process. The Swift driver lives under drivers/apple/TalesAppleDriver/ and exposes a small HTTP/JSON surface for hierarchy, tap, input text, clear text, screenshot, launch, and terminate operations.
Allowed runtime deps
xcrunxcrun simctlxcodebuildXCTest / XCUITestExplicitly not used
Maestro-style architecture can be useful inspiration, but Tales does not vendor or execute Maestro code.
The demo app lives under e2e/ios/demoapp/. It is a minimal SwiftUI app with bundle id org.taleslabs.tales.demo.
Screens:
welcome.title, welcome.registerregister.screen, register.email, register.password, register.submit, register.errorverify.screen, verify.code, verify.submit, verify.errorhome.screen, home.title, home.emailThe verification code is intentionally hardcoded to A1B2C3 so the mobile e2e flow is deterministic.
Cross-platform CI targets remain platform-neutral:
make testmake lintmake e2emake e2e-failuremacOS / Xcode-only targets:
make doctor-iosmake build-ios-demomake e2e-iosmake e2e-ios-failuremake doctor-ios prints system, Xcode, simctl, and iOS-related environment state without requiring optional variables to be set. Run it first when a local simulator behaves strangely after an Xcode upgrade. Prefer tales doctor when the Tales binary is available, it covers the same ground and adds embedded-driver cache introspection.
make build-ios-demo:
xcodebuild and xcrune2e/ios/demoapp/TalesDemoApp.xcodeprojbuild/ios/demoappTalesDemoApp.app for iOS Simulatorbuild/ios/demoapp/app_path.txtmake e2e-ios:
IOS_APP_PATH, IOS_BUNDLE_ID, and IOS_DEVICE_NAME (defaults to IOS_DEVICE_NAME="iPhone 17")tales test ./e2e/ios/pass --seed 1234 --parallel 1build/reportsbuild/artifactsmake e2e-ios-failure runs the failing iOS suite, expects exit code 1, and verifies that the failure is the expected missing_element visibility failure, not a simulator/driver environment failure.
IOS_DEVICE_NAME="iPhone 17" make e2e-iosIOS_DEVICE_NAME="iPhone 17 Pro" make e2e-ios-failureThe application under test must be built for the iOS Simulator. A physical-device .app bundle will not install into the simulator.
Tales V1 only auto-builds the repository demo app via make build-ios-demo. User applications should be built by the owning project and passed through:
IOS_APP_PATH=/path/to/MyApp.app \IOS_BUNDLE_ID=com.example.MyApp \IOS_DEVICE_NAME="iPhone 17" \ tales test ./my/mobile/suite --seed 1234config { mobile = { targets = { iphone = { platform = "ios" device_name = env("IOS_DEVICE_NAME", "iPhone 17") app = env("IOS_APP_PATH") bundle_id = env("IOS_BUNDLE_ID", "org.taleslabs.tales.demo") driver = { host = env("IOS_DRIVER_HOST", "127.0.0.1") port = 9080 // optional in embedded mode; omit to auto-allocate a free port external = false // Embedded mode is the default: project/scheme omitted. // The driver is extracted from the tales binary and built once. } } } }}
scenario "iOS register demo app" { # Guard the scenario so cross-platform CI runs do not try to # exercise the mobile provider on Linux / Windows. skip_unless { os = ["darwin"] env_set = ["IOS_APP_PATH"] reason = "iOS tests require macOS and IOS_APP_PATH pointing at a simulator-built app" }
step "mobile" "launch" { platform = "ios" target = "iphone" launch { clear_state = true } expect { visible { id = "welcome.register"; timeout = "20s" } } }
step "mobile" "open_register" { platform = "ios" target = "iphone" actions { tap { id = "welcome.register" } } expect { visible { id = "register.screen"; timeout = "10s" } } }}Element-targeted actions:
tap { id = "..." }double_tap { id = "..." }long_press { id = "..." duration = "1s" }, duration optional (default 1s).input_text { id = "..." value = "..." secure = true }clear_text { id = "..." }swipe { id = "..." direction = "up" distance = 0.6 duration = "300ms" }, drags one finger across the element. direction is the finger travel (up / down / left / right); distance (optional, a fraction in (0, 1], default 0.6) is the travel as a share of the element’s relevant dimension; duration optional (default 300ms).scroll { id = "..." direction = "down" }, scrolls the element’s content. direction is the content direction to reveal (the finger travels the opposite way). Accepts the same optional distance / duration as swipe.Device-level actions take no id:
press_key { key = "return" }, presses a hardware keyboard key. key is one of return, enter, tab, space, escape, delete. When key is return or enter and a soft keyboard is up, the driver routes the press to the keyboard’s submit button (Return / Done / Send / locale variants) rather than the text-input event-synthesis path — synthesizing \r while a SwiftUI TextField is the first responder crashes the XCTest runner on iOS 26.x, so the keyboard-button path is both safer and semantically more accurate (it fires the field’s submitLabel).press_button { button = "home" }, presses a device button (home or lock).set_orientation { orientation = "landscape_left" }, rotates the device. orientation is one of portrait, landscape_left, landscape_right, upside_down.dismiss_keyboard {}, dismisses the soft keyboard if one is up. Idempotent (a no-op when no keyboard is present). Useful before a snapshot-heavy step on a tall SwiftUI form: see Snapshot scope for why dismissing the keyboard first matters on iOS 26.x.scroll_to { id = "..." } (or label = "..."), scrolls the element into the viewport so a follow-up tap / input_text can hit it. Idempotent (no-op when the element is already in the safe area). Useful before input_text on offscreen fields on iOS 26.x: a focus tap that misses the actual input affordance leaves no first responder, the synth path then trips Failed to synthesize event: Neither element nor any descendant has keyboard focus, and that XCTest API violation tears the runner down mid-scenario. Tales’ input_text driver handler (both paste mode for SecureField and the regular TextField path) auto-attempts one scroll + retap before bailing with a 500, so most cases work without an explicit scroll_to; reach for it when the auto-scroll is not enough (deeply nested scroll containers, custom keyboard accessories) or to make the test intent explicit.Actions have an implicit wait of 10s with 250ms polling. Each action may set a per-call timeout.
Hierarchy snapshots are scoped to the app’s first window (app.windows.firstMatch.snapshot()) rather than the full application (app.snapshot()). The application snapshot pulls in every connected accessory the runner can reach, including the iOS keyboard daemon process, whose accessibility tree on a focused SwiftUI TextField is enormous (predictive bar, every key, modifiers, hardware-key passthroughs). On tall SwiftUI forms with the keyboard up, that subtree alone can push a single XCUIElement.snapshot() past the driver’s 8s bounded timeout. Scoping to the window stays inside the app’s own process and skips the keyboard daemon tree.
Modal sheets, SwiftUI .sheet content, and UIAlertController-style alerts all live inside that same window via the presentation hierarchy, so locators on them keep resolving. Window-scoped snapshot falls back to app.snapshot() when no window is exposed yet (between terminate and the next launch, or while the app is mid-launch).
If your scenario types into a long form and then needs a hierarchy snapshot, insert a dismiss_keyboard {} before the next assertion — it removes the keyboard from the window subtree and unbounds the snapshot time for follow-up steps.
first = true)Every element-targeted action (tap, double_tap, long_press, input_text, clear_text, swipe, scroll, wait_visible, wait_not_visible) accepts an optional first = true attribute. By default the resolver requires that an id matches a single logical element: a node and a same-id descendant collapse to one match (the SwiftUI wrapper + inner control case), but two genuinely distinct sibling matches surface multiple elements share the same id. That strict default is deliberate — it keeps explicit expect assertions backed by a real uniqueness guarantee.
System pickers break that contract: iOS’s PhotosPicker, the file importer and similar surfaces expose every cell of a grid under the same accessibility identifier (e.g. PXGGridLayout-Info), as siblings inside a collection view rather than as a parent-child pair. first = true opts the action into pre-order first-match resolution — the same semantics as XCUITest’s descendants(matching:).matching(identifier:).firstMatch — so the first cell wins instead of failing with ErrDuplicate.
actions { wait_visible { id = "PXGGridLayout-Info" first = true timeout = "15s" } tap { id = "PXGGridLayout-Info" first = true }}The strict default is intentionally kept on every expect { visible | not_visible | text | value | enabled | disabled } block: assertions still error when an id resolves to two distinct subtrees. Use first = true only at the action level, where the goal is to act on the picker, not to assert its shape.
accessibilityLabel (label = "...")Every element-targeted action (tap, double_tap, long_press, input_text, clear_text, swipe, scroll, wait_visible, wait_not_visible) and every expectation block (visible, not_visible, text, value, enabled, disabled) accepts an optional label = "<accessibilityLabel>" attribute as an alternative to id. The two are mutually exclusive at the parse layer: setting both is rejected with Conflicting element locator, setting neither with Missing element locator.
id matches accessibilityIdentifier — the canonical, locale-independent locator authored on the element. label matches accessibilityLabel — the user-facing string that screen readers announce. Use id whenever the app defines one; reach for label to interact with iOS system controllers that don’t expose an identifier:
PHPickerViewController (the SwiftUI PhotosPicker) — Done / Cancel.UIDocumentPickerViewController — file importer.UIActivityViewController — the share sheet.MFMailComposeViewController / MFMessageComposeViewController.actions { wait_visible { label = "Done" timeout = "10s" } tap { label = "Done" }}
expect { visible { label = "Done" }}Resolution is XCUIElement.descendants(matching: .any).matching(NSPredicate(format: "label == %@", label)).firstMatch on the driver side, mirrored by tree.FindFirstByLabel on the Go side for snapshot-only expectations. Match is exact and case-sensitive; firstMatch is implicit, so first = true next to label is redundant but accepted. Because system labels change with the simulator locale (Done vs Terminé), keep label for cases where id is unavailable — prefer id whenever the app under test defines one.
Unlike first, label IS accepted in expect blocks: the goal is to assert on system controllers as well as interact with them. The uniqueness guarantee of strict id resolution does not extend to label resolution.
expect { visible { id = "..." timeout = "10s" } not_visible { id = "..." timeout = "10s" } text { id = "..." value = contains("Welcome") } value { id = "..." value = "..." } enabled { id = "..." } disabled { id = "..." }}Expectations default to 10s with 250ms polling.
A step-level permissions { <service> = "allow" | "deny" } block sets privacy permissions via simctl privacy after install and before the app launches. Service names are simctl privacy services, camera, photos, location, contacts, microphone, calendar, reminders, motion, media-library, etc.
step "mobile" "launch" { permissions { camera = "allow" photos = "deny" } launch { clear_state = true }}value("id"), element attribute value at capture timetext("id"), element text content at capture timerequest.actions[N].value, the evaluated action value at index Nstep "mobile" "launch" { launch { clear_state = true } actions { wait_visible { id = "welcome.signin" } } capture { password = generate("password_gen") # generated once, real value }}
step "mobile" "fill" { depends_on = ["launch"] actions { input_text { id = "form.password", value = result.launch.password, secure = true } input_text { id = "form.password_confirm", value = result.launch.password, secure = true } }}Calling generate(...) twice produces two different values (the seed mixer includes the expression path), so the capture-once pattern is the only way to get matching values.
A scenario-level record { ... } block instructs the iOS provider to capture a screen recording for the duration of the scenario using xcrun simctl io <UDID> recordVideo. Use it to produce App Store preview videos or to capture flaky-bug repros.
scenario "app_store_preview" { tags = ["video"]
skip_unless { env_set = ["TALES_RECORD"] reason = "Set TALES_RECORD=1 to record an App Store preview" }
record { output = "preview.mp4" # required, relative to scenario.workdir codec = "h264" # optional: h264 (default), hevc mask = "black" # optional: ignored (default), alpha, black force = true # optional, default true }
step "mobile" "intro" { ... } step "mobile" "feature" { ... }}Attributes (all forwarded to simctl io recordVideo unchanged when set):
output (required): path relative to scenario.workdir. The workspace resolver rejects any attempt to escape the per-scenario directory.codec: h264 (App Store-friendly) or hevc. Defaults to simctl’s own default when omitted.mask: ignored, alpha, or black. Controls device-mask handling for notched / Dynamic Island simulators.display: internal or external. Selects which display to capture on devices that expose more than one.target: the mobile target name (config.mobile.targets.<name>) the recording should bind to. Omit when the scenario only drives one target.force: defaults to true; passes --force so simctl overwrites a stale file from a previous run instead of erroring.The recording is wired through a generic provider capability (ScenarioHook), so:
step "mobile" runs in the scenario, which is the earliest point where the simulator is guaranteed booted and a UDID is known.teardown { step ... } runs, so the recording shows the app under test (not the terminate {} / cleanup that closes it) and so a teardown step can assert on the produced file. The recorder sends SIGINT only and waits for simctl to flush the MP4 moov atom; a forceful kill would leave the file unplayable. A panic or early return falls back to a defer so the recorder is always stopped.type = "recording") in console, JSONL (scenario.artifacts[]), JUnit (<system-out>), and the embedded JSON payload of the visual HTML report.There is no dedicated CLI flag. Combine the existing primitives:
scenario "my_recording" { tags = ["video"] skip_unless { env_set = ["TALES_RECORD"] } record { output = "preview.mp4" } # steps...}Run with TALES_RECORD=1 tales test e2e/ios-record; without that variable, the scenario is reported as SKIPPED and xcrun simctl is never invoked. The e2e-ios-record Make target wraps the same invocation for the demo app.
ffmpeg after Tales finishes.--parallel 1 (or --scenario "..." / --tag video) to avoid the conflict.<video> element; the recording path is in the embedded JSON, so external tooling can pick it up.The driver block selects one of three execution modes:
| Configuration | Mode |
|---|---|
external = false, no source_path | Embedded (default). Extract + build + cache. |
external = false, source_path = "..." | Developer override. Same pipeline, local source. |
external = true | External. Health-check only; never spawn or kill. |
No extra fields are required. Tales:
<source-hash>-xcode-<version>-sdk-<version>-dev-<DEVELOPER_DIR>-ios-<runtime>-mac-<major>.<cache>/source/ atomically (rename-after-write).xcodebuild build-for-testing once, capturing output to <cache>/logs/build.log and writing a build.ok marker on success.xcodebuild test-without-building -xctestrun ... on every subsequent session./health failure: invalidates build.ok and rebuilds from scratch before failing the test.When iterating on the Swift driver, point source_path at a local checkout:
driver = { external = false source_path = "/path/to/drivers/apple/TalesAppleDriver"}The cache key still includes the source hash, so edits invalidate the cache automatically.
When you launch xcodebuild test yourself (for example to attach a debugger or capture detailed logs), point Tales at the existing endpoint:
driver = { external = true host = "127.0.0.1" port = 9080}Tales only health-checks the URL; it never spawns or kills an external driver.
In embedded modes (default and developer override) driver.port is optional. When omitted,
Tales auto-allocates a free host port per target at session start. Set it explicitly only when you
need a fixed port (for example to attach external tooling). In external mode the port (and host)
must point at the already-running driver, so set them explicitly.
This matters for multiple simulators in parallel: the simulator binds the driver’s HTTP server on
the host’s shared loopback, so two drivers running at once must use distinct ports. Define one target
per simulator and leave port omitted, and each gets a distinct port automatically, with no manual
bookkeeping:
config { mobile = { targets = { iphone = { platform = "ios", device_name = "iPhone 17", app = env("IOS_APP_PATH"), bundle_id = "com.example.app" } ipad = { platform = "ios", device_name = "iPad Pro 13-inch (M4)", app = env("IOS_APP_PATH"), bundle_id = "com.example.app" } } }}~/Library/Caches/tales/apple-driver/<cache-key>/ on macOS.TALES_DRIVER_CACHE_DIR to a directory of your choice (used as the final base, no extra suffix). Useful in CI to share or pin a cache.~/Library/Caches/tales/apple-driver/<cache-key>/ source/ extracted Swift driver source TalesAppleDriver.xcodeproj/ ... derived-data/ xcodebuild -derivedDataPath logs/ build.log build-for-testing stdout+stderr extract.ok marker, written after a successful extract build.ok marker, contains the cached .xctestrun path metadata.json source_hash, xcode_version, ios_runtime, ... .lock cross-process flock to serialize parallel talesmake clean-ios-driver-cache# or, for a custom base:rm -rf "$TALES_DRIVER_CACHE_DIR"Wipe the cache after a major Xcode upgrade, when you suspect a corrupted build, or before single-binary smoke testing.
Use tales doctor for a one-screen view of everything that influences the embedded driver pipeline.
| Stream | Path |
|---|---|
| Embedded driver build | <cache>/logs/build.log |
| Runtime driver process | build/artifacts/mobile/driver/<target>/driver.log |
| Failure-step screenshots | build/artifacts/mobile/<scenario>-<hash>/<step>/<phase>/attempt-N/screenshot.png |
| Failure-step hierarchy | build/artifacts/mobile/<scenario>-<hash>/<step>/<phase>/attempt-N/hierarchy.json |
When duplicate simulator names exist across runtimes, Tales selects deterministically:
The selected simulator name, UDID, and runtime are printed before the session is used. This avoids accidentally choosing an older duplicate runtime when Xcode ships several simulator runtimes.
Tales selectors are accessibility identifiers only. Do not rely on visible text as a selector.
TextField("Email", text: $email) .accessibilityIdentifier("register.email")
SecureField("Password", text: $password) .accessibilityIdentifier("register.password")
Button("Register") { submit()}.accessibilityIdentifier("register.submit")Every element used by Tales should have a stable identifier. Duplicate IDs are reported as errors instead of guessed.
The provider serializes mobile step execution per target name. Two scenarios using the same target (for example iphone) cannot clear state or terminate the app while each other is tapping or asserting. Different targets may still run in parallel when configured separately, each driving its own simulator on its own auto-allocated driver port.
On mobile step failure Tales writes:
build/artifacts/mobile/<scenario>-<file-hash>/<step>/<phase>/attempt-<n>/screenshot.pngbuild/artifacts/mobile/<scenario>-<file-hash>/<step>/<phase>/attempt-<n>/hierarchy.jsonThe file hash prevents collisions when two files contain scenarios with the same name. Paths are included in console, JUnit, and JSONL reports when available.
When Tales starts the managed Apple driver, stdout and stderr are written to build/artifacts/mobile/driver/<target>/driver.log. If the driver does not become healthy, the failure message includes this log path and suggests make doctor-ios.
When the XCUITest runner crashes during an action (the driver socket closes, the next Tales request sees connect: connection refused / EOF / broken pipe), the failure message switches to the diagnostic-files form: the original transport-level error is preserved, then Tales appends the absolute paths of the three files that hold the real post-mortem information:
… connect: connection refuseddriver process appears to have terminated mid-scenario; diagnostic files: driver log: build/artifacts/mobile/driver/iphone/driver.log xcresult bundle dir: ~/Library/Caches/tales/apple-driver/<key>/derived-data/Logs/Test (open <path>/*.xcresult in Xcode for the full XCTest crash report) build log: ~/Library/Caches/tales/apple-driver/<key>/logs/build.logThe same paths are attached to the step report as driver_log, xcresult_dir, and driver_build_log artifacts (JSONL + visual HTML report).
driver.log holds the live Swift driver log, including the request log
added in this release. Each handled HTTP request now emits
[tales-driver] request: METHOD /path + [tales-driver] response: METHOD /path status=<n> elapsed=<ms>ms, so the last request before XCTest tore down is the
one that triggered the crash.<xcresult dir>/*.xcresult is the Xcode test results bundle.
open ~/Library/Caches/tales/apple-driver/<key>/derived-data/Logs/Test/*.xcresult opens it in Xcode where you get the full failure chain, including
XCTest API violations, SIGABRT, runaway accessibility-engine queries, and any
Test Suite … failed line with its underlying reason.build.log is only useful when the runner died because the embedded
driver build was wrong, which is rare but cheap to expose.External drivers (driver.external = true) skip every Tales-owned path: the
diagnostic block is omitted because Tales does not control the runner there.
tales doctor (or make doctor-ios) to collect system, Xcode, runtime, device, and environment diagnostics.IOS_DEVICE_NAME with xcrun simctl list devices.make build-ios-demo or set IOS_APP_PATH to a simulator .app bundle.-sdk iphonesimulator.IOS_BUNDLE_ID matches the app’s PRODUCT_BUNDLE_IDENTIFIER.<cache>/logs/build.log (path printed in the error). Common causes: SDK no longer installed, signing config drift, stale derived data. make clean-ios-driver-cache then retry.build/artifacts/mobile/driver/<target>/driver.log.sudo xcodebuild -runFirstLaunch, then xcrun simctl shutdown all, then killall -9 com.apple.CoreSimulator.CoreSimulatorService || true, then xcrun simctl list devices. Optionally make clean-ios-driver-cache to force a fresh driver build..accessibilityIdentifier(...) and inspect hierarchy.json..topBarLeading / .topBarTrailing toolbar item under one identifier twice (an other wrapper and the inner button). Tales collapses that nested pair to a single element, so a unique id resolves; only two genuinely separate elements sharing an id are rejected as a duplicate./hierarchy returns a retryable error and Tales keeps polling until the app frees up. Give such a step a timeout (or retry) comfortably above the refresh duration; the driver no longer wedges or drops other requests while one snapshot is slow.xcrun simctl io screenshot availability.driver.port so Tales auto-allocates a free port, or set a different explicit port. See Driver port.