Automation

Automation is a very big part of Winedroid, for two reasons. First, it makes Winedroid development itself faster and more deterministic: apps can be exercised the same way on every run instead of relying on a human clicking through them. Second, it makes Android apps more accessible in general — scripting and driving apps to do things that are usually restricted on Android.

If you have an idea or a feature that would improve automation, please open a GitHub issue.

Winedroid can drive a running app from a timed input script: tap, type, wait for text, assert on the UI, take screenshots, and dump the view tree. This is how apps are smoke-tested without a human clicking through them.

The automation driver runs on the macOS window backend and in headless mode, which needs no window or display at all.

Running a script

Automation is enabled through environment variables on a normal launch:

WINEDROID_AUTOMATION_SCRIPT=<file> — path to the script to run.
WINEDROID_AUTOMATION_LOG=<csv> — optional; write a per-step latency log with the columns step,t_inject_ms,action,latency_ms,result.
WINEDROID_WINDOW_AUTOCLOSE_MS=<ms> — optional but usual; close the window after N ms so the run terminates on its own.
WINEDROID_AUTOMATION_INTERACTIVE=1 — optional; allow real mouse and keyboard input on the window while automation runs (see below).

While automation is active (a script or autoclick targets), real mouse and keyboard input on the window is ignored by default, so a stray click cannot corrupt a run. Each ignored event prints a Host ... ignored during automation line. Set WINEDROID_AUTOMATION_INTERACTIVE=1 if you want to interact with the window during an automation run.

Example run against a throwaway prefix:

env WINEDROID_PREFIX=/private/tmp/winedroid-auto \
    WINEDROID_AUTOMATION_SCRIPT=/private/tmp/calc.auto \
    WINEDROID_AUTOMATION_LOG=/private/tmp/calc.csv \
    WINEDROID_WINDOW_AUTOCLOSE_MS=2000 \
    winedroid /path/to/app.apk

Each step is echoed to stdout as it runs:

[automation] step 3 t=1250ms tap 58:406 latency=12ms -> ok

Steps run in order. A failing step (a timed-out wait, a failed assert) stops the script and the launch exits with an error.

Headless mode

WINEDROID_HEADLESS=1 runs the app with no window at all — the full runtime and renderer still run, just nothing is shown on screen. This is meant for servers and CI: the entire automation harness works unchanged, including screenshot, dump ui, WINEDROID_RENDER_DUMP_PNG, autoclick targets, and autoclose. Screenshots are rendered from the same surface state the window would show.

env WINEDROID_HEADLESS=1 \
    WINEDROID_PREFIX=/private/tmp/winedroid-ci \
    WINEDROID_AUTOMATION_SCRIPT=/private/tmp/calc.auto \
    WINEDROID_WINDOW_AUTOCLOSE_MS=20000 \
    winedroid /path/to/app.apk

Without a script or autoclose, a headless launch keeps running until it is killed. Window-only features (the Cmd+S screenshot shortcut, native controls) do not exist headless; everything else behaves identically.

Script format

One step per line:

<delay_ms> <verb> [args]

delay_ms — how long to wait after the previous step finished before running this one.
# starts a comment; blank lines are ignored.
Arguments with spaces are wrapped in double quotes: waittext "Add task".
Coordinates are written X:Y in top-origin screen pixels. Use dump ui or WINEDROID_UI_DUMP to find them rather than guessing from screenshots.

Example script:

0    waittext "AC" 8000
300  tap 58:406      # 7
300  tap 361:608     # +
300  tap 159:406     # 8
300  tap 361:709     # =
0    waitidle 3000
0    assert text "15"
0    screenshot /private/tmp/calc.png

Verbs

Input

Verb	Meaning
`tap X:Y [hold_ms]`	Click at a point. With `hold_ms`, dispatch a long-press (a DOWN, a hold of that duration, then UP) to a runtime canvas target.
`scroll <delta_y> [X:Y]`	Scroll the surface (dialog lists or clipped list content). Positive scrolls down; the point defaults to the surface center.
`swipe X1:Y1 X2:Y2 ms [steps]`	Dispatch a real `ACTION_DOWN` / interpolated `ACTION_MOVE`s / `ACTION_UP` MotionEvent stream to the runtime view root.
`key <name>`	Inject a key: `back`, `home`, `menu`, `enter`, `del`, `tab`, `space`, `volup`, `voldown`.
`text "string"`	Type a string into the focused view.
`down / move / up [ptr] X:Y`	Raw pointer primitives; not implemented yet (they need stateful pointer sessions) and fail with a clear error.

Clicking a view by resource id is not a script verb; use WINEDROID_WINDOW_AUTOCLICK_TARGETS=view:<resource-id> instead.

Synchronization

Verb	Meaning
`waitidle [timeout_ms]`	Wait until the runtime settles (no more frames requested). Default timeout 5000 ms; times out with an error.
`waittext "text" [timeout_ms]`	Wait until the text appears on the active surface. Default timeout 5000 ms.

Assertions

Verb	Meaning
`assert text "s"`	The active surface shows the text.
`assert notext "s"`	The active surface does not show the text.
`assert view <resource-id>`	A view with this id exists.
`assert display "s"`	The display/result view shows the text.

Output

Verb	Meaning
`screenshot [path]`	Save the active surface as a PNG (default `automation-screenshot.png`).
`dump [bounds\|surface\|ui] [path]`	`dump ui` writes a readable text tree (nodes, overlays, dialogs, top-origin bounds, clickable flags) to the path, or stdout if no path is given; `bounds` and `surface` print trace-style output. Only `dump ui` accepts a path.

Prefer dump ui over a screenshot when you only need to know what is on screen and where.

Tips

Use a fresh WINEDROID_PREFIX=/private/tmp/... prefix for clean app state; use the real prefix only to check persisted state.
Set WINEDROID_MISSING_PRIMITIVE_LOG=<path> to collect every missing-primitive error a run hits, with hit counts.
Pair scripts with WINEDROID_TRACE=ui,click,bounds when a step does not do what you expect.