feat(simu): unify UI automation harness

[bultodepapas]

Fixes: N/A

Summary

This PR adds a unified, opt-in UI automation system for the native EdgeTX
simulator.

Its purpose is to turn a manual simulator reproduction into a repeatable flow:
start from a known fixture, discover what the selected target supports, drive
the real EdgeTX runtime, wait for firmware-rendered LCD frames, capture
framebuffer artifacts, and preserve enough evidence to reproduce or diagnose a
failure.

Normal simulator use remains unchanged unless automation is explicitly enabled.
The automation surface is excluded from physical firmware, WASI, and Emscripten
builds.

Why this is useful

UI validation currently depends heavily on opening the simulator, navigating
screens by hand, and capturing the host window. Those results can vary with
window size, scaling, focus, timing, and local radio state.

This harness works against the simulator runtime and framebuffer instead of
automating the host window. It provides:

• repeatable input and simulator-state transitions;
• synchronization with actual firmware LCD updates;
• captures independent of host-window scaling;
• isolated fixture copies for declarative runs;
• structured manifests, streamed protocol evidence, hashes, and diagnostics;
• bounded cleanup of inputs, processes, threads, and temporary files; and
• the same host-side lifecycle on Windows and Linux.

This gives contributors and maintainers a reproducible foundation for UI
development, bug reproduction, and future regression testing.

What is included

Native simulator automation

• An opt-in mode using --automation-stdio and an explicit artifact root.
• A bounded protocol with monotonic request IDs and correlated JSON responses.
• Strict UTF-8, request-size, argument, queue, path, and response validation.
• Runtime discovery of commands, capabilities, inputs, bounds, and LCD geometry.
• Key, rotary, touch, switch, analog, and telemetry control through existing
  simulator and firmware ownership paths.
• Firmware-context telemetry operations and Lua reload.
• LCD frame barriers, warm task restart, and host-composed cold process restart.
• Fresh RGB565 framebuffer capture, including firmware-context invalidation of
  static LVGL screens and exclusive no-overwrite PPM publication.
• Terminal stop, EOF, broken-pipe, cancellation, and backpressure behavior.

Host harness

• A standard-library-only Python session and edgetx-ui CLI.
• Strict declarative flows with pre-launch schema validation and runtime target
  validation through describe.
• Immutable TX16S fixtures copied into a unique writable directory for each
  declarative run.
• One-command smoke and hardening workflows.
• Canonical PPM validation and deterministic host-side PNG conversion.
• Manifests, hashes, bounded stderr, and streamed JSONL protocol evidence.
• Child-process reaping and reader/writer-thread cleanup across normal exit,
  timeout, crash, terminate, and kill paths.

How it works

CLI / declarative flow
          |
          v
Python session and lifecycle
  - fixture isolation
  - request deadlines
  - artifact verification
  - process and thread cleanup
          |
          | bounded stdin requests / JSON-line responses
          v
native simulator transport
          |
          v
SDL automation executor
  - keys / rotary / touch / switches
  - atomic analog overrides
          |
          +--> bounded firmware mailbox
          |      - telemetry
          |      - Lua reload
          |
          +--> LCD sequence and framebuffer snapshot
                         |
                         v
                 capture worker -> PPM
                         |
                         v
                 host PNG + metadata

Timed actions such as presses, taps, and drags are composed by the host, so the
native command path does not sleep. Firmware-owned work crosses an explicit
mailbox. The LCD callback only publishes sequence and snapshot state and
performs no filesystem work.

While the response channel remains available, every admitted request owns
exactly one terminal result. Stop acts as an admission barrier and queued
requests that have not executed are completed with session_stopping. Request
sizes, queues, asynchronous work, timeouts, scenarios, diagnostics, and retained
in-memory evidence are bounded.

Developer workflow

The reference TX16S flow can be configured, built, run, and verified with:

python tools/ui-harness/edgetx-ui smoke \
  --build-dir build/ui-harness/tx16s

Every declarative execution receives a writable copy of the checked-in fixture
and creates a unique run directory containing settings and SD-card copies,
captures, metadata, hashes, a manifest, streamed protocol evidence, and bounded
simulator diagnostics.

Once execution has started, failures preserve the evidence observed before
failure and return a nonzero exit code. Pre-launch preparation failures remove
their incomplete run directory. Existing runs and artifacts are never silently
reused or overwritten.

TX16S is the first supported end-to-end profile. The native discovery model is
generic, but each additional target requires an explicit profile, fixture,
capability set, and representative flow before the declarative runner accepts
it.

Provenance and consolidation

This work consolidates the directions explored in EdgeTX PRs #7337 and #7646.
It is a substantial redesign rather than a mechanical merge.

PR #7337, developed by Mateusz Urban (onliner10), established the reusable
host-harness direction: a CLI, declarative flows, fixture-based startup,
target-aware interaction, and host-side framebuffer tooling. The TX16S fixture
in this PR is derived from and migrated from that work.

PR #7646 explored Windows-native transport requirements, explicit input
transitions, switch/analog/telemetry injection, Lua reload, restart behavior,
capture after a real LCD refresh, and forced invalidation of static LVGL screens.

The consolidated implementation keeps one generic activation path, one bounded
protocol, and one cross-platform host lifecycle. It deliberately does not carry
forward the append-only command-file transport, a simulator-only Lua control
API, native PNG encoding, direct cross-thread mutation, or a second automation
protocol.

Thank you to Mateusz for the original harness direction and fixture foundation.

Validation

Validation for the current head includes:

• 94/94 host tests passing on Windows.
• 94/94 host tests passing in the pinned Linux environment.
• 53/53 focused native automation tests passing under ASan and UBSan.
• Dedicated Windows and Ubuntu host-test jobs passing.
• Native sanitizer and WASI-isolation jobs passing.
• The complete repository check set passing: 51/51, with no
  pending or failed checks.
• Strict documentation build, Python byte-compilation, and whitespace
  validation passing.
• Additional real-simulator smoke and hardening performed during development
  covered protocol correlation, lifecycle, deterministic static capture,
  deliberate visual mutation, Lua generations, restart epochs, fixture
  integrity, temporary files, and process/thread reaping.

The evidence above does not include exact Linux-versus-Windows RGB equality or
a separately built and inspected Emscripten artifact.

Scope and limits

This PR does not:

• redesign an EdgeTX screen or widget;
• change model storage or EEPROM formats;
• add a public or simulator-only Lua automation API;
• add a network listener, MCP adapter, or concurrent-client protocol;
• provide a golden-image approval or image-diff policy;
• support every radio profile immediately;
• capture non-RGB565 displays;
• make warm task restart equivalent to a new process; or
• claim that the output directory is an OS sandbox against another same-user
  process modifying it concurrently.

Future value

This establishes one automation boundary that later work can extend without
creating another simulator control surface. Possible follow-ups include:

• additional target profiles and deterministic fixtures;
• reusable UI regression scenarios;
• golden-image review, image comparison, and masks;
• CTest or JUnit reporting;
• adapters built on the existing Python session;
• richer telemetry scenarios; and
• eventual alignment with hardware-in-the-loop workflows.

The implementation is ready for maintainer review. Passing checks do not
replace maintainer acceptance of the protocol and integration boundaries.

Suggested review order

1. Protocol bounds, response ownership, stop, and EOF behavior.
2. SDL-thread and firmware-thread ownership.
3. Frame freshness, capture publication, and artifact containment.
4. Host deadlines, cleanup, evidence bounds, and flow validation.
5. Native-only build isolation and the dedicated CI workflow.

[bultodepapas]

Correction to the delivery rule above: this effort now stays deliberately simple.

Implementation proceeds in this same draft PR and branch; there is no second implementation branch and no branch per phase. G0 remains an explicit review requirement before the PR leaves draft or is merged, but it does not block small implementation commits.

Phase 1 is now concrete at 5139415071686414f19fa6227e9bdd0835da8080: one pure bounded protocol/state module plus focused tests, with no SDL transport or runtime activation yet. Local validation passed 122/122 tests with ASan, and the complete GitHub test/build/documentation/packaging matrix is green.

@onliner10 and @pfeerick are invited to review this concrete slice and shape the next one. Credit and provenance from #7337 and #7646 remain explicit, and neither source PR needs to close until parity is demonstrated.

[bultodepapas]

Phase 2 is now published at c1be4bd5fb on this same branch and draft PR.

This slice adds the opt-in cross-platform stdio transport and SDL-loop activation while keeping the scope narrow: ping and stop work now; input, telemetry, Lua, restart, and framebuffer capture stay for later phases.

Validation completed:

• 129/129 gtests-radio tests passed with ASan in the official Linux environment;
• native Linux simulator built and passed real-pipe probes, including clean JSON-only stdout, EOF, malformed input, queue limits, paths with spaces, and dormant normal mode;
• parser/protocol/transport compiled on Windows with both Clang and MinGW under -Wall -Wextra -Werror;
• POSIX and Windows protocol vectors produced the same byte hash;
• strict documentation build and git diff --check passed.

The implementation intentionally does not add a permanent Python client yet; a disposable probe was enough to validate this transport slice. @onliner10 and @pfeerick remain invited to review the contract and retained behavior before the draft is marked ready.

[bultodepapas]

Phase 3 slice 3A is now published at 49f2e3db9b on this same branch and draft PR.

This keeps the scope intentionally narrow: a dependency-free, binary-pipe Python session and minimal edgetx-ui probe for the native commands that exist today (ping and stop). It adds strict ID/JSON validation, separate bounded stderr draining, command deadlines, crash detection, and guaranteed process/thread cleanup without select or startup sleeps.

Validation:

• 22/22 focused tests pass with warnings treated as errors on Windows Python 3.11 and official-image Linux Python 3.10;
• the suite includes malformed/partial stdout, wrong/repeated IDs, stderr flooding, command errors, crash, timeout, terminate fallback, CLI failure exit, and 100 subprocess cycles on each host;
• the real Linux TX16S simulator built and completed 100 start/ping/stop cycles with exit code zero and no live reader thread;
• the exact CLI also passed end-to-end with SDL offscreen and paths containing spaces;
• strict docs and git diff --check pass.

Credit: the reusable host-session and CLI direction comes from #7337 by @onliner10. I reviewed that implementation, but this protocol-v1/Windows-safe lifecycle is a substantial rewrite with no coherent source block copied; the provenance ledger records that distinction. A co-author trailer remains intentionally absent unless approved.

Phase 3 remains open for a real Windows-simulator run, explicit kill-fallback injection, and status/frame/capability readiness. @onliner10 and @pfeerick are invited to review this concrete host boundary before those pieces are added.

CI update: the complete GitHub test, firmware-build, documentation, and packaging matrix passed for this head (45/45).

[bultodepapas]

Phase 3 is now complete at f6c104e on this same branch and draft PR.

This final slice adds only the readiness boundary needed before inputs: bounded status/describe responses, target/LCD/command/capability validation, and startup completion after the simulator owns a real first LCD frame. It intentionally does not advertise or implement input, capture, telemetry, Lua, or restart yet.

Validation completed:

• 29/29 host tests pass with warnings as errors on Windows Python 3.11 and official-image Linux Python 3.10, including 100 lifecycle cycles on each host and forced kill-then-wait cleanup;
• 131/131 native TX16S tests pass;
• the real Linux simulator completes 100 readiness/stop cycles without an orphan;
• the exact current Windows simulator build completes ping/describe/first-frame-ready/stop with paths containing spaces and exit code zero;
• git diff --check passes.

Thank you again, @onliner10 and @pfeerick. This slice combines the reusable host-session direction from #7337 with the guarded epoch/first-frame model from #7646. Please review the resulting boundary and help shape Phase 4 inputs; G0 remains open before this draft can be marked ready.

CI update: the complete current-head GitHub Actions and CodeRabbit matrix passed 45/45 for f6c104e.

[bultodepapas]

Hi @onliner10 — Phase 4 of the consolidated simulator UI automation work is now available in this PR.

Thank you again for the work and direction you established in #7337, especially the reusable host-side harness and action model. Those ideas have materially shaped this consolidation, and I have tried to keep that provenance clear in both the plan and the PR history.

I would genuinely value your review of the protocol, host API, and retained behavior before this draft is marked ready. Please feel free to challenge the design or suggest a simpler direction where appropriate.

If you are comfortable with commit-level attribution for the parts substantially derived from your work, I would also be happy to add a Co-authored-by trailer. I will not rewrite commit history or use an email address without your approval; if you would like that attribution, please confirm the GitHub-associated email you prefer (a GitHub noreply address is completely fine).

No pressure either way — your review and feedback would already be greatly appreciated.

[bultodepapas]

Phase 5 is now published at bfd59db236 on this same branch and draft PR.

This slice adds one focused capability: render-complete framebuffer capture. It combines the portable PPM and host-conversion direction from #7337 with the static-screen invalidation requirement explored in #7646. Native capture waits for a strictly newer RGB565 framebuffer, publishes without replacing an existing artifact, and keeps file I/O out of the LCD notification. The Python client validates the PPM, converts and independently decodes PNG, and writes deterministic SHA-256 metadata.

Validation is complete:

• 34/34 focused native tests pass under AddressSanitizer;
• 49/49 host tests pass;
• the Windows TX16S simulator compiles, links, and captures through real binary pipes;
• 20 static captures are byte-identical with increasing display sequences;
• an ENTER interaction produces a different framebuffer hash;
• Unicode-and-space paths work, metadata hashes match, and no temporary artifacts remain;
• the complete current-head GitHub matrix passes 45/45.

Thank you, @onliner10, for the original harness and capture direction in #7337. I would especially value your review of the freshness boundary, artifact contract, and host API. @pfeerick, your review of the simulator/LVGL integration would also be very welcome. G0 remains open, so the PR stays draft.

[bultodepapas]

Phase 6 is now published at ada9972144 on this same branch and draft PR. This slice completes the state/lifecycle layer without adding another control surface: discovered switches and analogs use the existing simulator drivers, firmware-owned telemetry and Lua work cross a bounded SPSC mailbox, warm restart completes on the first frame of a new epoch, and cold restart returns a new process backed by fresh fixture copies. Automation uses deterministic no-splash startup, while normal simulator behavior and physical/WASM builds remain outside the change. Final local validation: - 38/38 focused native SimuAutomation* tests pass under AddressSanitizer; - 57/57 host protocol/session tests pass; - the Windows TX16S simulator compiles and links; - a real run passed switch and analog control, telemetry create/update, generation-observed Lua reload, warm restart, and cold restart with a new PID and a fully reaped old child; - formatting, compileall, and git diff --check pass. Thank you again, @onliner10, for the reusable harness/session direction established in #7337. Phase 6 deliberately keeps that direction while integrating the state-injection and restart goals from #7646. I would genuinely value your review of the host API, mailbox boundary, and restart semantics, and any simplification or correction you would like to propose. @pfeerick, review of the firmware/UI hook placement would also be very welcome. Current-head GitHub Actions, firmware builds, documentation, packaging, and CodeRabbit passed 45/45. G0 remains open, so the PR stays draft.

[pfeerick]

Just a friendly FYI ... tell your AI assistant to be less eager to tag and mention people... it's the quickest way to get yourself ignored if you are pinging people every couple of hours. Commits generally also speak for themselves if well written and described, so iteratively updating the initial PR body post would have been more than sufficient here, and better than individual progress comments with each commit, as the posts here also flow into things like discord channels, meaning there are will be several effectively "noise" posts there about this PR.

[Copilot]

Pull request overview

This PR consolidates EdgeTX simulator UI automation into a single, cross-platform harness: a bounded stdio protocol implemented in the native simulator plus a dependency-free Python host runner for flows, artifact capture/verification, and hardening.

Changes:

• Adds a versioned simulator automation protocol over stdin/stdout (JSON responses/events) with capture, input injection, Lua reload, restart, and frame barriers.
• Introduces a dependency-free Python UI harness (CLI + strict flow runner + protocol/PPM/PNG tooling) with extensive unit and hardening tests.
• Wires automation into the SDL simulator build/args and adds CI to exercise host tests, native ASan/UBSan radio tests, and WASI isolation checks.

Reviewed changes

Copilot reviewed 44 out of 45 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
tools/ui-harness/tests/test_protocol.py	Adds protocol encoding/decoding and schema validation tests (wire limits, duplicates, strict shapes).
tools/ui-harness/tests/test_ppm.py	Tests strict PPM parsing and deterministic, dependency-free PNG writing/reading.
tools/ui-harness/tests/test_hardening.py	Adds lifecycle/stress/hardening runner tests using a fake simulator fixture.
tools/ui-harness/tests/test_flow.py	Adds strict flow schema validation and manifest/protocol evidence tests.
tools/ui-harness/tests/fake_simulator.py	Provides a portable subprocess fixture for protocol/session lifecycle testing.
tools/ui-harness/README.md	Documents harness usage (smoke, harden, probe) and output/evidence contracts.
tools/ui-harness/flows/tx16s-smoke.json	Adds the checked-in TX16S smoke flow scenario.
tools/ui-harness/fixtures/tx16s/settings/RADIO/radio.yml	Adds TX16S radio settings fixture template (CRLF-managed via attributes).
tools/ui-harness/fixtures/tx16s/settings/MODELS/model1.yml	Adds a deterministic model fixture for simulator startup.
tools/ui-harness/fixtures/tx16s/settings/MODELS/labels.yml	Adds model label metadata fixture.
tools/ui-harness/fixtures/tx16s/sdcard/.gitkeep	Ensures an empty sdcard fixture directory is tracked.
tools/ui-harness/fixtures/README.md	Documents fixture provenance and immutability/copy semantics.
tools/ui-harness/fixtures/.gitattributes	Forces CRLF for fixture YAML to match simulator-produced checksums.
tools/ui-harness/edgetx-ui	Adds a Python entrypoint wrapper script for the harness CLI.
tools/ui-harness/edgetx_ui/protocol.py	Implements strict request encoding and message parsing/decoding for protocol v1.
tools/ui-harness/edgetx_ui/ppm.py	Implements strict PPM handling plus deterministic PNG encode/decode and hashing.
tools/ui-harness/edgetx_ui/cli.py	Adds CLI commands for probe/run-flow/smoke/harden and optional simulator build.
tools/ui-harness/edgetx_ui/init.py	Exposes the harness public API surface.
radio/src/tests/simu_automation_runtime.cpp	Adds gtests for firmware mailbox, telemetry edge cases, and tuple correctness.
radio/src/tests/simu_automation_capture.cpp	Adds gtests for capture path validation, writer behavior, cancellation, and stability.
radio/src/tests/simu_arg_parser.cpp	Adds gtests for automation argument parsing and stdout/stderr behavior.
radio/src/tests/CMakeLists.txt	Wires simulator automation sources into native gtests build with SIMU_AUTOMATION.
radio/src/targets/simu/sdl_simu.cpp	Integrates stdio automation, input/capture hooks, restarts, and analog override reads.
radio/src/targets/simu/CMakeLists.txt	Adds SIMU_AUTOMATION build gating (native-only) and sources for protocol/capture/stdio.
radio/src/targets/simu/automation_stdio.h	Declares the stdio automation transport/pump and request processing interface.
radio/src/targets/simu/automation_runtime.h	Declares firmware-periodic automation runtime mailbox and analog override APIs.
radio/src/targets/simu/automation_runtime.cpp	Implements firmware mailbox, telemetry/Lua operations, epochs/generations, and hooks.
radio/src/targets/simu/automation_protocol.h	Defines protocol constants, parsing/serialization structures, and session state model.
radio/src/targets/simu/automation_capture.h	Declares capture validation, snapshot/worker pipeline, and atomic cancellation contract.
radio/src/targets/simu/automation_capture.cpp	Implements capture path containment rules and canonical RGB565->PPM writer pipeline.
radio/src/targets/simu/arg_parser.h	Extends simulator args for --automation-stdio/--automation-output (SIMU_AUTOMATION).
radio/src/targets/simu/arg_parser.cpp	Implements automation option validation and stdout/stderr routing rules.
radio/src/main.cpp	Calls firmware-periodic automation hooks around UI execution paths.
radio/src/gui/colorlcd/LvglWrapper.cpp	Adds LVGL invalidation hook for deterministic capture of static screens.
mkdocs.yml	Adds simulator UI automation plan docs to the developer documentation nav.
.github/workflows/simulator_ui_automation.yml	Adds CI workflow for host tests, native ASan/UBSan automation tests, and WASI isolation.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.