Repository navigation
Formatting fixes - #412
Merged
Merged
Formatting fixes#412
Conversation
Device containers in the devices and mac-devices roles now receive STORAGE_TOKEN, so screenshot uploads and app-install blob fetches pass the authenticated STF storage routes. The token is read from the environment to keep the value out of this tracked repository, and it must match STORAGE_TOKEN in the STF server variables.env. DEVICE_VERSION moves to 3.0-hardening-2026.08 so providers run the security-remediated device image that expects the token.
Updates the provider image reference and adds the new default values needed by the updated STF image.
Honour the 'wireless' field that has existed in mcloud-devices.txt since
it was introduced but was never read (fields 2-10, 12 and 13 are read;
11 is skipped).
Motivation
----------
Apple TV 4K has no diagnostic port. 'ios listen' therefore never emits a
connect event for it, so create_containers() is never called and the
device cannot join the farm. The device is reachable over Apple's
RemoteXPC tunnel, but that tunnel needs uid 0 and a tun interface on the
host, so it lives outside the container set.
Everything else in the stack already works for such a device, because the
pieces that matter are HTTP and env driven:
* the appium image builds appium:webDriverAgentUrl as
http://${WDA_HOST}:${WDA_PORT} (files/zbr-default-caps-gen.sh)
* the device image registers with STF as --serial ${DEVICE_UDID} from
env, and health-checks WDA over HTTP
* appium-xcuitest-driver explicitly supports running off macOS when
given webDriverAgentUrl (assertWdaHostSessionCapsSupported)
So the only thing that genuinely differs is where WebDriverAgent lives.
Change
------
For rows with wireless=true:
* skip the connector container -- it exists only to share usbmuxd
* omit the --link to it from the device and appium containers
* set WDA_HOST / WDA_PORT from two new device-list fields (14, 15)
* pass WIRELESS=true to the device and appium containers, consumed by
zebrunner/mcloud-device#210 and zebrunner/appium#439
Rows without wireless=true are untouched: same --link, same
WDA_HOST=connector, same connector container. Verified by simulation that
the docker arguments generated for a usb row are byte-identical.
WDA_PORT is now passed explicitly on the usb path too; both images
already default it to 8100, so that is a no-op.
Fails fast with an explicit message if wireless=true but no wda_host is
given, rather than starting containers that cannot work.
Notes
-----
Empty-array expansion uses ${arr[@]+"${arr[@]}"} because macOS ships
bash 3.2, where "${arr[@]}" on an empty array is an unbound-variable
error under 'set -u'.
No change is needed in listen()/on-usb-update: 'start <udid>' already
exists as a CLI verb, so an external owner of the tunnel can start and
stop these devices directly.
Verified end to end on an Apple TV 4K (AppleTV14,1) on tvOS 18.4 and
26.6: a Linux container with no Xcode, no macOS and no tunnel access
created an Appium session and drove the device (page source, screenshot,
button press, active app).
Support wireless (portless) iOS/tvOS devices by honouring the existing 'wireless' field
values for "wireless" were changed to lowercase
Wireless Apple devices setup fix
Change the way sudo permissions provided for ansible
Stf platform enhancement
- tests/zebrunner_test.sh runs zebrunner.sh on a copy of the repo with a temporary HOME and stubbed sudo/launchctl/docker/ansible-playbook/uname: help, setup, version, ansible arguments, status, backup/restore and the Linux and macOS shutdown (sudo only records its calls) - tests/utility_test.sh: confirm, random_string, delimiter, replace and the message helpers Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- lint job plus a test job on ubuntu-latest and macos-latest running tests/run_all.sh (listener and usbmuxd_watch tests need macOS) - tests/Dockerfile: Linux test environment to run the Linux tests on macOS - README: how to run the tests Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
actions/checkout@v4 and actions/setup-python@v5 target the deprecated Node.js 20; v7 of both run on Node.js 24 and do not change the inputs used. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Pushes to and pull requests into develop, main and master (the current default branch) trigger the tests; other branches are tested through their pull requests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- workflow_dispatch to run the tests on any branch manually - a new push to a pull request cancels its outdated run; runs of develop/main/master are never cancelled - the GITHUB_TOKEN only reads the repository contents - job timeouts stop hung runs Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Only ids and names were checked, so a missing obligatory field, a wrong os or port range, or a host port shared by devices passed the deploy and broke the containers start later. The prechecks validate obligatory fields and port ranges, that the published host ports (appium_port and the min_port-max_port range) are not shared, and on macOS that a wireless device has wda_host. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
logs/listener.log grew without limit. newsyslog cannot truncate in place, and a file kept open by launchd (StandardOutPath) would keep being written after the rotation, so the listener appends its messages and every zebrunner-farm action output to the log per write now. The deploy installs a newsyslog rule (10 MB, 7 bzip2 archives, no signal) into /etc/newsyslog.d, shutdown removes it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
README suggested --extra-vars "ansible_sudo_pass=PSWD", which leaves the password in the shell history and the processes list. The examples rely on the interactive --ask-become-pass now, and the non-interactive run keeps the password in an ansible-vault encrypted become.yml (ignored by git). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`tr < /dev/urandom | head` never ended when SIGPIPE is ignored, as for the processes of CI runners, so `zebrunner.sh backup` hung on the macOS runner; and LC_CTYPE=C did not override a UTF-8 LC_ALL, so tr failed with "Illegal byte sequence" and returned an empty string. random_string reads finite chunks of /dev/urandom with LC_ALL=C now. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The roles use ansible_facts['system'] only, while the full facts gathering runs system_profiler on macOS, which took about 70 seconds per playbook on the GitHub macOS runner and made the ansible tests run for 23 minutes. The test plays use the same setting. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CI, the tests Docker image and local runs install the same versions of the linters (shellcheck too: the runner had 0.9.0, newer versions are stricter). tests/lint.sh reports missing tools with the install command. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
.yamllint extends the default rules with 160 chars lines, true/false only booleans and the *.yml.original settings examples; local settings ignored by git are skipped. The findings were fixed: yes/no booleans, comments without a space after #, too long comments and the workflow document start. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ruff.toml: 120 chars lines, pycodestyle, pyflakes, imports order, bugbear, pyupgrade and simplify rules. tests/fake_usbmuxd.py was formatted. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
DL3008 (pinned apt versions) is ignored: package versions of ubuntu:24.04 change with the security updates. ruff runs without its cache, so the linters work on a read-only checkout (e.g. mounted into the tests image). The hadolint-py wheel for macOS is broken, README suggests brew or docker. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
.pymarkdown disables the line length rule and allows the HTML of the centered logo and the requirements table. README formatting was fixed: consistent headings, blank lines around headings, lists and code blocks, code block languages and list indentation. A bare http://<PUBLIC_IP>/stf was rendered by GitHub as http:///stf, it is quoted as code now. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The roles pass the production profile now: modules by their full names, named plays, Jinja spacing, the role variables prefixed by the role name, docker pull by command, launchctl calls as argv lists (too long lines) and the two intended ignore_errors with a reason. ansible-lint formatted the YAML (double quotes, no blank line after ---), .yamllint got the settings it requires. Skipped: var-naming[pattern] and role-name, as the existing farms set the upper case variables in roles/mac-devices/vars/main.yml. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
.editorconfig keeps the style for shfmt and editors: 2 spaces indents and a space after redirections, the closest to the existing code. The scripts and the zebrunner-farm/listener templates were formatted; the Jinja expressions of the templates did not change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The macOS runner still spent about 70 seconds per playbook with the minimal facts (c985f33 did not help: the hardware facts were not the cause, a slow hostname lookup by socket.getfqdn() is suspected), so the ansible tests hit the 20 minutes job timeout. The facts are gathered by the first playbook and taken from a jsonfile cache by the others. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A diagnostic step before the tests times socket.gethostname(), socket.getfqdn() and the minimal Ansible facts, to find out what takes about 70 seconds per playbook on the macOS runner. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
tests/diagnose_facts.sh prints the host names and DNS settings and times the hostname lookups, the commands and Python calls of the minimal facts, every minimal facts collector alone, the Python interpreter discovery and playbooks with and without facts. The CI step may fail or time out (12 min) without failing the tests. Both are to be removed once the cause is known. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The CI diagnostics showed that on the GitHub macOS runner every reverse lookup and the lookup of the own host name time out after 35 seconds, so socket.getfqdn() of the Ansible platform facts takes 70 seconds; the other collectors, the interpreter discovery and ansible-playbook itself are fast. The ansible tests keep gathering the facts once and caching them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Makefile installs the linters of tests/requirements-lint.txt into .venv (again when the requirements change; on macOS hadolint comes from brew, the hadolint-py wheel is broken), then runs tests/lint.sh and tests/run_all.sh on this machine and in the Linux docker image. make lint, make test and make docker run a part of the checks; make lists the targets. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With JUNIT_DIR set, every check is written into ${JUNIT_DIR}/<test>.xml:
the section in the test case name, the expected and actual values of a
failure, skipped tests and checks, and an exit before finishing as a
failure; the text is XML escaped, control characters are dropped.
tests/lint.sh runs every linter as a check and reports all failures
instead of stopping at the first one.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The lint and tests jobs write JUnit XML and publish it, also when a check failed: mikepenz/action-junit-report (pinned by SHA) adds annotations and a detailed job summary without a check run, so the read-only token is enough also for pull requests from forks; upload-artifact keeps the XML files. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- macOS 13+ on Apple Silicon and Intel (universal usbmuxd_watch) - official docker and ansible install docs, ansible 2.10+ (ansible.builtin module names), go-ios purpose - the devices settings validation of the deploy - the macOS devices listener (USB devices only) and its log rotation - WebDriverAgent has to be installed in advance on macOS: the connector installs it on Linux only - zebrunner.sh commands; udev and syslog troubleshooting marked as Linux Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… calculations was edited
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.