Skip to content

Latest commit

 

History

History
154 lines (112 loc) · 10.1 KB

README.md

File metadata and controls

154 lines (112 loc) · 10.1 KB

Zebrunner Device Farm - Selenium Hub

Enhanced Selenium/Appium Grid for automating Android and iOS devices including TVs (Android, Tizen, Apple) and emulators/simulators. It is a Selenium 3 hub with a mobile proxy (MobileRemoteProxy) and capability matcher (MobileCapabilityMatcher): one Appium node is one device, devices are reserved in STF for the time of a session.

Feel free to support the development with a donation for the next improvements.

Zebrunner

Follow the installation and configuration guide in MCloud to reuse this image effectively.

Build

docker build . -t zebrunner/mcloud-grid:latest

The image build compiles and packages the hub; tests and linters run separately (see Development).

Run

docker run -d -p 4444:4444 -e GRID_NEW_SESSION_WAIT_TIMEOUT=240000 \
  -e GRID_TIMEOUT=60 -e GRID_BROWSER_TIMEOUT=60 \
  -e STF_URL=http://stf.example.com -e STF_TOKEN=<token> \
  --name mcloud-grid zebrunner/mcloud-grid:latest

The hub runs as an unprivileged user and stops gracefully on docker stop.

Configuration

Grid

Env var Default Meaning
GRID_NEW_SESSION_WAIT_TIMEOUT 600000 How long a new session request waits in the queue, ms
GRID_TIMEOUT 150 Client inactivity timeout of a session, s
GRID_BROWSER_TIMEOUT 0 Timeout of a command on the node, s (0 = none)
GRID_CLEAN_UP_CYCLE 5000 How often the hub checks timed out sessions, ms
GRID_THROW_ON_CAPABILITY_NOT_PRESENT true Reject requests no registered device can serve; see Queueing
GRID_JETTY_MAX_THREADS -1 Jetty threads of the hub (-1 = default)
GRID_DEBUG false Debug logging of the hub
GRID_PROXY, GRID_CAPABILITY_MATCHER mobile proxy and matcher Classes of the node proxy and capability matcher
JAVA_HEAP_OPTS -Xms1G -Xmx4G JVM heap of the hub
JAVA_OPTS Other JVM options
SE_OPTS Extra hub options, e.g. -debug; -servlets replaces the servlets of the hub config
CHECK_NODE_REACHABILITY true Reject the registration of a node the hub cannot connect to
NODE_REACHABILITY_TIMEOUT 2 Connection timeout of that check, s
NODE_ALLOWED_NETWORKS Comma separated CIDR ranges or IPs nodes may register from, e.g. 10.0.0.0/8, 192.168.1.15; the address a node registers with is checked; not set = any node (an invalid value rejects every node)
MAX_NEW_COMMAND_TIMEOUT Upper limit of appium:newCommandTimeout, s; bigger and disabled (0) values are limited (no limit when not set)
MCLOUD_LOG_LEVEL INFO Log level of the grid code, FINE for details; Selenium logs stay as they are

STF and device health

STF integration is enabled when both STF_URL and STF_TOKEN are set.

Env var Default Meaning
STF_URL STF address
STF_TOKEN Access token of the STF user that reserves devices for automation; when the user is an STF admin, a device that does not answer the reservation is also marked unhealthy in STF
STF_TIMEOUT 3600 Reservation timeout of a device in STF, s
CHECK_APPIUM_STATUS false Check /status-adb (Android) or /status-wda (iOS) of the node before a session
UNHEALTHY_MOBILE_TIMEOUT 60 A device failing the Appium status check is skipped for, s
INACTIVITY_RELEASE_TIMEOUT 60 A device whose session timed out before it started is skipped for, s
STF_DEVICE_INVALID_RESPONSE_IGNORE_TIMEOUT 600 A device with an invalid STF status or failed reservation is skipped for, s
STF_DEVICE_UNAUTHORIZED_IGNORE_TIMEOUT 600 A device unauthorized in STF is skipped for, s
STF_DEVICE_UNHEALTHY_IGNORE_TIMEOUT 60 An unhealthy or not ready device is skipped for, s
STF_DEVICE_MANUALLY_RESERVED_TIMEOUT 180 A device reserved in STF by another user is skipped for, s

Capabilities

Capability Meaning
platformName Android, iOS, ...
appium:platformVersion Exact (13), range (11-13), minimum (12+) or list (12,14); 7 matches 7.0
appium:deviceName, appium:udid One value or a comma separated list
zebrunner:deviceType phone, tablet, tv, tvOS, ...; tvOS devices get platformName=tvOS
zebrunner:STF_TOKEN Personal STF token: the device is reserved as that user and is not returned after the session
zebrunner:STF_TIMEOUT Reservation timeout in STF for this session, s
zebrunner:enableAdb Android: wait for the remote ADB connection of STF before the session

The node capabilities of the reserved device are passed to the session as zebrunner:slotCapabilities.

Queueing

With GRID_THROW_ON_CAPABILITY_NOT_PRESENT=true a request for a device that is not registered fails at once (cannot find : Capabilities {...}, or Empty pool of VM for setup when no device is registered at all). With false it waits in the queue for up to GRID_NEW_SESSION_WAIT_TIMEOUT and gets a device that registers meanwhile (a restarted device, a new emulator), otherwise it fails with Request timed out waiting for a node to become available.

Endpoints

Endpoint Content
/grid/console Selenium grid console, with the UDID of every device
/grid/admin/DevicesServlet Devices as JSON: platform, type, node, status free/busy/ignored/down, why and until when a device is ignored, its session (Appium session id, start, inactivity, last command)
/grid/admin/AllSessionsServlet Active sessions as JSON with the requested capabilities and the device
/grid/admin/MetricsServlet Prometheus metrics: mcloud_grid_devices{platform,status}, mcloud_grid_sessions, mcloud_grid_new_session_requests (the queue)
POST /grid/admin/TerminateSessionServlet?udid=<udid> or ?sessionId=<id> Terminates the session of a device and releases it in STF, see below
/wd/hub/status Hub status, also the healthcheck of the image

Terminating a session

curl -X POST -H "Authorization: Bearer <STF access token>" "http://grid:4444/grid/admin/TerminateSessionServlet?udid=emulator-5554"

The hub releases the device in STF with the given key first and terminates the session only when STF allows it. STF releases a device for the user it is reserved by or for an STF admin. Devices are reserved by the grid user (STF_TOKEN), so the key of a regular STF user may not work (403, the session keeps running) unless the session was started with that user's zebrunner:STF_TOKEN; an STF admin key works. Without the STF integration the endpoint answers 501.

The servlets are registered in the hub config by generate_config. com.zebrunner.mcloud.grid.servlets.ProxyInfo (registration requests of the nodes) is not registered by default.

Logs

docker logs mcloud-grid outputs JSON Lines: one JSON object per event, including Selenium, grid and startup/shutdown logs. Common fields are timestamp (UTC), level, logger, component, category and message. Device events also have udid and sessionId (the hub's internal session ID):

{"timestamp":"2026-10-09T12:00:00Z","level":"INFO","logger":"com.zebrunner.mcloud.grid.MobileRemoteProxy","component":"mcloud-grid","category":"device","udid":"emulator-5554","sessionId":"0f85...","message":"Device 'Pixel 7' (ANDROID 14) is selected for the session, starting the Appium session."}

Use docker logs mcloud-grid 2>&1 | jq -c 'select(.udid == "emulator-5554")' to see one device, select(.category == "grid") for hub-wide events or select(.component == "selenium") for Selenium itself. Selenium's own records have no device ID unless Selenium supplies it. Exceptions are escaped into one JSON field. MCLOUD_LOG_LEVEL=FINE adds grid details (skipped devices, STF device data, remoteConnect steps); SE_OPTS=-debug turns on Selenium's debug logs without changing the grid log level. The generated config remains available at /opt/selenium/config.json, but is not dumped to logs.

For interactive viewing of history and live logs with lnav:

docker logs --follow --timestamps mcloud-grid 2>&1 | lnav

Development

Requirements: docker, python 3 (for the linters), hadolint on macOS (brew install hadolint). tests/mvn.sh runs the pinned Maven + JDK toolchain in maven:3.9.16-eclipse-temurin-11, so host-wide mvn and Java installs are optional.

make check   # everything CI runs
make lint    # yaml, GitHub workflows, Dockerfile, markdown, shell scripts, the text files, checkstyle, spotbugs
make test    # unit and STF integration tests, coverage in target/site/jacoco
make docker  # docker image checks: user, config generation, options, endpoints, graceful shutdown
make load    # short load test of the image with fake Appium nodes

tests/lint.sh and tests/docker_test.sh write JUnit XML into $JUNIT_DIR when it is set, maven writes it into target/surefire-reports. Tests with STF enabled (TestNG group stf) run in their own JVM against a WireMock STF stub. Load tests against real grids are described in LOAD_TESTING.md. CI (.github/workflows/ci.yml) runs the same checks on pushes and pull requests: jobs lint, tests and docker (the image checks and a load test with fake Appium nodes); dependabot proposes updates weekly.

The conventions of the style, the formatting and the tooling of this repository are in CONVENTIONS.md, the same in all the MCloud repositories.

Support

License

Apache License 2.0