ares-do: send key events and take screenshots - #15
Open
mariotaku wants to merge 7 commits into
Open
Conversation
Adds the tenth tool. It presses buttons and captures the screen, so a UI flow can be scripted and replayed instead of walked by hand — the work that motivated it (webosbrew/samples#1) meant running luna-send once per keypress and diffing PNGs by hand. Two undocumented private-bus services do the work: test/sendKeyCode on com.webos.service.networkinput, and executeOneShot on com.webos.service.capture, falling back to com.webos.service.tv.capture (a 49LK5900 only has the latter). Both need root, so a uid probe runs before anything is sent rather than after three keys have gone nowhere. sendKeyCode reaches UInputWriter::sendKeyPress, which writes to /dev/uinput. That settles two things: the codes are ordinary evdev and stop at KEY_MAX, and there is no held modifier, so capitals cannot be typed at all. `text` refuses them up front, naming every character, in preference to a --shift flag that would quietly do nothing. Which code each remote button sends is LG's choice and often not the mainline name. /usr/share/X11/xkb/keycodes/lg in the firmware is the authority, and the surprises are now aliases with notes: the remote's Back is 412, not KEY_BACK (158). Confirmed on hardware — 158 moved 970 pixels, 412 dismissed the launcher and moved 1.4M. The flow language is not invented: a line is tokenized and handed to the same clap Command the shell uses, so the two cannot drift. The whole file parses before anything runs, and every error is reported at once. Ergonomics follow adb and xdotool rather than inventing conventions: stdout is data and stderr is for people, so `screenshot > shot.png` works; --json emits NDJSON; exit codes are a documented table; and `run -` replays a flow from stdin over one SSH handshake instead of one per key. `keys` and --dry-run need no device at all. Also fixes LunaError in ares-connection-lib, which read luna-send's stdout and stderr and then threw both away, collapsing "no such service", "not allowed on this bus" and "the channel died" into one value. New non_exhaustive Command variant carries them. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Gde2tcn5SbUHKhesuTNB7a
Pre-existing code that a newer rustfmt lays out differently. No behaviour change; split out of the ares-do commit so that diff stays about ares-do. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Gde2tcn5SbUHKhesuTNB7a
…at was never made Both found by running --method VIDEO against a TV with no tuner signal. com.webos.service.tv.capture answers that with `"errorCode":"CAPTURE_ERROR_09"` — a string. LunaReply typed errorCode as i32, so deserializing the reply failed and a perfectly clear error came out as "the reply was not JSON", pointing at the transport instead of at the TV. errorCode is now either shape, rendered rather than typed. The message is now what the service actually said: "Could not capture in no signal state (CAPTURE_ERROR_09)". RemoteTemp also warned that it could not delete the staging file after a capture the service had refused — there was no file, because the capture never happened. It is now armed only once the service accepts, so the guard still covers every path that can leave a file, and stays quiet about the ones that cannot. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Gde2tcn5SbUHKhesuTNB7a
… everything
Three gaps, all of which the internals already had the machinery for.
`launch`/`close` now take -p, in the two shapes ares-launch accepts —
key=value and a whole {json}, repeatable and merged — so the same input
works with either tool. Unlike ares-launch, a param that is neither shape
is an error: a launch missing half its parameters is worse than one that
did not happen.
`luna` makes one call and puts the reply on stdout so it pipes into jq.
Private bus by default like the rest of ares-do, --public to switch. A
returnValue:false reply exits 7, which makes a `luna` line usable as an
assertion inside a flow, with --allow-false when probing for a service.
`exec` runs one command and exits with its status, the way ares-shell
does. That is the one place ares-do departs from its own exit-code table,
so --json also carries exitCode.
And the reason all three needed work rather than a wrapper: nothing sent
to the device was bounded. Luna::call reads to EOF, so a command that
never exits, or a luna method that subscribes instead of replying, hung
forever — the one failure an unattended run cannot recover from. Every
device command now goes through one exec path built on
Channel::read_timeout, draining stdout and stderr together so a full
stderr window cannot deadlock a wait on stdout. --timeout bounds each
command (30s default, 0 waits forever), verified: `exec 'sleep 300'`
--timeout 3s gives up after 3.3s with exit 9.
A side effect worth noting: ares-do no longer calls Luna::call at all, so
LunaError::LunaUnavailable is gone from its error enum. It builds the
luna-send line itself with the same snailquote escaping, and can now tell
"luna-send exited non-zero, and here is what it said" from "the reply was
not a reply".
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gde2tcn5SbUHKhesuTNB7a
… around it The previous commit bounded ares-do's device calls by building the luna-send line itself. That fixed one tool and left the same hang in every other one, which is backwards — the unbounded read was never ares-do's problem, it was Luna::call's. So the bound moves into the library. A new `exec` module offers a public `Exec::exec_timeout` that drains stdout and stderr together against a deadline, and `Luna` gains `call_with`/`subscribe_with` taking a `LunaOptions` that carries the bus and the timeout. `call(uri, payload, public)` and `subscribe` stay, now as defaulted delegates, so ares-launch and ares-install did not change a line and are bounded anyway — verified against a device. Two things this makes possible that the workaround did not. Transfer's own exec is still private and unbounded, but there is now something public to point it at. And LunaError can finally say which of the four things went wrong: NotAvailable is back to meaning only "it failed and said nothing", with Command carrying luna-send's output, Reply carrying what came back when it was not a reply, and Timeout carrying how long we waited. ares-do drops its own exec.rs and its hand-built command line, and goes back to being a caller. Its three luna error variants collapse into one that wraps LunaError, so the mapping to an exit code is in one place — a timeout is 9 rather than 6, because a caller retries those differently. Default is 30s, `--timeout 0` waits forever. Verified: `exec 'sleep 300'` with --timeout 3s still gives up after 3.3s. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Gde2tcn5SbUHKhesuTNB7a
The aliases for buttons whose evdev code is not the obvious one were hand-picked - GOBACK, GUIDE, SOURCE, VOICE - which made them my names rather than LG's, and covered whatever happened to come up. They are now generated from /usr/share/X11/xkb/keycodes/lg, which is LG's own answer to "which code does this button send", under a REMOTE_ prefix. That is not what LG calls them internally, where the word is RCU (com.webos.service.mrcu, "LGE M-RCU", "Smart Remote RCU Input", 83 files), but nobody outside the platform knows what an RCU is and REMOTE_BACK needs no glossary. Each button appears under LG's name for it and, where that differs from what LG says it means, under both: the Input button is REMOTE_TV_VIDEO and REMOTE_INPUT_SOURCE, and 139 is REMOTE_MENU and REMOTE_SETTINGS. The meanings come from the Qt::Key_webOS_* names in the same file. 44 names for 27 buttons. The generator drops anything above KEY_MAX, because sendKeyCode reaches UInputWriter::sendKeyPress and that writes to /dev/uinput - so 149 of LG's 176 entries cannot be delivered however they are spelled, the remote's own Home button (773) among them. The prefix also settles the collision that started this. 158 is a real key - KEY_BACK, XF86Back in xkb - which is exactly why it looks like the Back button and is not; it is swallowed before any window sees it. Now BACK is 158 and REMOTE_BACK is 412, and neither has to pretend to be the other. Re-measured on a 49LK5900: BACK moved 31k pixels, REMOTE_BACK moved 489k and dismissed the launcher. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Gde2tcn5SbUHKhesuTNB7a
Bare BACK resolved to 158, the kernel's KEY_BACK and XF86Back in xkb. That is a real key, and the platform swallows it before any window sees it, so the tool reported success and nothing happened - which cost another agent an afternoon. The remote's Back button is 412. BACK now refuses and says which spellings mean what. Both readings stay reachable, just not by the ambiguous one: REMOTE_BACK for the button, KEY_BACK or XF86BACK for 158. The XF86 prefix is accepted for any key, since that is the name xkb uses and it says "the kernel's key, and I mean it" as clearly as KEY_ does. The rule is checked rather than remembered: a test walks every REMOTE_X whose bare X is a different code and fails unless it is declared ambiguous. Today BACK is the only one - EXIT, MENU, TV, SEARCH, INFO and the transport keys all agree between the two tables. Note this does not catch every trap, only the ones where both tables name the same button. HOME is still 102 while the shell's home is HOMEPAGE (172), and whether 102 reaches anything is untested. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Gde2tcn5SbUHKhesuTNB7a
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
ares-do, a tenth tool: it presses buttons and captures the screen, so a UI flow can be scripted and replayed instead of walked by hand. The work that motivated it (webosbrew/samples#1) meant runningluna-sendonce per keypress and diffing PNGs by hand.Everything is host-side over SSH. Nothing is installed on the device and nothing is written to its filesystem beyond one staging file per screenshot, which is deleted again.
Two undocumented services
test/sendKeyCodeoncom.webos.service.networkinput, andexecuteOneShotoncom.webos.service.capturefalling back tocom.webos.service.tv.capture— a 49LK5900 only has the latter, so the fallback is not theoretical. Both are on the private bus, so a uid probe runs before anything is sent rather than after three keys have gone nowhere.sendKeyCodereachesUInputWriter::sendKeyPress, which writes to/dev/uinput. That settles two things: the codes are ordinary evdev and stop atKEY_MAX, and there is no held modifier, so capitals cannot be typed at all.textrefuses them up front naming every character, rather than a--shiftflag that would quietly do nothing.Which code each button sends
LG's choice, and often not the mainline name.
/usr/share/X11/xkb/keycodes/lgin the firmware is the authority, and the surprises are aliases with notes inares-do keys. The remote's Back is 412, notKEY_BACK(158) — confirmed on hardware, where 158 moved 970 pixels and 412 dismissed the launcher and moved 1.4M. Likewise the remote's Home isHOMEPAGE(172), notKEY_HOME.The flow language is the CLI
A line is tokenized and handed to the same clap
Commandthe shell uses, so the two cannot drift and the flags are identical in both. The whole file parses before anything runs, and every error is reported at once with line numbers.Ergonomics
Follows
adbandxdotoolrather than inventing conventions: stdout is data and stderr is for people, soscreenshot > shot.pngworks;--jsonemits NDJSON ending with anerrorobject whosecodeis the exit code; exit codes are a documented table; andrun -replays a flow from stdin over one SSH handshake instead of one per key.keysand--dry-runneed no device at all.Also fixes the shared Luna client
Luna::callreadluna-send's stdout and stderr and then threw both away, collapsing "no such service", "not allowed on this bus" and "the channel died" into one value — and it read to EOF with no bound, so a method that subscribes instead of replying hung forever.ares-connection-libgains a publicExec::exec_timeoutthat drains both streams against a deadline, andLunagainscall_with/subscribe_withtaking aLunaOptionscarrying the bus and the timeout.call(uri, payload, public)stays as a defaulted delegate, soares-launchandares-installdid not change a line and are bounded anyway — verified against a device.Worth review: the 30s default is a behaviour change for those two, which previously waited forever.
subscribe_withis deliberately left unbounded, since a subscription is meant to stay open.LunaErrorgained variants and is now#[non_exhaustive], which protects future ones but not this change — a downstream exhaustive match (dev-manager-desktop) would need a wildcard.Not included
Pointer/
tapis parked onbackup/ares-do-tap. It works and lands on the pixel, but the TV runs dropbear, which has nodirect-streamlocal, so a unix socket cannot be forwarded and it needs a helper on the far end. That is a bigger departure than this tool should make.Verified
On a 49LK5900 (webOS 4.4.3): key injection, both capture services, PNG to stdout, a multi-step flow from stdin, remote temp cleanup, and every documented exit code. 67 unit tests, clippy clean at workspace
pedantic,fmt --checkclean, and the committed keycode table matches its generator.The second commit is unrelated
rustfmtchurn from a newer nightly, split out so theares-dodiff stays aboutares-do.🤖 Generated with Claude Code
https://claude.ai/code/session_01Gde2tcn5SbUHKhesuTNB7a