Skip to content

parker-sites: use rx traffic as a second liveness check (carry) - #66

Merged
awlx merged 2 commits into
parker-release-2026.11from
awlx-parker-traffic-liveness
Nov 21, 2026
Merged

awlx merged 2 commits into
parker-release-2026.11from
awlx-parker-traffic-liveness

Conversation

@awlx

@awlx awlx commented Nov 21, 2026

Copy link
Copy Markdown
Member

Problem

On a lossy or bloated uplink (for example LTE during a big download), the 1 s ping to the pgw gets lost or comes late. The node counts 3 misses, moves to another pgw or site, and the download breaks. The pgw works.

Change

  • New parker/carry.lua. A tunnel "carries" when the WireGuard rx of the current pgw grew by 48 KB or more in the last 3 s, and by 16 KB or more in at least 2 of these 3 s. Only rx is used. WireGuard counts rx only for authenticated packets. TX proves nothing.
  • While a tunnel of a site that is up carries, a missed ping is not counted. The ping of a busy tunnel waits up to 3 s in the background. A late reply counts as an answer.
  • After 30 s without an answer, misses count again (max defer).
  • Carry never makes a pgw or site up. A tunnel without traffic works as before. carry_bytes = 0 disables carry.
  • status shows carry= and carried=. Values are in uci parker.sites: carry_bytes, carry_window, carry_ticks, carry_max, probe_wait_busy.
  • README: new section "Carry".

Tests

Unit tests (from ffbs-mesh-vpn-parker/):

  • lua tests/carry-test.lua (Lua 5.5) and luajit tests/carry-test.lua: 15 cases, all pass.
  • luacheck luasrc/usr/share/lua/parker/carry.lua tests/carry-test.lua: 0 warnings, 0 errors.
  • The case "drain and trickle after a cut do not carry" fails without commit 3a51649.

Lab (QEMU arm64 mesh lab, node u1, image v2026.11.1-parker5+cp0798007-lab, armsr-armv8):

  • The u1 WAN went through a userspace impairment proxy: rate limit, drop-tail queue (bufferbloat), delay, loss. All impairment was on the lab side only.
  • Load: parallel downloads of a 2 GB file on the client behind u1, or a UDP stream from outside to the client.
  • Cuts: iptables/ip6tables on u1 for the WireGuard endpoint (both ways, or node to pgw only).
  • C-1 ran on the old build (parker5+cp5d8b362). C-2 (first run), C-3 and C-4 ran on 0798007. The first C-5 runs showed a slow failover. Commit 3a51649 fixes this. The other tests ran with carry.lua of 3a51649 copied to u1 (md5 checked). The rest of the image is 0798007.
Test Result
C-1 old build, 3/1 Mbit, 3 s queue, 16 downloads, 10 min Active MUC: 29 pgw switches, 4 site down. 10 active-site changes, 101 s without a site, 126 MB
C-2 carry, same as C-1 Active MUC: 0 pgw switches, 0 site down. 0 active-site changes, 0 s without a site, 190 MB
C-3 cut active pgw both ways under load Next pgw at +5 s
C-4 cut all MUC pgws under load VIE active at +8 s. MUC active again 15 s after the uncut
C-5 cut node to pgw only, TCP download Next pgw at +6 s (+7 to +8 s before 3a51649)
C-6 cut node to pgw only, UDP 40 KB/s from outside Carry holds, next pgw at +32 s (max defer)
C-7 cut node to pgw only, UDP trickle 8 KB/s No carry, next pgw at +4 s
C-8 idle, cut both ways No carry, next pgw at +4 s
C-9 switch-back after site cut Not faster than 3 hits + 10 s dwell (from C-4)
C-10 5 s full WAN loss every 25 s, under load, 10 min Active MUC: 4 pgw switches, 0 site down, 0 active-site changes. Idle VIE: flap hold-down as before

Not tested:

  • A real LTE uplink and real gateway failures. No change was made on live gateways.
  • An image that contains 3a51649. The lab used the 0798007 image with the carry.lua of 3a51649.
  • The idle standby site still switches pgws on a bloated uplink, because it has no rx. This is as designed.

awlx added 2 commits November 21, 2026 08:51
On a lossy or bloated uplink, the 1 s ping to the pgw gets lost or comes
late during big downloads. The node then moves to another pgw or site,
and the download breaks.

While rx of the current pgw grows by 48 KB in 3 s (2 of 3 ticks), a
missed ping of a site that is up is not counted, for at most 30 s after
the last answer. The ping then waits up to 3 s in the background, and a
late reply counts as an answer. Carry never makes a pgw or site up.
Without traffic, nothing changes.
A tick with only a small rx growth counted as a carry tick. After a pgw
stops during a download, the data in flight and the retransmits then kept
the window active for 3 more ticks, and the failover took 7-8 s instead
of about 5 s (lab: one-way cut under load). Now a tick counts only when rx
grew by at least carry_bytes / carry_window (16 KB).
@awlx
awlx merged commit 6b8cae6 into parker-release-2026.11 Nov 21, 2026
11 of 12 checks passed
@awlx
awlx deleted the awlx-parker-traffic-liveness branch November 21, 2026 11:46
awlx added a commit to freifunkMUC/site-ffm that referenced this pull request Nov 21, 2026
Includes:
- freifunkMUC/community-packages#60: tests: fix ShellCheck SC2317
- freifunkMUC/community-packages#61: count the clients of routing nodes
- freifunkMUC/community-packages#66: rx traffic as a second liveness check
- freifunkMUC/community-packages#67: report the CLAT traffic through respondd
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant