bug(shell): private install kernel check rejects a valid systemd LUKS credential mountpoint #5818

Pre-flight Checklist

• I have searched existing issues and this is not a duplicate
• I understand that PRs will be rejected if the linked issue does not have status:approved

📝 Bug Description

The private (Separate, unit-based) Gentle Shell installation from the gentle-ai TUI is refused before any change on a host whose root volume is unlocked by systemd-cryptsetup.

The delegated unit launch probe runs the kernel check, which parses every line of /proc/self/mountinfo and requires each mountpoint to pass privateHierarchyPath (internal/shellinstaller/user_runtime_primitives_unix.go):

func privateHierarchyPath(path string) bool {
	return utf8.ValidString(path) && filepath.IsAbs(path) && filepath.Clean(path) == path && !strings.ContainsRune(path, '\\') && strings.IndexFunc(path, unicode.IsControl) == -1
}

and privateMountPath (internal/shellinstaller/user_runtime_primitives_linux.go) only decodes the \040 space escape:

func privateMountPath(field string) (string, bool) {
	// Only the defined space escape is supported. Refuse all other escapes,
	// including encoded separators and control characters, rather than guess.
	path := strings.ReplaceAll(field, `\040`, " ")
	return path, privateHierarchyPath(path)
}

On a LUKS host, systemd mounts a credentials tmpfs for the cryptsetup unit. Because the unit name is systemd-escaped, - becomes the literal text \x2d, so the directory name itself contains backslashes. The kernel renders each one as the octal escape \134 in mountinfo:

55 53 0:35 / /run/credentials/systemd-cryptsetup@luks\134x2d11111111\134x2d1111\134x2d1111\134x2d1111\134x2d111111111111.service rw,nosuid,nodev,noexec,relatime,nosymfollow - tmpfs none ro,seclabel,size=1024k,nr_inodes=1024,mode=700,inode64,noswap

The corresponding mountpoint really exists on disk (Fedora 44, / is /dev/mapper/luks-…[/root] on btrfs):

systemd-cryptsetup@luks\x2d11111111\x2d1111\x2d1111\x2d1111\x2d111111111111.service
systemd-journald.service
systemd-resolved.service

\134 is not decoded, so the field keeps a backslash and privateHierarchyPath returns false, refusing the whole private installation as unavailable. The mount is a legitimate read-only systemd tmpfs; nothing on the host is misconfigured.

This is the same class as #5733 (unrelated mountinfo entries aborting the private kernel check, fixed by #5734). #5734 stopped reading the unrelated root field, but every mountpoint is still forced through the strict canonicality predicate, so a valid mountpoint containing an escaped backslash still aborts the install.

🔄 Steps to Reproduce

1. Use a Linux host with a LUKS volume unlocked by systemd-cryptsetup at boot (systemd >= 254, delegated user manager available). Reproduced on Fedora Linux 44, kernel 7.2.11, / on /dev/mapper/luks-… btrfs.
2. Confirm the mount exists: grep credentials /proc/self/mountinfo → a tmpfs mountpoint under /run/credentials/ whose name contains \134x2d.
3. Run gentle-ai and start the Gentle Shell installation (private / Separate mode) from the TUI.
4. The run fails in step 1, "Validate the selection and target".

✅ Expected Behavior

The private kernel check consumes only the cgroup2 hierarchy it actually needs (/sys/fs/cgroup) and tolerates unrelated mounts. Either classification is limited to entries whose mountpoint is /sys/fs/cgroup or beneath it, or the full mountinfo escape set (\040, \011, \012, \134) is decoded and a valid mountpoint that legitimately contains a backslash is accepted.

Note that decoding alone is not sufficient: the decoded path still contains a literal backslash, which privateHierarchyPath rejects. Both changes are needed, or the strict predicate must apply only to the trusted cgroup2 entry.

❌ Actual Behavior

The installation is refused before any change, and the delegated unit launch probe exits 1:

Installation failed  [░░░░░░░░░░░░] 0/5

  [!] Validate the selection and target              failed
  [ ] Prepare the private stage and preferences      pending
  [ ] Install the runtime and pinned suppliers       pending
  [ ] Apply optional runtimes and the SDK channel    pending
  [ ] Publish, verify and register the installation  pending

No installation was applied: the process did not complete and no target was published.
The installation stopped before publishing a verified result:
  installation refused before any change: the delegated unit launch probe failed: private installation: uncertain: private installation: uncertain: exit status 1
  unit kernel check failed: private installation: unavailable: mountinfo mountpoint "/run/credentials/systemd-cryptsetup@luks\134x2d11111111\134x2d1111\134x2d1111\134x2d1111\134x2d111111111111.service" is not canonical
  Preserve evidence: workspace="<unit-workspace>" destination/unit="gentle-shell-<redacted>"

The existing Gentle Shell installation keeps working: the refusal happens before any write.

Gentle AI Version

4.3.1 (latest release as of 2026-10-27)

Operating System

Linux (Fedora/RHEL)

AI Agent / Client

Other

📋 Affected Area

Installation Pipeline

💡 Logs / Error Output

unit kernel check failed: private installation: unavailable: mountinfo mountpoint "/run/credentials/systemd-cryptsetup@luks\134x2d11111111\134x2d1111\134x2d1111\134x2d1111\134x2d111111111111.service" is not canonical

Call path: installer TUI → delegated unit launch probe → UserKernelCheck() (user_install_linux.go) → privateKernel() → privateCgroupPath() → privateMountPath()/privateHierarchyPath().

Additional Context

• Search performed before filing: no existing issue mentions systemd-cryptsetup, \134, or mountinfo escape decoding.
• No user-side workaround found. --mode shared does not avoid the gate on Linux: user_install_flow_unix.go runs UserKernelCheck() on the non-direct path, and the direct path (userDirectKernelCheck(), only /proc/self/status) is not selectable by the operator. There is no flag to bypass the private kernel check.
• The mount has no bearing on the values the check actually consumes (cgroup2 membership and limits); it is a false positive caused solely by a display escape.
• A host without a systemd credentials mount does not reproduce, which is why this survived fix(shell): tolerate unrelated nsfs mountinfo entries in the private kernel check #5734.