Repository navigation
feat(compute-hosts): per-host path_namespace for runs on hosts with a shadowed NAS path - #725
Merged
dmlguq456 merged 5 commits intoOct 21, 2026
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
On mpwav,
/home/nasis now the company NAS stacked over the lab NAS; the lab NAS is at/home/nas_IIP. The company NAS also has/home/nas/user/Uihyeop. compute-hosts runs there therefore saw the wrong tree: three TASLP trainings died on 2026-10-21 withFileNotFoundErrorand left noexit_code, so no completion notice came. Any write would have landed on the company NAS.This adds an optional per-host inventory key:
For such a host, only the run-directory creation and the runner/payload (log, exit_code, the training) run inside a per-process user+mount namespace that rbinds the source onto the target. The uid/gid map is the caller's own (no
unshare -r, no uid 0). The shared tmux server,probe,claim,stopand has-session stay in the host namespace. Hosts without the key are unchanged.Behavior
path_namespace refused: <reason>and a synchronous exit 1 when: the source is missing or not a mount point, the logical run_root's(st_dev, st_ino)inside differs from the physical source path, a bound target's identity differs, or unshare, the map writes, the private mount or the rbind fail (EPERM included). If the runner's own re-check fails, it writesstop_reasonand thenexit_code=125through a descriptor of the verified source-side run dir, so the launcher's watcher still gets a terminal. It never falls back to the logical target, creates parents or overwrites a natural exit.mkdirs the run dir before tmux exists. The detached runner re-enters and re-verifies, because the pane inherits the server's namespace. The runner is written to<lease state dir>/compute-runners/<run_id>.pyon the host's local disk (refused if that lies under a mapped target) and unlinks itself on start. This keeps the tmux command short; an inline runner exceeded tmux's message limit.launch_compute: acquire, then the wrapper-bind wait, then release on every failure.path_namespace, local run_root commands (runincl.--dry-run,claim,runs,tail,stop,watch-run) refuse.list/probeskip local claims.route_parent_close.linked_resourcesdoes not scan run_root.ssh_host. A valid but unavailable source never blockslist/probe.gpu_leases.pykeeps its old bytes as a prefix before# BEGIN PATH_NAMESPACE EXTENSION. Unconfigured launches ship only that prefix.Docs:
core/OPERATIONS.md§5.13 and the installer template comment.Test
tools/run-tests.py --select utilities/owner_resource_continuation.test.py --select utilities/route_parent_close.test.py --select utilities/compute_hosts.test.py --select utilities/gpu_leases.test.py: PASS=4 (fingerprint7b9040f1656f0fe8); clean env 79/79, 40/40, 45/45, 6/6.56bdad1ac: the compute-hosts stand-in inowner_resource_continuation.test.pygains_namespace_self, whichroute_parent_close.linked_resourcesnow calls. Shard 2 of the first run hard-failed on this.7a855f24a(cycle evidence, not a repo golden): run with nested tmux and nohup runners, probe, claim, stop kill and has-session for an unconfigured host — all exact.tools/generate.py --check,tools/check-adaptation-boundary.sh: pass.mpwav-20261021-175708-ns-smoke2exit 0. The log shows/home/nassource163.239.13.40:/volume1/nas1_data, uid 1004, run_root visible and/home/DBresolving. A temp inventory with a missing source refused in 1.95 s with no directory on either NAS. Nothing new appeared on the company NAS (one pre-existing foreign leftover is untouched).Notes
path_namespace, so mpwav is protected only after this is released and installed.nvidia-smi/cuInitwere checked during framing).🤖 Generated with Claude Code