node: carry the stat watcher refs as OwnedRef values - #37591
Conversation
WalkthroughChangesThe stat watcher replaces local reference guards with Stat watcher reference ownership
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
Comment |
There was a problem hiding this comment.
LGTM — mechanical swap of two file-local RAII guards for the shared ScopedRef.
What was reviewed:
- Confirmed
ScopedRef::adoptstoresNonNullwithout bumping and itsDropcallsT::rc_deref, which the#[derive(ThreadSafeRefCounted)]bridge routes toThreadSafeRefCount::<T>::deref— byte-identical to the deletedDropimpls. - Checked the added non-null precondition (
NonNull::new_uncheckedvs the old*mut): all 4 call sites alreadyNonNull::new(this).expect(..)on the next line, sothisis provably non-null. - The explicit
drop(_ref_guard)ordering inwork_pool_callbackbeforeembedded_work_finishedis preserved.
Extended reasoning...
Overview
This PR deletes two private RAII types (SchedulerRefGuard, WatcherRefGuard) from src/runtime/node/node_fs_stat_watcher.rs and replaces their 4 uses with bun_ptr::ScopedRef::<T>::adopt. Net +7/-61 lines in one file. Both host types already #[derive(bun_ptr::ThreadSafeRefCounted)], and I verified in src/bun_core_macros/lib.rs that this derive emits an AnyRefCounted impl with DestructorCtx = () and rc_deref_with_context forwarding to ThreadSafeRefCount::<Self>::deref — exactly what the deleted Drop bodies called. ScopedRef::adopt (src/ptr/ref_count.rs:992) does not bump the refcount, matching the deleted adopt constructors.
Security risks
None. This is a pure refactor of an internal RAII type with no user-facing surface, no new unsafe (net -4 unsafe items), and no change to refcount semantics or ordering.
Level of scrutiny
Low-to-medium. Refcount code is memory-safety-sensitive, so I traced the semantics rather than trusting the description: ScopedRef<T> requires T: AnyRefCounted with DestructorCtx: Default, both satisfied by the derive; adopt uses NonNull::new_unchecked (adding a non-null precondition the old *mut field lacked), but every call site immediately follows with NonNull::new(this).expect(...) for ParentRef, so non-null is already an established invariant. The one site with an explicit drop(_ref_guard) before embedded_work_finished still drops at the same point since ScopedRef implements Drop the same way.
Other factors
The PR description is precise and matches the code. This follows REVIEW.md's "grep for the in-tree helper before hand-writing anything" — ScopedRef is already used across ~20 files in the runtime for this exact pattern. fs.watchFile.test.ts passes per the description. No prior reviews or outstanding comments on the timeline.
|
Updated 4:05 AM PT - Aug 12th, 2026
❌ @robobun, your commit 4accd62 has some failures in 🧪 To try this PR locally: bunx bun-pr 37591That installs a local version of the PR into your bun-37591 --bun |
Jarred-Sumner
left a comment
There was a problem hiding this comment.
this is marginally better but is there an even cleaner thing we can do here?
|
The cleaner version is to not adopt anything in the callbacks: the ref taken when the work-pool task / restat is scheduled should travel as an owned value on the task itself, released when the task is dropped after the callback runs, so the four callbacks stop knowing about refcounts at all. That needs the same owning ref type I am adding for #37618; I will restructure this PR on top of it rather than keep the guards. |
8a2c988 to
5a2628f
Compare
|
Reworked in 5a2628f, now stacked on #37665 ( |
There was a problem hiding this comment.
Beyond the inline nit, I traced the refcount balance on every rewritten path (queue push/pop, post_to_js_thread → take_hop_ref/release_unrun, InitialStatTask::run_owned's Box partial-move, shutdown_for_exit, finalize_js_box) and each ref is released exactly once; the this.loop_handle read after into_raw() in post_to_js_thread matches the pre-PR shape, and WatcherBatch::size_hint is sound (BatchIterator::next decrements batch.count). Given this is a cross-thread refcount-ownership rewrite, a human pass is still worthwhile — note the PR description is stale (describes the earlier +7/-61 ScopedRef swap, not the current OwnedRef restructure).
Extended reasoning...
This is a memory-safety refactor of fs.watchFile's stat-watcher that replaces manual ThreadSafeRefCount::ref_/deref pairing with typed OwnedRef<T> values carried through the queue, work-pool tasks, and JS-thread hops. I verified against src/ptr/ref_count.rs that OwnedRef::acquire bumps, from_raw/into_raw transfer without touching the count, Clone bumps, and Drop releases; and against src/threading/unbounded_queue.rs that BatchIterator decrements count per yield so the ExactSizeIterator impl is correct. The one finding (no Drop on WatcherBatch) is defensive only — both call sites fully drain and the in-loop panics are unreachable — and is not a regression from the pre-PR raw-loop shape. Deferring rather than approving because this touches the most-blocked category (cross-thread intrusive-refcount lifetime across ~15 rewritten functions) and the PR body no longer matches the diff.
node_fs_stat_watcher.rs took and released its refs by hand. StatWatcherScheduler::append called ref_() and pushed a raw pointer into the intrusive queue; work_pool_callback and shutdown_for_exit called ThreadSafeRefCount::deref on the pointers they popped; InitialStatTask stored a raw pointer whose ref it released on the closed path and handed to the JS thread otherwise; restat() called ref_() before posting; the three JS-thread continuations adopted those refs into two file-local guard types (SchedulerRefGuard, WatcherRefGuard) whose Drop called deref, and release_hop called deref directly. The refs are now OwnedRef values. A private WatcherQueue wraps the intrusive queue so that push takes an OwnedRef<StatWatcher> (into_raw inside) and pop_batch yields OwnedRef values (from_raw inside): append takes the queue's ref by value, a closed watcher leaves the scheduler by being dropped, and shutdown_for_exit closes and drops. InitialStatTask.watcher is an OwnedRef that drops with the task on the closed path; post_to_js_thread takes the OwnedRef the hop carries and take_hop_ref gives it back on the JS thread (run_hop and Taskable::release_unrun), so each direction of the hop hand-off is written once. The callers of append and restat() mint the extra ref with clone() on the ref they already hold instead of ref_() on a pointer, and init() takes the initial-stat task's ref with one audited OwnedRef::acquire. shutdown_for_exit adopts the RareData ref as a local OwnedRef, finalize uses finalize_js_box, and the ref_/deref wrappers on StatWatcher, the deref wrapper on the scheduler and both guard types are deleted. The scheduler's own ref across its intrusive work-pool task has no task value to live in, so that one site uses ScopedRef::adopt. Every ref is still taken and released at the same point and on the same thread as before, including the release of the initial-stat task's ref before embedded_work_finished() and of the scheduler's ref before its embedded_work_finished(). OwnedRef is a single pointer and its constructors and Drop inline to the same ThreadSafeRefCount ref_/deref calls, so nothing changes at runtime. The file goes from 34 unsafe blocks to 27.
5a2628f to
4accd62
Compare
There was a problem hiding this comment.
Re-reviewed after the Drop for WatcherBatch addition — that closes the one gap I flagged, and this pass found nothing further. Given this restructures seven cross-thread refcount hand-offs (queue push/pop, initial-stat task, pool→JS hops, RareData slot) with load-bearing drop ordering in run_owned and post_to_js_thread, a human look would still be worthwhile.
What was reviewed:
- Each
OwnedRefacquire/drop traced against the removedref_()/deref()pair — same points, same threads, including the_finished-before-watcherordering inrun_owned. post_to_js_thread'sthis.loop_handleread afterinto_raw()— every caller still holds a second ref (queue's or JS wrapper's), same as the pre-PR&selfshape.WatcherBatch::size_hintreadsbatch.count, whichBatchIterator::nextdecrements, solen()and the newDropdrain agree.finalize→finalize_js_boxmatches the old body's leak-then-shared-borrow-then-deref sequence.
Extended reasoning...
Overview
Single-file refactor of src/runtime/node/node_fs_stat_watcher.rs (net −107 lines, 34→27 unsafe blocks). Replaces seven hand-paired ref_()/deref() sites with typed OwnedRef<StatWatcher> values carried by their owners: a private WatcherQueue newtype whose push/pop convert to/from raw for the intrusive queue, InitialStatTask.watcher as an owned field, pool→JS hops that move an OwnedRef into the posted task and reconstitute it in take_hop_ref, and shutdown_for_exit adopting the RareData slot's ref. Deletes the two hand-rolled SchedulerRefGuard/WatcherRefGuard types, the StatWatcher::ref_/deref and StatWatcherScheduler::deref wrappers, as_ctx_ptr, and release_hop. Stacked on #37665 which introduces OwnedRef.
Security risks
None identified. No user-facing surface change, no parsing of untrusted input, no auth/crypto. The risk class here is memory safety (UAF/leak from an unbalanced intrusive refcount across the work-pool ↔ JS-thread boundary), not security.
Level of scrutiny
High. This is exactly the category REVIEW.md flags as most-blocked: intrusive refcounts across threads with multiple terminal paths (closed-early, VM teardown, shutdown, success/error hops). Each hand-off's release point moved from an explicit deref() call to a value's drop site, and drop ordering is load-bearing in at least two places (run_owned declares watcher after the _finished guard so the closed-path drop precedes embedded_work_finished(); run_hop releases the hop's ref on return rather than inside each continuation). I traced each new drop against the removed deref() and found them equivalent, but a maintainer familiar with the scheduler's lifetime should confirm.
Other factors
- My prior 🟡 nit (missing
DroponWatcherBatch) was addressed in the current revision —dropnow drains un-yielded nodes viaself.by_ref().for_each(drop), andsize_hintcorrectly tracksBatchIterator's decrementingbatch.count. finalizenow uses the sharedbun_ptr::finalize_js_boxidiom; checked it matches the removed body (leak Box → run closure with&T→rc_deref).post_to_js_threadreadsthis.loop_handlethrough a non-owningthis_ptr()afterinto_raw()— verified every call site (restatvia queue's ref,run_ownedvia JS wrapper's Strong ref) still holds a second ref across the post, same as the pre-PR&selfreceiver did.- The one unresolved comment-cop nag on line 127 targets the new two-line
Dropdoc comment; the author already resolved 18 identical bot fires on the previous revision with the same rationale (ownership documentation, not a workaround justification). fs.watchFile.test.ts/fs.watch.test.tspass per the description; no new test is expected for a behaviour-preserving type-system refactor.
|
CI status for this revision (build 92872): every lane passed except the two "darwin 26 aarch64" test jobs, which expired in the queue without running (single machine on that tier, long backlog); the same lanes passed on the base PR #37665 (build 92785). The test failures listed on the build all passed on retry. Not retriggering, since a new build would only join the same queue; the two jobs can be retried once the lane drains. |
|
Heads-up: #37857 converts |
Stacked on #37665 (the base of this PR is that branch). The first version of this PR replaced the two hand-rolled ref guards in
src/runtime/node/node_fs_stat_watcher.rswithScopedRef; review asked whether there was something cleaner than guards, and this version is the answer: the refs themselves become values.What
fs.watchFilekeeps eachStatWatcheralive through several intrusive refs: the scheduler's queue holds one per watcher, the initial-stat work-pool task holds one, and every hop posted from the pool to the JS thread carries one. Until now all of them wereref_()calls paired withderef()calls (or guards adopting them) at the other end, seven pairs spread over the file. Now each of them is anOwnedRef<StatWatcher>held by the thing that owns it:UnboundedQueue<StatWatcher>is wrapped in a privateWatcherQueuewhosepushtakes anOwnedRef(the oneinto_rawfor the queue) and whosepop_batchyieldsOwnedRefs back (the onefrom_raw, with the invariant stated there).WatcherBatchdrains itself on drop, so a batch that is not consumed to the end releases the rest instead of leaking it;appendtakes the queue's ref by value;work_pool_callbackis afor watcher in batchloop thatcontinues past closed watchers (dropping the queue's ref is what removes them) and pushes the value back otherwise;shutdown_for_exitcloses and drops. The two hand-writtenThreadSafeRefCount::<StatWatcher>::derefcalls and theref_()inappendare gone.InitialStatTask.watcheris anOwnedRef, taken with the oneOwnedRef::acquireininit()where the freshly created watcher is known live, socreate_and_scheduleis a safe fn; on the closed pathrun_ownedsimply returns (the field drops before theembedded_work_finishedguard, the order the explicitderefhad), otherwise the value moves into the hop.post_to_js_thread(watcher: OwnedRef<Self>, hop)moves the ref into the posted task (the oneinto_rawin that direction) andtake_hop_refis the onefrom_rawin the other;run_hopcalls it and dispatches the three continuations with the value, which therefore take&OwnedRef<Self>(the two initial-stat ones clone it to hand the queue its ref) or plain&self, and the hop's ref is released whenrun_hopreturns instead of at the end of each continuation, the next statement on the same thread.Taskable::release_unrun, which releases a hop the VM tears down unrun, isdrop(take_hop_ref(this)).restatpostswatcher.clone(); theas_ctx_ptrhelper it needed is gone.shutdown_for_exitadopts the RareData slot's ref on the scheduler as anOwnedReflocal and lets it drop at the end, whereSelf::derefused to be, so that wrapper is deleted too;StatWatcher::finalizeuses the existingbun_ptr::finalize_js_boxidiom, which let theStatWatcher::ref_/derefwrappers go as well.Two things are deliberately unchanged: the scheduler's own ref across its work-pool hop still uses
ScopedRef::adopt(the task node is intrusive, so there is no value to carry the ref; parking one in a field would need a lock or an ordering argument plus an unsafe acquire on the other side, which is not cleaner), and each watcher'sRefPtrto the scheduler (converting it would move that release fromfinalizeto the destructor, a behaviour change). Every ref is still taken and released at the same points and on the same threads as before, including the order relative toembedded_work_finished(); the file goes from 34unsafeblocks to 27, net minus 99 lines.Why
Which holder owns which ref is now written in the types: a popped watcher, a task's field and a hop's value each carry their ref, a release is a drop, and the raw hand-offs the intrusive queue and the task queue force are confined to one push/pop pair and one post/take pair, each stating its invariant once.
OwnedRefis aNonNull, so the queue nodes, the task and the posted pointer are exactly the words they were, and each drop is the same atomic decrement the removedderefwas.Part of a series of small type-system hardening changes.
Verification
cargo checkandcargo clippyare clean forbun_runtime; the source-tree lints in test/internal/source-lints pass (82 pass). Debug build succeeds.bun bd test test/js/node/watch/fs.watchFile.test.ts test/js/node/watch/fs.watch.test.ts: 53 pass, 8 skip, 0 fail (re-run on the final revision); the Node test suite's test-fs-watchfile.js, test-fs-watchfile-bigint.js and test-fs-watchfile-ref-unref.js, run under the debug build the way the Node test runner runs them, all exit 0.cargo check --target x86_64-pc-windows-msvcalso passes forbun_runtimeon this branch, so no platform-gated caller of the removed hand-written ref/deref entry points remains.