bun_ptr: add OwnedRef, an intrusive ref that is released on drop - #37665
bun_ptr: add OwnedRef, an intrusive ref that is released on drop#37665robobun wants to merge 1 commit into
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (2)
WalkthroughChangesThe PR adds Owned reference lifecycle
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
Comment |
There was a problem hiding this comment.
I re-reviewed this and found no bugs. My earlier note about the unsafe impl Sync SAFETY comment doesn't apply to this revision — OwnedRef has no Send/Sync impl here (it stays !Send/!Sync via the inner RefPtr's NonNull), and the doc comment states that. Given this adds a new public unsafe-backed primitive to the foundational bun_ptr crate that follow-up PRs will build on, a human sign-off on the API shape is still worthwhile.
What was reviewed:
- The two
ManuallyDrop::takeexits (Dropandinto_ref_ptr) are mutually exclusive —into_ref_ptrwrapsselfinManuallyDropfirst, so no double-take. new/adopt/Clone/Derefforward to existingRefPtrmethods with the same safety contracts asScopedRef; debug tracking is preserved.Option<OwnedRef<T>>niche and theCell<Option<_>>::takerelease-once pattern are covered by the Miri tests.
Extended reasoning...
Overview
Adds OwnedRef<T> to src/ptr/ref_count.rs (a ManuallyDrop<RefPtr<T>> newtype with a Drop impl that calls RefPtr::deref()), re-exports it from src/ptr/lib.rs, adds ThisPtr::owned_ref() alongside ref_guard(), updates a few doc comments to point at the new type, and adds seven Miri-run unit tests plus a count() test helper. No existing callers change; the type is not yet used anywhere in the tree.
Change since my prior comment
My earlier inline finding referenced an unsafe impl Sync for OwnedRef<T> and its SAFETY comment at lines ~1193-1195. That impl is not present in this revision (commit 16d2d0f titled "OwnedRef stays !Send/!Sync like RefPtr"), and the type's doc comment now states it is !Send/!Sync for the same reason RefPtr is. So that concern is resolved — there is no thread-safety obligation on the debug tracker introduced here.
Security risks
None user-facing. The risk surface is memory-safety soundness of a new unsafe-backed smart pointer. The two unsafe blocks in the type body (ManuallyDrop::take in Drop and in into_ref_ptr) are exclusive by construction; the unsafe fn constructors forward the exact caller contracts of RefPtr::init_ref/from_raw and ScopedRef::new/adopt. Tests pass under Miri (Tree Borrows) per the PR description.
Level of scrutiny
High. bun_ptr is a T0 foundation crate; this is a new public API that three named follow-up PRs will adopt for request queues, timers, and socket self-refs. The implementation is small and mechanically forwards to RefPtr, but the API shape (new vs adopt, into_raw/into_ref_ptr, whether Deref is appropriate for a shared-ownership handle, staying !Send) is a design decision a maintainer should ratify before dependents stack on it.
Other factors
The comment-cop bot flagged ~13 lines; those are doc comments and required SAFETY comments on a public unsafe API, not workaround justifications, so I did not treat them as blocking findings — but they are outstanding automated comments the author may want to address or dismiss. The bug-hunting pass this run found nothing.
|
Revised in d817546, in case anyone read the first version: |
d817546 to
a07d452
Compare
Same idea and API as the OwnedRef in #31173 (new, acquire, from_raw, into_raw), built on the raw count so it can be Send and Sync for thread-safe hosts, plus RefPtr::into_owned and ThisPtr::owned_ref. RefPtr's debug holder tracking now takes &self and keeps its tables behind its lock, so a RefPtr on one thread and OwnedRefs on others no longer form a &mut to a thread-safe count that other threads are using. Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
a07d452 to
bca06d7
Compare
There was a problem hiding this comment.
The latest push implements the fix from the third review round: debug_data_ptr and DebugDataOps::acquire/release now take &self, the holder tables live behind bun_core::Mutex<Holders>, and noop_debug_data is a plain static — so no path from a RefPtr on one thread forms a &mut to the count while an OwnedRef on another thread reads it. The new ref_ptr_tracking_coexists_with_owned_refs_on_other_threads test exercises exactly that shape, and the Send SAFETY comment now covers it. That closes the concern at line 866; I found nothing further this round.
Given this adds a new public handle type to a T0 foundation crate with hand-written unsafe impl Send + Sync and reshapes DebugData's locking, a maintainer sign-off would still be worthwhile before it lands.
Also checked: alloc_id moving outside the lock is fine (AtomicU32::fetch_add); deinit keeping &mut self is fine (last-release only, uses get_mut()); the derive at bun_core_macros/lib.rs:479 now auto-refs to &ThreadSafeRefCount instead of &mut.
Extended reasoning...
Overview
Adds OwnedRef<T> to bun_ptr — an RAII intrusive-ref handle that releases on Drop, with Deref/Clone, Send + Sync under Arc's bounds, plus RefPtr::into_owned and ThisPtr::owned_ref bridges. To make the Send/Sync impls sound in debug builds, this revision also refactors DebugData: debug_data_ptr and DebugDataOps::acquire/release now take &self, and the live/frees maps move into a Holders struct behind bun_core::Mutex. Eight new unit tests, two of them cross-thread; all run under Miri.
Security risks
None user-facing. This is a memory-safety primitive; the risk class is soundness of the unsafe impl Send + Sync. Three prior review rounds identified aliasing routes through the debug tracker's &mut self methods; the current revision closes them at the layer that owned the invariant (the tracker itself) rather than by removing bridges one at a time, which was the fix recommended in each round. I re-traced the previous scenario against the new code: both the RefPtr side (rc_debug_data → debug_data_ptr(&self) → acquire/release(&self) → holders.lock()) and the OwnedRef side (rc_ref/rc_deref → &*get_ref_count(...)) now form only shared borrows of ThreadSafeRefCount/DebugData, with the mutex serialising the map writes. bun_core::Mutex wraps std::sync::Mutex, so it is a real lock. deinit still takes &mut self but only runs after the atomic 1→0 transition, which is pre-existing behaviour.
Level of scrutiny
High. This is new public API in the T0 bun_ptr foundation crate, with hand-written unsafe impl Send + Sync, intended as the base for stacked follow-ups (#37591, #37594, #37618). REVIEW.md asks for maintainer agreement on new cross-cutting abstractions; while #31173 and prior review requests establish that the type is wanted, the Send/Sync surface and the DebugData locking refactor warrant a human look.
Other factors
The comment-cop bot's five unresolved flags on this revision are the same false positives as prior rounds (rustdoc on public API, a doc comment on DebugDataOps::acquire, and a note in the test module). No runtime code uses OwnedRef yet, so behavioural risk is limited to the DebugData refactor, which is behaviour-preserving (same insert/remove semantics; alloc_id was already atomic and just moved outside the lock).
Problem
bun_ptrhas no handle that can sit in a field or container and release its ref on drop.RefPtr<T>owns a ref but has noDrop, so every holder callsderef()by hand on every path;ScopedRef<T>releases on drop but cannot be dereferenced or stored.NonNull<T>or*mut Tand must release it at every site that removes it. Review on http_jsc: park the WebSocket upgrade client refs as OwnedRef fields (and Cell handshake state) #37594 and mysql: hold the request queue refs as OwnedRef<JSMySQLQuery> #37618 asked for an RAII type instead.RefPtr's debug-build holder tracker took&mut selfon acquire and release, so aRefPtrused on one thread while another handle to the same object was released on a second thread would have formed a&mutover a count the other thread was reading.Fix
OwnedRef<T>: one non-null pointer that takes a ref on construction (newallocates,acquiretakes a ref,from_rawadopts one), derefs to&T, takes another ref onclone, and releases on drop.into_rawis the only way to give the ref up without releasing it.RefPtr::into_ownedandThisPtr::owned_refconvert existing handles.rc_derefinDrop, andinto_rawis the only escape, so a holder cannot forget or double release without naming it. Those are the same calls the hand-written sites make, so there is no runtime cost. It gives upRefPtr's debug holder tracking, asScopedRefalready does.SendandSyncfollowArc's bounds (T: Send + Sync). Only a host with the atomic count can meet them, sinceRefCountandCellRefCountedare!Sync. So that aRefPtron another thread is safe too, the debug tracker's methods now take&selfand its tables sit behind the mutex it already had; debug builds only.cargo miri test -p bun_ptrpasses all 27. No call sites outsidebun_ptrchange;cargo check -p bun_runtimeconfirms the derive output still compiles, and mysql: hold the request queue refs as OwnedRef<JSMySQLQuery> #37618, rebuilt on this branch, exercises it through the MySQL suites.Background
RefCount,CellRefCounted, or the atomicThreadSafeRefCount, added by derive macros).rc_ref/rc_derefbump it through a raw pointer, and the release that reaches zero runs the destructor and frees the allocation.AnyRefCountedis the derive-emitted trait that lets one generic handle work over all three counts.ScopedRef<T>takes a ref for one scope and releases it on drop; it keeps an object alive across a re-entrant call and cannot be dereferenced or moved.RefPtr<T>is the storable handle, released by an explicitderef(); its docs say dropping one leaks.ThisPtr<T>wraps thethispointer a callback is dispatched with and asserts it points at a live object.ref_guard()turns it into aScopedRef; the newowned_ref()is the storable version.RefPtrrecords an id per holder in tables inside the count'sDebugData, so a double release can be diagnosed. It is compiled out of release builds, which is why the locking change costs nothing there.*mut Trather than&T(the contractScopedRef::newalready has), and it checks the cross-thread tests for data races.Original description
What
bun_ptrhas two handles over an intrusive refcount, and neither is the one a struct field or a container element wants:RefPtr<T>owns a ref but has noDropimpl. Its own docs say that dropping one leaks, so every holder releases by hand on every path.ScopedRef<T>releases on drop, but cannot be dereferenced or stored; it brackets one re-entrant call.So code that parks a ref somewhere (a request queue, a task, an in-flight socket or timer) stores a
NonNull<T>or*mut T, remembers that it stands for a ref, and callsderef()/deref_nn()at every site that removes it. Review on #37594 and #37618 asked for an RAII type instead. #31173 proposed exactly this type in May and never landed; this PR is that type again, with the same name and constructor vocabulary (Jarred is credited as co-author), on current main and with the three things the reworks stacked on it need:Send/Syncfor thread-safe hosts,as_non_null/this_ptrfor the call sites that key on the pointer, andRefPtr::into_ownedfor migrating existing handles. If you would rather land #31173 itself I can restack the consumers on it instead; the consumers only use the shared surface.It is
ScopedRefas a value: the samerc_refon construction andrc_derefon drop, plusDeref,Cloneand the hand-offs, so it works for all three count flavours (RefCounted,CellRefCounted,ThreadSafeRefCounted) through theAnyRefCountedbridge the derives already emit. It is one pointer with a null niche (Option<OwnedRef<T>>is pointer sized; there is a test for that).ThisPtr::owned_ref()sits next toref_guard()as the safe way for a callback to take one. There is no conversion fromOwnedRefback toRefPtr; nothing needs one.Send/SyncfollowArc's bounds. That is what lets a work-pool task carry the ref it holds on aThreadSafeRefCountedhost and release it on the other thread, which is the shape the stat watcher (#37591) needs. The bounds are sound because the only count that can live inside aSynchost is the atomic one:RefCountand theCell<u32>behindCellRefCountedare!Sync, so hosts embedding them never satisfy the bounds; andThreadSafeRefCount'sref_/derefare already the operations such hosts use from several threads today. The one thing that was not ready for this wasRefPtr's debug holder tracking: itsacquire/releasetook&mut self(anddebug_data_ptrtook&mutof the whole count), so aRefPtrdriven on one thread while anOwnedRefto the same object was released on another would have formed a&mutover a count the other thread was reading, which review pointed out. Those methods now take&selfand the two tables live behind the lock the struct already had (DebugDatais debug-build only, so this costs nothing in release);noop_debug_databecomes a plain static. Two tests cover the threading: one clones anOwnedRefon a thread-safe host into four threads, drops the clones there and destroys it from a fifth, the other keeps aRefPtrrecording and releasing holder entries on the main thread while four threads releaseOwnedRefclones of the same object. The whole test module runs under Miri, which also checks these for data races.The constructors take
*mut Trather than&Ton purpose. A first version took&T; Miri (Tree Borrows, which is how this crate's tests run) rejected it, because a ref taken through a pointer derived from a shared borrow may be the one whose release destroys the object, and the destroying release writes to the count's debug data and frees the allocation.ScopedRef::newandRefPtr::init_refhave the same*mut Tcontract for the same reason, so the per-typeref_guard(&self)style helpers that already exist forScopedRefare where the one audited cast belongs (the first consumer, #37618, addsJSMySQLQuery::owned_ref(&self)next to itsref_guard).new(value)needs no unsafe at all and replaces theBoxplus initial-count hand-off that types like the proxy tunnel do today.What it gives up relative to
RefPtr(and to the #31173 version) is the debug-build holder tracking, whichScopedRefdoes not have either; in the current port that tracker only records ids, and it is not written for concurrent use, so being able to send the ref is worth more than keeping it.RefPtritself gainsinto_owned, the&selftracker described above and a doc pointer here.No call sites outside
bun_ptrchange (the derives' generated code callsdebug_data_ptrthrough a reference, which now auto-borrows shared;bun_runtime, which has thread-safe hosts, was checked). The PR adds the type, its docs and nine unit tests (take/release, last one out destroys,new/into_raw/from_rawround trip,Cell<Option<_>>::takereleases exactly once,RefPtr::into_ownedleaves the count alone,ThisPtr::owned_refoutlives the callback, the two cross-thread tests, the niche).Why
With the ref represented as a value, "this field holds a ref" is stated by its type, releasing it is
= None,take(), or popping it out of a container, and a path that forgets to release, or releases twice, does not compile unless it namesinto_raw. The follow-ups that use it are the reworks requested in review: #37618 (the MySQL request queue becomes a fifo ofOwnedRef<JSMySQLQuery>, removing the fivederef_nnsites), #37591 (the stat watcher's tasks carry the ref instead of the callbacks adopting it) and #37594 (the WebSocket upgrade client's socket, C++ owner and tunnel refs becomeOption<OwnedRef<Self>>fields); they are stacked on this branch. There is no runtime cost relative to the hand-written pairs: construction andDropare the samerc_ref/rc_derefcalls the hand-written sites make, inlined.Part of a series of small type-system hardening changes.
Verification
cargo check,cargo clippyandcargo fmt --checkare clean forbun_ptr.cargo miri test -p bun_ptr(Tree Borrows, asbun run rust:miriruns it): 27 passed, including the 9 new tests.cargo check -p bun_runtimeconfirms the derive-generated code is unaffected by the&selfchange; #37618, rebuilt on top of this branch, exercises it through the MySQL suites.