Skip to content

mysql: hold the request queue refs as OwnedRef<JSMySQLQuery> - #37618

Open
robobun wants to merge 1 commit into
farm/c83f5856/ptr-owned-reffrom
farm/c83f5856/mysql-request-queue-nonnull
Open

mysql: hold the request queue refs as OwnedRef<JSMySQLQuery>#37618
robobun wants to merge 1 commit into
farm/c83f5856/ptr-owned-reffrom
farm/c83f5856/mysql-request-queue-nonnull

Conversation

@robobun

@robobun robobun commented Aug 11, 2026

Copy link
Copy Markdown
Collaborator

Stacked on #37665 (the base of this PR is that branch; only the last commit is this change).

What

The MySQL connection's request queue (src/sql_jsc/mysql/MySQLRequestQueue.rs) holds one intrusive ref on every JSMySQLQuery it contains. Until now the elements were raw pointers (*mut, then NonNull in the first version of this PR), the ref was taken with ref_() in add(), and it was released by hand at the five places an element leaves the queue (three in advance(), one in clean(), one in Drop), each paired with a discard(1) or read_item() and a comment saying which ref it balanced. Review asked why this was not an RAII type.

Now the element type is the ref:

type Queue = LinearFifo<OwnedRef<JSMySQLQuery>, DynamicBuffer<OwnedRef<JSMySQLQuery>>>;

pub(crate) fn add(&mut self, request: OwnedRef<JSMySQLQuery>)
  • JSMySQLQuery::do_run passes this.owned_ref() to enqueue_request; owned_ref(&self) is a one line helper next to the existing ref_guard(&self), and is the only place a ref is taken. add() no longer calls ref_(); it just writes the value into the fifo.
  • Every removal is read_item(), and dropping the value it returns is the release, so the five deref / deref_nn calls and their comments are gone. In advance() the two sites that used to discard(1) and then deref now drop(requests.with_mut(|q| q.read_item())), which keeps the same order (element out of the fifo, borrow of the fifo ended, then the release, which may run the query's destructor); clean() and Drop drain with read_item() as before and the value drops at the end of each iteration, where the deref used to be.
  • current() / current_ref() read the head element through a new LinearFifo::peek_item_ref (peek_item for move-only element types; peek_item now delegates to it) and return the same NonNull / ThisPtr as before, built with OwnedRef::as_non_null / this_ptr; the unsafe { ThisPtr::new(..) } in current_ref is gone, because the element proves the query is live. advance() copies the head's identity out the same way before running a request, so no borrow of the fifo is held across a call that may re-enter the queue, and its run-failure path compares against current() instead of re-peeking.
  • LinearFifo does not drop its contents (it is a bag of bits that supports move-only elements through read_item), so the comment on Queue says that elements are only ever removed with read_item and that Drop drains; clean() already swapped the fifo out and drained it for re-entrancy reasons, which is unchanged.

MySQLConnection::enqueue_request and JSMySQLConnection::enqueue_request change their parameter type; nothing else outside the queue changes. The file loses all of its unsafe and its CellRefCounted import; the one unsafe added is the OwnedRef::acquire inside owned_ref(), which is the same cast ref_guard() already makes.

Why

The invariant "the queue holds a ref on each element" is now carried by the element type instead of by five comments: a path that removes an element cannot forget to release, cannot release twice, and cannot release something it did not remove. It is zero-cost: OwnedRef<T> is a NonNull<T>, so the fifo's layout is unchanged, add() performs the same increment it did through ref_(), and each drop is the same decrement the removed deref_nn performed, inlined. peek_item_ref is the body peek_item already had, minus the copy.

Part of a series of small type-system hardening changes.

Verification

cargo check and cargo clippy are clean for bun_collections and bun_sql_jsc. Debug build succeeds. Against the container's MariaDB, a local smoke script (not committed) drove the queue through its paths on the debug build: 200 concurrent queries plus writes and a transaction on one connection, a mix of succeeding and failing queries (the run-failure release in advance()), close() with 50 queries still queued (clean()), and repeated close-with-queued-work (Drop); all four pass under ASAN. bun bd test test/js/sql/sql-mysql-clean-reentry.test.ts test/js/sql/sql-mysql-prepare-ok-zero-statement-id.test.ts test/js/sql/sql-mysql-tls-plaintext-injection.test.ts test/js/sql/sql-mysql.test.ts: 4 pass, 0 fail (the Docker-backed cases in sql-mysql.test.ts do not run here; they run in CI).

cargo check --target x86_64-pc-windows-msvc also passes for bun_sql_jsc on this branch, so no platform-gated caller of the removed hand-written ref/deref entry points remains.

@robobun
robobun requested a review from alii August 11, 2026 21:35
@coderabbitai

coderabbitai Bot commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

@robobun, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 11 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 09b63dfd-63b6-43f9-85e3-f6ab3340cefd

📥 Commits

Reviewing files that changed from the base of the PR and between 108e412 and 32eca06.

📒 Files selected for processing (4)
  • src/sql_jsc/mysql/JSMySQLConnection.rs
  • src/sql_jsc/mysql/JSMySQLQuery.rs
  • src/sql_jsc/mysql/MySQLConnection.rs
  • src/sql_jsc/mysql/MySQLRequestQueue.rs

Comment @coderabbitai help to get the list of available commands.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — mechanical *mutNonNull tightening with no behavioral change.

What was reviewed:

  • deref_nn is exactly unsafe { deref(p.as_ptr()) } (ref_count.rs:637), so all 5 release sites are byte-identical to before.
  • impl_field_parent! split: the nonnull arm returns NonNull from the same from_mut(self) provenance the old get_js_connection used; both call sites (advance, upgrade_to_tls) updated, no others exist.
  • NonNull::from(this) in do_run carries the same shared provenance as the old as_ctx_ptr(); the queue only forms ParentRef (&T) and deref_nn from it.
  • peek_item(0) == request still compares by address (NonNull PartialEq).
Extended reasoning...

Overview

Changes the MySQLRequestQueue element type from *mut JSMySQLQuery to NonNull<JSMySQLQuery> and threads that through add/current/advance and the two enqueue_request wrappers. The 5 release sites switch from unsafe { JSMySQLQuery::deref(p) } to the safe deref_nn(p) trait method, and the 6 NonNull::new(p).expect(...) re-checks disappear. get_js_connection is regenerated via the nonnull arm of impl_field_parent! so advance and upgrade_to_tls receive a NonNull<JSMySQLConnection> directly.

Security risks

None. No I/O, protocol, or auth logic touched — pure pointer-type plumbing inside the request queue.

Level of scrutiny

Low-to-medium. This is native code with intrusive refcounting, so I verified each substitution is semantics-preserving: deref_nn at ref_count.rs:637 is literally unsafe { Self::deref(this.as_ptr()) }; the nonnull macro arm at bun_core/lib.rs:783 wraps the same from_field_ptr!(.., from_mut(self)) the previous two-arm form used, so provenance is unchanged; NonNull::from(&Self) in do_run yields the same shared-provenance address as the prior as_ctx_ptr() (from_ref(self).cast_mut()); NonNull<T> has identical size/repr to *mut T so the LinearFifo layout is unchanged; and NonNull's PartialEq compares addresses, so the peek_item(0) == request head-check in advance behaves identically.

Other factors

Grepped for other get_js_connection callers — only the two updated in this diff exist. The pattern (two impl_field_parent! invocations for ref + nonnull forms) already appears elsewhere (FileReader, Execution). Net -5 unsafe blocks, -6 expect, +0 unsafe. Existing MySQL tests pass per the PR description; this change has no observable behavior to add a new test for.

@Jarred-Sumner Jarred-Sumner left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to call deref_nn? Why isn't this an RAII type?

@robobun

robobun commented Aug 11, 2026

Copy link
Copy Markdown
Collaborator Author

Agreed. The reason it is not one today is that bun_ptr has no owning RAII handle for intrusive counts: RefPtr/IntrusiveRc deliberately has no Drop, and ScopedRef is a scope guard without Deref. I will add one (an owned ref that holds exactly one count, derefs to &T, clones by taking another count, releases on Drop, with from_raw/into_raw for the FFI hand-offs) in a small base PR, and then make this queue a LinearFifo of those so popping or clearing releases the query without any deref_nn. Will update this PR on top of that.

@robobun robobun changed the title mysql: store NonNull in the request queue and release via deref_nn mysql: hold the request queue refs as OwnedRef<JSMySQLQuery> Aug 12, 2026
@robobun
robobun changed the base branch from main to farm/c83f5856/ptr-owned-ref August 12, 2026 01:31
@robobun
robobun force-pushed the farm/c83f5856/mysql-request-queue-nonnull branch from 32eca06 to 84cb1ad Compare August 12, 2026 01:31
@robobun

robobun commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator Author
Updated 1:43 AM PT - Aug 12th, 2026

@robobun, your commit 282ba3329e33700b6a2b1157b5ca2ebc3fab93f5 passed in Build #92786! 🎉


🧪   To try this PR locally:

bunx bun-pr 37618

That installs a local version of the PR into your bun-37618 executable, so you can run:

bun-37618 --bun

Comment thread src/collections/linear_fifo.rs
Comment thread src/sql_jsc/mysql/JSMySQLQuery.rs
Comment thread src/sql_jsc/mysql/MySQLConnection.rs
Comment thread src/sql_jsc/mysql/MySQLRequestQueue.rs
Comment thread src/sql_jsc/mysql/MySQLRequestQueue.rs
Comment thread src/sql_jsc/mysql/MySQLRequestQueue.rs
Comment thread src/sql_jsc/mysql/MySQLRequestQueue.rs
Comment thread src/sql_jsc/mysql/MySQLRequestQueue.rs
Comment thread src/sql_jsc/mysql/MySQLRequestQueue.rs
@robobun

robobun commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator Author

Reworked as promised, now that the owning handle exists (#37665, which this PR is stacked on): the fifo holds OwnedRef<JSMySQLQuery> values, do_run passes this.owned_ref() in, and every removal is a read_item() whose result is dropped, so the five deref_nn calls are gone along with the unsafe in current_ref. Description rewritten to match; the queue paths (run failure, clean() on close with queued work, Drop) were exercised against a local MariaDB on the debug build in addition to the existing test files.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed this PR and didn't find any bugs. Because it reworks intrusive-refcount ownership in native code (the queue now relies on LinearFifo's bag-of-bits move semantics interacting correctly with OwnedRef::Drop on every removal path), a human look would still be worthwhile.

What was reviewed

  • Refcount balance: every removed deref maps to exactly one read_item() + drop; add() no longer double-refs; OOM in write_item drops the moved-in OwnedRef so no leak.
  • LinearFifo never drops elements — verified no remaining discard() calls on the OwnedRef fifo; realign/ensure_total_capacity bit-move + MaybeUninit drop of the old buffer don't double-free; clean() and Drop still drain fully.
  • Re-entrancy in advance(): the run-failure identity check now uses current() == Some(request) (address compare, no deref of a possibly-freed pointer); req is not touched after on_error returns.
  • peek_item_ref is the previous peek_item body minus the copy; peek_item delegates to it, so existing callers are unchanged.
Extended reasoning...

Overview

This PR converts MySQLRequestQueue's element type from raw *mut JSMySQLQuery to OwnedRef<JSMySQLQuery>, so the queue's intrusive ref on each request is released by RAII rather than by five hand-paired deref() calls. It adds LinearFifo::peek_item_ref (a &T-returning peek for move-only element types), a one-line JSMySQLQuery::owned_ref() helper, and threads NonNull<JSMySQLConnection> through advance()/get_js_connection() in place of *mut. The enqueue_request signature on both the JS wrapper and the protocol struct changes to take the owned ref by value.

Security risks

None identified. This is a type-level refactor of an existing ownership pattern; no new attack surface, no protocol/parsing changes, no user-controlled input handling touched.

Level of scrutiny

High. Per REVIEW.md, native memory safety is the most-blocked category, and this PR sits squarely in it: it changes who releases each intrusive ref and when, and it stores a Drop type inside LinearFifo, which explicitly does not run destructors on its contents. The correctness argument depends on the invariant that every element leaves the fifo via read_item() (never discard(), never a silent buffer drop). I checked that this holds on all paths in MySQLRequestQueue.rs, but the invariant is documented only in a comment on the type Queue alias — a future discard() or ordered_remove_item() call would silently leak.

Other factors

  • No new automated test ships in this PR. It's a behavior-preserving refactor covered by the existing sql-mysql*.test.ts suite, and the description reports ASAN-clean smoke runs exercising each removal path — but a maintainer should confirm that's acceptable for a refcount-balance change.
  • The comment-cop github-actions bot has left eight inline flags on multi-line comments in the diff. Most look like doc/SAFETY comments the bot fires on indiscriminately, but a maintainer may want to triage them.
  • The current() return type changed from Option<*mut> to Option<NonNull<>>; I checked the two external callers (is_idle, can_flush) and they only use .is_none() / current_ref(), so no behavior change.
  • Drop ordering in advance() is preserved: with_mut(|q| q.read_item()) returns the OwnedRef after the fifo borrow ends, then the explicit drop() releases the ref (which may run the query's destructor and re-enter). This matches the old discard(1)-then-deref sequence.

@robobun
robobun force-pushed the farm/c83f5856/ptr-owned-ref branch from a07d452 to bca06d7 Compare August 12, 2026 03:30
@robobun
robobun force-pushed the farm/c83f5856/mysql-request-queue-nonnull branch from 84cb1ad to 282ba33 Compare August 12, 2026 03:32

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed this and didn't find any bugs. Since it reworks intrusive-refcount ownership on a re-entrant JS-facing path, a human pass on the drop ordering would still be worthwhile.

What was reviewed:

  • Refcount balance: owned_ref() takes the same increment add() used to; every removal is now read_item() whose returned OwnedRef drops after the fifo borrow ends — matches the old discard(1) + deref order at all five sites.
  • Run-failure path in advance(): current() == Some(request) subsumes the old length + peek check; request is a copied NonNull used only for identity comparison after on_error, so re-entrant clean() can't cause a UAF or double-release there.
  • LinearFifo doesn't drop elements — confirmed MySQLRequestQueue::Drop and clean() still drain via read_item(), and write_item's OOM early-return drops the moved-in OwnedRef so no leak.
  • peek_item_ref is the old peek_item body minus the copy; get_js_connection()NonNull and current()Option<NonNull> have no other callers.
Extended reasoning...

Overview

Converts MySQLRequestQueue's element type from raw *mut JSMySQLQuery to OwnedRef<JSMySQLQuery> (the RAII intrusive-ref handle added in #37665), so removing an element from the fifo and dropping it is what releases the ref. Removes five hand-paired unsafe deref calls and the unsafe ThisPtr::new in current_ref. Adds LinearFifo::peek_item_ref (same body as peek_item, returns &T), tightens MySQLRequestQueue::advance's param from *mut to NonNull, and splits impl_field_parent! so get_js_connection() returns NonNull. Signature-only changes to enqueue_request in JSMySQLConnection / MySQLConnection.

Security risks

None. No untrusted-input parsing, auth, or crypto is touched; this is an internal ownership refactor.

Level of scrutiny

High. This is native intrusive-refcount management on a JS-cell payload with re-entrant callbacks (on_error, reject) that can synchronously mutate or drain the same queue — the exact class REVIEW.md calls out as most-blocked. I traced the drop order at each removal site (with_mut(|q| q.read_item()) returns the OwnedRef after the fifo borrow ends, so the destructor can't re-enter under a live borrow), the run-failure re-entry case (only pointer identity is compared after on_error; no deref of a possibly-freed request), and the OOM path in write_item (the moved OwnedRef is dropped by the early return, so no leak). The LinearFifo no-drop caveat is handled: Drop and clean() drain with read_item, and no discard() calls remain on this element type. peek_item_ref only accesses slots at offset < count, so the pre-existing assume_init_slice UB note in linear_fifo.rs is not made worse (OwnedRef is NonNull-bearing, but so were several existing element types the file already documents).

Other factors

The base PR (#37665, OwnedRef) is merged. Verification was ASAN + local MariaDB smoke over the four queue paths plus the existing mysql test files, but no new automated test in this PR (it's a behavior-preserving type change). The comment-cop bot flags are all resolved; the added comments are short doc/ordering notes, not workaround justifications. I'm deferring rather than approving because refcount lifetime changes across re-entrant JS boundaries are the category where a second pair of eyes on drop ordering has the highest payoff, even when the mechanical trace checks out.

@robobun
robobun requested a review from Jarred-Sumner August 12, 2026 04:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants