node:sqlite: implement the module and pass the Node v26.3.0 test suite - #32498
Conversation
|
Updated 4:45 PM PT - Jul 16th, 2026
⏳ @robobun, your commit 2546808 is still building in
|
|
Found 3 issues this PR may fix:
🤖 Generated with Claude Code |
|
This PR may be a duplicate of:
🤖 Generated with Claude Code |
|
Not a coincidence — this PR includes #29821's branch as a merge (its C++ implementation is unchanged) and adds what it still needed to land on current main: registration in the Rust module-resolver tables, a bundled SQLite 3.53.2 update (fixes a crash on malformed changesets that the v26.3.0 test suite exercises), the SQLITE_ENABLE_PERCENTILE define, and the vendored test files synced to the Node v26.3.0 tag. If this one merges, #29821 can be closed as superseded. |
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughImplements Changesnode:sqlite built-in module implementation
Possibly related PRs
Suggested reviewers
🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
Comment |
There was a problem hiding this comment.
Actionable comments posted: 5
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/js/node/test.ts`:
- Around line 71-72: The reset() and restoreAll() methods in the test tracker
are currently empty implementations that do nothing, which means spy state is
never actually cleaned up even though these calls appear to succeed. Implement
these methods to actually reset and restore the tracker's internal spy state
respectively. Since the tracker instance returned at lines 162-166 is shared,
ensure these cleanup methods properly clear all tracked spies and restore their
original behavior instead of being no-op methods.
- Around line 51-52: The fn method accepts original and impl parameters without
validating that they are callable functions, causing errors to surface later at
the $apply method instead of immediately. Add validation guards at the beginning
of the fn method to check that both original and impl parameters are functions,
and throw an appropriate error if either is not callable. This should occur
before creating the MockFunctionContext instance to catch invalid inputs
upfront.
In `@src/jsc/bindings/sqlite/JSSQLStatement.cpp`:
- Around line 283-287: Replace the sqlite3_close(db->db) call with
sqlite3_close_v2(db->db) to properly defer cleanup in garbage-collected
environments. The sqlite3_close_v2() function always succeeds (no return-code
check needed) and marks unreferenceable connections as zombie connections for
cleanup during VM teardown, matching the pattern already used in
VersionSqlite3::release(). This ensures that when db->db is nulled on the
following line, the finalizer's cleanup path can still defer proper resource
release.
In `@src/jsc/bindings/sqlite/NodeSqlite.h`:
- Around line 162-173: The BusyScope struct can be accidentally copied or moved,
which would cause multiple instances to exist and each would decrement
m_busyDepth in their destructors, leading to counter underflow and bypassing
re-entrancy protection. Make BusyScope non-copyable and non-movable by
explicitly deleting the copy constructor, copy assignment operator, move
constructor, and move assignment operator. Add these deleted member function
declarations to the BusyScope struct definition.
In `@test/js/node/test/parallel/test-sqlite-session.js`:
- Line 607: The test `concurrent applyChangeset with workers` has an explicit
timeout of 120 seconds which violates the no-timeout rule. Remove the `{
timeout: 120_000 }` configuration from this test. Instead of extending the
timeout, reduce the test complexity by either decreasing the number of
iterations from 10 to 3-5, or splitting the test into separate test cases with
fewer workers per test to keep execution within the fast test budget without
requiring explicit timeouts.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 8fa6b59e-be07-4812-aace-2c53082f8f1b
📒 Files selected for processing (46)
scripts/build/deps/sqlite.tsscripts/build/unified.tssrc/js/node/test.tssrc/jsc/bindings/BunProcess.cppsrc/jsc/bindings/ErrorCode.tssrc/jsc/bindings/ZigGlobalObject.cppsrc/jsc/bindings/ZigGlobalObject.hsrc/jsc/bindings/isBuiltinModule.cppsrc/jsc/bindings/sqlite/JSSQLStatement.cppsrc/jsc/bindings/sqlite/NodeSqlite.cppsrc/jsc/bindings/sqlite/NodeSqlite.hsrc/jsc/bindings/sqlite/sqlite3.csrc/jsc/bindings/sqlite/sqlite3_local.hsrc/jsc/bindings/webcore/DOMClientIsoSubspaces.hsrc/jsc/bindings/webcore/DOMIsoSubspaces.hsrc/jsc/modules/NodeModuleModule.cppsrc/jsc/modules/NodeSqliteModule.hsrc/jsc/modules/_NativeModule.hsrc/resolve_builtins/HardcodedModule.rssrc/resolve_builtins/HardcodedModule.zigtest/js/node/module/node-module-module.test.jstest/js/node/process/process.test.jstest/js/node/sqlite/node-sqlite.test.tstest/js/node/test/common/index.jstest/js/node/test/common/index.mjstest/js/node/test/parallel/test-sqlite-aggregate-function.mjstest/js/node/test/parallel/test-sqlite-authz.jstest/js/node/test/parallel/test-sqlite-backup.mjstest/js/node/test/parallel/test-sqlite-config.jstest/js/node/test/parallel/test-sqlite-custom-functions.jstest/js/node/test/parallel/test-sqlite-data-types.jstest/js/node/test/parallel/test-sqlite-database-sync.jstest/js/node/test/parallel/test-sqlite-limits.jstest/js/node/test/parallel/test-sqlite-named-parameters.jstest/js/node/test/parallel/test-sqlite-serialize.jstest/js/node/test/parallel/test-sqlite-session.jstest/js/node/test/parallel/test-sqlite-statement-sync-columns.jstest/js/node/test/parallel/test-sqlite-statement-sync.jstest/js/node/test/parallel/test-sqlite-template-tag.jstest/js/node/test/parallel/test-sqlite-timeout.jstest/js/node/test/parallel/test-sqlite-transactions.jstest/js/node/test/parallel/test-sqlite-typed-array-and-data-view.jstest/js/node/test/parallel/test-sqlite.jstest/js/node/test/sqlite/next-db.jstest/js/node/test/sqlite/worker.jstest/regression/issue/25707.test.ts
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/jsc/ErrorCode.rs`:
- Around line 706-710: The error code index alignment between Rust and
TypeScript is broken. In the Rust ErrorCode.rs file, the SQLITE_ERROR constant
is defined at index 324 with a total COUNT of 325 entries, but the TypeScript
errors array only contains 321 entries with SQLITE_ERROR at index 320, and is
additionally missing 51 error codes that exist in Rust's CODE_STR. Update the
TypeScript errors array to include all 325 error code entries in the exact same
order as they appear in Rust's CODE_STR constant, ensuring that SQLITE_ERROR
ends up at index 324 to maintain the index-alignment invariant required by the
C++ memory-safety constraint referenced in the code comment.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 350fc0c6-eff9-4d06-bc58-2d58738e9b44
📒 Files selected for processing (4)
src/jsc/ErrorCode.rssrc/jsc/bindings/ErrorCode.tssrc/jsc/bindings/sqlite/JSSQLStatement.cppsrc/jsc/bindings/sqlite/NodeSqlite.h
There was a problem hiding this comment.
I didn't find any bugs, but this is a large feature PR adding an entire new built-in module (node:sqlite) with ~4k lines of new native C++ JSC bindings, GC/IsoSubspace wiring, a bundled SQLite version bump, and module-resolver changes across Rust/Zig/C++ — it warrants a human review.
Extended reasoning...
Overview
This PR implements the node:sqlite built-in module end-to-end and brings it to parity with Node v26.3.0's test suite. It touches 47 files, including:
- New native bindings:
NodeSqlite.h(758 lines) declaring six newJSDestructibleObjectcell types (JSDatabaseSync,JSStatementSync,JSStatementSyncIterator,JSNodeSqliteSession,JSNodeSqliteLimits,JSNodeSqliteTagStore) plus their prototypes/constructors, andNodeSqlite.cpp(not shown in the diff header but referenced as a new no-unify TU) implementing them. - GC/VM wiring: six new
LazyClassStructuremembers inZigGlobalObject.h, sixinitLaterblocks infinishCreation, and matchingIsoSubspace/GCClient::IsoSubspaceslots inDOMIsoSubspaces.h/DOMClientIsoSubspaces.h. - Module registration:
HardcodedModule.rs,HardcodedModule.zig,_NativeModule.h,NodeModuleModule.cpp,isBuiltinModule.cpp,NodeSqliteModule.h. - Bundled SQLite bump 3.53.0 → 3.53.2: large amalgamation diff in
sqlite3.c/sqlite3_local.h(upstream-generated), plus six new compile-timeSQLITE_ENABLE_*flags and making the sqlite dep unconditionally built. bun:sqliteteardown fix:Bun__closeAllSQLiteDatabasesForTerminationnow usessqlite3_close_v2and nulls the handle.node:testshim: a minimalMockTracker/MockFunctionContextso vendored Node tests usingt.mock.fn()work.- Error-code table append:
ERR_SQLITE_ERRORadded at the end ofErrorCode.tsand mirrored inErrorCode.rs(index 324, COUNT bumped to 325). - Tests: 19 vendored upstream
test-sqlite-*files, a 1100-line Bun-native test, common/index.{js,mjs} helpers, and a regression-fixture swap fromnode:sqlite→node:quic.
Security risks
The implementation surface is memory-safety-sensitive: raw sqlite3* / sqlite3_stmt* / sqlite3_session* ownership across GC-managed wrappers, re-entrancy guards (BusyScope) protecting against close() from inside UDF/authorizer/option-getter callbacks, ABA-style open-generation tracking, and buffer-detach defenses around applyChangeset/deserialize. loadExtension is gated on allowExtension per Node semantics. The header design is careful and well-commented (e.g., sessions freed before sqlite3_close_v2, deleteTrackedSessions() invariant, structure-cache invalidation on schema change), and the test suite explicitly exercises several UAF/double-free scenarios under ASAN. I did not spot a concrete vulnerability, but the .cpp implementation is large and not fully visible in the diff, and any of these invariants being subtly wrong is a heap-UAF in production.
The SQLite 3.53.2 bump is an upstream patch release and includes the session-extension hardening the PR description calls out; the amalgamation diff is mechanical.
Level of scrutiny
High. This is a brand-new public API surface backed by thousands of lines of new native code that manages C resource lifetimes against a concurrent GC, wires new IsoSubspaces, and changes the module-resolution tables that every require() consults. It also incorporates another open PR (#29821) by merge. None of this is mechanical or pattern-following; it's exactly the kind of change a maintainer should read.
Other factors
- The bug-hunting system found nothing.
- My earlier inline nit (
require('../common/index.mjs')intest-sqlite-config.js) was answered — the file is byte-identical to upstream Node v26.3.0, intentionally kept verbatim. - The author already addressed two CodeRabbit findings in commit 4481f87 (
BusyScopemade non-copyable/non-movable; termination path switched tosqlite3_close_v2). The remaining CodeRabbit comments (mock.fn input validation, mock.reset/restoreAll no-ops, and the explicit per-test timeout on the worker-based session test) are minor and the author can decide whether to act on them. - Test coverage is extensive (319 upstream subtests passing per the description, plus a dedicated Bun suite covering the GC/re-entrancy edge cases), which raises confidence but doesn't substitute for a human pass over the native lifetime management and the GC visit/subspace plumbing.
|
CI status after the error-code alignment fix (build #63406): 282/286 jobs passed. Notes on the remaining 4 failed jobs — none are sqlite failures:
|
|
Build #63424 (latest commit): 281/286 jobs passed; the binary-size step now soft-fails as an intentional, documented increase. The 4 remaining job failures are all pre-existing or flaky and unrelated to this PR — puppeteer's Chrome download failing on the darwin-26 runner (next-pages tests), the pre-existing test-tls-client-destroy-soon on darwin-14, the cross-branch test-net-connect-memleak GC flake on Alpine, and a grpc-js flake. I've retried those 4 jobs to get a clean run. |
|
Build #63467 (latest commit, all review feedback applied): 281/286 jobs passed. The binary-size step soft-fails as the documented intentional increase, and the 4 hard-failed jobs are the same recurring flaky/pre-existing tests already triaged earlier in this PR — test-net-connect-memleak.js on the two Alpine shards (GC-timing flake also seen on unrelated branches), test-tls-client-destroy-soon.js on darwin-14 (pre-existing, also fails on main), and napi.test.ts on Windows (known flaky suite; this PR doesn't touch N-API). I've retried those 4 jobs to get a clean run. |
There was a problem hiding this comment.
No further issues found after the latest fixes — all prior findings look addressed. Deferring to a human reviewer given the scope (~4.5k lines of new GC-sensitive JSC bindings in NodeSqlite.{cpp,h}) and the macOS binary-size trade-off you flagged.
Extended reasoning...
Overview
This PR implements the node:sqlite built-in module end-to-end: ~3.7k lines of new C++ in NodeSqlite.cpp plus an 821-line header defining six JSC cell types (JSDatabaseSync, JSStatementSync, iterator, session, limits, tag-store), lazy class-structure / IsoSubspace wiring in ZigGlobalObject, module registration across the Rust/Zig/C++ resolver tables, a bundled SQLite 3.53.0→3.53.2 amalgamation bump, six new compile-time feature defines, an always-on sqlite build (previously macOS dlopen-only), a minimal node:test mock.fn() shim, process.versions.sqlite, a new ERR_SQLITE_ERROR code, and ~6k lines of vendored Node v26.3.0 tests plus ~1.3k lines of Bun-authored tests.
Security risks
The implementation handles native resource lifetimes (sqlite3*, sqlite3_stmt*, sqlite3_session*) across JSC GC boundaries with re-entrant JS callbacks (UDFs, authorizers, filter/conflict handlers, option getters). Over the review cycle I flagged and the author fixed four memory-safety issues in this area: a Strong<>-rooted UDF cycle leak, an orphaned-session unbounded-growth leak, a deferred-xDestroy use-after-free on zombified connections, and a stale-iterator return() that silently rewound a live cursor. Buffer-detachment via hostile option getters is now guarded in both deserialize() and applyChangeset(). Extension loading is gated behind allowExtension; the authorizer hook is exposed. These are exactly the surfaces a human should re-read in NodeSqlite.cpp — the diff omits that file's body due to size.
Level of scrutiny
High. This is a large new native module with subtle GC/finalizer ordering invariants, and the review history demonstrates that getting those invariants right took several iterations. Separately, the author explicitly raised a policy question for reviewers: bundling sqlite3.c on macOS grows the darwin binaries by ~1.8 MB (documented and [skip size check]-tagged), and they noted they're "happy to revisit if reviewers prefer a different trade-off."
Other factors
All 15+ inline findings I raised across eight review rounds have been applied and resolved; the latest revision (commits through 4d19528) produced no new findings from the bug-hunting system. CI on the most recent build is green except for documented pre-existing/flaky failures unrelated to this change. Test coverage is extensive (319 vendored Node subtests passing plus Bun-authored regression tests for each fixed lifetime bug). The remaining work is human sign-off on the implementation design, the lifetime-management approach in NodeSqlite.cpp, and the binary-size trade-off.
|
Final-state build #63476: 282/286 jobs passed; binary-size soft-fails as the documented intentional increase, and the only 3 hard-failed jobs are the recurring flakes already triaged (terminal.test.ts PTY timeout on darwin — also failing on 10+ unrelated branches — and test-net-connect-memleak.js on the two Alpine shards). Retried those 3 jobs to get the build green. Automated review has signed off ("no further issues"); ready for human review. |
|
Retry outcome on build #63476: the three flaky jobs failed again on the same tests, so the build stays at 282/286 with those two flake classes. For the record on why they're not from this PR: test-net-connect-memleak.js asserts a dropped socket is collected after one gc() — with conservative stack scanning that's sensitive to binary layout, it isn't exercising any sqlite code (node:sqlite isn't loaded there), it passed on the same Alpine shards on build #63456 of this PR, and it also fails on unrelated branches (e.g. #63365, #63344). The terminal.test.ts PTY timeout on darwin appears on 10+ recent unrelated branches. Happy to dig further if a maintainer suspects either is real. |
|
Brought the branch up to date with main (merge commit 13bdb62). Conflict notes: ERR_SQLITE_ERROR moved to the end of the error-code table after the codes main added (Rust mirror updated to index 327 / COUNT 328), and the PR's minimal node:test mock shim was dropped in favor of the MockTracker that landed on main — TestContext.mock now returns that tracker (dde35fb) since Node's sqlite tests drive callbacks through t.mock.fn(). Re-verified locally after the merge: all 19 vendored test-sqlite-* files pass, node-sqlite.test.ts (69) and bun:sqlite (80) pass, and the error-code mapping spot-checks are correct. |
|
Build #63956 (current head, post-main-merge): 284/286 — the only hard-failed job is darwin-26, which hit the known Next.js/puppeteer infra tests and the flaky s3 suite (binary-size soft-fails as the documented intentional increase). Retried that job; everything sqlite-related is green. |
|
Build #63956 is now green after the darwin-26 retry — 285/286 passed with only the binary-size soft-fail (documented intentional increase). CI is clean for the current head; ready for human review. |
90c2be6 to
3848ace
Compare
jsDatabaseSyncDeserialize calls deleteTrackedSessions() after taking its own BusyScope (m_busyDepth == 1), so the previous 'gated on m_busyDepth == 0' wording was literally inaccurate for that caller. The ASSERT is still valid: every caller enters with m_busyDepth == 0, and inUse is only set inside a nested BusyScope that completes synchronously.
… a pending exception sqlite3WalkExprNN maps WRC_Prune to continue for sibling columns inside one expression (SELECT a + b), so an authorizer that throws on column 'a' is re-invoked for 'b' with the exception still pending. Every sibling C->JS callback (xFunc, stepBase, valueBase, applyChangesetXConflict/XFilter) already checks scope.exception() before re-entering JS; add the same guard here. The post-call check still surfaces the original error either way; this is the consistency guard validateExceptionChecks targets.
Wrap the sqlite3_malloc'd output in an ArrayBuffer whose destructor runs sqlite3_free instead of createUninitialized + memcpy + free. Same technique bun:sqlite's serialize() already uses (JSSQLStatement.cpp via JSBuffer__bufferFromPointerAndLengthAndDeinit); under LAZY_LOAD_SQLITE sqlite3_free is the dlsym'd library's own free, so there is no allocator mismatch. adoptSqliteBuffer() handles the null/zero-length case so the three call sites stay one line.
| // Node's retry-forever semantics are preserved otherwise. The VM is | ||
| // being torn down, so just clean up and let the unwind happen — no | ||
| // JS allocation on a terminating VM. | ||
| if (vm.traps().needHandling(VMTraps::NeedTermination) || vm.hasPendingTerminationException()) [[unlikely]] { |
There was a problem hiding this comment.
@robobun we don't check for VMTraps NeedTermination like this anywhere. why is this needed?
There was a problem hiding this comment.
It's needed because vm.hasTerminationRequest() won't work here: m_hasTerminationRequest is only flipped by VMTraps::handleTraps() (which then calls vm.setHasTerminationRequest()), and that only runs at a JS safepoint. Worker.terminate() → notifyNeedTermination() just does traps().fireTrap(NeedTermination), which sets the atomic trap bit cross-thread and nothing else.
Without a progress callback this loop is sqlite3_backup_step → sqlite3_sleep(25ms) → repeat, never entering JS, so there's no safepoint to promote the trap bit into m_hasTerminationRequest or a pending termination exception. Polling hasTerminationRequest() here would read false forever and the worker would hang on a permanently-BUSY destination. test/js/node/sqlite/node-sqlite.test.ts "Worker.terminate() interrupts a backup() spinning on a locked destination" covers exactly that (parent holds a BEGIN IMMEDIATE lock on the dest, worker backup-spins on BUSY, terminate() must land).
The other hasTerminationRequest() call sites in src/jsc/bindings/ (NodeVMModule, NodeVMScript, BunProcess) all check after returning from a JS call where a safepoint has already handled the trap, so the bool is set. This is the only spot that sits in a pure-C loop with no JS re-entry.
That said, if reading the trap bits directly is undesirable, the alternative is to call JSC::call into a no-op on each iteration (giving the trap a safepoint to land at, then checking scope.exception()), or to route this through Bun's hasTerminationRequest Rust helper via an extern "C" that also peeks the trap bits. Happy to switch the shape if you'd prefer one of those.
| // Node retries SQLITE_BUSY/LOCKED indefinitely (BackupJob just calls | ||
| // ScheduleWork() again with no timeout), so match that: no invented | ||
| // busy budget. Back off between retries so a contended destination | ||
| // doesn't busy-spin at 100% CPU. Unlike Node this loop runs on the JS | ||
| // thread, so a permanently-locked destination would be an unrecoverable | ||
| // hang; the progress callback fires on BUSY/LOCKED too so a caller can | ||
| // throw from it to abort. (Node fires progress between retries too, but | ||
| // gated on remaining_pages != 0 — Bun fires unconditionally so the | ||
| // escape hatch works even before the first successful step.) | ||
| constexpr int kBusyRetrySleepMs = 25; |
There was a problem hiding this comment.
@robobun is it not possible to run this as a task in the thread pool like nodejs?
There was a problem hiding this comment.
Yes, and that's the right fix; the synchronous loop is a documented divergence, not a design choice. The header comment above jsNodeSqliteBackup says as much and names PhonyWorkQueue/ConcurrentCppTask__createAndRun as the in-tree precedent. Doing it would also remove the VMTraps::NeedTermination poll entirely, since the JS thread wouldn't be blocked.
What it needs, roughly mirroring Node's BackupJob:
- A heap-allocated job object holding
sqlite3_backup*, the destinationsqlite3*,rate, aStrong<JSPromise>, aStrong<JSDatabaseSync>(so the source cell and handle survive GC across the async gap), and aStrong<>forprogressFn. - Each tick: dispatch one
sqlite3_backup_stepviaConcurrentCppTask__createAndRun, then post back to the JS thread (ScriptExecutionContext::postTask) to runprogress, check fordb.close()/abort, and either re-dispatch or resolve/reject. BusyScopecan't stay stack-scoped; the job would hold an explicit busy count (or am_backupInFlightflag onJSDatabaseSync) thatcloseInternal()checks the same way it checksisBusy(), so a close during backup defers until the job finishes. Node handles this viaBaseObjectPtrkeeping theDatabaseSyncalive.sqlite3_backup_stepreading the source connection from a pool thread is safe underSQLITE_THREADSAFE=1(serialized) since it takes the source handle's mutex internally; that's the same assumption Node makes.
The current sync version passes all 18 vendored test-sqlite-* files (the upstream suite doesn't actually exercise the "source db usable during backup" promise), which is why it shipped this way with the divergence noted. Happy to implement the threadpool version here if you'd rather not carry the divergence; it's ~200 lines but not risky. If you'd prefer to land this PR as-is and take that as a follow-up I can open one.
There was a problem hiding this comment.
I didn't find any bugs, but this is a large feature PR (~4,100 lines of new C++ in NodeSqlite.cpp plus bindings/build/test changes across 50 files) implementing node:sqlite, so it warrants a human review.
What was reviewed:
- Exception-scope handling and GC rooting in the new JSC classes (DatabaseSync/StatementSync/Session) — WriteBarriers visited, ThrowScopes checked.
- SQLite lifetime/ownership: statements and sessions tracked on the DB and finalized on close; lazy-load dlsym gating on macOS.
- Parameter binding paths for Buffer/TypedArray/BigInt and the changeset callback re-entrancy guards.
- Module registration in ZigGlobalObject/HardcodedModule and
process.versions.sqlitewiring.
Extended reasoning...
Overview
This PR adds a native node:sqlite implementation to Bun. The core is a new 4,122-line C++ file (src/jsc/bindings/sqlite/NodeSqlite.cpp) plus a header, defining DatabaseSync, StatementSync, Session, backup, aggregate/user-defined functions, and changeset APIs. It touches ZigGlobalObject.{cpp,h}, IsoSubspace headers, BunProcess.cpp (process.versions.sqlite), isBuiltinModule.cpp, module registration (_NativeModule.h, NodeSqliteModule.h, HardcodedModule.rs), lazy_sqlite3.h and the vendored amalgamation config, build scripts, ErrorCode.ts, and ~20 test files ported from Node's parallel suite.
Security risks
The surface includes: user-controlled SQL passed to sqlite3_prepare_v2; loadExtension (dlopen of arbitrary paths, gated by an allowExtension flag); user JS callbacks invoked from inside SQLite (UDFs, aggregates, authorizer, changeset conflict handlers) with the attendant re-entrancy and GC hazards; and Buffer/TypedArray byte views bound into SQLite blobs. None of these are new attack surface relative to Node, but each is a place where a missed exception check, a stale pointer after a JS callback, or an unrooted JSValue would be exploitable.
Level of scrutiny
High. This is thousands of lines of new hand-written JSC C++ binding code interfacing with a C library, with manual memory management, WriteBarrier members, visitChildren implementations, and callbacks that cross the JS↔native boundary in both directions. Per the repo's own review guidance, memory safety in native bindings is the most-blocked PR category. It also adds fields to ZigGlobalObject and new IsoSubspaces.
Other factors
The PR ships an extensive Node-ported test suite (test-sqlite-*.js/mjs) which is a good signal, and the bug-hunting pass came back clean. But the sheer size and the number of independently risky subsystems (GC integration, dlopen-based lazy loading on macOS, session/changeset C API, build-system changes) put this well outside the bar for auto-approval. A maintainer should read at least the ownership/finalization paths in NodeSqlite.cpp and the ZigGlobalObject additions.
…er-v26 Resolve conflict in src/js/node/test.ts: keep this PR's per-node MockTracker for t.mock (supersedes #32498's module-level stopgap).
- sliceAnsi-fuzz: scale the O(n) time bound on ASAN too, not only debug (the release ASAN lane took the 1000-iteration path and hit the 5s bound). - stripANSI: the heapStats string count can drop between the two reads when GC collects an unrelated string, so the new "standalone C1 ST is not stripped" check is one-sided (<=) with a full GC before the baseline. - docs/bun.d.ts: `wordWrap: false` breaks every line at the column width, not "only at explicit newlines" -- that is what Bun and npm wrap-ansi both do and what the new test asserts. - wrapAnsi.test: cover C1 CSI hard wrap with a non-SGR final byte (`K`). No src/ changes. Rebased onto main for #32498 / #34418 / #34423.
Only src/js/node/worker_threads.ts conflicted, in three hunks where #34338 ("don't hang when captured stdout/stderr is never consumed") and this branch touch the same lines. #34338 removed the #stdoutAutoPipe/#stderrAutoPipe fields and moved the stdio port ref/unref out of ref()/unref() — ports now manage their own ref via makePortReadable's incrementsPortRef. This branch only added #hasRef bookkeeping there, so main's structure is taken wholesale and only the two `if (!this.#exited) this.#hasRef = ...` lines and the field are kept. async_hooks.ts (#31825) and VirtualMachine.rs (#34293, #32498) auto-merged. `git diff origin/main -- src/js/node/worker_threads.ts` is a pure addition: zero deleted lines, so nothing from #34338 or #31825 is reverted. Verified on the merge result: test-worker-hasref, test-worker-error-stack- getter-throws, test-perf-hooks-worker-timeorigin, test-diagnostics-channel- worker-threads and the new "online fires before the entry point finishes" all pass; #34338's own repro still exits 0 like node; BroadcastChannel ref()/unref() and the 'online' timing fix both still match node v26.3.0.
The SQLite selectors loaded `bun:sqlite`, which exists only under Bun. Moving the suites to Node did not fail them, it emptied them: one test became it.skip and eleven kept their names, kept passing, and ran no assertions behind `if (!sqlite) return`. The previous commit quarantined those four files onto a Bun-hosted vitest project. This replaces the quarantine with a loader that works on both runtimes. `sqliteDatabase()` now tries `node:sqlite` first and falls back to `bun:sqlite`: - node:sqlite is unflagged from Node v22.13.0 and is the module upstream pi uses (packages/storage/sqlite-node); it lets the selectors and their tests run under Node. - Bun 1.3.14 does not ship it. oven-sh/bun#32498 is merged but unreleased, and the shipped binary is Bun-compiled, so the fallback is what keeps that binary working. When Bun releases node:sqlite both runtimes take the first branch and the fallback can be deleted. - better-sqlite3 was evaluated and rejected: it segfaults Bun 1.3.14 on construction, which is worse than a catchable missing-module error. Two deltas are absorbed so callers see no change: node:sqlite spells the option readOnly and rejects it passed explicitly as undefined, and it refuses to bind the booleans bun:sqlite stores as integer 1/0 (normalizeSqliteWriteValue permits booleans). Removed with the quarantine: the agent-bun project and BUN_HOSTED_TESTS, the test:bun script, the separate CI step, and the hard-require test helper. Test fixtures now go through test/helpers/sqlite.ts, which mirrors the same preference order behind the bun:sqlite-shaped API the suites were written against. The CI contract is rewritten rather than dropped: it now asserts the loader order, a single vitest project, that no SQLite test is excluded from collection, and that no soft guard returns. Docs updated in AGENTS.md, docs/ci.md and development.md. Verified on both runtimes: the four SQLite files pass 47/47 under `vitest` (Node) and 47/47 under `bun --bun vitest`. Full coding-agent suite 2899 passed / 29 skipped, restoring the pre-migration skip count of 29 that the quarantine pass had left at 30. Root suites: unit 5402 passed / 2 skipped, integration 469 passed / 1 skipped, ci-contracts 34 passed, script tests pass, typecheck clean. Assistant-model: Claude Opus 5
…sts (#2079) * chore: take upstream pi's toolchain split for install, checks, and tests Adopt earendil-works/pi's task-for-task toolchain rather than only its test runner. npm installs, builds, checks, and runs the suites; vitest replaces `bun test` for the three root suites; Bun keeps exactly the two jobs pi also gives it, compiling release binaries and running `scripts/*.ts`. Install moves to `npm ci --ignore-scripts` against a regenerated `package-lock.json`, and `bun.lock` is deleted. The repository tracked both lockfiles and only verified one: `npm ci` failed on `main` because the tracked lock had drifted from package.json, while that same unverified lock is the input to the shrinkwrap published inside @bastani/atomic. One verified source of truth closes that gap. bunfig.toml's supply-chain gate ports 1:1 to a committed .npmrc (`min-release-age=3`, `min-release-age-exclude`, `save-exact`), with a matching dependabot `cooldown` so automated bumps cannot outrun it. The three root suites move to vitest projects sharing a pi-shaped `vitest.base.ts` that sets only `resolve.alias`. A `bun:test` alias lets 629 test files migrate unedited; the 95 files using `Bun.*`, `import.meta.dir`, or a Bun-spawning `process.execPath` move onto `test/helpers/runtime.ts`, whose helpers exist mainly to close the differences that fail silently (`Bun.write` creating parent directories, `spawnSync` returning `status` rather than `exitCode`, `Bun.spawn` refusing a missing binary synchronously). The duration guard is rewritten for vitest's JSON reporter rather than retired. Under Bun's stdout it scored 4288 of 4417 unit tests and mis-attributed barrel-re-exported files; it now scores 5396 of 5398 records with correct attribution, and its `blind` state finally means the harness broke instead of being unreachable. The flaky runner keeps every behaviour and changes only its input contract. Distinct test names are unchanged: 4417 unit, 289 integration, 32 -> 33 ci, with an itemised eight-rename allowlist proved by scripts/compare-test-inventory.mjs. Assistant-model: Claude Opus 5 * fix: keep the bun:sqlite suites on Bun and point the migration's own guard at them Repairs the toolchain migration against independent verification. The blocking loss was larger than reported. Moving packages/coding-agent from `bun --bun test` to Node did not skip one test; it silently emptied eleven. `src/core/tools/resource-selectors.ts` loads `bun:sqlite` and throws without it, and its tests guarded that with `if (!sqlite) return` or `? it : it.skip` — so under Node one declaration skipped and ten more kept their names, kept passing, and ran no assertions. Running the four affected files under Node with hard requires fails exactly 11 tests, which is the size of the hole. Those four files now form a Bun-hosted vitest project (`agent-bun`), run by a new `npm run test:bun --workspace=@bastani/atomic` and a new agent-suite CI step. The Node project (`agent`) excludes exactly the files the Bun one collects, so the second step is coverage rather than a repeat. The guards are hard requires again via test/helpers/bun-sqlite.ts, which throws and names the command to use. test/ci/ci-workflow-contracts.test.ts makes it structural: every test file naming `bun:sqlite` must be in BUN_HOSTED_TESTS, collected by `agent-bun`, excluded from `agent`, sharing one testTimeout, with no early-return guard or `? it : it.skip` left in it, and the CI step must exist. scripts/test-duration-guard.ts steps over a leading `bun`/`bunx` so the new step is scored like every other suite. Only the runtime's own leading flags are dropped — filtering every `-` argument swallowed `--project` and made the guard average whichever projects happened to agree. scripts/compare-test-inventory.mjs was committed unreferenced and was never pointed at the suite that regressed. It now takes a repeatable `--candidate` (a suite split across runtimes is compared as the union of its parts; either half alone reads as a loss), auto-detects a bun log or a vitest report as the baseline, and diffs the *skip* set as well as the name set — a test that ran before and skips now keeps its name and keeps the suite green. Its rules are covered by scripts/compare-test-inventory.test.mjs, which CI runs in static-checks via `npm run test:scripts`. AGENTS.md records the four invocations and why the comparison itself stays a migration-time gate. Evidence, all four suites, baseline captured at HEAD~1 under Bun: unit 4417 -> 4423 distinct names, 0 missing, 0 newly skipped, 2 skipped both sides, 8 reviewed renames integration 289 -> 289, 0 missing, 0 new, 0 newly skipped, 1 skipped both ci 32 -> 34, 0 missing, 0 newly skipped, 1 reviewed rename coding-agent 2893 -> 2893 (union of agent + agent-bun), 0 missing, 0 new, 0 newly skipped, 29 skipped both sides — not the 30 Node produced Also fixed: - test/unit/flaky-test-suite-runner.test.ts names REAL_VITEST_SUITE_TIMEOUT_MS at both structural call sites, as AGENTS.md requires and the PR did not do. - The aliased-declaration branch of `declarationPattern` has a fixture again (`const runTest = built ? test : test.skip`), in both argument shapes. - run-flaky-test-suite.ts treats an unreadable report as blind, not just a missing one, and two fixture modes cover the paths that had none: a suite that writes no report at all, and a corrupt report whose deterministic failure is found by the log scan in `findFailedDeterministicFile`. - bun-test-shim's `setDefaultTimeout` clamps with Math.min instead of claiming to. TEST_TIMEOUT_MS moved to a leaf module so the shim does not pull `vitest/config` into 629 test files' workers; the clamp is unit-tested. - test/unit/bump-version-script.test.ts's fixture root has a package-lock.json, so `bumpNpmLock` is entered: workspace entries and first-party ranges stamped, third-party pins and link entries untouched, plus the no-lockfile case. WARN_RATIO is unchanged and must stay so; AGENTS.md now says why, alongside the Bun.spawn fidelity gap in test/unit/web-access-subprocess.test.ts, which `installBunGlobal()` closes for the module's own logic but not for Bun's spawn. docs/ci.md, DEV_SETUP.md and packages/coding-agent/docs/development.md follow, and a stale paragraph describing the removed file-length gate is gone. Assistant-model: Claude Opus 5 * fix(tools): resolve SQLite from node:sqlite with a bun:sqlite fallback The SQLite selectors loaded `bun:sqlite`, which exists only under Bun. Moving the suites to Node did not fail them, it emptied them: one test became it.skip and eleven kept their names, kept passing, and ran no assertions behind `if (!sqlite) return`. The previous commit quarantined those four files onto a Bun-hosted vitest project. This replaces the quarantine with a loader that works on both runtimes. `sqliteDatabase()` now tries `node:sqlite` first and falls back to `bun:sqlite`: - node:sqlite is unflagged from Node v22.13.0 and is the module upstream pi uses (packages/storage/sqlite-node); it lets the selectors and their tests run under Node. - Bun 1.3.14 does not ship it. oven-sh/bun#32498 is merged but unreleased, and the shipped binary is Bun-compiled, so the fallback is what keeps that binary working. When Bun releases node:sqlite both runtimes take the first branch and the fallback can be deleted. - better-sqlite3 was evaluated and rejected: it segfaults Bun 1.3.14 on construction, which is worse than a catchable missing-module error. Two deltas are absorbed so callers see no change: node:sqlite spells the option readOnly and rejects it passed explicitly as undefined, and it refuses to bind the booleans bun:sqlite stores as integer 1/0 (normalizeSqliteWriteValue permits booleans). Removed with the quarantine: the agent-bun project and BUN_HOSTED_TESTS, the test:bun script, the separate CI step, and the hard-require test helper. Test fixtures now go through test/helpers/sqlite.ts, which mirrors the same preference order behind the bun:sqlite-shaped API the suites were written against. The CI contract is rewritten rather than dropped: it now asserts the loader order, a single vitest project, that no SQLite test is excluded from collection, and that no soft guard returns. Docs updated in AGENTS.md, docs/ci.md and development.md. Verified on both runtimes: the four SQLite files pass 47/47 under `vitest` (Node) and 47/47 under `bun --bun vitest`. Full coding-agent suite 2899 passed / 29 skipped, restoring the pre-migration skip count of 29 that the quarantine pass had left at 30. Root suites: unit 5402 passed / 2 skipped, integration 469 passed / 1 skipped, ci-contracts 34 passed, script tests pass, typecheck clean. Assistant-model: Claude Opus 5 * chore: drop the migration-only coverage comparison script scripts/compare-test-inventory.mjs proved this migration shed no test names, and it required a baseline captured from the runner being replaced. Once the migration lands there is no such runner, so the script cannot run again, and AGENTS.md already documented it as a migration-time gate rather than a CI step. Removed with its unit test and its AGENTS.md section. The coverage evidence it produced is recorded in the PR body. Assistant-model: Claude Opus 5 * refactor(test): import vitest directly and delete the bun:test shim Completes the pi practice the migration deferred: test files import from "vitest", not from "bun:test" through an alias. Codemodded 631 files. The shim was not a pure re-export, so the mechanical rename carried four adaptations with it: - `.serial` -> `.sequential` (241 declarations); vitest spells Bun's in-file ordering modifier differently. - `mock(...)` -> `vi.fn(...)`, `mock.restore()` -> `vi.restoreAllMocks()`. - `spyOn(...)` -> `vi.spyOn(...)`; vitest has no top-level export. - `setDefaultTimeout(30_000)` dropped: it equalled TEST_TIMEOUT_MS, so the shim's clamp made it a no-op already. Five files genuinely need Bun's module registry and keep `bun:test`, because they re-exec their bodies under `bun test` in a child process where the specifier resolves natively: - overlay-adapter-autowrap and overlay-adapter-hidden-render now take `mock` from a dynamic `await import("bun:test")` inside the child-only function, so the parent no longer needs the alias to load them. - mcp-oauth-lifecycle-reset, the session-manager preload fixture and the 2791 fswatch regression keep a static import, because theirs sits inside a generated child script or a Bun preload rather than in the parent module. Deleted test/helpers/bun-test-shim.ts, its unit test, and the `bun:test` alias in vitest.base.ts. Docs updated in AGENTS.md, DEV_SETUP.md and the vitest configs; the AGENTS.md example now imports from "vitest". Also aligned the root `test` script with pi's shape: it was running only the unit project, and now runs `test:scripts`, every vitest project, and the workspace suites, matching `npm run test:scripts && npm run test --workspaces`. Verified: unit 579 files / 5400 passed / 2 skipped (exactly the two tests of the deleted shim's own suite fewer, nothing else moved), integration 469 passed / 1 skipped, ci-contracts 34 passed, all projects together 5903 passed / 3 skipped, coding-agent 2899 passed / 29 skipped, script tests 3 passed, typecheck and check clean. The four SQLite selector files still pass 47/47 under both Node and `bun --bun vitest`, and all four child-process files pass. Assistant-model: Claude Opus 5 * docs: drop follow-up notes from the contributor docs AGENTS.md and DEV_SETUP.md are instructions, not a backlog. Assistant-model: Claude Opus 5 * style: adopt biome formatting Adds biome.json modelled on upstream pi: tab indent width 3, line width 120, recommended lints with the same handful of overrides pi disables. Scope follows pi as well -- package sources and tests, root suites, scripts -- excluding generated files, fixtures and vendored skills. This commit is the formatter pass only, so the lint fixes that follow are reviewable apart from 2550 whitespace changes. tsc --noEmit is clean across the 1748 reformatted files. Assistant-model: Claude Opus 5 * style: apply every biome lint fix and enforce biome in check Completes the Biome adoption. The rule set is upstream pi's exactly: the recommended preset plus the same six overrides (noNonNullAssertion, useConst, useNodejsImportProtocol, noExplicitAny, noControlCharactersInRegex, noEmptyInterface). Nothing else is disabled. `biome check` is now the first step of `npm run check`, so the prek hook and the CI static-checks job enforce it without further wiring. `npm run format` applies the formatter. Roughly 2100 findings are resolved. Most were mechanical, but one class was not: biome's noConfusingVoidType autofix rewrites `T | void` to `T | undefined`, and this repository uses `T | void` deliberately at its SDK boundary -- the host ExtensionAPI's `on()` returns void while the internal event bus returns an unsubscribe function, so the union is what accepted both. Rewriting it broke 47 type contracts. Rather than disable the rule and lose parity, the affected surfaces now type those returns as `unknown` and narrow with a typeof guard at the call site, which satisfies both the rule and the compiler: - WorkflowEventSurface and PiResultIntercomExtensionAPI `on` - the ExtensionAPI event-handler return in public-types - run-tool-execution-tracker's drain result Test doubles that returned Promise<void> now return undefined explicitly so they satisfy the signatures they implement. No test was skipped, weakened or deleted, and no `as any`, ts-ignore or biome-ignore suppression was added anywhere. biome.json is migrated to the 2.5.5 schema, where `recommended` is spelled `preset`. Verified: biome check clean across 2279 files, tsc --noEmit clean, unit 5400 passed / 2 skipped, integration 469 passed / 1 skipped, ci-contracts 34 passed, script tests 3 passed, coding-agent 2899 passed / 29 skipped, and the SQLite selector files still 47/47 under both Node and `bun --bun vitest`. Every count is identical to before the lint pass. Assistant-model: Claude Opus 5 * fix(ci): repair three failures the npm toolchain switch introduced All three only appear in CI, which is why they survived local validation. 1. Windows suites and agent-suite: ENOENT uv_spawn 'npm'. test/helpers/runtime.ts resolved the executable to decide whether to throw ENOENT, then spawned the bare name anyway. Bun resolved and ran Windows `.cmd` shims itself; Node does not, and since 20.12 (CVE-2024-27980) it refuses to exec a `.cmd` or `.bat` without a shell. Spawn the resolved path, and use a shell only for a `.cmd`/`.bat` shim. 2. release-archive: "Required runtime dependency not found: css-select". scripts/build-binaries.sh installs every platform's native binding with `npm install --no-save --force`. On a versionless base those resolve to the 0.0.0 placeholder, which is not published, so npm fails with ETARGET -- but not before mutating node_modules and pruning real runtime dependencies. Skip the fetch entirely at the placeholder version, and restore the tree with `npm ci` if it fails for any other reason. 3. static-checks: "must be run in a directory where a docs.json file exists". That message is misleading; mintlify actually refuses to start on Node 25+, and static-checks installs no Node toolchain, so npx picked up the runner's Node 26. Run mintlify through `bunx --bun` as it was before, which hosts it regardless of the runner's Node. Verified locally: npm run check, test:unit 5400 passed / 2 skipped, test:ci-contracts 34 passed, script tests 3 passed, and an end-to-end run-flaky-test-suite.ts invocation that spawns npm through the repaired helper. Assistant-model: Claude Opus 5 * fix(ci): resolve nested npm dependency layouts and Windows .cmd shims Two more npm-vs-bun differences the first pass did not reach. 1. release-archive: "Required runtime dependency not found: css-select". Skipping the 0.0.0 binding fetch was necessary but not sufficient -- the dependency was never in the root node_modules to begin with. `bun install` runs with `linker = "hoisted"`, so every transitive dependency lands at the root; npm nests on a version conflict, at arbitrary depth. css-select resolves to node_modules/linkedom/node_modules/css-select, and that copy's own boolbase sits beside it rather than beneath it. copy-runtime-dependencies.ts now resolves each package by walking up through every ancestor node_modules exactly as require.resolve would, instead of assuming a single hoisted root. It copies 270 packages where it previously aborted at 151, and ./scripts/build-binaries.sh now produces a complete darwin-arm64 archive locally. 2. Windows suites: ENOENT uv_spawn on the resolved npm path. The previous fix resolved the executable but preferred the extensionless match. On Windows npm ships as both `npm` -- a POSIX shell script Windows cannot exec -- and `npm.cmd` beside it, so the resolver returned the one that cannot run. Try PATHEXT candidates before the bare name. Verified: npm run check, test:unit 5401 passed / 1 skipped, test:ci-contracts 34 passed, a full copy-runtime-dependencies run with no missing packages, and a complete ./scripts/build-binaries.sh --platform darwin-arm64 archive build. Assistant-model: Claude Opus 5 * fix(ci): match pi's .npmrc and Node version exactly The .npmrc this migration wrote claimed to replace bunfig's supply-chain gate one-for-one, but it did not hold: `min-release-age-exclude` only exists in npm 11.17.0 and later. CI runs the npm bundled with its Node, which is older, so npm reported "Unknown project config" and silently ignored the key. The exemption was doing nothing there, and npm warns it will stop working entirely in the next major. Upstream pi's entire .npmrc is two lines: save-exact=true min-release-age=2 Ours is now byte-identical to that. The exclusion list is gone, which does change behaviour: @earendil-works/pi-* releases were exempt from the age gate under bunfig because they are consumed same-day, and they no longer are. A same-day pi bump now needs an explicit `--min-release-age=0` on that one install. pi itself carries no exemption, so this is the parity cost. Node in CI moves 24 -> 22 to match pi's workflows. Both repositories already declare `engines.node >= 22.19.0`, and Bun was already pinned to 1.3.14 everywhere, which is pi's pin too. The dependabot `cooldown` is realigned 3 -> 2 so it still matches the .npmrc gate; the two exist to cover each other and drifting apart would leave a hole. Docs corrected in AGENTS.md and docs/ci.md. Verified: npm ci --dry-run emits no unknown-config warning, npm run check passes, test:ci-contracts 34 passed, and both workflows validate. Assistant-model: Claude Opus 5 * fix(security): close two ReDoS paths, a zip-slip, and a DOM-global misbind Addresses every open CodeQL and code-quality finding on this PR. All four are real; none is a false positive. Security (CodeQL): - resource-selectors.ts, polynomial regex on uncontrolled data. The scheme scanner `/[a-z][a-z0-9+.-]*:\/\/.../gi` is unanchored under /g, and its two character classes overlap, so every interior position of a long `[a-z0-9+.-]` run is retried. A `(?<![a-z0-9+.-])` lookbehind pins each match to a real scheme boundary. Measured on a 40,000-character adversarial string: 793 ms before, 0 ms after. Match results are identical on realistic input. - resource-selectors.ts, the same class in the skill:// parser. `([^/]+)\/?(.*)$` is ambiguous between the optional separator and the tail; `([^/]+)(?:\/(.*))?$` is not. Verified equivalent across the empty tail, trailing slash, nested path and non-matching cases. - assert-builtin-archive-set.test.ts, zip slip. A tar entry name is attacker-controlled and may contain `..`, so `join(root, header.name)` can write outside the extraction root. Entries are now resolved and rejected unless they stay under it. Code quality: - examples/extensions/subagent/index.ts called `new Text(text, 0, 0)` with no `Text` in scope, so it bound to the DOM global, which takes one argument. CodeQL reported it three times as superfluous trailing arguments. The intended class is pi-tui's `Text`; it is now imported. This was a genuine bug in the example, not a lint artifact. Verified: npm run check clean, test:unit 5401 passed / 1 skipped, test:ci-contracts 34 passed, the archive suite and the three SQLite selector suites pass, and a regex-equivalence probe confirms both patterns match exactly as before. Assistant-model: Claude Opus 5 * fix(test): keep the foreground subagent tests hermetic under Node Two tests in subagents-workflow-session-persistence.test.ts failed in CI with "No API key found for the selected model" while passing locally, because they were reading the developer's real ~/.atomic/agent credentials. The cause is a runtime assumption the toolchain migration removed. The tests hand the executor a `.ts` stub CLI via piArgv1, and `resolvePiCliScript` accepts a `.ts` entry only when the runtime is Bun. Under `bun test` that held, and on main these pass in 169ms with no network. Under vitest's Node worker the stub is rejected, the executor falls through to the real installed `atomic`, and the test makes a live provider call -- 4.4s and a 401 once a dummy key was supplied. A unit test reaching the network is worse than one that fails, so raising the timeout or injecting a key would both have been wrong. The background test in this same file already had the answer: run the child in a runtime that can execute the entry. Both foreground tests now drive the executor through a Bun child the same way, which keeps the stub in play. Coverage is unchanged -- all three still assert the persisted session header's workflow classification and that no session leaks into the listing. Verified with HOME pointed at an empty directory and OPENAI_API_KEY and ANTHROPIC_API_KEY unset, reproducing the CI environment: 3 passed, no network. Full suite 5401 passed / 1 skipped, npm run check clean. Assistant-model: Claude Opus 5 * fix(ci): run the flake-runner fixture through Bun and harden the zip-slip guard Windows suites: ten tests in flaky-test-suite-runner.test.ts failed while the two real-vitest cases passed. The fixture writes an extensionless `vitest` file carrying a `#!/usr/bin/env node` shebang and invokes it as `./vitest`. Windows has no shebang support, so Node cannot exec it; Bun.spawn used to paper over this before the runner moved to node:child_process. The fixture now runs that fake suite through Bun explicitly. The duration guard already steps over a leading Bun runtime before matching `basename` against `vitest`, so the budget still resolves and the gate stays on -- which the budget assertions in the same file continue to prove. CodeQL zip slip: the previous guard compared the resolved target against the resolved root with startsWith, which is correct but is not the shape CodeQL recognises as a barrier, so the alert stayed open. Replaced with the canonical form from CodeQL's own remediation guidance: compute the path relative to the root and reject when it is empty, escapes with `..`, or is absolute. Verified: flaky-test-suite-runner 12 passed, the archive suite 2 passed, full unit suite 5401 passed / 1 skipped, npm run check clean. Assistant-model: Claude Opus 5 * fix(security): use the barrier shape CodeQL recognises for the zip-slip guard js/zipslip stayed open through two attempts. The check was functionally correct each time; the problem was that CodeQL did not recognise it as a barrier, so the tainted flow was still reported at the `resolve(root, header.name)` call. The first attempt guarded with `target !== containedRoot && !target.startsWith( containedRoot + sep)`. The extra disjunct is what broke recognition. The second attempt switched to a `relative()` check, which CodeQL does not model for this query either. This restores the exact shape the js/zipslip remediation documents: one condition, `startsWith` against the resolved root plus a separator. The `!== root` disjunct is unnecessary anyway, since a tar entry never names the extraction root itself. Behaviour verified directly rather than assumed: builtin/ok.txt -> allow nested/../ok.txt -> allow (contains .. but stays inside) ../escape.txt -> block ../../etc/passwd -> block /abs/evil -> block Archive suite 2 passed, unit suite 5401 passed / 1 skipped, npm run check clean. Assistant-model: Claude Opus 5 * fix(ci): match runtime binaries case-insensitively in the duration guard Windows resolves executables through PATHEXT, whose entries are conventionally uppercase, so the fixture's resolved Bun runtime arrives as `...\bun.EXE`. The guard's case-sensitive binary regexes then failed to step over the runtime prefix, no budget resolved, and the headroom gate silently disabled — failing the five guard-dependent flake-runner fixture tests on the Windows suites job. Windows filename matching is case-insensitive; the npm/vitest/bun binary checks now match the same way, with a regression assertion covering the uppercase-extension shape. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * fix(security): resolve the CodeQL alerts flagged on the toolchain-parity PR Three fixes for the github-advanced-security annotations: - js/polynomial-redos (resource-selectors.ts x2): the archive and SQLite selector regexes carried an ambiguous greedy `(.+\.ext)` prefix that backtracks polynomially on adversarial input. Both parsers now use a linear right-to-left scan for the extension split and reuse the unchanged, unambiguous suffix grammar. Verified behaviorally identical to the old regexes across ~214k generated inputs, including drive letters, multi-colon members, and case variants. - js/zipslip (assert-builtin-archive-set.test.ts): the startsWith barrier was sound but CodeQL barrier recognition does not follow the guarded target variable into a separate .then callback, which kept the alert open across two earlier revisions. The guard and the filesystem writes now live in the same function. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * fix(security): resolve the remaining open CodeQL alerts on the PR ref Code fixes for seven pre-existing alerts, preserving behavior for all valid inputs: - js/insecure-randomness (live-browser-dom.js, and the same class in live-browser-session.js): live-edit session ids fall back to crypto.getRandomValues instead of Math.random when crypto.randomUUID is unavailable (plain-http LAN preview). - js/resource-exhaustion (live-server.mjs): the client-supplied poll timeout is clamped to the 10-minute default ceiling; NaN falls back to the default instead of firing immediately. - js/regex-injection (live-accept.mjs): the digit-validated --variant value is additionally regex-escaped before splicing into a RegExp. - js/double-escaping (live-manual-edit-evidence.mjs): decodeBasicHtml now decodes & last, so crafted input such as &lt; decodes to the literal < instead of being double-unescaped to <. - js/incomplete-sanitization (auth-storage-01.suite.ts x2): the test's shell-path escaping handles backslash and quote in one pass; output is unchanged because the preceding slash conversion removes every backslash. The impeccable script changes extend the vendored tree's existing CodeQL-fix divergences from upstream. The two js/shell-command-injection-from-environment alerts on pi's spawnProcess wrapper were dismissed as false positives instead: command and args are an argument vector with no shell-string concatenation, and cross-spawn exists to escape arguments safely for Windows .cmd shims. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
What
Adds the
node:sqlitemodule and brings the full Node v26.3.0test-sqlite-*suite into the repo, passing.This builds on #29821 (merged into this branch unchanged): native
DatabaseSync,StatementSync,backup(), sessions/changesets, custom scalar/aggregate/window functions, the authorizer, andprocess.versions.sqlite, all implemented in C++ against the bundled sqlite3 amalgamation. On top of that, this PR adds what was still missing to actually ship it on current main and to match Node v26.3.0:src/resolve_builtins/HardcodedModule.rs): the module resolver no longer reads the Zig tables, sonode:sqliteis registered there as a prefix-only builtin (same treatment asnode:test). This is the line that makesrequire("node:sqlite")resolve.scripts/update-sqlite-amalgamation.sh 3530200 2026). 3.53.2 contains the session-extension hardening (SQLite check-ine807d4e379) that makessqlite3changeset_apply()returnSQLITE_CORRUPTon malformed UPDATE changesets instead of dereferencing a NULLsqlite3_value*. Node v26.3.0 ships 3.53.1 plus a cherry-pick of the same check-in, andtest-sqlite-session.jsexercises it.SQLITE_ENABLE_PERCENTILEadded to the sqlite build defines, matching Node's compile-time feature set, so thepercentile()/median()SQL functions exist (asserted by upstreamtest-sqlite.js).test-sqlite-database-sync.js,test-sqlite-backup.mjs, andtest-sqlite-session.jsrefreshed to the upstream copies (using-based cleanup, the backup keep-alive GC test, the malformed-changeset test);test-sqlite-database-sync-dispose.jsremoved because upstream merged that suite intotest-sqlite-database-sync.js; thepercentile is enabledtest now gates on a runtime probe of the loaded library (sqlite_compileoption_used) rather than an unconditional skip, so it runs for real wherever the bundled amalgamation is linked.common/index.jsnow treats--experimental-sqlite/--no-experimental-sqliteas no-ops under Bun instead of re-spawning the test, so upstream// Flags:headers stay verbatim.Test results (debug build, Linux x64)
test/js/node/test/parallel/test-sqlite-*.{js,mjs}files pass when run the way CI runs them (bun test --config=bunfig.node-test.toml): 319 subtests pass, 0 fail, 4 skip. Node v26.3.0 has no sqlite tests undertest/sequential/.online-timing in the busy-timeout test, the error shape forrequire("sqlite")without thenode:prefix, the--no-experimental-sqliteflag, and one upstream-conditional skip in the backup test.percentile(), geopoly, rbu,load_extension) gate on a runtimesqlite_compileoption_usedprobe of the loaded library, so they skip there rather than run; pointingDatabase.setCustomSQLite()at a full-featured build makes them execute. The counts above are Linux x64, where the bundled amalgamation is linked.test/js/node/sqlite/node-sqlite.test.ts,test/js/bun/sqlite/,test/js/sql/sqlite-sql.test.ts,test/js/node/process/process.test.js,test/js/node/module/node-module-module.test.js, andtest/regression/issue/25707.test.tspass with the bumped SQLite.Fixes from review
JSDatabaseSynchandles live only as GC cells and Bun does not destruct the VM on a normal exit, so an unclosed file-backed database never reachedsqlite3_close_v2()and its WAL was never checkpointed — unlike Node (~DatabaseSyncon environment teardown) andbun:sqlite.open()now registers the handle (mapped to its owningJSC::VM*, captured on the owning thread) in a registry mirroringbun:sqlite's,closeInternal()unregisters it, andBun__closeAllNodeSqliteDatabasesForTermination()walks it from the same exit-handler call site. The exit walk filters on the stored VM pointer, so it never dereferences a cell another thread's heap may be sweeping, and is realm-agnostic (node:vmcontexts,bun --hotreload globals). Busy connections —process.exit()called from inside a UDF/authorizer withsqlite3_step()still on the stack — are skipped by both the walker and the destructor, since closing them is the same UAF a busyclose()refuses. On that destructor branch the non-SQLite bookkeeping still runs: each tracked session record is flaggeddbGone(so~JSNodeSqliteSessionnever follows its raw pointer into the now-swept database cell) and the handle is unregistered. Becausesqlite3_close_v2only zombifies a connection that still has un-finalized statements (deferring the checkpoint to a finalize that never comes), the walker first runs an explicitSQLITE_CHECKPOINT_TRUNCATEso the data always lands in the main file; in that case the emptied-wal/-shmfiles still exist, since truly closing would require centrally finalizing statements that~JSStatementSynclater finalizes again. Two spawned regression tests cover the exec-only shape (sidecars removed) and the prepare-held shape (no un-checkpointed data stranded in a-wal).sqlite3_bind_text/blobandsqlite3_result_text/blobtake anintlength, so asize_toverINT_MAXwrapped to a negative or small value (undefined behaviour for the 32-bit API, or silent truncation). Switched to the*64variants, which check the un-truncated length againstSQLITE_LIMIT_LENGTHand fail with a cleanSQLITE_TOOBIG.sqlite3changeset_applyhas no*64variant, so it gets an explicit size guard before the (now provably in-range) narrowing. This has no CI-constructible test — the trigger needs a >2 GiB allocation — and no observable effect for any input SQLite accepts (SQLITE_MAX_LENGTHcaps well below 2³¹); the existing bind/result suite covers every swapped call site.Not vendored on purpose:
test-permission-sqlite-load-extension.js(requires Node's--permissionmodel) andtest-webstorage-without-sqlite.js(covers Node's webstorage global, notnode:sqlite).Binary size
darwin: unchanged (dlopen system libsqlite3, same as bun:sqlite). Linux/Windows: +~200 KB from the six new
SQLITE_ENABLE_*compile defines.macOS caveat
Bun uses the system libsqlite3.dylib on macOS.
loadExtension()andpercentile()/geopoly/rbu (and the session extension on older macOS) require pointing at a full-featured build viaDatabase.setCustomSQLite().Deliberate divergences from Node v26.3.0
Each is commented at its site in
NodeSqlite.cpp:sqlite3_step()exhausts the iterator. Node neither resets nor marks the iterator done, so catching the error and callingnext()again silently re-yields from row 1 (SQLite auto-resets a halted statement). We reset and mark it done.iterate()'sreturn()on a finalized statement is a no-op. Node throwsERR_INVALID_STATE. Throwing there turns a benignfor (r of stmt.iterate()) { db.close(); break; }into an exception, because for-of runs IteratorClose on break.return()only resets the statement if the iterator still owns it. Node resets unconditionally, which rewinds a newer iterator's cursor when one has since taken over the statement.sqlite3_prepare_v3(SQLITE_PREPARE_PERSISTENT)where Node usesprepare_v2. Cached statements are exactly the long-lived reused case the flag documents; the hint is allocator-only and not observable.Test plan
bun test --config=bunfig.node-test.toml ./test/js/node/test/parallel/test-sqlite-session.jspasses (covers the malformed-changeset fix from the SQLite bump)bun test test/js/node/sqlite/node-sqlite.test.tsandbun test test/js/bun/sqlite/pass (regression coverage for the bundled SQLite update and the exit-time close)Fixes #31402
Fixes #20412
(Not marking #24255 as fixed: the
MockTrackeritself landed onmainseparately; this PR only makesTestContext.mockreturn it so the sqlite tests can uset.mock.fn().)no test proof · iteration 5 · Platform-specific test(s) that do not run on this machine. Deferring to CI, which covers all platforms: test/js/bun/sqlite/sqlite.test.js test/js/node/process/process.test.js test/js/node/sqlite/node-sqlite.test.ts