Skip to content

Fix IDBMirrorVFS keeping blocks past the end of a shrunken database - #353

Open
lalexdotcom wants to merge 2 commits into
rhashimoto:masterfrom
lalexdotcom:fix/idb-mirror-block-leak-clean
Open

lalexdotcom wants to merge 2 commits into
rhashimoto:masterfrom
lalexdotcom:fix/idb-mirror-block-leak-clean

Conversation

@lalexdotcom

Copy link
Copy Markdown

What happens

A database that grows and then shrinks keeps the blocks it no longer uses. Grown to 531 pages, then emptied and VACUUMed back to 2, it leaves 531 blocks in IndexedDB for the 2 it uses.

Two places keep them.

#acceptTx truncates the in-memory view from tx.fileSize + blockSize:

let truncated = tx.fileSize + file.blockSize;
while (file.blocks.delete(truncated)) {
  truncated += file.blockSize;
}

The first block past the end of the file starts at tx.fileSize, not one block beyond it, so one block survives every shrink. And because Map.delete returns false on a missing key, the loop stops at the first absent block, so blocks past the end that are not contiguous survive too.

#commitTx writes the transaction's blocks to IndexedDB and deletes none, so everything a transaction wrote before shrinking stays in the store.

The database stays correct: reopening reads the right rows, PRAGMA integrity_check passes, page_count is right. What it costs is storage — a database that repeatedly grows and shrinks holds its high-water mark for ever.

Why deleting from the store is safe

Blocks are not versioned. The key is [path, offset], so a block has exactly one value, and a connection learns about changes through the BroadcastChannel, which carries the block data itself rather than a pointer into IndexedDB. The store is read on open and written on commit.

#getOldestTxInUse guards the tx store, whose records are what lets a connection catch up, and it is untouched here.

The change

#acceptTx truncates from the first block that is actually past the end, and walks the keys instead of stopping at the first gap. #commitTx deletes the range past the end of the file, in the transaction it already has open.

Test

test/vfs_leak.js, wired into test/IDBMirrorVFS.test.js: grow the database well past the page cache, empty it, VACUUM, and compare the number of blocks IndexedDB holds with PRAGMA page_count.

It shrinks the database without rolling anything back, which is what lets it run on master unchanged — a rollback reaches the same truncation, but through a path where the database does not survive (that is #352, and this PR does not depend on it).

Against master it fails with 531 blocks kept for a 2-page database. With the change, the file's tests pass and the full suite is 13 files, 0 failures.

Checklist

  • I grant to recipients of this Project distribution a perpetual,
    non-exclusive, royalty-free, irrevocable copyright license to reproduce, prepare
    derivative works of, publicly display, sublicense, and distribute this
    Contribution and such derivative works.
  • I certify that I am legally entitled to grant this license, and that this
    Contribution contains no content requiring a license from any third party.

lalexdotcom and others added 2 commits September 18, 2026 15:28
Two places keep them. #acceptTx truncates the in-memory view from
tx.fileSize + blockSize, but the first block past the end of the file
starts at tx.fileSize, so one block survives every shrink; its loop also
stops at the first absent block, so scattered blocks past the end
survive as well. And #commitTx writes the transaction's blocks to
IndexedDB without deleting any, so everything a transaction wrote before
shrinking stays in the store.

A database grown to 531 pages and then emptied and VACUUMed keeps 531
blocks for the 2 it uses. The database itself stays correct -- reopening
reads the right rows and integrity_check passes -- so this costs storage
quota rather than data: a database that repeatedly grows and shrinks
holds the high-water mark for ever.

Deleting from the store is safe here because blocks are not versioned:
the key is [path, offset], so a block has one value, and connections
learn about changes through the BroadcastChannel, which carries the
block data itself. IndexedDB is read on open and written on commit.
#getOldestTxInUse governs the tx store, whose records are what a
connection needs to catch up, and it is untouched.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The database is grown well past the page cache, then emptied and
VACUUMed, and the number of blocks IndexedDB holds is compared with
PRAGMA page_count.

It shrinks the database without rolling anything back, so it reaches the
truncation on its own: nothing in it depends on a transaction being
undone, and it runs on master as it stands. Against the previous
behaviour it fails with 531 blocks kept for a 2-page database.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant