[JSC] Make the bytecode cache format independent of the encoding platform - #389
Draft
dylan-conway wants to merge 2 commits into
Draft
[JSC] Make the bytecode cache format independent of the encoding platform#389dylan-conway wants to merge 2 commits into
dylan-conway wants to merge 2 commits into
Conversation
…form The cache is decoded by Bun executables cross-compiled for a different OS/CPU than the one that produced it, but two things in it depended on the encoder's C++ ABI: - CachedMetadataTable serialized the finalized metadata offset table, whose entries are byte offsets computed from sizeof(Op::Metadata). Those sizes differ between ABIs (everything embedding CallLinkInfo is 16 bytes larger under MSVC), so a table encoded on Linux/macOS and linked on Windows had LLInt indexing past each entry. Serialize per-opcode entry counts instead and lay the table out again on decode via the normal finalize() path. - CachedStringSourceProvider's Bun-only m_sourceLength sat in the base class's tail padding under Itanium but not under MSVC. It didn't validate anything SourceCodeKey doesn't already (the key compares the source hash), so drop it and hand back the provider being decoded against. Also make the encoder's output a pure function of its input so it can be compared across platforms: fixed allocation alignment and page size instead of alignof(max_align_t)/pageSize(), zeroed pages so struct padding is not heap garbage, string switch tables encoded in index order rather than StringImpl-pointer hash order, and UnlinkedHandlerInfo encoded field-wise rather than copied with 30 bits of bit-field slack. The two allocation entry points now static_assert what the compiler can check about a type's portability (alignment, no destructor/array cookie, and for verbatim-copied types: no padding or bit-field slack, no long/wchar_t/long double).
The verbatim-copy check rejected `long` by name to catch the LP64/LLP64 size difference, but int64_t (EncodedJSValue) is `long` under glibc. Only reject `long` where it is the 32-bit outlier.
Preview Builds
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Bun cross-compiles
bun build --compile --bytecodeexecutables, so the JSC bytecode cache is decoded on a different OS/CPU than encoded it. The cache is the in-memory image of theCached*objects, and two parts of it depended on the encoder's C++ ABI:CachedMetadataTablestored byte offsets computed fromsizeof(Op::Metadata)(every opcode embeddingCallLinkInfois 16 bytes larger under MSVC, so a POSIX-built table linked on Windows had LLInt indexing past each entry — a crash inllint_entry), and the Bun-onlyCachedStringSourceProvider::m_sourceLengthlanded at a different offset under MSVC. The table is now serialized as per-opcode entry counts and laid out again by the decoder through the normalfinalize()path;m_sourceLengthis gone (the key already compares the source hash and length).The encoder's output is also made a pure function of its input — fixed allocation alignment/page size, zeroed pages, string switch tables in index order instead of
StringImpl*hash order,UnlinkedHandlerInfoencoded field-wise instead of with bit-field slack — so Bun can assert byte-identical output on every platform in CI. The two allocation entry pointsstatic_assertwhat the compiler can check about a serialized type's portability. All behavior changes are underUSE(BUN_JSC_ADDITIONS).Test plan
In progress — will fill in before marking ready: Release and Debug builds on macOS arm64 are done and Bun built against this decodes its own
--bytecodeoutput (new cross-platform snapshot test stable across runs); currently runningJSTests/stressin disk-cache mode (JSC_diskCachePaththenJSC_forceDiskCache) with the Debug shell against a baseline shell, Linux/Windows builds, cross-OS decode of one executable, and startup/size A/B against the current pin.