Skip to content

Startup snapshots for compiled executables (bun build --snapshot) - #37225

Draft
Jarred-Sumner wants to merge 354 commits into
mainfrom
claude/lowmem-cc
Draft

Startup snapshots for compiled executables (bun build --snapshot)#37225
Jarred-Sumner wants to merge 354 commits into
mainfrom
claude/lowmem-cc

Conversation

@Jarred-Sumner

@Jarred-Sumner Jarred-Sumner commented Aug 9, 2026

Copy link
Copy Markdown
Collaborator

Combined branch, kept open only for its CI matrix. The reviewable form is the stack: #37259 (deps) → #37260 (runtime + Bun.startupSnapshot) → #37261 (bun build --snapshot) → #37262 (tooling); this PR closes once that is in. Depends on oven-sh/mimalloc#13 and oven-sh/WebKit#397; the branch pins to preview artifacts, so it builds as-is.

What it does

bun build --compile --snapshot builds the executable, runs it once, and when startup has drained the runtime writes a snapshot of the started-up process (evaluated modules, everything startup built, JIT code) and embeds it, as is, in the executable. Later launches map the executable's own pages and continue in the event loop instead of booting; process.on('restore') runs first. Docs: docs/bundler/startup-snapshots.mdx.

Surface
bun build --compile --snapshot[=auto|manual] --snapshot-io=strict|local|network auto: taken once startup drains; manual: the app calls take(). Default io policy refuses fs/spawn/dns/net while the snapshot is taken; local/network allow and report every use with call sites. The build also reports env reads, process.env copies and std streams set up before the snapshot. No snapshot → the build fails and says why.
bun build --snapshot --outfile exe The snapshot step alone, on an executable built earlier, in place and re-runnable (cross-compile anywhere, snapshot on the target). The build tells the executable to take its snapshot through a marking in its own payload; nothing goes through env or argv.
Bun.build({ compile, snapshot: true | { mode, io } }) Same thing programmatically; throws without compile.
Bun.startupSnapshot.main(fn) For tools: imports go into the snapshot, fn runs after restore with the launch's own argv/cwd/env/stdio; such a snapshot is valid for any argv and keeps compiled code.
Bun.startupSnapshot.take({ timers, envGate }), isBuildingSnapshot(), epoch(), reclean(), process.on('restore') Manual mode and introspection.
Area Files
Take + restore src/jsc/bindings/StartupSnapshot.{h,cpp}: region dump (adjacent runs merged, JIT pages re-counted after the write), __DATA overlay (Linux: minus the payload), extern-library fixups, argv key, env gate, page-size/build identity, dlopen replay, verbose timing marks
What a resumed process re-derives run_command.rs restore sequence + hooks in io, http, threading, jsc/{event_loop,rare_data,VirtualMachine}, runtime/{timer,dns_jsc,node/*,socket}, bun_core/{startup_snapshot,util,env_var,output}: clocks, PRNG, timers (rebased), DNS cache, dead-fd hangups, env caches, thread pools, fs watchers, stdio (tty flags, termios, colors, Bun.std*, process.std* — terminal streams kept in place and resized, others rebuilt), signal handlers behind process.on(signal)
Env hygiene JSEnvironmentVariableMap, SharedEnvStore.h, BunProcess.cpp: store-backed process.env during the build so it can be refilled in place at restore; build-time report of reads and copies
API src/runtime/api/StartupSnapshotObject.rs, bun.d.ts
Build Arguments.rs, build_command.rs, JSBundler.rs, js_bundle_completion_task.rs, StandaloneModuleGraph.rs (payload marking, embed/re-embed), exe_format/elf.rs (re-injection into an executable that already carries a payload)
Support predicate Compiled in on macOS (with the mimalloc zone override, BUN_MIMALLOC_ZONE_OVERRIDE) and glibc Linux; off under ASAN; stubs elsewhere. bun build --snapshot reports unavailability up front; the tests probe it.
Tooling StartupSnapshotTooling.cpp, compiled only with -DBUN_STARTUP_SNAPSHOT_TOOLING=1
Tests test/js/bun/startup-snapshot/: 21 tests — round trips, embedded restore, GC after restore, timers, spawnSync, RNG/time, DNS, polls, fs.watch, env gate, auto mode, split step, Bun.build, io policy, main(), stdio (three cases), signals, wasm, wasm tier-up race

Numbers

Cost when unused, on matched baselines: JetStream2 at parity (+0.05% on medians, 5 interleaved pairs); allocation-churn peak RSS equal or lower; hello-world +0.1 MB; full JSTests/stress differential against a stock shell identical (5,599 tests). Plain compiled executables with bytecode get ~20 MB smaller because instruction streams alias the embedded cache.

A large interactive application (one real turn, then idle): 228–232 MB / ~1.4–2.1 s CPU plain → 144–158 MB / 0.7–1.1 s from the snapshot; +174 MB binary.

Command-line tools, each built with --bytecode versus the same build plus a snapshot (Linux: Debian 12 arm64 container, CI binary; private memory is RssAnon at exit):

tool wall clock private memory snapshot
babel + preset-env 50.9 → 26.8 ms 38.5 → 20.2 MB 50 MB
postcss + autoprefixer 45.5 → 17.3 ms 32.6 → 17.4 MB 35 MB
sass 36.0 → 14.0 ms 33.1 → 17.2 MB 36 MB
typescript 29.4 → 12.0 ms 32.0 → 16.2 MB 31 MB
prettier (5 plugins) 21.9 → 13.8 ms 20.6 → 11.2 MB 25 MB
markdown-it 9.6 → 7.9 ms 7.7 → 6.6 MB 8 MB
yaml 8.6 → 8.2 ms 7.2 → 7.3 MB 7 MB
esbuild-wasm, module warmed 117 → 24 ms (plain) 135 MB
noop (floor) 2.1 → 2.1 ms (plain) 6 MB

macOS numbers for the same tools are smaller wins on wall clock (a snapshot launch there is one extra exec, whose price is machine-dependent: typescript −20%, babel −37%, autoprefixer −43%, prettier/markdown-it/yaml flat to +20%) with the same 60–65% drop in user CPU; typescript's footprint goes 37 → 24 MB.

Not in this PR

The macOS build option (mimalloc as the process malloc zone) that snapshots need is not enabled in official builds; that is a separate decision. musl support is gated off until its dump crash is debugged. Table evacuation at restore is still unconditional (a memory-first choice that a main()-style build could skip).

…ed heap image is embedded in the __BUN/.bun section payload (pass 2 reuses the exact pass-1 graph bytes; Offsets.image), and the executable restores from itself by mapping regions out of its own file; sidecar .img / BUN_IMAGE_IN kept for debugging; test
…sting compiled executable (pass 2 for Bun.build({compile}) pipelines); Claude Code runs as a single self-restoring file
…ng (fetch gate) so app-driven snapshots can go quiet
…-in (BUN_IMAGE_FRESHARENA); Linux uses the general path (sealed image arenas + hint floor)
…uilder's hint pointer came with __DATA; fresh OS memory must go above the image, never kernel-placed)
…-segment overlay (both paths), before anything can allocate
…ght after the region loop — before the fresh heap is created — else it wipes the allocator state the fresh heap was registered in (Linux)
… hint pointer and is honored when the pointer restarts (fixes fresh reserves hinting MI_HINT_BASE after the overlay on Linux)
…area (it was landing mimalloc's pointer inside WTF OSAllocator's deterministic window)
…defaults, tests, tools); drain JIT plans before delete/freeze; CPU-feature word in the header with a superset check at restore
…v reload allocates (GC's sanitizeStackForVM asserted with the builder's stack top under ASLR)
… ELF headers (which change when the image payload is appended)
…plain pread (ipread adds the embedded-image base offset -> no linker-owned ranges -> GOT overwritten)
… defined in bun_core::image, bumped by the restore sequence) + CPU-dispatch re-probe for highway/simdutf/BoringSSL at restore
…d — network (connect/listen/dns), spawn and node:fs are disabled while an image is being built
Jarred-Sumner and others added 30 commits August 10, 2026 04:38
…ocal set up after the point restore diverges at, so restored processes had none: deep transpiler input crashed; test). main() registered twice is an error in both modes (test); the Bun object is refreshed only if it was ever made; fork-detection generation advances past the builder's on restore
…chine's) instead of the zone frozen in from the build; the date caches are reset either way. Test covers a different TZ and no TZ
…e launch (importing from "bun" reifies everything, so the default S3 client — the builder's credentials — was in practically every snapshot; the stdio blobs' slots are re-armed as well); the builder's secrets are dropped before the refresh so the client is really rebuilt. Test also pins that captured env references see the launch's variables
…test); unlinked-code-block jettisoning applies to compiled executables and snapshot runs only, leaving plain runs as before
… a zone ICU rejects resolves as it does at boot (test builds under a non-UTC zone so every launch shape is distinguishable); a throwing 'restore' listener is reported before main() runs and no longer skips the reclean timer; the -Wformat suppressions are confined to their files
… configurations where the file is only stubs (the previous placement broke musl, FreeBSD and Android builds)
…the blob predicate now yields the class; refused under strict and local; test shows the request used to go out); remaking a reified Bun property that throws for this launch is reported and left undefined instead of leaving an exception that kept 'restore' from firing (test); fixture uses the harness's fill idiom
…t is resident — pointers into an all-zero table exist (found by an allocator update whose page-map tables can be untouched at build time; the first thread started after restore then faulted). Test starts a thread after restore
…hans arming thread is a thread-local re-armed at restore; .env values the builder's environ shadowed reach launches lacking the key; drop dead Linux includes in the tooling file

The strict-I/O sweep had covered the Blob accessors but not delete()/stat(), the S3File prototype's stat, or the S3Client instance methods that Bun.s3 is; the fixture now exercises each family. The arming thread was a ThreadId in a static, overlaid from the builder and never equal to the restored main thread, so restored no-orphans processes spawned unprotected children; a thread-local flag has nothing to overlay and is cheaper per spawn. A .env value shadowed by the builder's environ was discarded at parse time, so a launch without that variable saw undefined where a normal boot reads the file; builders stash those and the restore reload fills in whatever the launch's environ left absent.
…nScope, which is what stops the allocators first
…ing file sheds unused and duplicate includes
…t-watcher scheduler) are re-seated at restore; dns gate moves to the shared resolver helper and covers lookupService
…lears the exception before printing it; trap report names dropped faults; document .env build-time semantics and the burned-key trade-off
…ithout an IPC channel gets none even if the builder had one; tooling: shared freeze index, known sample rate in the report, accurate trap-disabled wording
…ke(); two docs re-anchored to their functions; tooling block sizes are 64-bit throughout
… boot; tooling: last narrowed block size widened
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants