mimalloc: keep the idle sweep's state on the tld instead of in __thread variables (fixes Android SIGSEGV)

[robobun]

Superseded as a patch by oven-sh/mimalloc#17, at the maintainers' request. This PR stays as a draft and will turn into the pin bump once that one is merged; the analysis below still describes the bug and the change.

Problem

• Android arm64 builds SIGSEGV intermittently (30-70% of runs of bunmicro -profile) since the mimalloc pin bump in Update mimalloc to the upstream dev3 (v3.4.3) sync #36431 (Intermittent SIGSEGV on Android (arm64) since mimalloc dev3/v3.4.3 sync (#36431) #38051).
• Reproduced on linux-x64 by building mimalloc the way the Android target builds it (see below). The crash is a stack overflow from unbounded recursion; every level looks like this:

  _mi_malloc_generic (size=16)                       page.c:1963
  __emutls_get_address
  mi_page_free_collect_ex                            page.c:1213   <- reads the __thread sweep guard
  _mi_page_free_collect / mi_page_queue_find_free_ex page.c:1754
  mi_find_free_page / mi_find_page                   page.c:1865 / 1951
  _mi_malloc_generic (size=16)                       page.c:2001
  __emutls_get_address
  ...
  

• Cause: the fork's hole-purging sweep keeps its re-entrancy guard and per-sweep counters in __thread variables (vendor/mimalloc/src/page.c:478-489) and reads the guard from mi_page_free_collect_ex (page.c:1213), i.e. from inside the allocator. Bun's Android target is aarch64-linux-android28, and for API < 29 clang lowers every __thread to emulated TLS: __emutls_get_address materializes a thread's variables on first access with malloc(), which on that target is mimalloc itself. Clang also speculates the read ahead of the last word of the mi_page_has_purged check, so it fires the first time a thread collects any page with an empty free list. If the size class emulated TLS asks for (128 bytes for its per-thread array, 16 for the flag) is itself sitting on an exhausted page at that moment, serving it collects that page, reads the still-unmaterialized guard again, and recurses until the stack is gone.
• Why Update mimalloc to the upstream dev3 (v3.4.3) sync #36431 exposed it: the previous pin wrote another __thread variable (__mi_theap_main) during thread init, while the thread had no pages yet, which materialized each thread's emulated-TLS state at a harmless moment (confirmed with a breakpoint on __emutls_get_address: malloc(128) + malloc(23) from _mi_theap_default_set). The dev3 sync removed that variable, so the first emulated-TLS allocation moved into the collect path. glibc/musl/macOS/Windows use native TLS and are unaffected.

Fix

• patches/mimalloc/sweep-state-on-tld.patch (applied through the existing patches: mechanism in scripts/build/deps/mimalloc.ts; robobun cannot push to oven-sh/mimalloc, so this is carried the same way earlier mimalloc fixes were until the pin moves): the state of a running sweep moves onto the mi_tld_t being swept (holes_sweeping, holes_sweep_full, holes_sweep_skipped, holes_sweep_visited, appended to the zero-initialized tail of the struct). _mi_page_purge_holes_begin/end and _mi_page_purge_holes take that tld; both callers (theap.c, arena.c) already have it.
• _mi_page_purge_holes_in_progress() now answers "is the calling thread inside a sweep of its own heaps" by reading _mi_theap_default()->tld->holes_sweeping. The default theap is the one thread-local every TLS model reads without allocating (a pthread key on Android). On the owner path this is exactly the old semantics; on the scavenger path the guard only ever protected the scavenger's own (never swept) pages, so reading false there is harmless.
• The sweep's own collects (theap.c, arena.c) call _mi_page_free_collect_no_unpurge explicitly instead of relying on the guard, so the "do not un-purge what we are about to purge" behavior no longer depends on thread-local state at all.
• Result: the only __thread variable mimalloc still touches on Android is the message-output guard in options.c (recurse), which is only reached when mimalloc prints a warning or error. mi_page_free_collect_ex contains no __emutls_get_address call; the binary references it only from mi_recurse_enter_prim/mi_recurse_exit_prim.
• Verification (all on linux-x64, mimalloc built as the Android target builds it: -femulated-tls, MI_MALLOC_OVERRIDE, MI_TLS_MODEL_PTHREADS, compiler-rt's emutls):
  • reproducer below: current pin SIGSEGV 10/10, previous pin (acd9924a) ok 10/10, current pin + this patch ok 10/10
  • thread-churn stress (threads creating/destroying heaps, parking, TLS destructors allocating after thread exit): current pin 13/60 SIGSEGV, previous pin 0/60, patched 0/60
  • native TLS, both MI_TLS_MODEL_LOCAL and MI_TLS_MODEL_PTHREADS, release and MI_DEBUG=3: a sweep discards 6 hole runs and the following allocations hand all 6 back (reuse_calls=6, purged_blocks=0), identical numbers before and after the patch; the stress runs clean under the new assertions
  • bun bd (MI_DEBUG=3) builds with the patch applied; test/js/bun/jsc/heapStats-mimalloc.test.ts passes; a script that churns memory on the main thread and a Worker runs clean with the scavenger sweeping the parked threads, with MIMALLOC_SCAVENGER=0 (inline owner sweeps, 36 passes observed via breakpoints), and with MIMALLOC_PURGE_HOLES_MIN_INTERVAL=0 MIMALLOC_PURGE_HOLES_FULL_EVERY=1; process.versions.mimalloc is unchanged
• No test/ change: every CI platform uses native TLS, where the bug is unreachable, so nothing under bun test can fail before and pass after. The C reproducer below is the regression test; it needs a from-source build of mimalloc with -femulated-tls. @jjtseng93, if you can run your 30x bunmicro -profile loop on an Android build of this branch, that would confirm it on the real target.
• mimalloc: fold every thread's theap into the subproc stats aggregate #34739 also adds a patches: entry to mimalloc.ts; whichever lands second needs a one-line merge.

Background

• Emulated TLS: for targets without native ELF TLS (Android before API 29), the compiler turns each __thread variable into a descriptor and replaces every access with a call to __emutls_get_address (compiler-rt), which keeps a per-thread array in a pthread key and allocates both the array and each variable's storage with malloc() the first time a thread touches them. Native TLS is a register-relative load and never allocates, which is why the same code is fine on every other Bun target.
• theap / tld: in this mimalloc a thread's state is a mi_tld_t (one per thread) owning one mi_theap_t per heap the thread allocates from; pages belong to a theap.
• Idle sweep (fork feature): mi_on_thread_idle() walks a thread's pages and discards ("purges") the memory of free blocks that fill whole OS pages, taking those blocks off the free lists. When a page with purged holes later runs out of free blocks, mi_page_free_collect_ex brings one run back (_mi_page_unpurge_run). The guard exists so that a mi_malloc nested inside the sweep itself (reachable from a warning's output hook) does not un-purge a page the sweep is in the middle of rewriting.
• Scavenger: a background thread; Bun's event loop parks via mi_on_thread_idle_start() around epoll_wait, and the scavenger runs the sweep for parked threads. That is why the sweep already carried the target tld everywhere, and why per-thread state of the sweeping thread was the wrong place for this data even on native TLS.

Reproducer (linux-x64, no device needed)

repro.c:

#include <pthread.h>
#include <stdio.h>
#include <stdlib.h>
#define EXTEND 4096   // mimalloc formats one OS page worth of blocks at a time
static void* victim(void* arg) {
  (void)arg;
  static void *a128[EXTEND / 128], *a16[EXTEND / 16], *a8[EXTEND / 8 + 1];
  // use up exactly the formatted blocks of a 128-byte page (emulated TLS's per-thread array)
  // and a 16-byte page (the guard's own storage); nothing has been collected yet
  for (size_t i = 0; i < EXTEND / 128; i++) a128[i] = malloc(128);
  for (size_t i = 0; i < EXTEND / 16;  i++) a16[i]  = malloc(16);
  // exhaust an 8-byte page and ask for one more: collects it -> first guard access on this thread
  for (size_t i = 0; i < EXTEND / 8 + 1; i++) a8[i] = malloc(8);
  for (size_t i = 0; i < EXTEND / 8 + 1; i++) free(a8[i]);
  for (size_t i = 0; i < EXTEND / 16;  i++) free(a16[i]);
  for (size_t i = 0; i < EXTEND / 128; i++) free(a128[i]);
  return NULL;
}
int main(void) {
  pthread_t t;
  pthread_create(&t, NULL, victim, NULL);
  pthread_join(t, NULL);
  puts("ok");
  return 0;
}

Build against a mimalloc tree (vendor/mimalloc after bun bd has the patch applied; a plain extract of the pin does not):

clang++ -x c++ -std=gnu++20 -O2 -DNDEBUG -DMI_BUILD_RELEASE -DMI_CMAKE_BUILD_TYPE=release \
  -DMI_STATIC_LIB -DMI_SKIP_COLLECT_ON_EXIT=1 -DMI_NO_PROCESS_DETACH=1 -DMI_DEFAULT_ALLOW_THP=0 \
  -DMI_MALLOC_OVERRIDE -fno-builtin-malloc -DMI_TLS_MODEL_PTHREADS=1 \
  -femulated-tls -fPIC -pthread -I$MI/include -c $MI/src/static.c -o mimalloc.o
clang -O1 -pthread -c repro.c -o repro.o
clang++ -pthread --rtlib=compiler-rt repro.o mimalloc.o -o repro   # --rtlib=compiler-rt: emutls from libclang_rt, as on the NDK
./repro

(MI_TLS_MODEL_PTHREADS is what prim-tls.h selects for __ANDROID__; the previous pin spells it MI_TLS_MODEL_DYNAMIC_PTHREADS.) nm repro | grep __emutls_v lists the __thread variables that ended up emulated: 6 on the current pin, 2 after the patch (recurse and the never-read __mi_thread_id_helper).

Results: current pin, exit 139 x10; previous pin, ok x10; patched, ok x10.

---

no test proof · iteration 0 · docs-only change; test-proof not applicable

[coderabbitai]

Warning

Review limit reached

@robobun, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 13 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 8126e6a6-5613-4797-b4cf-39414cab7b0b

📥 Commits

Reviewing files that changed from the base of the PR and between b7a0431 and f4817d4.

📒 Files selected for processing (2)

• patches/mimalloc/sweep-state-on-tld.patch
• scripts/build/deps/mimalloc.ts

---

_{Comment @coderabbitai help to get the list of available commands.}

[robobun]

^{Updated 5:05 AM PT - Aug 13th, 2026}

❌ @robobun, your commit f4817d4 has some failures in Build #94635 (All Failures)

---

🧪   To try this PR locally:

bunx bun-pr 38168

That installs a local version of the PR into your bun-38168 executable, so you can run:

bun-38168 --bun

[claude]

I reviewed this PR and didn't find any bugs. Because it patches the process-wide allocator's multi-threaded sweep path (including a semantic change to how _mi_page_purge_holes_in_progress() reads its guard on the scavenger thread), a human familiar with the fork's hole-purging feature should sign off.

Checked: patch applies cleanly to the pinned commit; every caller of _mi_page_purge_holes/_begin/_end is updated; _mi_page_free_collect_no_unpurge already exists in the pin.
The new mi_tld_t fields land in the zero-initialized tail — tld_empty/mi_process_tld_main positional initializers stop at memid, so zero (false/0) is correct for all four.
The scavenger-path guard weakening is compensated by the explicit _no_unpurge collects in theap.c/arena.c; a nested malloc on the scavenger only touches its own theaps, not the parked thread's pages.

Extended reasoning...

Overview

Adds a vendored patch to oven-sh/mimalloc (applied via the existing patches: mechanism in scripts/build/deps/mimalloc.ts) that moves four __thread variables backing the fork's idle hole-purging sweep — mi_purging_holes, mi_holes_sweep_full, mi_holes_sweep_skipped, mi_holes_sweep_visited — onto the mi_tld_t being swept, and threads that tld through _mi_page_purge_holes, _mi_page_purge_holes_begin/end, and mi_page_purge_holes_walk. The two sweep-internal collects switch from _mi_page_free_collect to the existing _mi_page_free_collect_no_unpurge. Root cause: on Android API < 29, __thread compiles to emulated TLS, whose first-touch allocation re-enters mimalloc from inside mi_page_free_collect_ex, causing unbounded recursion.

Security risks

None identified. This is an internal allocator refactor with no attacker-controlled inputs; the state relocation does not widen any trust boundary.

Level of scrutiny

High. This is the process-wide memory allocator, and the change touches concurrent state shared between an owner thread and a background scavenger. The scavenger-path behavior of _mi_page_purge_holes_in_progress() changes (it now reads the calling thread's own tld via _mi_theap_default(), so on the scavenger it returns false while sweeping a parked tld). The PR argues this is safe because (a) the sweep's own collects no longer depend on the guard, and (b) a nested malloc on the scavenger allocates from the scavenger's own theaps, not the pages under sweep. That argument reads correctly to me and I traced it through page.c:1213 and both callers, but it is exactly the kind of invariant a maintainer of this fork feature should confirm.

Other factors

• Verified the patch applies cleanly (patch -p1 --dry-run) against the pinned source; no fuzz, no offsets.
• Checked init.c: tld_empty and mi_process_tld_main initialize positionally through memid and rely on zero-init for the tail, which is correct for the four new fields.
• Confirmed the only remaining mi_decl_thread in page.c post-patch is gone (all four removed); _mi_page_holes_count_page_freed/_ineligible use process-wide atomics directly, not thread-locals.
• The removed parg->tld != NULL check in mi_arena_page_purge_holes_at is safe: the sole caller now early-returns on tld == NULL, and the only call site (theap.c:238) always passes a non-NULL tld.
• No automated test (justified: no CI target uses emulated TLS); the PR provides a linux-x64 reproducer and requests on-device confirmation.

[robobun]

For whoever does the sign-off on the hole-purging side, the one semantic change is what _mi_page_purge_holes_in_progress() returns on the scavenger thread, so here is the invariant to check (line numbers are the pinned, unpatched sources):

• Before the patch the guard had exactly three readers: the sweep's own two collects (theap.c:174, arena.c:1357) and the nested-allocation check in mi_page_free_collect_ex (page.c:1213). The patch makes the first two _mi_page_free_collect_no_unpurge, the same call the pre-sweep collect already uses (theap.c:107), so they no longer depend on the guard at all. page.c:1213 is the only reader left.
• Owner path: mi_on_thread_idle sweeps _mi_theap_default()->tld (theap.c:286), which is the tld _begin marks, so a nested allocation on the owner reads back the same flag the old __thread variable held. No change in behavior.
• Scavenger path: the flag is set on the parked tld, and a nested allocation on the scavenger reads its own (absent) tld, so it gets false. What it could un-purge is limited to pages that allocation can reach: the scavenger's own theap pages, since the parked thread's pages are still owned by that thread and are not in the abandoned map, and every abandoned page the arena pass touches is held out of the map and owned for the duration of its purge (arena.c:1347-1350), so a reclaim on the scavenger cannot pick it up. Nothing the sweep is rewriting is reachable from there.
• The MI_DEBUG build asserts tld->holes_sweeping inside _mi_page_purge_holes, i.e. that begin/end bracket the tld actually being swept on both paths. That assertion held in the thread-churn stress linked from the description and in bun bd runs with both the scavenger sweeping parked threads and MIMALLOC_SCAVENGER=0 inline sweeps.

Nested allocation inside a sweep is only reachable through a user output function on a mimalloc warning in the first place; on native-TLS targets the change is otherwise behavior-preserving (same discard and reuse counts before and after, see the description).

[jjtseng93]

Confirmed

• On real Android/Bionic hardware (not emulated) — this fixes it.

Setup: an actual Android phone, no VM/emulator involved. The cross-compile toolchain (LLVM 21 + NDK r27c, bun scripts/build.ts --profile=android-release --android-ndk=...) runs inside a proot (chroot-like, Debian) userland on that same physical device. The resulting aarch64-linux-android28 binary was run both from inside that proot shell (syscalls still going through proot's ptrace-based translation) and, to rule that out as a confound, from a completely native Termux shell reached over a local websocket bridge (jsgotty) — no proot involved at all, confirmed via whoami returning a real Android app UID (u0_a512) rather than the proot user.

Built patches/mimalloc/sweep-state-on-tld.patch via the patches: mechanism with that pipeline, on top of two different Bun trees: the commit that originally introduced the regression (c08f665367, #36431) and this PR's own base (main @ b7a0431032). Ran the bunmicro -profile loop plus a wasmoon-only.js script I isolated down from it (below) on-device every time:

Build	Execution context	bunmicro -profile	wasmoon-only.js
c08f665367 (the original pin bump, unpatched)	proot	30-70% crash (baseline)	16/30 crash
c08f665367 + this patch	proot	0/30	0/60 (ran twice)
main @ b7a0431032 (this PR's base) + this patch	proot	0/100	0/100
main @ b7a0431032 (this PR's base) + this patch	native, no proot	0/100	0/100

Before applying, checked whether the patch's target files (page.c, theap.c, arena.c, internal.h, types.h) differ between the two mimalloc pins (d078ad06 vs 1803341d) — they don't, so the patch applies with identical semantics on both. git apply applied cleanly with no rejects/fuzz in both builds.

wasmoon-only.js loads wasmoon@1.16.0 — same version bunmicro itself pins ("wasmoon": "^1.16.0" in its package.json, resolving to 1.16.0), isolated down to just the Lua VM instantiation so it's not tangled up with anything else bunmicro -profile touches:

const wasmoon = await import("wasmoon"); // wasmoon@1.16.0
const factory = new wasmoon.LuaFactory();
const lua = await factory.createEngine();
lua.doStringSync("x = 1 + 1");
console.log("lua ok, x =", lua.global.get("x"));
lua.global.close();

Thanks for tracking this down — the emulated-TLS-recursion explanation matches everything I was seeing.

[Jarred-Sumner]

Robobun push this as a PR to our mimalloc fork instead

[robobun]

Done: oven-sh/mimalloc#17 (against bun-dev3-v2; it also applies cleanly on top of claude/sync-dev3, so it can land before or after #15). It carries the same change plus a regression test in the fork's suite, test-emulated-tls, which builds the allocator once more with -femulated-tls as malloc and fails three different ways on the current branch (stack overflow in optimized builds, stack overflow in unoptimized builds, scavenger assertion in debug builds) and passes with the change. I could not assign it, so it is not assigned to anyone.

Converted this PR to a draft. Once #17 is merged this becomes the pin bump (drop patches/mimalloc/, move MIMALLOC_COMMIT, update the sha in test/js/node/process/process.test.js); feel free to close it instead if you would rather do the bump yourself.

[Jarred-Sumner]

Folded into #37367: its pin (oven-sh/mimalloc be7eb3ff1) has this fix in the fork itself, so the .patch stop-gap isn't needed.

[robobun]

Confirmed: be7eb3ff1 carries the merged oven-sh/mimalloc#17 (tld-held sweep state, no thread-locals left in page.c, and test-emulated-tls is in the fork's suite), so #37367 delivers the fix and nothing from this PR is needed. Repointed the #38051 status at #37367.