Skip to content

Stop allocating on every guest callback, and fix two callback bugs - #123

Merged
syrusakbary merged 4 commits into
mainfrom
perf/guest-callback-path
Oct 28, 2026
Merged

syrusakbary merged 4 commits into
mainfrom
perf/guest-callback-path

Conversation

@syrusakbary

@syrusakbary syrusakbary commented Oct 28, 2026 •

Copy link
Copy Markdown
Member

JavaScript calls into a guest function cost more than they need to: the bridge allocates and frees memory several times per call. Two bugs also sit on the same path. This PR fixes all three, one commit each, after a commit that adds the benchmark used to measure them.

1. Stop allocating on every JavaScript call into the guest

In a profile of a guest function that reads one argument and returns a value (the shape of a small DOM method), about 27% of each call was in malloc and free:

  • Scope frames. Every frame allocated its list of value ids when its first value was stored, and freed it when the frame was popped; each callback opens one. A popped frame now hands its emptied list to the next frame. One PushScopeFrame helper, used by all five places that open a frame, does this. Each env keeps up to 8 lists of at most 256 entries.
  • Callback info. Each callback inserted its napi_callback_info into an unordered_map and erased it on return. Callbacks nest, so a vector used as a stack does the same job. Lookups and removal search from the innermost entry. Ids are still never reused.
  • napi_get_cb_info allocated twice on the Rust side: to read argc, and for the argument ids. Reading argc no longer allocates, and the ids are a SmallVec of 16. smallvec is already in both lock files through Wasmer; this adds it as a direct dependency.

napi_get_cb_info also no longer creates ids for this or the arguments when the guest passes no pointer for them. And the owner-thread check from #107 remembers the env this thread resolved last, so a thread driving one env skips the hash lookup.

tests/perf_calls.rs (new, ignored) runs each case through real Wasmer dispatch, the bridge and V8, with one guest per thread. Numbers are median ns per call over 6 rounds. Builds of main (6545fd6) and of this PR ran alternately. Release build, macOS arm64.

Case Threads main This PR
Guest import (napi_get_value_int32) 1 24.0 23.0
4 26.0 25.7
JavaScript calls an empty guest function 1 90.6 82.7
4 101.2 83.1
JavaScript calls f(x): napi_get_cb_info, napi_get_value_int32, napi_create_int32 1 311.5 207.3
4 358.4 237.8

I measured each piece by building without it. The frame lists (22% of the f(x) call), the callback-info stack (10%), the Rust allocations (9%) and the last-env check (6%; more with 4 threads) each pay for themselves. Two other changes I tried didn't measurably help, so they aren't here: one memory view for all of napi_get_cb_info's writes, and a stack buffer for the C++ argument array.

In the empty-function case, about half of what remains is Wasmer entering and leaving the guest (TypedFunction::call: installing and pausing the store, the trap handler). Going lower needs work in Wasmer.

2. Fill missing callback arguments with undefined

When a guest asked napi_get_cb_info for more arguments than the call passed, the slots past the actual count were left unwritten in guest memory. N-API says they hold undefined, so an addon checking an optional argument with napi_typeof(argv[i]) read garbage.

  • Native: the engine already fills those slots. The bridge now hands all of them to the guest, with one id for undefined that every missing slot shares.
  • Host-JS: the bridge fills them the same way. It also now reports the actual argument count in *argc, as N-API does, instead of the smaller of the actual and requested counts. Its guest layer stops creating argument ids when the guest passes no argv, and writes argv in one memory write.

3. Escape a callback's return value before closing scopes it left open

When a guest callback returned, the bridge first closed the scopes the guest had left open, then escaped the return value. If the value was created in one of those scopes, it was read from a closed scope whose handle block V8 may already have freed. It worked only because the freed memory still held the value. The value is now escaped first. The escape slot belongs to the scope around the callback's, so closing the inner scopes and then the callback's own leaves it intact.

Tests

  • tests/guest_callbacks.rs (new) runs JavaScript calling guest functions through real Wasmer dispatch and V8:

    • callbacks nested 40 deep through JavaScript, each reading its own callback info again after the inner ones have returned;
    • scopes a callback leaves open are closed, and ids created in them stop resolving;
    • 3, 16, 17, 18 and 20 arguments;
    • every out pointer of napi_get_cb_info null;
    • envs created and released one after another on one thread;
    • missing_arguments_are_undefined checks argc and the type of the extra slots. It fails on main.
    • a_return_value_from_a_scope_left_open_survives returns a value created after 3,000 others in a scope the callback leaves open. On main it passes by luck, and crashes when freed memory is overwritten (MallocScribble=1 on macOS). With this PR it passes either way.
  • A wasm-bindgen test checks the host-JS argv and argc for calls with fewer and with more arguments than requested.

  • Results (macOS arm64):

    Suite Result
    cargo test --lib 54 passed
    --test concurrent_envs 10 passed
    --test guest_callbacks 4 passed, also with MallocScribble=1
    Host-JS snapi_js::tests (wasm-bindgen) 11 passed
    Manifest tests, native + WASIX 36 programs, same output
    cargo clippy --lib + the new tests no warnings
  • Each commit builds and passes the tests that exist at that point. Two independent reviews (of the first version and of this one) found no bugs; their suggestions are applied.

CI needs one more line, which I couldn't push (my token lacks the workflow scope). napi-native-concurrency.yml runs only concurrent_envs. To also run the new tests, add - "tests/guest_callbacks.rs" to both paths lists and change the test step to:

        run: ./cargo-standalone.sh test --locked --test concurrent_envs --test guest_callbacks
        env:
          MALLOC_PERTURB_: 165

MALLOC_PERTURB_ makes glibc overwrite freed memory, like MallocScribble on macOS, so a regression of fix 3 would fail on Linux too. I couldn't verify this on Linux.

Found while testing, not changed here

These happen the same way on main:

  • Releasing an env from inside one of its own callbacks crashes. The isolate is gone when the provider's FunctionTrampoline resumes. DisposeBridgeState should refuse the release while a guest callback of that env is running, returning the existing refused status.
  • Two live envs on one thread crash. Running JavaScript in one env while another env created later on the same thread is alive crashes V8 with a SIGSEGV in Isolate::main_thread_local_heap. Each env keeps a persistent Isolate::Scope.

🤖 Generated with Claude Code

syrusakbary and others added 4 commits October 27, 2026 19:07
tests/perf_calls.rs is an ignored test that times, through real Wasmer
dispatch, the bridge and V8, with 1, 2 and 4 guests on their own
threads:

- a guest N-API import (napi_get_value_int32, napi_get_undefined);
- JavaScript calling an empty guest function;
- JavaScript calling a guest function that reads one argument and
  `this`, and returns a new int32, like a small DOM method.

Run it with
`./cargo-standalone.sh test --release --test perf_calls -- --ignored --nocapture`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Profiling JavaScript calls into a guest function that reads one argument
and returns a value showed about 27% of the time in malloc and free, all
from bookkeeping the bridge redoes on every call:

- Every scope frame allocated its list of value ids when its first value
  was stored and freed it when the frame was popped. Popped frames now
  give their emptied list to the next frame, through one PushScopeFrame
  helper that all five places opening a frame use. Each env keeps up to
  8 lists of at most 256 entries.
- Each callback inserted its napi_callback_info into an unordered_map
  and erased it on return. Callbacks nest, so a vector used as a stack
  does the same; lookups and removal search from the innermost entry.
- napi_get_cb_info allocated for the argc read and the argument ids on
  the Rust side. The argc read no longer allocates, and the ids are a
  SmallVec of 16 (smallvec is already in the lock files through Wasmer).

napi_get_cb_info also no longer mints ids for `this` or the arguments
when the guest passes no pointer for them. And the owner-thread check
remembers the env the thread resolved last, so a thread driving one env
skips the hash lookup.

tests/perf_calls.rs, median ns per call (release, macOS arm64):

| Case                               | Threads | Before | After |
|------------------------------------|---------|--------|-------|
| Guest import                       | 1       | 24.0   | 23.0  |
|                                    | 4       | 26.0   | 25.7  |
| JavaScript calls an empty function | 1       | 90.6   | 82.7  |
|                                    | 4       | 101.2  | 83.1  |
| JavaScript calls f(x)              | 1       | 311.5  | 207.3 |
|                                    | 4       | 358.4  | 237.8 |

tests/guest_callbacks.rs checks the paths this changes: callbacks
nested 40 deep through JavaScript that re-read their own callback info,
scopes a callback leaves open, argument counts on both sides of 16, null
out pointers, and envs created and released one after another on one
thread.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When a guest asked napi_get_cb_info for more arguments than the call
passed, the slots past the actual count were left unwritten in guest
memory. N-API fills them with undefined, so an addon that checks an
optional argument with napi_typeof(argv[i]) read whatever was there.

- Native: the engine already fills those slots with undefined; the
  bridge now hands all of them to the guest, minting one id for
  undefined that every missing slot shares.
- Host-JS: the bridge fills them the same way, and now reports the
  actual argument count in *argc, as N-API does, instead of the smaller
  of the actual and requested counts. Its guest layer also stops minting
  argument ids when the guest passes no argv buffer, and writes argv
  with one memory write.

Tests: missing_arguments_are_undefined (tests/guest_callbacks.rs) fails
on main, reading the unwritten slot; a wasm-bindgen test checks the
host-JS argv and argc for calls with fewer and more arguments than
requested.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When a guest callback returned, the bridge first closed the scopes the
guest had left open and then escaped the return value. If the value was
created in one of those scopes, it was read from a closed scope, whose
handle block V8 may already have freed. It worked only because the freed
memory still held the value.

Escape the value first: the escape slot belongs to the scope around the
callback's, so closing the inner scopes and then the callback's own
leaves it intact. This only runs while the callback's frame is open.

a_return_value_from_a_scope_left_open_survives returns a value created
after 3000 others in a scope the callback leaves open. On main it passes
by luck, and crashes when freed memory is overwritten (MallocScribble=1
on macOS); with this change it passes either way.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@syrusakbary
syrusakbary force-pushed the perf/guest-callback-path branch from fa4b988 to 3282d92 Compare October 28, 2026 02:53
@syrusakbary syrusakbary changed the title Stop allocating on every JavaScript call into the guest Stop allocating on every guest callback, and fix two callback bugs Oct 28, 2026
@syrusakbary
syrusakbary merged commit d109823 into main Oct 28, 2026
11 checks passed
syrusakbary added a commit that referenced this pull request Oct 28, 2026
Brings in #123 (faster guest callback path). Scope frames and callback
invocations live on the agent on this branch, so #123's pool of emptied
frame buffers and its stack of callback invocations do too.

PushScopeFrame, now the one helper that opens a frame, pushes onto the
agent's stack, reuses a pooled buffer and records the env that opened
the scope, as PushFrame did. PopCurrentFrame returns buffers to the
agent's pool and still closes each scope through the env that opened it.
generic_wasm_callback escapes the return value before closing the scopes
the guest left open, on the agent's frames.

host-JS: #123's get_cb_info and its test reach the handle table and
callback infos through `state.agent`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
syrusakbary added a commit that referenced this pull request Oct 28, 2026
Brings in #123, #108 and #124 through Phase 0.

Rust: #108 allocates native env ids per application, so a realm env takes
its id from the application's registry too, and never shares one with
another store's env. It still gets no registry entry. Terminate resolves
through the registry since #108, so terminating one of the caller's realm
envs goes through its principal env, which stops every env of the
isolate as before.

host-JS: releasing an env first asks whether guest code is running
(#124), then releases its realm envs as Phase 1 does through free_env.

The bridge needed no change: #124's count of running guest calls is per
agent, so it covers realm envs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant