feat: libsignal wasm - #2067
Open
jlucaso1 wants to merge 75 commits into
Open
Conversation
Contributor
|
Thanks for opening this pull request and contributing to the project! The next step is for the maintainers to review your changes. If everything looks good, it will be approved and merged into the main branch. In the meantime, anyone in the community is encouraged to test this pull request and provide feedback. ✅ How to confirm it worksIf you’ve tested this PR, please comment below with: This helps us speed up the review and merge process. 📦 To test this PR locally:If you encounter any issues or have feedback, feel free to comment as well. |
jlucaso1
force-pushed
the
feat-libsignal-wasm
branch
from
November 16, 2025 00:33
cc9ae9b to
779ad19
Compare
|
I'm following along and testing it out 🫡🙂 |
Salientekill
pushed a commit
to Salientekill/Baileys
that referenced
this pull request
Nov 17, 2025
- Add WebAssembly version of libsignal - Improve performance with WASM implementation - Update dependencies for libsignal WASM support 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Member
|
Testing in scale this week |
Salientekill
pushed a commit
to Salientekill/Baileys
that referenced
this pull request
Nov 19, 2025
…ets#1969 to latest Baileys Applied PRs: - WhiskeySockets#2067: libsignal wasm - WhiskeySockets#2057: emit setting events - WhiskeySockets#1969: improve retry logic Note: PRs WhiskeySockets#1991, WhiskeySockets#1981, WhiskeySockets#1906, WhiskeySockets#1892 have conflicts with latest Baileys version and were skipped. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Collaborator
Author
|
Good news: The PR is ready for testing (no need to delete sessions or sender keys anymore, this will be auto migrated, only in case reverting this is necessary) |
jlucaso1
marked this pull request as ready for review
November 20, 2025 06:52
Salientekill
pushed a commit
to Salientekill/Baileys
that referenced
this pull request
Nov 20, 2025
… v0.4.0-alpha.3 Updated libsignal WASM implementation with latest whatsapp-rust-bridge version. Changes: - Updated whatsapp-rust-bridge: 0.4.0-alpha.2 → 0.4.0-alpha.3 - Core WASM implementation was already in place from previous PR - Provides 10x faster message encryption/decryption performance Performance benefits: - Native-speed cryptographic operations (WASM) - Efficient group message handling - Better memory management Build and dependencies verified successfully. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Contributor
|
I've tested this in a production environment. There are several cases where the bot suddenly stops receiving upsert messages, and it only starts working again after rescan. I'm still monitoring it, but for now that's the main issue I've encountered. |
It's not due to that PR, try implementing that PR and see if it continues: #2070 |
|
It was merged but the purpshell reverted. |
Contributor
Ah, I see. That was my mistake. another one of my instances running v7.0.0-rc8 had the same issue, so it wasn’t related to this PR. |
|
This implementation is not stable, after testing it, I noticed that Then I continue to keep getting
|
Collaborator
Author
This occurs in a fresh or existing session? Is there any error log that you can share? |
Contributor
|
I'm getting this error when using an existing (old) session in this pr. The issue disappears when I revert back to v7.0.0-rc6. |
Collaborator
Author
Thank you, I'll check soon |
Occurs only on existing session I'm yet to test on a fresh session |
Collaborator
Author
|
@4relial @astrox11 Please test again with this updated version (I've just improved the error logging, will be more precise to identify what is wrong). And thank you guys for helping <3 |
Salientekill
pushed a commit
to Salientekill/Baileys
that referenced
this pull request
Nov 21, 2025
… v0.4.0-alpha.4 Updated to latest whatsapp-rust-bridge version released 14 minutes ago. Changes: - whatsapp-rust-bridge: 0.4.0-alpha.3 → 0.4.0-alpha.4 Maintains 10x faster WASM-based cryptographic operations. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Member
we need to handle it differently. |
Member
|
Would love if the logging architecture was implemented the same way |
Member
Related to this @Salientekill (even though this is off topic) check out rc9 |
Ksksksks I answered you discord already (shiroe) |
Member
@jlucaso1 This was the same .unwrap() kind of error that took cloudflare offline, be careful man 😆 |
Collaborator
Author
Yeah hahaha. I've just searched and removed all .unwrap from code |
Every field of a sender-key state has a v1 counterpart, so unlike a session this conversion loses nothing: the group format already persists seeds rather than derived material. The record is exposed as a function instead of being applied inside store_sender_key, so the bridge keeps writing native bytes and the caller decides what shape its storage keeps. Reading a legacy record also stopped carrying an empty private signing key as present. The JS backend wrote an empty Buffer for every sender key received from someone else, and Some(empty) fails validation the moment anything reads the record's components.
The last row that still diverged from what an older release reads. With this, rolling back keeps groups working instead of needing every sender key redistributed, verified by running a group through this branch and then handing the storage to develop: it read the pending message, kept sending, and adopted a new key without any conversion step. Also drops the empty record written before processing a distribution message. The builder creates one when the row is absent, and that pre-write was the only remaining path storing a sender key outside the policy above.
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
All reported issues were addressed across 4 files (changes from recent commits).
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
The JS backend wrote senderSigningKey.private as an empty Buffer for every sender key received from someone else, so omitting the field made the row distinguishable from one it produced. It reads either shape, but the point of this projection is to be indistinguishable, not merely readable. The rollback suite's header still claimed sender keys had no projection and could not be read by the old build, which the tests below it now disprove.
A CPU profile of the release artifact shows wasm-function[N]: the release profile strips, and wasm-opt drops the name section. wasm-pack's own --profiling still builds the release profile, so `pnpm build:profile` drives cargo and wasm-bindgen directly, skips wasm-opt, and resolves the wasm-bindgen matching the crate's schema from wasm-pack's cache. The release path is untouched: same flags, same output size.
Contributor
There was a problem hiding this comment.
All reported issues were addressed across 3 files (changes from recent commits).
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
Falling through to a global wasm-bindgen is worse than stopping: it is almost never the version the crate was linked against, and the error it produces talks about schema numbers instead of what to do. It now names the version and the two ways to get it. The wasm-pack metadata profile went with it. That path drives cargo directly, so the section was never read.
A stored sender key is legacy JSON, and the buffers inside it can be written two ways: as a byte array by plain JSON.stringify, or as base64 text by Baileys' own BufferJSON.replacer. The decoder only accepted the array, so a store that round-trips its rows through BufferJSON read every key as absent. Absent then defaulted to empty, which built a record with no chain seed and no signing key. That record fails to deserialize, and the failure surfaced as "protobuf encoding was invalid" with nothing pointing at the row it came from. Accept both shapes, and stop defaulting the fields the record cannot be built without so a bad row names itself instead of failing later.
The core keeps the current state at the front of the record and prunes from the back. The legacy shape is the mirror of that: the JS backend reads the last entry as the current state and drops the first one when the record overflows. Both conversions carried the core order through unchanged, so a record with more than one state was written upside down. Rolling back to a pre-WASM release would send under a stale key, and the sixth distribution message would evict the freshest state instead of the oldest. Reverse on the way out and on the way back in.
The error told the reader to install the exact wasm-bindgen-cli and put it on PATH, but the lookup only scanned the wasm-pack cache. Following that advice on a clean machine hit the same throw, leaving build:profile unusable until an unrelated normal build populated the cache. Scan PATH after the cache, with the same exact-version check, so a mismatched global build still cannot slip in.
mitata already defaults to collecting once before a benchmark; asking for "inner" collects between every iteration instead. That penalises the WASM side out of proportion, because each collection also drains the FinalizationRegistry that frees wasm-bindgen handles, and the cost lands inside the measurement. The numbers were not just noisy, they pointed the wrong way: encodeNode read as 11.45x slower than the JS encoder and measures 2.24x faster without the forced collection, and decodeNode goes from 11.98x slower to 2.56x faster.
The core leases 64 outbound counters ahead of durability so the send path only needs a flush once per batch. Carrying that ceiling forward means persisting it, and the record shape we write has nowhere to put it: the component export materializes the reservation, so the whole batch burned on every operation instead of once per batch. Consecutive sends landed on the wire at counters 0, 64, 128, 192, and the peer derived 63 skipped keys for each one. After 20 group messages the stored row was 199,724 bytes and a decrypt cost 3,488 us against 97 us for the first; a DM row passed MAX_LEGACY_MESSAGE_KEYS around the 32nd message and fell back to bridge bytes, which is the rollback guarantee going away. We hand the changeset back to the caller, which persists it before the ciphertext reaches the wire, so there is nothing left for the lease to protect. Every record goes through counter_lease.rs rather than calling the core directly: a path that forgets goes back to leasing without failing, and the symptom shows up somewhere else entirely. Group decrypt is now flat at ~82 us with a 441 byte row, and a DM row after 40 messages is 919 bytes and still legacy.
Contributor
There was a problem hiding this comment.
2 issues found across 15 files (changes from recent commits).
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="packages/baileys/src/__tests__/Signal/group-sender-key.test.ts">
<violation number="1" location="packages/baileys/src/__tests__/Signal/group-sender-key.test.ts:112">
P2: The 'reads a stored key whose buffers came back as base64 text' test doesn't actually produce the {type:'Buffer', data:'<base64>'} shape it aims to cover. JSON.parse without BufferJSON.reviver turns the record's buffers into plain number arrays, which BufferJSON.replacer ignores (it only rewrites real Buffer/Uint8Array instances), so the row is rewritten unchanged and the decrypt never hits base64-text handling — the test passes trivially and gives false confidence in that path. Parse with BufferJSON.reviver first so the replacer reconstructs the base64 Buffer objects.</violation>
</file>
<file name="packages/whatsapp-rust-bridge/src/storage_adapter.rs">
<violation number="1" location="packages/whatsapp-rust-bridge/src/storage_adapter.rs:861">
P1: Reused cipher instances can still retain a leased counter ceiling because records written by `store_session`/`store_sender_key` bypass this new waiver on their subsequent cache hits; after a crash/reload, that persisted lease fast-forwards the chain and recreates skipped message counters. Applying the waiver before serialization and cache insertion on both store paths would make the policy consistent.</violation>
</file>
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
Records were waived where they are loaded, but a record the protocol
builds for itself never passes a load: the store path is the only place
that sees it before it is serialized and cached. Both stores kept the
default lease on such a record, so the policy read differently depending
on where a record came from.
No such record can reach a send today, because the builders and the
ciphers hold separate adapters and therefore separate caches. Verified:
five sends through one reused cipher, over a record the group builder
created, leave the iteration at 5 either way. That is a property of the
call sites though, not of the policy, and the whole point of routing
records through one place is that a path which forgets fails silently.
Also pin the shape in the base64 sender-key test. It rewrites the row
with BufferJSON.replacer, which rewrites a { type: 'Buffer', data: [...] }
object as well as a real Buffer, so the assertion now states that rather
than leaving a reader to work it out.
A sender-key state memoizes the signing key's basepoint multiplication and the verifier's Edwards entries, but the memo lives in the record and we rebuild the record from storage on every operation. Every group message repaid a derivation whose inputs never changed. Keeping the record alive instead is the obvious fix and the wrong one: it would go stale the moment anything rotated the key, and encrypting under a chain the peer already dropped is worse than encrypting slowly. Keying on the signing key's public bytes has no such window. A rotated key is a different entry by construction, so a stale entry is unreachable rather than wrong, and the core's prewarm setters verify that what they are handed belongs to the state before installing it. Measured on the group path, cipher rebuilt per message against one kept warm: the gap closes from 14.8 to 1.9 us on encrypt and from 10.2 to 5.9 on decrypt. End to end, encryptGroupMessage goes from 71.8 to 62.0 us. Needs oxidezap/whatsapp-rust#1213, which added the setters.
The adapter would migrate a pre-WASM session object field by field when the store handed it one instead of bytes. Baileys never hands it one: `loadSession` converts a legacy record through the core's typed model first, or reports no session, so the branch was unreachable from the only consumer. It was also wrong. The fallback stored a message key's seed as its cipher key and zeroed the mac and iv, so the first ciphertext the old build enciphered failed its MAC. That is why the conversion moved to the typed model in the first place, and the comment at the call site has said so since. Its test asserted that the session opened and that encrypt returned a WhisperMessage, never that the peer could read it, so it would not have caught any of that. Removing both leaves one way to read a legacy record, the one that works.
The snapshot calls replaced SessionCipher and SessionBuilder for the whole direct-message path, and Baileys is the only consumer, so nothing reached them any more. Removing the pair took the adapter's session, identity, pre-key and signed-pre-key stores with it: those existed to serve those two types. The adapter now backs the group path alone, which needs sender keys and nothing else. The cascade the compiler then found came out too: five payload structs, the address and identity caches, the raw-store probe, and the extern declarations for eleven JS callbacks the bridge no longer calls. The SignalStorage interface goes from twelve members to two. wasm-opt output drops 7.6%, from 1,038,376 to 959,313 bytes for the SIMD build and 1,095,158 to 1,012,934 for the other. Three test files went with the API. Their scenarios did not: the bundle round trip now runs through processBundleWithSnapshot, and simultaneous initiation, injection followed by an incoming prekey message, and the guarantee that reprocessing a bundle does not discard a live session are expressed against the snapshot calls in snapshot_api.test.ts. What is left of storage_adapter.test.ts is the one case that still applies: a rejected write must not be cached as if it had succeeded.
The rc.9 fixture was 2,270 lines of pretty-printed JSON, most of it whitespace around base64. Minified and stripped of the four fields no test reads, it is one line and 10,878 bytes, down from 27,637. The three wire parity suites no longer import baileys@7.0.0-rc.9 at all. They call the same functions through test/helpers/legacy-wire.ts, which serves what rc.9 produced for each node from a recorded file. Parity is still checked against the JS implementation, just not by running it: a node with no recorded vector throws rather than passing quietly, and the header says how to re-record. Attribute order is part of the wire format, so the vector key preserves key order. Sorting it collapsed the two nodes in the order-sensitivity test onto one entry, which passed for the wrong reason until the replay run caught it. crypto-parity now compares against node's own crypto: MD5 and HKDF are standard, so the reference is the algorithm, not another implementation. appstate-parity and noise-session still need the dependency, because LT_HASH_ANTI_TAMPERING and makeNoiseHandler are client logic rather than primitives, and porting 1,600 lines of them is its own change. Also widen signalStorage's return type: the bridge's SignalStorage went from twelve members to two, and this file still calls the rest itself.
Contributor
There was a problem hiding this comment.
All reported issues were addressed across 18 files (changes from recent commits).
Not reviewed (too large): packages/baileys/src/__tests__/fixtures/legacy-session-rc9.json (~2,271 lines) - if these are generated or fixture files, add them to ignored paths to exclude them from future reviews.
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
Three problems with the vector harness, one of which broke CI. `decodeBinaryNode` returned `unknown`, so every caller reading `.tag` or `.attrs` failed the test typecheck. Jest never caught it because ts-jest does not typecheck; the CI step that runs tsc did. The buffer handling was inert. `Buffer.toJSON` runs before any replacer, so the `__b` branch was unreachable and the revive side looked for a shape that is never produced. Nothing broke yet because no recorded node carries binary content, but the first one that did would have come back as a plain object instead of a Buffer. Store and revive the shape that actually appears. The merge on write is read-modify-write, so recording has to be single-threaded. Say so where the instructions are, and write through a temp file so an interrupted run cannot leave truncated JSON behind. Also rename crypto-parity's helpers to nodeMd5/nodeHkdf and say node:crypto in the titles: the reference stopped being the JS client when the comparison moved to the standard algorithms.
Contributor
There was a problem hiding this comment.
All reported issues were addressed across 4 files (changes from recent commits).
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
The revive guard matched any object with type "Buffer" and an array under data. Buffer.from truncates out-of-range values rather than rejecting them, so a hand-edited or truncated vector would have revived as wrong bytes and the comparison would have run against them. Check the byte range and the exact key set, and throw when something claims to be a Buffer but is not one, so the failure names itself.
hashify, hmac, prost, serde-wasm-bindgen and simd-adler32 are declared but never referenced. prost in particular is not a leftover from the core: the core does not depend on it at all, so it entered the lockfile purely through this declaration. Two that cargo-machete also flagged stay, and now say why. getrandom is never called; it is there to turn on the wasm_js backend, without which the wasm32 target refuses to build. rand's sys_rng is what make_rng falls back to for seeding, and while another crate happens to enable it today, key generation must not depend on that staying true. Nothing changes in the artifact: dead code elimination was already dropping these, so the wasm is byte-identical at 959,313. The lockfile loses 26 lines and two crates.
libsignal's deserialize runs a v1 migration that copies a record-level registrationId onto every entry missing one, and haveOpenSession then requires the entry to have it. An auth state written before that migration still keeps the id only at the top. We required it on the entry, so for such a record every entry looked unusable: hasOpenLegacySession returned false, readSessionBytes reported no session, and pending whisper messages failed with SessionNotFound after the upgrade. The session was live the whole time. Fall back to the record-level id in both the open-session check and the conversion, which is the same migration libsignal does.
The ternary in entryRegistrationId needed parentheses. AGENTS.md still described Signal/ as wrapping libsignal-node, which this branch replaced with the bridge.
The only warning left in the crate, in a test helper that dropped the Result on the floor. Now the build is clean under clippy with -D warnings across every target and feature.
Dropping SessionCipher and SessionBuilder left this bench importing them, so `pnpm bench` died on the second file. It now holds the snapshot itself and applies each changeset, which is what a consumer does; the wrapper keeps the old method names so the benchmark bodies are unchanged. Still 7.5x, 9.6x and 4.0x faster than libsignal-node on encrypt, decrypt and round trip.
Contributor
There was a problem hiding this comment.
All reported issues were addressed across 1 file (changes from recent commits).
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
…aced `SnapshotCipher` models what the Baileys store does with a changeset, and it ignored `sessionCleared`. The core sets that when a peer's identity key is replaced: the ratchet built on the old key is void, so it returns no new session and asks the caller to drop the row instead. Applying only `changes.session` left the cipher holding bytes the core had disowned, and every later operation would have run on them. Not reachable from the benchmark as written, which never rotates an identity, but the point of this class is to mirror `applyChanges`, and it did not. The production path in `Signal/libsignal.ts` was already correct.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
9 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replaces the JS libsignal with the Rust core compiled to WebAssembly, keeping the auth state on disk exactly as it is today.
What changed
Session operations run as pure functions: the caller reads a snapshot, the protocol runs in memory, and the mutations come back as an explicit changeset applied in one write. Nothing calls back into JS mid-operation, so an operation cannot re-enter the caller's transaction or take locks in a conflicting order. That re-entrancy is what produced
message with old counterfailures under load.The identity row is locked alongside the session, because applying a changeset writes both and the group path does a read-modify-write on that same row.
Storage is unchanged
Sessions and group sender keys are written in the shape a pre-WASM release reads, not as the bridge's bytes. Rolling back is swapping the package: no conversion step, no migration. Identity keys keep the same 33-byte prefixed form, and pre-keys are untouched.
This costs about 0.3 ms per message against storing the native bytes. Serialization is worth its own change, with the breaking note that deserves, rather than extra surface area here.
One caveat: a record the v1 model cannot express keeps its bridge bytes. Reads accept both shapes, so that costs compatibility for one row instead of failing the write. Reaching it takes genuine out-of-order delivery.
Outbound counters
The core leases 64 outbound counters ahead of durability, and the legacy record has nowhere to carry that ceiling, so the export materialized the reservation and the whole batch burned on every operation. Sends landed on the wire at counters 0, 64, 128, 192, and the peer derived 63 skipped keys for each. After 20 group messages the stored row was 199,724 bytes and a decrypt cost 3,488 us; a direct-message row passed the skipped-key limit around the 32nd message and fell back to bridge bytes, losing the rollback guarantee above.
This branch waives the lease, which the core exposes as a per-record declaration (oxidezap/whatsapp-rust#1211). That is sound because the changeset is persisted before the ciphertext reaches the wire, and it is the guarantee pre-WASM releases gave. A direct-message row after 40 messages is now 919 bytes and still legacy.
Derivation cache
A sender-key state memoizes two curve derivations, but the memo lives in the record, which we rebuild per operation. Keeping the record alive instead would go stale the moment anything rotates the key. The bridge keys on the signing key's public bytes, which has no such window, and hands the result back through the core's prewarm setters (oxidezap/whatsapp-rust#1213), which verify it belongs to the state.
encryptGroupMessagegoes from 71.8 to 62.0 us.Numbers
wabenchagainst the same server, three runs per side, interleaved, median reported. Intel Core Ultra 9 275HX, 24 threads,--allocator stable --cpu-policy auto. Certificate verification is neutralised in both benchmark worktrees, since the test server cannot sign with WhatsApp's certificate key.Direct messages, 3000 at 300/s:
Only this branch holds the offered 300/s;
developsettles below it and accumulates lag.Group messages, 500 at 50/s, which is what
developsustains:Past that rate
developstops following: offered 100/s it delivers 53.4/s at a 2,013 ms median. This branch holds 300/s with a 1.05 ms median.Group receive is not covered by either scenario, and it is where the counter fix lands hardest: a microbenchmark of consecutive decrypts goes from 3,488 us on the twentieth message to a flat 82 us, with the stored row dropping from 199,724 to 441 bytes.
Fixes found along the way
getSenderKeyDistributionMessageseeded an empty record before asking the builder to create one, which loads as a state-less session and fails. Distributing your own sender key was broken, so no group could be sent to.Array::from(undefined)threw across the boundary on a legacy sender-key state stored withoutsenderMessageKeys.BufferJSON. Only the array was accepted, so a store that round-trips through it read every key as absent and surfaced asprotobuf encoding was invalid.Cleanup
SessionCipherandSessionBuilderare gone, along with the adapter's session, identity and pre-key stores that existed to serve them, and the legacy JSON session fallback, which was unreachable and stored a message key's seed as its cipher key. TheSignalStorageinterface goes from twelve members to two, and the SIMD wasm drops 7.6% to 959,313 bytes.Compatibility
The rc.9 fixture holds real sessions and ciphertexts produced by baileys@7.0.0-rc.9, with no Rust in the pipeline. The suite decrypts them, keeps the conversation going, and checks stored records are untouched when only read. The three wire-parity suites in the bridge now replay recorded rc.9 vectors instead of importing it.
Validation
583 tests in the Baileys package, 168 in the bridge plus 21 Rust unit tests under
wasm-pack, lint clean.The bridge tracks the Rust core at the commit that merged the prewarm setters, so it has to be built from source or published before this lands.