Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions api/store.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -521,6 +521,32 @@ impl SharedSqlite {
`read` takes a permit from a separate reader pool, sized by [`SqliteStoreConfig::read_pool_size`](/concepts/storage#memory-and-thread-tuning-sqlitestoreconfig). A burst of reads can then run alongside a pending write instead of queueing behind `run`'s write-path permits. If `read_pool_size` is `0` (the default) or the underlying connection isn't WAL, `read` falls back to queueing on the same permits as `run` — it's always safe to call. Wrapping the closure in a deferred transaction also means a `read` that issues more than one statement (e.g. resolve a chat's identity keys, then query by them) sees one consistent snapshot across all of them, rather than possibly straddling a write that commits in between. Prefer `read` over `run` for anything that only queries. [`whatsapp-rust-chat-store`](/api/chat-store)'s query paths do, and so does most of `SqliteStore`'s own `SignalStore`/`AppSyncStore`/`ProtocolStore`/`DeviceStore` surface as of [whatsapp-rust#1222](https://github.com/oxidezap/whatsapp-rust/pull/1222) — session, identity, sender-key, and pre-key lookups among them. A handful of reads are deliberately kept on `run` instead: a stale answer for these would go out on the wire, fail an operation outright, or overwrite a cache unconditionally (app-state sync key lookups, `messageSecret` reads, `get_devices`).
</Note>

### Sharing the pool with sibling devices

```rust
pub fn share_for_device(&self, device_id: i32) -> Self;
```

`SqliteStore::share_for_device()` returns a new `SqliteStore` for a *sibling device* in the same database file, cloning this store's pool, write-serialization semaphore, and reader pool instead of opening another connection. Every constructor builds its own r2d2 pool, so a process holding N sessions against one database file today opens N connections — and a connection costs memory before it reads a single row: a fixed ~46.9 KiB lookaside slab (`SQLITE_DEFAULT_LOOKASIDE`, not reducible via any pragma — `SQLITE_DBCONFIG_LOOKASIDE` is C-API only and unreachable through diesel) plus a page cache that grows to [`cache_size_kib`](#database-configuration). Since every query already carries a `device_id`, sibling sessions on one database only ever needed that field to differ.
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Outdated

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Split the pool-cost paragraph into concise sentences

Break this paragraph into shorter sentences. It combines pool construction, connection counts, lookaside configuration, page-cache behavior, and device scoping in one dense line, which violates the project's requirement to keep sentences concise and cover one idea at a time. The similarly long performance sentence below should be split as part of the same correction.

AGENTS.md reference: AGENTS.md:L25-L25

Useful? React with 👍 / 👎.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Qualify the lookaside allocation as build-dependent

Qualify the 46.9 KiB figure as applying to the tested bundled SQLite build. SQLITE_DEFAULT_LOOKASIDE is a compile-time setting, so users who follow the preceding instructions to link a system SQLite—or the later SQLCipher instructions—can have a different default size or no lookaside allocation at all; presenting this as a fixed cost for every connection makes the memory guidance inaccurate in those supported configurations.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Describe sharing as removing a pool rather than one connection

Scope the claim that N sessions open N connections to the default configuration. with_config_for_device can set pool_size above one and add read_pool_size, so each session can own multiple physical connections and sharing removes that entire pool. The current wording, also repeated as “removes the per-session connection” in concepts/storage.mdx, gives an incorrect connection and memory baseline for supported custom configurations.

Useful? React with 👍 / 👎.


```rust
use whatsapp_rust_sqlite_storage::SqliteStore;

let device_1 = SqliteStore::new_for_device("whatsapp.db", 1).await?;
// One pool, one connection, two sessions.
let device_2 = device_1.share_for_device(2);
```

The returned store owns clones of the pool handles, so it stays usable for as long as it lives — dropping the store it came from closes nothing.

What it does **not** do:

- **Create the device row.** It only stamps queries with `device_id`; the row still comes from the usual provisioning path (`create_new_device` or restore), same as a store built with `new_for_device`.
- **Isolate writes.** Siblings share the write permits set by [`SqliteStoreConfig::pool_size`](#database-configuration) — at the default of `1`, their writes serialize against each other. Measured on a burst where every sibling writes continuously, sharing costs roughly 2.5x the aggregate write throughput of a pool per session, in exchange for FIFO-fair scheduling across siblings (a private connection per session instead leaves fairness to SQLite's busy handler, which backs off randomly — about 2x the spread between the fastest and slowest session).

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Remove the claim that SQLite randomizes busy backoff

Correct the fairness comparison for the documented PRAGMA busy_timeout configuration. SQLite's standard timeout handler performs repeated sleeps until the configured timeout is exhausted; it does not provide the randomized backoff claimed here. The same assertion is repeated in concepts/storage.mdx, so both passages currently give readers a false explanation for the measured fairness difference.

Useful? React with 👍 / 👎.

- **Split `resource_report()`.** Siblings share one pool, so every handle reports the same whole-pool estimate — when summing across a fleet of siblings, count it once per pool, not once per handle.

Because of the write-serialization trade, this is for **mostly-idle fleets** — sessions that are connected but not writing continuously, which is the common shape — rather than a default replacement for a store per session. See [Memory and Thread Tuning](/concepts/storage#memory-and-thread-tuning-sqlitestoreconfig) for measured numbers.
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Outdated

### Features

- **Connection pooling** - Uses Diesel r2d2 with pool size of 2
Expand Down
12 changes: 12 additions & 0 deletions concepts/storage.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -979,6 +979,18 @@ As of [whatsapp-rust#1222](https://github.com/oxidezap/whatsapp-rust/pull/1222),

At ~100 sessions the r2d2 thread count drops from ~300 to 2.

### Sharing one pool across sibling devices

The measurements above still assume one `SqliteStore` per session. If those sessions share the same database file, [`SqliteStore::share_for_device`](/api/store#sharing-the-pool-with-sibling-devices) removes the per-session connection entirely instead of just shrinking its cost: a sibling store clones the base store's pool, write semaphore, and reader pool rather than opening its own.

Measured on an idle session (a couple of point reads, `cache_size_kib` at its 512 KiB default):

| | pool per session (marginal) | shared pool (marginal) |
|---|---|---|
| RSS per additional session | ~122.6 KiB | ~0.0 KiB |

The trade is on the write side: siblings share the write permit set by `pool_size`, so at the default of `1` their writes serialize against each other — roughly 2.5x slower aggregate write throughput on a continuous-write burst, in exchange for FIFO-fair scheduling across siblings instead of SQLite's busy-handler backoff. That makes it a fit for mostly-idle fleets rather than sessions that write continuously. See [Sharing the pool with sibling devices](/api/store#sharing-the-pool-with-sibling-devices) for the full trade-off and what the method does not do.

## DeviceCommand Pattern

**Location:** `src/store/commands.rs`, `wacore/src/store/commands.rs`
Expand Down