Skip to content
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions api/store.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -521,6 +521,32 @@ impl SharedSqlite {
`read` takes a permit from a separate reader pool, sized by [`SqliteStoreConfig::read_pool_size`](/concepts/storage#memory-and-thread-tuning-sqlitestoreconfig). A burst of reads can then run alongside a pending write instead of queueing behind `run`'s write-path permits. If `read_pool_size` is `0` (the default) or the underlying connection isn't WAL, `read` falls back to queueing on the same permits as `run` — it's always safe to call. Wrapping the closure in a deferred transaction also means a `read` that issues more than one statement (e.g. resolve a chat's identity keys, then query by them) sees one consistent snapshot across all of them, rather than possibly straddling a write that commits in between. Prefer `read` over `run` for anything that only queries. [`whatsapp-rust-chat-store`](/api/chat-store)'s query paths do, and so does most of `SqliteStore`'s own `SignalStore`/`AppSyncStore`/`ProtocolStore`/`DeviceStore` surface as of [whatsapp-rust#1222](https://github.com/oxidezap/whatsapp-rust/pull/1222) — session, identity, sender-key, and pre-key lookups among them. A handful of reads are deliberately kept on `run` instead: a stale answer for these would go out on the wire, fail an operation outright, or overwrite a cache unconditionally (app-state sync key lookups, `messageSecret` reads, `get_devices`).
</Note>

### Sharing the pool with sibling devices

```rust
pub fn share_for_device(&self, device_id: i32) -> Self;
```

Call `SqliteStore::share_for_device()` to get a new `SqliteStore` for a *sibling device* in the same database file. It clones this store's pool, write-serialization semaphore, and reader pool instead of opening another connection. Without it, every constructor builds its own r2d2 pool, so a process holding N sessions against one database file opens N connections. A connection costs memory before it reads a single row: a fixed ~46.9 KiB lookaside slab (`SQLITE_DEFAULT_LOOKASIDE`), plus a page cache that grows to [`cache_size_kib`](#database-configuration). You can't reduce the lookaside slab with a pragma — `SQLITE_DBCONFIG_LOOKASIDE` is C-API only and diesel doesn't expose it. Since every query already carries a `device_id`, sibling sessions on one database only ever needed that field to differ.

```rust
use whatsapp_rust_sqlite_storage::SqliteStore;

let device_1 = SqliteStore::new_for_device("whatsapp.db", 1).await?;
// One pool, one connection, two sessions.
let device_2 = device_1.share_for_device(2);
```

The returned store owns clones of the pool handles, so you can keep using it for as long as it lives — dropping the store it came from closes nothing.

What it does **not** do:

- **Create the device row.** It only stamps queries with `device_id`. You still need to provision the row through the usual path — `create_new_device` or a restore — the same as with a store built from `new_for_device`.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Remove the manual provisioning requirement

When you pass the returned store through the normal PersistenceManager initialization path, it already checks backend.exists() and calls backend.create() when the scoped row is absent (concepts/storage.mdx lines 59–64). Telling users they must provision the row themselves is therefore incorrect, and the suggested create_new_device API is not documented anywhere in the repository; clarify that share_for_device itself does not create the row, but normal client initialization does.

Useful? React with 👍 / 👎.

- **Isolate writes.** Siblings share the write permits set by [`SqliteStoreConfig::pool_size`](#database-configuration). At the default of `1`, their writes serialize against each other. On a burst where every sibling writes continuously, sharing costs roughly 2.5x the aggregate write throughput of a pool per session. In exchange you get FIFO-fair scheduling across siblings — a private connection per session instead leaves fairness to SQLite's busy handler, which backs off randomly and produces about 2x the spread between the fastest and slowest session.
- **Split `resource_report()`.** Siblings share one pool, so every handle reports the same whole-pool estimate. When you sum across a fleet of siblings, count it once per pool, not once per handle.

Because of the write-serialization trade, reach for this with **mostly-idle fleets** — sessions that are connected but not writing continuously, which is the common shape — rather than as a default replacement for a store per session. See [Memory and Thread Tuning](/concepts/storage#memory-and-thread-tuning-sqlitestoreconfig) for measured numbers.

### Features

- **Connection pooling** - Uses Diesel r2d2 with pool size of 2
Expand Down
12 changes: 12 additions & 0 deletions concepts/storage.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -979,6 +979,18 @@ As of [whatsapp-rust#1222](https://github.com/oxidezap/whatsapp-rust/pull/1222),

At ~100 sessions the r2d2 thread count drops from ~300 to 2.

### Sharing one pool across sibling devices

The measurements above still assume one `SqliteStore` per session. If your sessions share the same database file, call [`SqliteStore::share_for_device`](/api/store#sharing-the-pool-with-sibling-devices) instead. It removes the per-session connection entirely rather than just shrinking its cost — a sibling store clones the base store's pool, write semaphore, and reader pool instead of opening its own.

Measured on an idle session (a couple of point reads, `cache_size_kib` at its 512 KiB default):

| | pool per session (marginal) | shared pool (marginal) |
|---|---|---|
| RSS per additional session | ~122.6 KiB | ~0.0 KiB |

You pay for this on the write side. Siblings share the write permit set by `pool_size`, so at the default of `1` their writes serialize against each other. On a continuous-write burst that costs roughly 2.5x slower aggregate write throughput. In exchange you get FIFO-fair scheduling across siblings instead of SQLite's busy-handler backoff. Reach for it when your fleet is mostly idle, not when sessions write continuously. See [Sharing the pool with sibling devices](/api/store#sharing-the-pool-with-sibling-devices) for the full trade-off and what the method does not do.

## DeviceCommand Pattern

**Location:** `src/store/commands.rs`, `wacore/src/store/commands.rs`
Expand Down