Skip to content

docs: add a storage pools concept and usage guide - #3648

Open
halfprice wants to merge 4 commits into
mainfrom
zhewu/storage_pool_doc
Open

docs: add a storage pools concept and usage guide#3648
halfprice wants to merge 4 commits into
mainfrom
zhewu/storage_pool_doc

Conversation

@halfprice

@halfprice halfprice commented Aug 16, 2026

Copy link
Copy Markdown
Collaborator

Description

Adds a documentation page for the storage pool preview feature, which currently has no coverage in the docs beyond release-note mentions.

The new page (docs/content/system-overview/storage-pools.mdx, added to the Concepts sidebar next to Quilt) covers:

  • At a glance: a side-by-side comparison of regular blobs and pooled blobs.
  • How pools work: the shared reservation model, shared lifetime, reusable capacity, encoded-byte accounting (including that the fixed per-blob metadata overhead still applies, so Quilt remains the tool for small blobs), and the active/expired state model with silent expiry.
  • Costs: a per-operation fee breakdown (which operations pay the storage fee versus the write fee), the pay-for-reserved-not-used property, and the no-refunds rule.
  • When to use pools: guidance plus a decision-guide matrix in the same style as the Quilt page.
  • Rust SDK usage: full lifecycle examples — create, store through reserve_and_store_blobs_in_storage_pool with result handling, inspect with storage_pool_status and list_pooled_blob_ids, delete, extend, and increase capacity — including the SDK's documented caveats (no automatic lifetime extension, no automatic epoch-change retry, automatic capacity increase, PostStoreAction::Keep only, deletable by default).
  • Move contracts: the walrus::system entry-point signatures, Move-only capabilities (pool creation from an existing Storage, capacity decrease, burn and destroy, pooled-blob metadata), the blob_bucket wrapper, and a danger admonition about sharing a raw StoragePool.
  • Events: a reference table of the five pool events for indexers, noting which operations emit nothing.
  • Constraints and common pitfalls.

Also adds "storage pool" and "pooled blob" entries to the glossary so the terms auto-link site-wide.

The page is marked with a warning that the feature is a preview available only through the Rust SDK and Move contracts, with no CLI, HTTP API, or TypeScript SDK support yet.

Test plan

  • node src/scripts/audit-docs.mjs: the new page passes all frontmatter, link, code-fence, and goal checks.
  • pnpm build in docs/site: succeeds; no broken links or anchors introduced by the new page.
  • Code examples verified against crates/walrus-sdk (StoreBlobsInStoragePoolApi, PooledBlobStoreResult, StoreArgs defaults) and the e2e tests in crates/walrus-e2e-tests/tests/test_client.rs.

@github-actions

github-actions Bot commented Aug 16, 2026

Copy link
Copy Markdown
Contributor
PR Preview Action v1.8.1

QR code for preview link

🚀 View preview at
https://MystenLabs.github.io/walrus/pr-preview/pr-3648/

Built to branch gh-pages at 2026-08-19 20:33 UTC.
Preview will be ready when the GitHub Pages deployment is complete.

@github-actions

github-actions Bot commented Aug 16, 2026

Copy link
Copy Markdown
Contributor

📋 afdocs check results

URL: https://MystenLabs.github.io/walrus/pr-preview/pr-3648/

Running checks on mystenlabs.github.io/walrus/pr-preview/pr-3648/...

Agent-Friendly Docs Check: https://MystenLabs.github.io/walrus/pr-preview/pr-3648/
Timestamp: 8/19/2026, 8:37:55 PM

content-discoverability
  ✓ llms-txt-exists: llms.txt found at https://MystenLabs.github.io/walrus/pr-preview/pr-3648/llms.txt
  ✓ llms-txt-valid: llms.txt follows the proposed structure (H1, blockquote, heading-delimited link sections)
  ⚠ llms-txt-size: llms.txt is 67,985 characters (between 50,000 and 100,000; consider splitting)
      Learn more: https://agentdocsspec.com/spec/#llms-txt-size
  ○ llms-txt-links-resolve: llms.txt contains 197 links, but none are under /walrus/pr-preview/pr-3648
  ○ llms-txt-links-markdown: llms.txt contains 197 links, but none are under /walrus/pr-preview/pr-3648
  ✓ llms-txt-directive-html: llms.txt directive found in HTML of all 205 pages, near the top of content
  ✗ llms-txt-directive-md: Could not fetch markdown for any of 205 pages; 205 had no markdown version
      Learn more: https://agentdocsspec.com/spec/#llms-txt-directive-md

markdown-availability
  ✗ markdown-url-support: No pages support .md URLs (0/205 tested)
      Learn more: https://agentdocsspec.com/spec/#markdown-url-support
  ✗ content-negotiation: Server ignores Accept: text/markdown header (0/205 pages return markdown)
      Learn more: https://agentdocsspec.com/spec/#content-negotiation

page-size
  ✓ rendering-strategy: All 205 pages contain server-rendered content
  ○ page-size-markdown: Skipped: dependency check did not pass
  ⚠ page-size-html: 1 of 205 pages convert to 50K–100K chars (max 214K HTML → 80K markdown (80% boilerplate))
      Learn more: https://agentdocsspec.com/spec/#page-size-html
  ✗ content-start-position: 12 of 205 pages have content starting past 50% (worst 64%)
      Learn more: https://agentdocsspec.com/spec/#content-start-position

content-structure
  ✓ tabbed-content-serialization: 65 tab group(s) across 38 of 205 pages; all serialize under 50K chars
  ✓ section-header-quality: 3 page(s) with tab headers checked; headers include variant context
  ✓ markdown-code-fence-validity: All 0 code fences properly closed across 1 pages

url-stability
  ✓ http-status-codes: All 205 pages return proper error codes for bad URLs
  ✓ redirect-behavior: No redirects detected across 205 pages

observability
  ✗ llms-txt-coverage: llms.txt covers 0/197 sitemap doc pages (0%); 197 missing
      Learn more: https://agentdocsspec.com/spec/#llms-txt-coverage
  ○ markdown-content-parity: Skipped: dependency check did not pass
  ✓ cache-header-hygiene: All 206 endpoints have appropriate cache headers

authentication
  ⚠ auth-gate-detection: 1 of 205 pages require authentication (204 accessible)
      Learn more: https://agentdocsspec.com/spec/#auth-gate-detection
  ✓ auth-alternative-access: Alternative access detected (public-llms-txt, partial-public-access) for site with 1 auth-gated pages

Summary
  11 passed, 3 warnings, 5 failed, 4 skipped (23 total)

Full spec: https://agentdocsspec.com/spec/

@jessiemongeon1

jessiemongeon1 commented Aug 16, 2026

Copy link
Copy Markdown
Contributor

Style Guide Audit

Audited 1 file(s) against the Sui Documentation Style Guide.

6 violation(s) found. All must be fixed before merge.

docs/content/system-overview/storage-pools.mdx (6 violation(s))

6 violation(s) (0 regex, 6 claude)

  • Line 79 — no-italic-text
    • Current: *encoded* bytes
    • Fix: **encoded** bytes or plain text
  • Line 167 — no-italic-text
    • Current: *encoded* size
    • Fix: **encoded** bytes or plain text
  • Line 334 — passive-voice
    • Current: A pool is created.
    • Fix: You create a pool.
  • Line 335 — passive-voice
    • Current: A blob is registered in a pool.
    • Fix: You register a blob in a pool.
  • Line 336 — passive-voice
    • Current: A pooled blob is certified.
    • Fix: You certify a pooled blob.
  • Line 337 — passive-voice
    • Current: A pooled blob is deleted from an active pool.
    • Fix: You delete a pooled blob from an active pool.

Automated audit using the Sui Documentation Style Guide.

@halfprice

Copy link
Copy Markdown
Collaborator Author

Style Guide Audit

Audited 1 file(s) against the Sui Documentation Style Guide.

10 violation(s) found. All must be fixed before merge.

docs/content/system-overview/storage-pools.mdx (10 violation(s))

10 violation(s) (10 regex, 0 claude)

  • Line 100 — No em dashes in prose

    • Current:
    • Fix: Rewrite with comma, parentheses, or split sentence
  • Line 101 — No em dashes in prose

    • Current:
    • Fix: Rewrite with comma, parentheses, or split sentence
  • Line 102 — No em dashes in prose

    • Current:
    • Fix: Rewrite with comma, parentheses, or split sentence
  • Line 103 — No em dashes in prose

    • Current:
    • Fix: Rewrite with comma, parentheses, or split sentence
  • Line 104 — No em dashes in prose

    • Current:
    • Fix: Rewrite with comma, parentheses, or split sentence
  • Line 143 — No em dashes in prose

    • Current:
    • Fix: Rewrite with comma, parentheses, or split sentence
  • Line 144 — No em dashes in prose

    • Current:
    • Fix: Rewrite with comma, parentheses, or split sentence
  • Line 147 — No em dashes in prose

    • Current:
    • Fix: Rewrite with comma, parentheses, or split sentence
  • Line 148 — No em dashes in prose

    • Current:
    • Fix: Rewrite with comma, parentheses, or split sentence
  • Line 149 — No em dashes in prose

    • Current:
    • Fix: Rewrite with comma, parentheses, or split sentence

Automated audit using the Sui Documentation Style Guide.

Addressed


## How storage pools work

A storage pool is a Sui object that wraps a storage resource with a fixed encoded capacity and a lifetime spanning a start epoch up to, but not including, an end epoch. The pool tracks three things: its reserved encoded capacity, the encoded bytes currently in use, and the number of registered blobs. Blobs stored in a pool are represented by `PooledBlob` objects that live inside the pool rather than as independently owned `Blob` objects.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Passive: "Blobs stored in a pool are represented by PooledBlob objects that live inside the pool rather than as independently owned Blob objects." I recommend changing to: "The pool represents each blob as a PooledBlob object that lives inside the pool rather than as an independently owned Blob object."

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, applied the suggested wording.

.await?;
```

When sizing the capacity, remember that it is consumed by *encoded* blob sizes. You can compute a blob's exact encoded size with `walrus_core::encoding::encoded_blob_length_for_n_shards`, or check it for a sample file with `walrus store --dry-run`. The pool's lifetime is bounded by the system's maximum blob lifetime, currently 53 epochs, the same limit that applies to [regular blob lifetimes](/docs/walrus-client/managing-blobs).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Passive: "remember that it is consumed by encoded blob sizes." I recommend changing to: "remember that blobs consume it at their encoded size."

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, applied the suggested wording.


## Common pitfalls

- **Sizing the pool in raw bytes.** Reserved capacity is consumed by encoded sizes, about 4.5x the raw size plus roughly 64 MB of metadata per blob. A pool sized to the sum of your raw file sizes runs out of capacity almost immediately.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Passive: "Reserved capacity is consumed by encoded sizes, about 4.5x the raw size..." I recommend changing to: "Blobs consume reserved capacity at their encoded size, about 4.5x the raw size plus roughly 64 MB of metadata per blob."

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, applied the suggested wording.

halfprice and others added 4 commits August 19, 2026 13:31
Documents the storage pool preview feature: what a pool is, its shared
lifetime and reusable-capacity model, cost structure, when to prefer it
over individual blobs or Quilt, the pool operation surface, and how to
use it through the Rust SDK and the Move contracts.
Adds an at-a-glance comparison with regular blobs, a pool state table,
a per-operation fee breakdown, a decision guide, expanded Rust SDK
examples covering the full lifecycle, the Move entry-point signatures,
an events reference for indexers, and common pitfalls. Also adds
storage pool and pooled blob glossary entries so the terms auto-link
site-wide.
Addresses the style guide audit on the storage pools page, which
disallows em dashes; the empty fee cells in the costs and operations
tables now say None.
Applies review suggestions to use active voice for how pooled blobs
are represented and how blobs consume reserved capacity.
@halfprice
halfprice force-pushed the zhewu/storage_pool_doc branch from 3e014ba to f3f62db Compare August 19, 2026 20:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants