Skip to content

feat(examples): pluggable grc20 ledger storage + p/nt/hashmap (flat gas for large ledgers)#5965

Draft
omarsy wants to merge 14 commits into
gnolang:masterfrom
omarsy:feat/grc20-hashmap-kv-backend
Draft

feat(examples): pluggable grc20 ledger storage + p/nt/hashmap (flat gas for large ledgers)#5965
omarsy wants to merge 14 commits into
gnolang:masterfrom
omarsy:feat/grc20-hashmap-kv-backend

Conversation

@omarsy

@omarsy omarsy commented Jul 15, 2026

Copy link
Copy Markdown
Member

Summary

Adds a KV storage-backend seam to p/demo/tokens/grc20 and a new
p/nt/hashmap/v0 package so token ledgers can opt into storage-access gas
that stays flat as the ledger grows
, instead of growing with holder count.

This addresses the state-scaling half of #5906. The default backend is
unchanged (avl.Tree), so this is fully backwards-compatible and opt-in.

Why

A cold GRC20 transfer costs ~6M gas at 1 holder but ~18M at ~1,000,000
holders
. The cause is the data structure, not the contract: avl.Tree
materializes its search path as one persisted object per node, and the base
store charges a flat ~59k gas per cold object read, so a transfer loads
O(log N) objects. Measured cold (fresh keeper, empty object cache — the state
a validator is in after cache GC), same transfer op:

ledger size avl hashmap (1024) bptree (fanout 128)
20,000 13.8M 4.3M 7.8M
100,000 15.5M 4.5M 10.7M
1,000,000 18.6M 5.7M (−69%) 11.7M (−37%)
  • avl at 1M reproduces the real on-chain test_atone transfer (18.3M) —
    independent confirmation of the mechanism.
  • hashmap (a native-map bucket array; each bucket is a single persisted
    object regardless of entry count) stays flat: O(1) object loads per op.
  • bptree (already in-tree) rides the same seam and keeps ordered
    iteration
    , which hashmap trades away.

Full analysis and 1..1M measurements are in the ADR.

Changes

New package

  • examples/gno.land/p/nt/hashmap/v0/hashmap.gno (Map: Get/Set/Remove/Has/Size/Iterate, FNV-1a bucketing, fixed power-of-two bucket count, sizing guidance), hashmap_test.gno, gnomod.toml.

grc20 seam (backwards-compatible, default avl)

  • examples/gno.land/p/demo/tokens/grc20/types.gno — add KV interface (satisfied by *avl.Tree, *hashmap.Map, *bptree.BPTree), Option, WithStorage; ledger fields become KV.
  • examples/gno.land/p/demo/tokens/grc20/token.gnoNewToken gains variadic opts ...Option; ledger built from the chosen backend (default avl.NewTree).
  • examples/gno.land/p/demo/tokens/grc20/storage_option_test.gno — full token flow over all three backends through the one seam.

ADR

  • gno.land/adr/prxxxx_hashmap_ledger.md — context, decision, measured results (incl. the 1M scaling and the bptree comparison), alternatives (native VM storage, block-level warm/cold gas pricing), consequences.

Notes / open discussion

  • This is one lever (data layer). The swap congestion that saturated test13
    is a different workload; block-level warm/cold gas pricing would help it
    and every contract automatically — flagged in the ADR as a separate,
    protocol-level proposal for maintainers to weigh.
  • Sizing was measured: the default 1024 buckets stays flat from ~1k to ~1M
    entries
    , and over-sizing (4096) is counter-productive (every op decodes the
    whole bucket-pointer array). Documented in the package.
  • FNV-1a uses a fixed seed (grindable); the bounded-impact limitation is
    documented in the package and ADR.

AI-assisted (investigation, implementation, and the cold-cache measurements);
reviewed and owned by the author.

🤖 Generated with Claude Code

Add gno.land/p/nt/hashmap/v0, a persistent string-keyed map with O(1)
persisted-object loads per operation, and a KV storage-backend seam on
p/demo/tokens/grc20 (a `WithStorage` option on `NewToken`) so token
ledgers can opt into flat storage-access gas as they grow. The default
backend stays avl.Tree, so existing tokens are unchanged and grc20 does
not import hashmap.

Addresses the state-scaling half of gnolang#5906: a cold GRC20 transfer grows
from ~6M gas at 1 holder to ~18M at ~1M holders because avl.Tree loads
O(log N) objects, each 59k gas cold. Measured cold, hashmap keeps a
transfer flat (~-69% vs avl at 1M); the in-tree bptree, via the same
seam, preserves ordered iteration at ~-37%. avl at 1M reproduces the
real on-chain 18.3M number. Details and full 1..1M measurements in
gno.land/adr/prxxxx_hashmap_ledger.md.
@github-actions github-actions Bot added 🧾 package/realm Tag used for new Realms or Packages. 📦 ⛰️ gno.land Issues or PRs gno.land package related labels Jul 15, 2026
@Gno2D2 Gno2D2 added the review/triage-pending PRs opened by external contributors that are waiting for the 1st review label Jul 15, 2026
@Gno2D2

Gno2D2 commented Jul 15, 2026

Copy link
Copy Markdown
Collaborator

🛠 PR Checks Summary

All Automated Checks passed. ✅

Manual Checks (for Reviewers):
  • IGNORE the bot requirements for this PR (force green CI check)
Read More

🤖 This bot helps streamline PR reviews by verifying automated checks and providing guidance for contributors and reviewers.

✅ Automated Checks (for Contributors):

🟢 Maintainers must be able to edit this pull request (more info)
🟢 Pending initial approval by a review team member, or review from tech-staff

☑️ Contributor Actions:
  1. Fix any issues flagged by automated checks.
  2. Follow the Contributor Checklist to ensure your PR is ready for review.
    • Add new tests, or document why they are unnecessary.
    • Provide clear examples/screenshots, if necessary.
    • Update documentation, if required.
    • Ensure no breaking changes, or include BREAKING CHANGE notes.
    • Link related issues/PRs, where applicable.
☑️ Reviewer Actions:
  1. Complete manual checks for the PR, including the guidelines and additional checks if applicable.
📚 Resources:
Debug
Automated Checks
Maintainers must be able to edit this pull request (more info)

If

🟢 Condition met
└── 🟢 And
    ├── 🟢 The base branch matches this pattern: ^master$
    └── 🟢 The pull request was created from a fork (head branch repo: omarsy/gno)

Then

🟢 Requirement satisfied
└── 🟢 Maintainer can modify this pull request

Pending initial approval by a review team member, or review from tech-staff

If

🟢 Condition met
└── 🟢 And
    ├── 🟢 The base branch matches this pattern: ^master$
    └── 🟢 Not (🔴 Pull request author is a member of the team: tech-staff)

Then

🟢 Requirement satisfied
└── 🟢 If
    ├── 🟢 Condition
    │   └── 🟢 Or
    │       ├── 🔴 At least one of these user(s) reviewed the pull request: [aronpark1007 davd-gzl jefft0 notJoon omarsy MikaelVallenet] (with state "APPROVED")
    │       ├── 🔴 At least 1 user(s) of the team tech-staff reviewed pull request
    │       └── 🟢 This pull request is a draft
    └── 🟢 Then
        └── 🟢 Not (🔴 This label is applied to pull request: review/triage-pending)

Manual Checks
**IGNORE** the bot requirements for this PR (force green CI check)

If

🟢 Condition met
└── 🟢 On every pull request

Can be checked by

  • Any user with comment edit permission

omarsy added 2 commits July 15, 2026 18:04
…ter merge

Master moved avl.Tree.Get (and bptree) to return a single any (nil = absent).
Align KV.Get and hashmap.Map.Get to match, so *avl.Tree/*hashmap.Map/*bptree
all satisfy KV and grc20's balanceOf/allowance nil-checks compile.
@davd-gzl
davd-gzl self-requested a review July 15, 2026 16:11
@omarsy
omarsy marked this pull request as draft July 15, 2026 16:35
@Gno2D2 Gno2D2 removed the review/triage-pending PRs opened by external contributors that are waiting for the 1st review label Jul 15, 2026
omarsy added 3 commits July 15, 2026 18:45
grc20's interface-typed ledger fields grow a token's persisted footprint;
gov/dao/v3's treasury imports grc20, so the genesis deposit test1 pays rises
~1.6M ugnot, shifting the two exact-balance assertions. Update to new values.
grc20's larger package (KV seam) raises the gas to deploy/run realms that
import it. Two tight-budget txtars hit out-of-gas: bump gas-wanted for the
mitm addpkg (wugnot) and change-law proposal run (govdao).
Measured persisted bytes / deposit for avl vs hashmap vs bptree at
deploy-base and 20k/100k/1M holders. hashmap is -93% storage vs avl at
100k holders (~144 vs ~2090 bytes/holder), so the backend swap cuts both
per-tx gas and storage deposit.
omarsy added 4 commits July 15, 2026 23:21
…NV-1a

bucketIndex now derives the bucket from crypto/sha256.Sum256 (low 64 bits)
rather than an FNV-1a loop written in Gno. Sum256 is a calibrated native
binding, so bucket selection no longer runs a per-byte hash loop in the
interpreter on every Get/Set/Has/Remove.

Measured cold, end-to-end grc20 transfer: ~-0.5M gas, flat from 20k to 1M
holders. The interpreted-hash removal saves -0.75M in the VM term, offset by
+4 fixed cold reads (+0.24M) for the sha256 package's first-use load — net
-0.5M. (A compute-only measurement shows ~-0.9M by omitting that package
load.)

Ships in this PR, not a follow-up: bucket placement is a function of the
digest, so changing the hash after any map is persisted relocates every key.
Store buckets in a two-level directory (√B pages of √B buckets, lazily
allocated) rather than one flat B-slot array. A flat array is a single object
that every op decodes in full (~190KB at 1024 slots, fixed at any entry count);
splitting it means an op decodes only the directory + one page + one bucket,
each a √B-slot array. This also makes a high bucket count cheap, so the default
rises to 4096 and each leaf bucket stays small as the ledger grows.

New()/NewWithBuckets(count) are unchanged in signature; count is now split
evenly across the two levels. Placement is fixed (digest + split), so this
ships before any map is persisted.

Measured cold, end-to-end 1M-holder grc20 transfer: 8.58M -> 6.59M (-1.99M,
~-23% vs the flat design; ~-65% vs today's avl). Nearly flat in N (6.17M at 20k
-> 6.59M at 1M). Read categorization: the bucket ArrayValue decode drops
571k -> 118k and the leaf MapValue decode 845k -> 212k; the rest is alloc gas
on the same shrunken objects. Unit tests and the full grc20 flow pass.
Independent review found no code defects; the actionable findings were all
stale documentation left from before the SHA-256 + two-level rewrite:
- ADR Consequences said FNV-1a / default 1024 / "over-sizing counter-productive"
  — the shipped design is SHA-256, default 4096, and two-level reverses the
  over-sizing finding.
- ADR headline "Measured results" table showed pre-two-level numbers
  (77 obj / 7.2-7.3M); updated to the shipped 85 / 6.17-6.59M.
- Renamed a stale `bucketIndex` reference to `locate`.
- Package doc: "√B × √B" is exact only for even exponents (softened to ≈√B and
  documented the 2:1 split for odd exponents); noted iteration order is
  preserved across reload and that the persisted image is insertion-order-
  dependent, not canonical like avl.Tree.

Comment/markdown only — no code change.
The ADR's storage-footprint section still showed the flat-1024 numbers. Measured
the shipped two-level design (rlm.Storage, same harness):
- deploy base 170KB -> 23KB: pages/buckets are allocated lazily, not pre-
  allocated, so an empty ledger is nearly free.
- at scale: 20k 4.42M / 100k 15.98M / 1M 145.6M bytes. Marginal ~144 bytes/
  holder (same as a flat map); the 4096-bucket object overhead amortizes by 1M.
- vs avl: -89% (20k) / -92% (100k) / -93% (1M); deposit at 100k ~1,598 GNOT vs
  avl ~20,863 GNOT.
Also tightened the NewWithBuckets doc on the bit-split. Comment/markdown only.
@notJoon

notJoon commented Jul 16, 2026

Copy link
Copy Markdown
Member

based on ADR, I measured the costs by running a local chain with gnodev. while the measured costs did not exactly match, I confirmed that they shoed the same trends described in docs

grc20_gnodev_gas_analysis

omarsy added 3 commits July 16, 2026 10:39
…ound

A second independent review pass (0 code defects again) caught internal
contradictions the first pass missed:
- ADR "~200-object package/code floor" was impossible against the ~85-object
  read floor (conflated total decodes incl. cached stdlib with paying reads) —
  corrected to "most of those ~85 reads".
- "subtract ~5.5M for a full token transfer" had the wrong sign and magnitude;
  a full transfer ADDS ~0.5-1M of grc20/package overhead over the KV-isolated
  numbers.
- Annotated the intro table's object counts as the initial-investigation harness
  (avl@20k 182 there vs 191 in the refined tables).
- "-0.9M" was mislabeled as the VM term (it's compute-only); "splits n evenly"
  softened (odd exponents give the directory the extra bit).
- Package doc: disambiguated the page/bucket bit-split wording, and dropped an
  overstated "avl.Tree image is canonical" claim (AVL shape is also history-
  dependent) while keeping the accurate "not key-sorted" contrast.

Comment/markdown only — no code change.
Third (convergence) review round passed all hard checks; these close the two
cleanest residual nits:
- a leaf bucket is a native map (~N/B entries), not a "√B-slot array" — only the
  directory and page are; reworded so the √B-slot characterization applies to
  the two arrays, with the bucket called out separately.
- clarified the SHA-256 net saving: −0.5M = −0.75M VM term less the +0.24M
  package cold-load; the "−0.9M" figure is a higher compute-only estimate.

Comment/markdown only — no code change.
…rcount)

The original ADR simulated production gas as VM_gas + reads*59k + writes*24k —
only 2 of the store's 7 gas dimensions and no depth-gas multiplier — undercounting
by ~2x. Re-measured by running the cold transfer through the real cache-wrapped,
gas-metered store (DefaultGasConfig + DefaultParams depth params, as app.go
configures) and reading GasConsumed():

- cold avl 27.9M (20k) / 29.8M (100k) / 32.0M (200k) / ~35M (1M extrap)
- cold two-level hashmap 16.2M / 16.5M / 18.8M -> -42/-45/-46% vs avl (was -65%)
- a fixed ~10.7M package-code depth floor dominates every cold transfer (all
  backends) -> package-code cold-load is now the biggest remaining lever
- depth/flat/per-byte are cache-miss-only, so warm transfers (txtar) ~7M, cold
  (congestion/GC) ~16-28M; on-chain test13 18.3M is a modest-ledger cold number,
  NOT avl@1M (that "18M@1M" was an undercount artifact)
- SHA-256 re-measured: -0.74M vs FNV-1a (clean; crypto/sha256 is stdlib-cached,
  no package penalty)

Independently corroborated by a maintainer's gnodev reproduction. Code and
storage-footprint numbers are unaffected; the relative hashmap-vs-avl win holds.
@omarsy

omarsy commented Jul 16, 2026

Copy link
Copy Markdown
Member Author

@notJoon, the gnodev run caught a real issue: my harness was undercounting by ~2×. It simulated only 2 of the store's 7 gas dimensions and skipped the depth-gas multiplier on cold IAVL reads. Re-measured through the real cache-wrapped, gas-metered store (DefaultGasConfig + DefaultParams), it now matches your gnodev (hashmap ≈16.2M vs your ~17.4M).

Corrected cold transfer:

holders avl two-level hashmap
20k 27.9M 16.2M (−42%)
100k 29.8M 16.5M (−45%)
1M ~35M 18.8M (−46%)

Itemizing that also surfaced why even hashmap floors at ~16M cold: a fixed ~10.7M IAVL cost, of which ~7.67M (half a cold tx) is the VM re-reading its 13 governance params as 13 separate depth-multiplied keys — backend-independent, on top of the ledger reads. Bundling those into one KV read is a measured flat −7.08M/tx; orthogonal to this PR, so I'll open it separately.

Direction and fix are unchanged — only the magnitudes (~2× low) and the old "18M @ 1M" anchor were wrong.

@omarsy

omarsy commented Jul 16, 2026

Copy link
Copy Markdown
Member Author

One more correction. I re-measured on current master, which has since merged the B+tree fast-index store (#5937/#5938). My numbers above were on a pre-fast-index base, so they're now too high: the fast-index cut cold reads 5-10x (a cold GET went 590k to 59k, param reads 590k to 118k).

Corrected cold transfer (real cache-store metering, master, bptree fast-index + Fixed depth pins applied):

holders avl hashmap
20k 19.66M 7.97M (-59%)
100k 21.59M 8.25M (-62%)

Two things flip vs my earlier post:

  1. This PR looks better, not worse. With the fixed floor now small, the hashmap advantage is -59% at 20k and -62% at 100k (was -42%), and it grows with ledger size. The dominant avl cost is now the base-store tree walk (~8.2M of 19.66M), which is exactly what the hashmap removes.
  2. The "param floor" I flagged is mostly gone. The fast-index already cut it from ~7.67M to ~1.65M, so the separate param-bundle idea is now a ~1.5M top-up rather than a ~7M lever.

Net: the thesis holds and the fix is stronger than even the corrected numbers suggested. I'd just been measuring on a base that predated the fast-index.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

📦 ⛰️ gno.land Issues or PRs gno.land package related 🧾 package/realm Tag used for new Realms or Packages.

Projects

Development

Successfully merging this pull request may close these issues.

3 participants