Skip to content

Finalize SQL Implementation#1291

Draft
yyforyongyu wants to merge 19 commits into
impl-secret-storefrom
impl-runtime-store-6
Draft

Finalize SQL Implementation#1291
yyforyongyu wants to merge 19 commits into
impl-secret-storefrom
impl-runtime-store-6

Conversation

@yyforyongyu

Copy link
Copy Markdown
Collaborator

Finalizes the store-backed (SQL) wallet implementation. Adds the kvdb legacy
lifecycle/store adapters, the runtime DBConfig (backend selection) and sync
state, and routes the legacy manager, signer, and syncer paths through the
db.Store instead of a raw walletdb handle. buildWallet then constructs and
runs the wallet from the selected store — kvdb via the legacy adapters, or a SQL
store — with recovery and full unit coverage.

Config.DB is retyped from a raw walletdb.DB handle to DBConfig. Because
that retype fuses every cfg.DB consumer with the mutually-dependent
construction pipeline, the wallet: construct store-backed runtime wallet
commit is intentionally atomic (no smaller subset compiles); the surrounding
commits stay small and each builds and test-compiles independently.

Stacked on impl-secret-store.

@coveralls

coveralls commented Jul 13, 2026

Copy link
Copy Markdown

Coverage Report for CI Build 29839445611

Warning

No base build found for commit 5799790 on impl-secret-store.
Coverage changes can't be calculated without a base build.
If a base build is processing, this comment will update automatically when it completes.

Coverage: 57.464%

Details

  • Patch coverage: 606 uncovered changes across 28 files (1894 of 2500 lines covered, 75.76%).

Uncovered Changes

Top 10 Files by Coverage Impact Changed Covered %
wallet/manager.go 766 555 72.45%
wallet/runtime_store.go 274 199 72.63%
wallet/syncer.go 383 314 81.98%
wallet/runtime_builder.go 105 61 58.1%
wallet/controller.go 205 184 89.76%
wallet/recovery.go 106 86 81.13%
wallet/internal/db/kvdb/driver.go 86 67 77.91%
wallet/internal/db/kvdb/lifecycle.go 72 54 75.0%
wallet/internal/db/kvdb/utxostore.go 47 30 63.83%
wallet/address_manager.go 22 9 40.91%
Total (40 files) 2500 1894 75.76%

Coverage Regressions

Requires a base build to compare against. How to fix this →


Coverage Stats

Coverage Status
Relevant Lines: 62551
Covered Lines: 35944
Line Coverage: 57.46%
Coverage Strength: 14406.79 hits per line

💛 - Coveralls

Comment thread wallet/internal/db/kvdb/signer.go Outdated
Comment thread wallet/internal/db/kvdb/driver.go
Comment thread wallet/internal/db/kvdb/driver.go
Comment thread wallet/db_ops.go
Comment thread waddrmgr/manager.go
Comment thread wallet/signer.go Outdated
Comment thread wallet/signer.go Outdated
Comment thread wallet/manager.go
Comment thread wallet/syncer.go Outdated
Comment thread wallet/syncer.go
Comment thread wallet/manager.go
Comment thread wallet/manager.go
Comment thread wallet/controller.go Outdated
Comment thread wallet/wallet.go Outdated
Comment thread wallet/runtime_store.go
Comment thread wallet/controller.go Outdated
Comment thread wallet/controller.go Outdated
Comment thread wallet/manager.go
Comment thread wallet/controller.go Outdated
Comment thread wallet/internal/db/kvdb/lifecycle.go
Comment thread wallet/syncer.go Outdated
Comment thread wallet/manager.go Outdated
Comment thread wallet/tx_reader.go Outdated
Comment thread wallet/db_ops.go Outdated
Comment thread wallet/syncer.go
Comment thread wallet/controller.go
Comment thread wallet/controller.go Outdated
Export the wallet birthday safety margin as BirthdaySafetyMargin so the
runtime store construction path can back a requested birthday off by the
same duration the legacy Create path uses, instead of hardcoding the
value in two places.
Define backend-specific runtime database settings, validation, and safe
defaults before wiring them into wallet construction.
Add backend-neutral runtime store handles and open paths for KVDB,
SQLite, and PostgreSQL without changing wallet construction yet.
Construct unpublished wallets from the selected runtime store while
retaining backend-specific legacy managers only for KVDB.
Add per-name construction guards so waiters cannot observe or rebuild a
wallet before its owner publishes a complete instance.
Add SQL runtime wallet creation and secret seeding helpers, and reject
watch-only master xpub creates that cannot yield accounts.
Route signer, recovery, rescan, transaction-read, and account paths
through Store. The shared stable account identity changes must move
together because recovery snapshots, target resolution, and scan
persistence use the same key.
Switch Manager and Config to runtime stores, migrate construction tests
and harnesses, and remove the legacy db_ops implementation after its
production callers have moved behind Store.
Set the wallet DBConfig with an explicit kvdb backend and the harness
walletdb path so the manager integration test drives the store-backed
construction path.
Make terminal teardown single-owner and ordered: clear secret material,
evict the manager cache entry, close stores, record the close result,
then publish Stopped to all waiters.
Serialize same-name Load calls with the manager construction guard, and
hold that guard through teardown so fresh stores cannot open before the
prior instance closes.
Guard forward-only synced-tip updates.

Preserve the last known-good tip when Store reads fail.

Expose TxStatus through wallet for migrated runtime callers.
Regenerate watch-output scope queries and add the create-complete wallet
column. Existing SQL wallets migrate as complete while new creates can
persist an incomplete state explicitly.
Key targeted rescans by durable account IDs.

Cover SQL and kvdb behavior for imported and derived accounts.
Persist incomplete state before account initialization and mark it
complete only after initialization succeeds. Complete wallets reject
over-create before mutation, while incomplete SQL and KVDB wallets
replay missing accounts after strict identity checks.
Add a Manager-scoped registry that normalizes SQL store identities and
hands create, load, and recovery paths reference-counted leases.

Serialize pool opens and final closes so concurrent wallets share one
store without closing peers or opening duplicate connection pools.
Bind incomplete creation to a durable normalized account manifest.

Reject mismatched retries and incomplete loads.

Complete only the original initialization plan.
Coordinate DB-wide SQL rollbacks across wallets sharing one runtime
store. Quiesce syncers before reset, preserve genesis on full resets,
and serialize global rewinds against ordinary SQL writes.

Reject invalid rescan anchors and immediately restart syncers after a
coordinated reset without lifecycle churn.
@yyforyongyu
yyforyongyu force-pushed the impl-runtime-store-6 branch from 950e3de to 99030fe Compare July 23, 2026 17:22
@yyforyongyu

Copy link
Copy Markdown
Collaborator Author

No issues found by glm-5.2 on re-review of the updated PR. 🛠️


r.mu.Unlock()

<-done

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The opening wait here blocks on <-done with no ctx.Done() branch, while the entry.closing wait right below does select on the caller's context — a caller whose context is canceled during a slow or stuck pool open stays blocked until the open finishes; make the opening wait context-aware like the closing one. k3

// metadata and not an ordinary block or an uninitialized block reference.
func ValidateGenesisBlock(genesis Block) error {
if genesis.Height != 0 {
return fmt.Errorf("%w: genesis height must be zero", ErrInvalidParam)

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line is 85 columns (limit is 80, counting a tab as 8), and the overflow repeats across the new code: block_common.go:34 (82), runtime_store_registry.go:143 (85)/:166 (84), reset_coordinator.go:127 (83), syncer.go:277/:486 (83), manager.go:376 (83) and ~15 more at 81-82; rewrap them. k3

@yyforyongyu

Copy link
Copy Markdown
Collaborator Author

No issues found by gpt-5.5 🐋


c.mu.Unlock()

err := req.apply(full, height, genesis)

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Participants remain syncStateSynced throughout quiescence and this DB reset, so CreateTransaction and FundPsbt still pass validateSynced and can select stale UTXOs because reads are not write-gated; mark every participant unsynced before cancellation and keep it unsynced through restart. (gpt-5.6-sol)


return lease, nil

default:

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After a failed open, opening is false while openErr and old reservations remain, so a fresh acquire enters this branch and clears that generation before its waiters finish, letting them return an uncounted or nil replacement lease; wait for failed reservations to drain or tag reservations by generation. (gpt-5.6-sol)

)
}

digest := sha256.Sum256([]byte(runtimeCfg.Postgres.DSN))

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hashing the raw DSN gives equivalent URL, keyword, or reordered-option spellings separate pools and reset coordinators for the same PostgreSQL database, allowing one alias to write during another alias’s coordinated rollback; derive identity from a canonicalized parsed configuration. (gpt-5.6-sol)

c.mu.Lock()

for syncer := range c.syncers {
syncer.resetAfterChainReset()

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ResetChain rewrites every affected wallet’s persisted birthday, but this callback only resets syncer state, leaving Wallet.birthdayBlock stale so Controller.Info can report a block the reset deleted indefinitely; refresh each participant’s wallet-level birthday cache before restart. (gpt-5.6-sol)

Comment thread wallet/manager.go
)
}

err = verifyCreateManifest(w.createManifest, manifest)

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The manifest excludes the private passphrase and only spendable SQL account seeding authenticates it, so watch-only shell and KVDB seed/xpub retries can complete with a passphrase that cannot unlock the persisted secrets; authenticate the retry passphrase before marking creation complete. (gpt-5.6-sol)

Comment thread wallet/tx_reader.go
errNilTxDetailMsgTx = errors.New("tx detail MsgTx is nil")
)

// TxStatus is the wallet-relative validity state of a transaction, exposed on

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This unrelated public TxStatus API addition was previously isolated but is now folded with two sync-tip fixes into the 838-line wallet: harden runtime sync state commit; restore it as its own atomic commit for reviewability and safe reverts. (gpt-5.6-sol)

func (s *Store) execWrite(ctx context.Context,
fn func(*sqlc.Queries) error) error {

s.writeGate.RLock()

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This RLock is acquired before ctx is observed, so a canceled write blocked behind a detached or stuck chain reset cannot return until that reset finishes; use a context-aware gate here and in the SQLite mirror. (gpt-5.6-sol)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants