test(scale): certify the final billion-edge lifecycle and portable-v2 round trip

[DecisionNerd]

Problem

Component tests cannot establish the milestone outcome. The integrated final tree needs one reproducible public-facade proof with exact scale, resource, recovery, portable-v2 verification, and clean-import evidence.

Developer laptops (including macOS Air-class machines) are not a valid system under test for this certification: peak RSS, disk, and wall-clock numbers from a laptop are not product evidence.

Objective

Run and publish the final billion-live-edge certification on a named Linux cloud / provisioned evidence host across construction, atomic publication, reopen, CSR, bounded queries, deterministic portable-v2 bundle export, full integrity/compatibility verification, clean import, reopen, and fingerprint comparison.

Requirements

• Freeze the exact integrated commit and declared cloud hardware/software environment (provider, SKU/vCPU/RAM/disk class, OS image, tool versions).
• Execute the certification only as an explicitly provisioned cloud/CI evidence job (or equivalent rented Linux host matching the declared SKU). Local #[ignore] / Make targets may be used for small ladder rungs and dry-runs; they must not be submitted as test(scale): certify the final billion-edge lifecycle and portable-v2 round trip #745 success evidence.
• Use the approved deterministic profile and lower-rung preflight (ladder contract from test(scale): define the billion-live-edge public-facade contract and first-fail ladder #736 / scale_g500_ladder.v1.json).
• Generate/import Graph500 SCALE26/edgefactor-16 data or a documented equivalent profile that still yields >=1,000,000,000 live persisted edges.
• Report raw attempts separately from self-loop/duplicate/rejected counts and live persisted counts.
• Build/use adjacency and execute deterministic 1-hop and 2-hop LIMIT queries through the public facade.
• Export the complete project as the canonical portable-v2 bundle, run a full feat(io): verify expanded and bundled portable projects #784 verification pass, import into a clean workspace, reopen, recount, rerun queries, and compare package/source/project fingerprints.
• Record semantic package digest separately from transport digest and source-generation fingerprint.
• Record elapsed time, peak RSS, and peak disk for every phase (generate, ingest, reopen/CSR, query, export, verify, import, reopen, query-after-import, negative drills).
• Run corruption, cancellation, resource-limit, and recovery drills at representative bounded scale.
• Consume the completed epic(io): ship portable project v2 packaging verification and promotion #740 conformance ledger for equivalent expanded/bundled identity, selective packages, OCI promotion, and cross-language parity; do not duplicate a billion-edge payload or require hosted-registry upload solely for this proof.

Host capacity (declared cloud SKU)

Stay within the machine class used for the run (not a laptop claim):

• Peak RSS ≤ 128 GiB
• Peak local NVMe ≤ 1 TiB

Wall-clock (useful certification fail-safe)

Replace the legacy 24 h host ceiling with a product-useful fail-safe on the declared cloud SKU:

• Full certification wall-clock ≤ 4 hours (timeout_s: 14400) total end-to-end on the frozen commit + declared SKU.
• 4 h is a runaway/kill envelope and provisional certification budget — not a marketing SLA and not a laptop expectation.
• Evidence must publish per-phase wall times so tighter product SLOs can be set from measured cloud dry-runs without pretending overnight completion is “normal usage.”
• If a phase or the full run exceeds the fail-safe, stop with the first failing phase documented; do not emit a success or scale claim.
• After the first successful cloud dry-run, revise phase budgets downward in the profile/docs via a follow-up PR if measurements support it.

Acceptance criteria

• The published evidence identifies exact commit, command/profile, cloud host SKU, tool versions, seed, package format/version, and policy.
• Evidence explicitly records that the SUT was provisioned Linux cloud (not a developer laptop).
• Live persisted edge count is >=1,000,000,000 after all policies.
• Full verification reports the final bundle complete, cryptographically valid, and semantically compatible before import.
• Source and imported project reopen with matching vertex/edge counts and ontology/capability authority.
• Required 1-hop and 2-hop query observations and canonical fingerprints match.
• Package digest, transport digest, source generation, and imported generation identities are distinct and reconciled.
• Every phase stays within the host capacity envelope and the ≤4 h wall-clock fail-safe, or the issue remains open with the first failing phase documented.
• Per-phase timings (generate → … → post-import query + drills) are present in the checked-in evidence artifact.
• Negative drills prove no partial publication after corruption, cancellation, resource failure, or interrupted finalization.
• epic(io): ship portable project v2 packaging verification and promotion #740's representative-scale conformance evidence covers expanded/bundle equivalence, selective privacy/closure, OCI digest pinning, and Rust/Python/Node/CLI parity.
• Exact-head CI and CI Gate are green for the final evidence/integration PR (small always-on rungs only in normal CI; SCALE-26 remains a dedicated evidence job).
• Evidence is checked into a durable, reviewable repository location without committing the large dataset, project package, secrets, or host-specific paths.

BDD completion scenarios

Scenario: Live scale is reported honestly

Given SCALE26 raw attempts contain removed rows
When counts are reported
Then raw, rejected, duplicate, self-loop, and live counts reconcile
And the live persisted count independently satisfies one billion.

Scenario: Verified round trip preserves the project

Given the complete portable-v2 bundle has been exported
When it is fully verified, cleanly imported, reopened, and queried
Then counts, ontology/capability authority, query observations, and canonical fingerprints match the source
And verification occurred before import mutation.

Scenario: Cloud SUT is required for certification

Given a run is proposed as #745 success evidence
When the SUT is a developer laptop or undeclared local machine
Then the evidence is rejected
And only a named Linux cloud / provisioned evidence host is accepted.

Scenario: An envelope violation remains a failure

Given a phase exceeds RSS, disk, the 4 h wall-clock fail-safe, or configured resource limits
When certification stops
Then the first failing phase and measurements are recorded
And no success, integrity, or scale claim is emitted.

Observability and security

Record aggregate phase timings, peak resources, row/entry/byte counts, package/transport/source fingerprints, and typed failures. Exclude graph/package content, UUID inventories, secrets, credentials, absolute host paths, and registry tokens.

Testing and documentation

Map the large run to the scale and full-bundle scenarios; run negative drills at bounded representative scale; validate the evidence schema; document exact cloud reproduction, interpretation, environmental caveats, and links to #740 child conformance evidence. Keep normal CI on S10 (and similar) only.

Non-goals

A benchmark leaderboard, a universal hardware-independent capacity claim, laptop certification, a 24 h “normal usage” story, duplicating the billion-edge payload into both local representations, or uploading the full proof to a hosted OCI registry.

Related issues

Canonical tracker: #735. Portable-v2 epic: #740. This final close gate is blocked by #736–#744 and #783–#787. Ladder profile/docs must match this cloud SUT + 4 h fail-safe (see docs/development/perf-g500-ladder.md and scale_g500_ladder.v1.json).