Skip to content

Integrate durable media delivery and saved-storage maintenance (candidate graph) - #2029

Draft
Brian Krabach (bkrabach) wants to merge 44 commits into
mainfrom
feat/durable-media-storage
Draft

Brian Krabach (bkrabach) wants to merge 44 commits into
mainfrom
feat/durable-media-storage

Conversation

@bkrabach

@bkrabach Brian Krabach (bkrabach) commented Nov 14, 2026 •

Copy link
Copy Markdown
Collaborator

Status

Draft: in progress, not ready for merge or release. Current head a947e96b0f389bd842c173daecb24f8230e51247, tree f7029ba74bff37fbbcf916d62cc66f9cc52d0355.

Integrated source

  • Accepted main ba4127997052df7b34045b58a374ca1e6d17c1ab merged as845768167, preserving accepted Windows/auth/ACP/managed-Docker/pricing work. All28 new-main nonclient/generated entries exact, all22 prior committed-reader entries unchanged. ExecutionEvents retains both pricing metadata and awaited media/options. Main release metadata is inherited, not a media release reservation.
  • Selects accepted PR2072 whole canonical client190217f55969112490e8e3e4039c4b14d7532fae / tree6fbf712baf10ef5711b55fa102af9f7d78376620, all663 tracked files. All16 accepted Optimize/repair paths remain exact17ec. Complete accepted build2cceae6c5a546047 / static tree9fc8cd8beb9b86e9e1ea652660c1dfb7eb16b8ea retained:315 manifest-listed assets plus receipt=316files. Normal prepare/verify and all asset hashes checked; no rebuild/chunk editing. Paired-server provenance remains explicitly unqualified, not actual installed pairing.
  • Exact Optimize browser owner/order is validated before subtracting its one addition from frozen historical inventories. Six negative controls reject altered/missing/duplicate owners; installation-repair ownership unchanged. Earlier missing-order/count/test-NameError attempts retained.
  • Real safe diagnostic export timestamp repair0139 keeps indexed original time without restoring private event_id/raw fields.
  • Exact capture checkpoint f45/cdcbe uses sealed CI batches and selected installed-slot proof. Unknown historical authority remains HELD; synthetic group fixtures are mechanism evidence, not production migration eligibility.
  • Committed T2 reader/inside-context ACK packet9decd+efaf is exact. New prospective canonical storage/bundle/rewind packet02b80+Linux-only test marker58ea is integrated518b6e85a+a947. It requires the actual live VersionedSessionBinding, explicit expected generation/stable intent/admit, publication lifetime and no unknown replay. No runtime caller files were imported from that packet.
  • Browser fixture now stages TWO real pending outbox rows through Diagnostics.configure/record/flush, scheduler stopped before destination, transport guard never called. Complete row/payload/idempotency equality and no extra enqueue are required; this is pending-outbox preservation, NOT successful remote delivery/ACK.

Focused evidence — separate immutable source inputs, not an aggregate

  • WHOLE Foundation88664390ed0d43d46e17cad89c17aaa5cbd8d833 + CI1ea source archives:81 passed/1 explicit real-runtime bundle opt-in skip,43.10s;65 capture/retention/exact-ACK passed/zero skips,8warnings,40.57s. Single whole Foundation contains neutral lifetimee2af and mode issuer12da; not a mixed overlay. Linux prospective new IDs only. Full installed caller qualification is pending.
  • Current accepted-main pricing/media/registry joins:11 passed/zero skips,1.12s. Separate frozen coverage10 passed/1 existing config warning,0.48s. Existing historical counts remain unchanged.
  • Real integrated public Optimize/expiry/records/export on b7c4/A03/CI1ea:partial1completed2held,5indexedrecords,2pendingrows unchanged,attempts0/transportCalls0. No service overlay, listener or rendered browser.
  • Earlier diagnostics41 passed/zero skips; earlier accepted-main17 passed/zero skips; reader21 passed/1 missing-old-source-input skip then that case separately1 passed with exact old42/496 inputs. Earlier74/52/102/118/42/76 and65 runs, failures and timeouts retain original inputs and limits.
  • Independent narrowed source reviews cover merge preservation, exact reader and five-file checkpoint joins, browser/export/outbox oracle and strict registry owner. Reviewers did not independently rerun these reported tests.

The actual media-worker owner reports a separate source-only fixture milestone09c98f5ce:two turns on one worker/provider/writer, explicit retirement then fresh-worker resume with private PNG and no replay. Manager read rawc2625337 and verified21 manifest hashes/7 host Git joins. It is NOT integrated here, installed parity is explicitly false, and three supplemental naming calls were refused attribution, not passed.

Remaining acceptance

Complete actual runtime root/finite/selected/nested/resumed media and prospective persistent-writer callers, cancellation/retirement and mode transitions; final exact host packet and accepted dependency graph in BOTH root and separately packaged worker; supported provider private-wire/privacy plus actual pixels; one installed guest with wheel/loaded-byte joins, real rendered Optimize/browser journey; remaining independent review and sole Release Manager publication.

Previous remote CI missing unreleased Foundation modules remains failed, not waived or green from source overlays. Current candidate upstream source is not an accepted merged graph. No Boolean/names-only/digest-only enablement, inferred append suffix, hidden canonical fallback or second writer.

Unknown historical origin/cardinality/order/original TTL stays HELD, separate from future refs-first delivery and eligible diagnostic expiry. No fabricated raw captures/receipt registry or net reclaimed-filesystem-bytes claim. Planned/unlinked/held/recovery values remain distinct.

Owned Unified guest0/cap1, program cap3 with separate Foundation/CI allocations. No duplicate guest, Core/new protocol, user live migration/cleanup/restart/adoption, Mac campaign, merge or media release. Current published releases remain independent; sole RM owns publication.

Checkpoint host API, explicit policies, per-file ledger and real CI retention adapter. Historical image/canonical bindings remain explicitly held pending their owning integration.

Generated with Amplifier

Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
Validate exact native ownership and indexed slices before replacement, preserve completed rebind receipts, hold unverifiable original image policies and exercise the real CI/Foundation diagnostic binding. Retain partial-stage and candidate-graph qualification limits.

Generated with Amplifier

Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
Keep combined selections explicit; a source with no qualified inline images goes through the existing expiry owner without staging a no-op conversion or weakening replay fences.

Generated with Amplifier

Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
Quarantine only the changed capture while unrelated diagnostics persist, classify exact Optimize receipt reads as passive Work recovery, propagate expiry metadata through coalescing/browser projection, and rename the operation-journal helper to distinguish it from app state publication. Actual client expiry rendering is a named canonical-client handoff.

Generated with Amplifier

Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
@bkrabach

Copy link
Copy Markdown
Collaborator Author

Candidate integration update at dc77a3f plus test-only 0ef1cf1:

  • Joined backend repair 144c3ea: 127 affected source cases passed, zero skipped, 17 warnings, 53.79s against immutable CI49ea460a + Foundation600b620f. Exact raw logs and JUnit retained locally.
  • New maintained real signed-transfer expiry-during-decode regression: old Foundation600b620f FAILS with expired-import metadata-only refusal after source release; candidate Foundationf427fc2c passes all three transfer controls (3 passed, zero skipped,22.36s). This is candidate cause-matched proof; final upstream security review/accepted graph is pending.
  • Current remote native run43539913624/job132277993107 remains FAILED: 3failed/397passed. Attachment positive probe fails because selected installed Foundation has no session.media yet; this must be resolved by the accepted dependency graph, not a compatibility fallback/skip. Two unchanged worker-parking cases also fail; their context.prepare fixture assertion is observed but not yet proven causal. Focused diagnosis is underway. Skipped default-installed/bootstrap lanes are not passed.

Draft is still not merge-ready. Producer graph qualification, compound conversion/expiry, old-origin policy, final client57 acceptance/mirror/build and isolated/installed checks remain pending; no release/live effects.

Reuse CI verified_optimize continuation without clearing its permanent replay fence. Checkpoint the completed image stage and preserve both maps; reconcile lost expiry acknowledgement without repeating prune, keep final diagnostic ACK inside writer/index fences, and report separate stage accounting with unknown aggregate contributions. Preserve quarantined references/outbox under ordinary age and count retirement. Exercise real owner stages, a lost post-replacement receipt and bounded 16 MiB raw diagnostic maintenance.

Generated with Amplifier

Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
@bkrabach

Copy link
Copy Markdown
Collaborator Author

Retained integration checkpoint: exact pushed head3bee560f9dcb596b46fe06582a1c3d0d78510862/tree7811f939f46529396d43732d788ac705a2e703d8. This remains DRAFT, dependency-blocked, NOT complete or merge-ready.

  • Compound image extraction -> diagnostic expiry -> second exact index mapping -> final diagnostic media ACK is now joined.131 affected source cases passed, zero skipped,56.78s at immutable CIab556384 + Foundation543f8875. Unknown aggregate bytes remain unknown; recovery originals/sidecars retained; no reclamation inferred.
  • Final reviewed Foundation543f8875 expiry guard passes the actual signed-transfer during-decode case. Old600b620f failure is retained; priorf427 three controls reused. No pin acceptance is inferred.
  • Fixture-only3bee answers the current context.prepare batch in warm/history probes;9 focused cases passed, independent narrow review found no actionables. It DOES NOT restore runtime parking or weaken the park assertions.
  • Cause diagnosis from actual native43539913624 artifact14092220382: loop-live7828753b removed live.park at upstream918528cf. Caught stale context.manifest fixture assertion was not the parking cause. Separate lifecycle owner/activation-contract repair is pending; no baseline-failure or green native claim.

Blocking external inputs: positive qualified post-hook graph/cooperative nested-route disposition (safe bridge remains always incomplete), original historical-provenance policy, named lifecycle owner, final parent Astra/Fable/upstream acceptance, accepted dependency graph/pins, canonical client57/676 acceptance then whole-copy/source-record/build/pairing, and assigned isolated test capacity. Runtime producers stay gated off; that is not forward-feature delivery. Old unknown-origin cohort remains held0eligible. No DTU, inference, live cleanup/migration/restart/adoption, merge, version or publication occurred. Finite evidence/branches/PRs are retained by manager152.

@bkrabach

Copy link
Copy Markdown
Collaborator Author

Narrow host mapping-boundary repair pushed at exact 03d4743 (tree22e2e6d22919b9524dfbcbbdca4d279e4bab298f). CI maps nonblank physical events, while the current host rebind requires contiguous LF-terminated spans. Fresh host admission now holds blank-span/unterminated layouts BEFORE replacement or raw expiry, using bounded chunks during the existing source hash pass. This does not rewrite gaps, invent events, clear retained holds or replay uncertain operations. Original source/stat/cursor/reference/outbox guards remain.

Cause-matched old3bee function overlay:9 FAIL leading/inter/trailing blanks after actual CI commit, retained. Final affected tests111 PASS/0skip/8warnings18.02s with exact task-owned CIe6de12c343a533187b836f4a99042c415eed2f18 + Foundation87d257e4057865688e9d0294ba170fe4ad1d1cec Git-archive imports; no installed graph/pin claim. Real CI stage/apply/combined and raw-expiry fixtures assert precommit refusal, source+sidecar/records/cursor/delivery bytes unchanged. Contiguous positives and lost-ACK reconciliation retained.16MiB single-event Python peak1,147,420B. Initial fixture IndexError and two overstrict VT/FF exception expectations retained/repaired honestly; CI may hold VT/FF before host callback. Independent exact changed-risk review clear, not separate test rerun or group qualification.

Raw final log e36d20a3df83c73a2e6686931cf98662729281f1156d3525c85438f35edb65bc; JUnit378c0b7d2c8eb4fe45f55eadf956340abcb81a06eb188c5cffc7ff73170b315c. Full task remains open/dependency-blocked: production old-image original provenance/cardinality missing0eligibleheld, no multi-group binding/accepted graph, positive bridge/cooperative contract, parking lifecycle, client acceptance/synchronization/build and isolated qualification still pending. No DTU/live operations, merge or release.

@bkrabach

Copy link
Copy Markdown
Collaborator Author

Current source checkpoint: 31c53bb, tree dbc72a55aec52d969640ced971fca3e4065144d5, now includes reviewed capture-lifetime repair b64cbcbbd. One native/flush fence and one worker thread cover conversion, combined expiry, rebind, final ACK and cancellation cleanup. Access-time-only self-read drift no longer causes false index rejection; content/identity/write/change witnesses remain. Actual focused source pairing with Foundation b279 and CI e6de: 118 passed, zero skips, 8 warnings, 18.67s. Earlier b279/current-host media/assets/storage/canonical pairing: 102 passed, zero skips, 65.87s. These are separate source-only mechanism runs, not installed or positive-hook-graph acceptance.

Parent explicitly accepted the truthful cohort boundary: refs-first future images and genuinely eligible diagnostic-expiry Optimize operations qualify separately; historical inline images missing original origin/group/TTL receipts remain held. No new raw captures or fabricated receipt registry was added to manufacture an old-image conversion pass. No reclaimed bytes or historical migration claimed. Positive Worker/child/transition and built-client browser qualification remain in progress. Prior remote missing-Foundation-module CI failures remain retained pending accepted root and worker dependency selection; no skips or protections were altered.

@bkrabach

Copy link
Copy Markdown
Collaborator Author

Independent shared-policy integration is pushed at 0afe9c1, tree a55c70a296c0e459a1bf3ebd009ba54096fe17c3. It consumes Foundation shared MediaPolicy() defaults rather than treating the upload chooser file-count/size/pixel settings as durable capture quotas. Upload chooser controls are unchanged. The real host store now admits a group exceeding that chooser count, publishes 12 distinct PNGs, saves and acknowledges the canonical references, and resolves every original byte sequence.

Actual focused source pairing with fixed Foundation d5451144ab095d6561ce0b04e9746b9f3c176545: 42 passed, zero skips, 16.54s across policy, host storage and assets. The contribution received independent source review; this is not an installed Worker/child graph or inference pass. Positive root/child/lifecycle integration, exact client acceptance/copy/build, rendered-browser qualification and Release Manager publication remain unfinished. The PR stays draft and the overall task stays active.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant