bucket_spec.md

marimo Notebooks — Object-Storage Backend

Schema Design, Storage Architecture & CRUD API

Scale target: 10 Projects × 100 Notebooks · Iceberg-inspired metadata layer · No database

---

1. Overview & Design Principles

This document defines the bucket schema for marimo notebook create, read, update, and delete operations. It uses the Bucket port without an external database. The content model follows the Apache Iceberg metadata indirection pattern. catalog.json is the only mutable content pointer, and immutable snapshots and versions retain history. _system/ also stores mutable coordination records outside the content snapshot chain.

Core invariant: catalog.json is the only content pointer overwritten in place. Every content-history write creates a new object. _system/ stores mutable sessions (§4.8), app claims (§4.8.1), editor claims (§4.8.2), identities (§4.10), and tokens (§4.11). These records do not participate in the snapshot chain. Each CAS-managed record is written through its owning core service.

Design Goals

• No database — the configured Bucket adapter is the persistence layer
• O(1) reads for listing projects and notebooks — no object scanning
• Immutable snapshots give audit history for free
• Self-describing notebooks — each notebook folder is a complete, portable unit
• A local notebook has a latest-only workspace/. A Git-synced notebook stores each pushed workspace inside its immutable version.
• Typed source pointer — local notebooks store source in the workspace. git notebooks receive pushed revisions from an external workflow.
• Append-only audit events — one immutable object per event
• Scales comfortably to 10 projects × 100 notebooks per project (1,000 total)

Storage Requirements

Because there is no database, the object store itself must provide the guarantees the design leans on:

• Strong read-after-write consistency — a read (or list) immediately following a write reflects the latest data, so a writer always sees the snapshot it just committed.
• Conditional writes (compare-and-swap) — If-Match on ETag for the atomic catalog.json swap, returning 412 Precondition Failed on mismatch; If-None-Match: * (create-if-absent) for first-write bootstrapping.
• Stable ETags — head, list, and get return the same non-empty ETag for an unchanged object, and the ETag changes whenever the contents change (a content-hash backend such as storage-fs may keep it across an identical rewrite). The per-process metadata and session caches, and the ensureInitialized catalog check, rely on this to skip downloads. A backend that cannot supply an ETag returns '', and none of these caches store or match such a value.

AWS S3 provides both (strong read-after-write consistency since 2020; If-Match / If-None-Match on PutObject since 2024). Verify these for any other S3-compatible target (MinIO, Ceph RGW, Backblaze B2, Tigris, …) — conditional-write support in particular varies by implementation and version. This is the single most important portability constraint for the design.

---

2. Metadata Layer

The _system/ namespace is the heart of the architecture. Understanding it first makes the rest of the schema obvious.

How the metadata layer works

catalog.json  →  snapshots/{id}.json  →  full project + notebook index
     ↑
only file ever mutated in place

catalog.json is a tiny pointer file — it names the current snapshot. Every write operation clones the current snapshot, applies changes, writes a new immutable snapshot, then atomically swaps catalog.json to point at it using a conditional PUT on ETag. If two writers race, one gets a 412 Precondition Failed and retries. No corruption is possible.

ensureInitialized caches a positive "catalog populated" result for 30 seconds per process. A wiped or reset bucket can take up to 30 seconds per replica to be detected and re-initialized.

A single GET on the current snapshot returns the complete project and notebook listing — all metadata, no code. At 10 projects × 100 notebooks with ~200 bytes of metadata per notebook, the snapshot is roughly 200KB. That's the entire index in one fast GET.

Snapshot schema versioning

Every snapshot carries a schema_version field. When the snapshot structure changes, the service upgrades the current snapshot in memory before it writes the next snapshot. It does not rewrite old snapshots.

---

3. Full Bucket Schema

3.1 Top-Level Layout

s3-bucket/
├── _system/
│   ├── catalog.json                        ← THE single mutable pointer
│   ├── snapshots/
│   │   └── {snapshot-id}.json              ← immutable, one per write
│   ├── sessions/
│   │   └── {project-id}/
│   │       └── {session-id}.json            ← kernel session record
│   ├── version-prune-cutoffs/
│   │   └── {project-id}/
│   │       └── {notebook-id}.json           ← monotonic version boundary for new sessions (CAS)
│   ├── proposal-payloads/
│   │   └── {project-id}/
│   │       └── {notebook-id}/
│   │           └── {proposal-id}.json       ← immutable payload-expiry marker
│   ├── app-pools/
│   │   └── {project-id}/
│   │       └── {notebook-id}.json           ← reservations, assignments, and visit leases (CAS)
│   ├── apps/
│   │   └── {project-id}/
│   │       └── {notebook-id}.json           ← legacy app-session claim (CAS)
│   ├── editors/
│   │   └── {project-id}/
│   │       └── {notebook-id}.json           ← persistent-editor claim (CAS)
│   ├── reconcile/
│   │   └── orphans/
│   │       └── {sandbox-id}.json            ← first-seen marker for an undated orphan
│   ├── job-runs/
│   │   └── {project-id}/
│   │       └── {run-id}.json                ← execution/finalization marker (deleted after finalization)
│   ├── identities/
│   │   └── {user-id}.json                  ← user display identity (mutable, CAS-managed)
│   ├── tokens/
│   │   └── {token-id}.json                 ← personal access token record (mutable, last-writer-wins)
│   ├── cli-authorizations/
│   │   └── {authorization-id}.json         ← short-lived PKCE login grant (CAS-claimed)
│   ├── cli-device-user-codes/
│   │   └── {user-code}.json                ← immutable lookup claim for a device grant
│   ├── oauth-clients/
│   │   └── {client-id}.json                ← immutable dynamic client registration
│   ├── oauth-authorizations/
│   │   └── {authorization-id}.json         ← short-lived MCP OAuth grant (CAS-claimed)
│   ├── oauth-rate-limits/
│   │   └── {endpoint}.json                 ← shared MCP OAuth request window (CAS)
│   ├── integrations/                       ← organization-wide integrations (§4.12)
│   │   ├── _names/
│   │   │   └── {name}.json                 ← name claim (CAS, app-claim pattern)
│   │   └── {integration-id}/
│   │       ├── integration.json            ← CAS-managed head
│   │       └── versions/
│   │           └── {000001}.json           ← immutable configuration version
│   ├── idempotency/
│   │   └── {digest}.json                   ← replay record with a 24h TTL
│   ├── events/
│   │   └── {YYYY-MM-DD}/
│   │       ├── {event-id}.json             ← one immutable object per event
│   │       └── _idempotency/
│   │           └── {key}.json              ← stable key mapped to one event
│   ├── sandbox-diagnostics/
│   │   └── {user-id}.json                  ← per-admin sandbox diagnostic lease (CAS)
│   ├── _maintenance.lock                   ← advisory maintenance lease
│   ├── _session_lifecycle.lock             ← advisory session-lifecycle lease
│   └── _jobs.lock                          ← advisory job-scheduler lease
│
└── projects/
    └── {project-id}/
        ├── project.json                    ← project metadata
        ├── alerts.json                     ← project alert destinations (CAS-owned, encrypted)
        ├── integrations/
        │   ├── _names/
        │   │   └── {name}.json             ← per-name singleton claim (CAS, app-claim pattern)
        │   └── {integration-id}/
        │       ├── integration.json        ← head: name/kind/enabled/current_version (CAS-managed)
        │       └── versions/
        │           └── {000001}.json       ← immutable config version (create-if-absent)
        └── notebooks/
            └── {notebook-id}/
                ├── meta.json               ← notebook metadata (title, author, tags…)
                ├── README.md               ← human description
                ├── source.json             ← `local` or `git` source metadata
                ├── integration_sync_token.json ← push-sync token hash (push-mode Git sources only)
                ├── fs_snapshot.json        ← optional provider filesystem-snapshot pointer
                ├── workspace/              ← local sources only: latest sandbox workspace
                │   ├── notebook.py          ← latest local code
                │   ├── pyproject.toml      ← latest local dependencies
                │   └── data/cars.csv       ← optional persisted runtime file
                ├── proposals/
                │   └── {proposal-id}/
                │       ├── proposal.json    ← immutable capture manifest
                │       ├── changes/
                │       │   └── {index}      ← temporary changed-file bytes
                │       └── publication.json ← CAS-managed publication state
                ├── jobs/                    ← headless runs (§4.13)
                │   └── {job-id}/
                │       ├── job.json         ← definition head (CAS, JobsService)
                │       ├── occurrences/
                │       │   └── {YYYYMMDDTHHmmZ}.json ← scheduled-fire claim (create-if-absent, immutable)
                │       ├── run-index/
                │       │   └── {reverse-ulid}.json ← empty, immutable newest-first history index
                │       └── runs/
                │           └── {run-id}/
                │               ├── run.json     ← run record (CAS, JobRunService)
                │               ├── output.html  ← write-once rendered output
                │               └── logs.txt     ← write-once stdout+stderr (capped)
                └── versions/
                    └── {version-id}/
                        ├── version.json
                        ├── notebook.py         ← local source snapshot
                        ├── pyproject.toml      ← local dependency snapshot
                        ├── workspace/          ← Git sources only: complete synced tree
                        ├── git/                ← pull-mode Git sources only: credential-free `.git` payload
                        ├── workspace.zip       ← optional packed startup copy of workspace + `.git`
                        ├── notebook.html       ← optional snapshot (see below)
                        └── session.json        ← optional snapshot (see below)

3.1.1 Local and Git workspaces

For a local source, the notebook-level workspace/ folder is the latest mirror of the sandbox working directory. It contains notebook.py and pyproject.toml. It can also contain runtime files when MARIMOHUB_PERSIST_WORKSPACE=workspace.

• Latest-only and non-versioned. workspace/ is overwritten in place on each save/teardown and is never touched by version pruning. The immutable per-version record lives under versions/{vid}/ (§4.7).
• Runtime files are opt-in. Under the default MARIMOHUB_PERSIST_WORKSPACE=source, workspace/ contains only notebook.py + pyproject.toml. Under workspace, the sandbox's non-source files are captured here on teardown and restored on the next session (§8).
• The mount is rooted at workspace/. When a sandbox mounts the bucket, the working dir maps to workspace/. Because meta.json / README.md / source.json / versions/ sit outside workspace/, control metadata is never exposed to user code in the sandbox.

For a git source, each sync writes a complete tree under versions/{vid}/workspace/. The source pointer selects one immutable version. The notebook has no mutable workspace mirror. A session receives a copy of the selected version and cannot write changes back. Each synced version keeps a best-effort workspace.zip copy of the final sandbox tree. Provisioning prefers this single object and falls back to the canonical workspace/ and git/ objects when it is absent or invalid. The packed copy is created for combined workspace and Git inputs up to 32 MiB.

3.2 Complete Path Reference

Path	Type	Description
_system/catalog.json	JSON	Single entry point. Points to the current snapshot. This is the only mutable content pointer.
_system/snapshots/{id}.json	JSON	Immutable index snapshot. Lists all projects and notebooks with metadata. Written once, never modified.
_system/sessions/{pid}/{sid}.json	JSON	Live session record, partitioned by project so a project-scoped read lists only _system/sessions/{pid}/. Created on notebook open, updated by heartbeat, and retired by an explicit stop or lifecycle deadline. Closing a browser tab does not immediately delete it.
_system/version-prune-cutoffs/{pid}/{nid}.json	JSON	Monotonic { cutoff_version_id } advanced before pruning old version folders. A new source-pinned session writes its record and then rejects itself if its version is at or before the cutoff; otherwise a later cutoff advance must see and protect that session. CAS updates are owned by SessionService. Hard deletion removes the record.
_system/proposal-payloads/{pid}/{nid}/{proposal-id}.json	JSON	Immutable expiry marker listing the temporary proposal change objects eligible for retention cleanup. It remains until every listed payload is deleted successfully.
_system/app-pools/{pid}/{nid}.json	JSON	Versioned pool record owned by AppPoolStore: reservations, visit assignments, visit leases, and deletion tombstones. See App pool records.
_system/apps/{pid}/{nid}.json	JSON	Legacy app-singleton claim, released through SessionService during pool migration. New app admissions use _system/app-pools/{pid}/{nid}.json. See §4.8.1.
_system/editors/{pid}/{nid}.json	JSON	Per-notebook persistent-editor claim. Records the shared/exclusive mode, names the only session allowed to save, and records idempotent takeover phases. All writes go through SessionService. See §4.8.2.
_system/identities/{user-id}.json	JSON	User display identity ({ id, email, name, picture_url?, suspended_at? }). Owned and CAS-updated by IdentityService. Resolves opaque author/user_id IDs to a person and gates suspended users at authentication time.
_system/tokens/{token-id}.json	JSON	Personal access token record ({ id, user_id, name, hash, … }) keyed by the ULID embedded in the presented mhub_pat_ bearer, so verification is a single GET. Stores only the SHA-256 of the secret. Mutable, last-writer-wins (coarse last_used_at refresh); revocation deletes the object. See §4.11.
_system/cli-authorizations/{authorization-id}.json	JSON	Ten-minute browser-to-CLI PKCE grant. Stores hashes/challenges, never a PAT or plaintext authorization secret. CliAuthorizationService CAS-claims it before minting one token, then deletes it. See §4.11.1.
_system/cli-device-user-codes/{user-code}.json	JSON	Create-if-absent lookup claim from an eight-letter device user code to its short-lived CLI authorization. CliAuthorizationService owns the claim. It deletes the claim after exchange and prunes it after ten minutes. See §4.11.1.
_system/oauth-clients/{client-id}.json	JSON	Immutable OAuth client registration. OAuthClientStore owns each record. Public clients expire after 90 days. See §4.11.2.
_system/oauth-authorizations/{authorization-id}.json	JSON	Ten-minute MCP OAuth authorization owned by OAuthAuthorizationService. Approval stores a one-time code hash with CAS. Exchange claims the record with CAS before it mints one scoped PAT and deletes the record. See §4.11.2.
_system/oauth-rate-limits/{endpoint}.json	JSON	Bounded sliding-window timestamps for an MCP OAuth endpoint. OAuthRateLimitService uses ETag CAS so all serving replicas enforce one deployment quota. Expired timestamps are removed on the next admitted request. See §4.11.2.
_system/events/{YYYY-MM-DD}/{event-id}.json	JSON	Structured event log. One immutable object per event, keyed by a monotonic ULID under a per-day prefix. Primary audit trail.
_system/events/{YYYY-MM-DD}/_idempotency/{key}.json	JSON	Create-if-absent mapping from a stable operation key to one event ID and serialized event body. A retry repairs a missing event object from this marker without changing its append position or payload.
_system/idempotency/{digest}.json	JSON	Recorded POST-create response for an Idempotency-Key, keyed by sha256(user:route\nkey). Replayed on retry; pruned after 24h. See idempotency.md.
_system/reconcile/orphans/{sandbox-id}.json	JSON	First-seen Unix timestamp for an active sandbox that has no session record and no provider creation time. The reconciler creates and deletes these markers.
_system/sandbox-diagnostics/{user-id}.json	JSON	Per-admin CAS lease for a sandbox startup diagnostic ({ sandbox_id, expires_at }). It prevents overlapping tests across serving replicas and makes the active sandbox visible to reconciliation. SandboxDiagnosticLease owns all writes; release CAS-writes a null free marker, and expiry recovers leases left by a crashed request.
_system/_maintenance.lock	JSON	Advisory lease for snapshot and event retention. MaintenanceLock owns this key.
_system/_session_lifecycle.lock	JSON	Advisory lease for the session lifecycle and reconciliation sweep. The lifecycle loop owns this key.
projects/{pid}/project.json	JSON	Project metadata: name, description, owner, members, tags. Optional security_labels (classification + compartments), written only by ProjectService.setSecurityLabels under super-admin standing; see §4.3.
projects/{pid}/alerts.json	JSON	Project-scoped Slack and signed-webhook destinations. CAS-managed and written only by ProjectAlertStore; endpoint URLs and signing secrets are path-bound managed ciphertext. The record is retained through the soft-delete window so the final project deletion alert can be delivered.
projects/{pid}/integrations/{iid}/integration.json	JSON	Integration head: kind, instance name, enabled, and the current_version pointer. CAS-managed via ETag mutateObject — written only by ProjectIntegrationsStore (a version bump must be atomic against a concurrent edit). See §4.12.
projects/{pid}/integrations/{iid}/versions/{n}.json	JSON	Immutable integration config version, keyed by zero-padded number so key order matches version order. Written create-if-absent. Secret fields contain managed ciphertext envelopes or external reference markers, never resolved plaintext. Sessions pin these version numbers, so history is never rewritten.
projects/{pid}/integrations/_names/{name}.json	JSON	Per-name singleton claim ({ integration_id, claimed_at }) anchoring integration-name uniqueness — the same claim class as the app claim, written only by ProjectIntegrationsStore via acquireSingletonClaim/releaseSingletonClaim. See §4.12.
_system/integrations/{iid}/integration.json	JSON	Organization-wide integration head. It has the same fields as a project integration head, except it has no project_id. OrgIntegrationsStore is its only writer. See §4.12.
_system/integrations/{iid}/versions/{n}.json	JSON	Immutable organization-wide configuration version. It follows the same create-if-absent and secret-handling rules as a project integration version.
_system/integrations/_names/{name}.json	JSON	Organization-wide name claim. Claims are unique within this tier, so a project integration can use the same name.
projects/{pid}/notebooks/{nid}/meta.json	JSON	Notebook metadata: title, description, status, author, tags, last_run_at. Never contains code or paths. Optional security_labels override, written only by NotebookService.setSecurityLabels; see §4.4.
projects/{pid}/notebooks/{nid}/README.md	Markdown	Human-readable description and usage notes.
projects/{pid}/notebooks/{nid}/source.json	JSON	Typed source record. local stores the current version pointer. git stores Git coordinates, mode, sync state, and the current synced version.
projects/{pid}/notebooks/{nid}/integration_sync_token.json	JSON	Push-sync credential for one notebook. The record stores a token hash, not the plaintext token. Present only for push-mode Git notebooks.
projects/{pid}/notebooks/{nid}/fs_snapshot.json	JSON	Optional. Pointer to the notebook's current CoreWeave-native filesystem snapshot ({ snapshot_id, captured_at, owner_user_id }). Mutable, last-writer-wins, written only by the teardown snapshot path. Exclusive editors restore snapshots only for the same owner; shared editors may restore across starters. Present only under MARIMOHUB_COMPUTE_COREWEAVE_FILESYSTEM_SNAPSHOT=true. Not recommended alongside MARIMOHUB_PERSIST_WORKSPACE=workspace — the two double-persist state.
projects/{pid}/notebooks/{nid}/proposals/{proposal-id}/proposal.json	JSON	Immutable proposal manifest containing capture strategy, author/session provenance, pinned Git source revision, and ordered change hashes. Created with create-if-absent.
projects/{pid}/notebooks/{nid}/proposals/{proposal-id}/changes/{index}	any	Temporary bytes for a non-delete proposal change. The index corresponds to proposal.json.changes; delete changes have no object. These objects may be deleted after retention expires even though their manifest remains.
projects/{pid}/notebooks/{nid}/proposals/{proposal-id}/publication.json	JSON	CAS-managed proposal publication state owned by NotebookProposalService. It records either pending state or the durable provider change-request result. For an updated change request, the root proposal's head_commit advances as the shared current-head pointer after each child result is durable.
projects/{pid}/notebooks/{nid}/workspace/notebook.py	Python	Latest local notebook code. Present only for a local source.
projects/{pid}/notebooks/{nid}/workspace/pyproject.toml	TOML	Latest local Python dependency manifest. Present only for a local source.
projects/{pid}/notebooks/{nid}/workspace/{path}	any	Files from an imported folder or a local notebook sandbox. Imported local workspaces always persist these files. Other local notebooks require MARIMOHUB_PERSIST_WORKSPACE=workspace. Latest-only and non-versioned.
projects/{pid}/notebooks/{nid}/workspace/{dir}/.marimohub-directory	empty	Directory marker that keeps an otherwise empty workspace directory listable. Written create-if-absent by NotebookWorkspaceService.createDirectory; hidden from listings, search, and sandbox restores; reserved as a path segment.
projects/{pid}/notebooks/{nid}/workspace_mutation_claim.json	JSON	Short-lived lease ({ holder, expires_at }) serializing workspace-browser mutations for one notebook. CAS-managed and written only by NotebookWorkspaceService; renewed during long copies and deletes, released to { holder: null, expires_at: null }, and replaced in place once expired.
projects/{pid}/notebooks/{nid}/versions/{vid}/notebook.py	Python	Immutable version snapshot of the code.
projects/{pid}/notebooks/{nid}/versions/{vid}/pyproject.toml	TOML	Dependency snapshot at time of version.
projects/{pid}/notebooks/{nid}/versions/{vid}/version.json	JSON	Version metadata: saved_at, author, message, parent_id, optional snapshot descriptors.
projects/{pid}/notebooks/{nid}/versions/{vid}/workspace/	any	Complete immutable tree from one Git sync. Present only for a git source version.
projects/{pid}/notebooks/{nid}/versions/{vid}/git/	any	Immutable credential-free .git payload for a pull-mode source version. Restored into the sandbox workdir at provision and pruned with the enclosing version. With a root_path, .git is restored to the Git root (the configured workdir) and the workspace to <workdir>/<root_path>.
projects/{pid}/notebooks/{nid}/versions/{vid}/workspace.zip	ZIP	Optional duplicate of the final synced sandbox tree used only as a startup fast path. Pull-mode Git metadata is stored under .git/. Canonical reads and fallback restores continue to use workspace/ and git/.
projects/{pid}/notebooks/{nid}/versions/{vid}/notebook.html	HTML	Optional. Rendered HTML snapshot, copied from __marimo__/notebook.html on session teardown. Immutable once written.
projects/{pid}/notebooks/{nid}/versions/{vid}/session.json	JSON	Optional. marimo session state (cell outputs), copied from __marimo__/session/{notebook}.py.json on teardown. Immutable.

---

4. File Schemas

4.1 _system/catalog.json

The single mutable entry point. Overwritten atomically using conditional PUT with If-Match on the current ETag.

{
	"version": 1,
	"updated_at": "2025-03-05T14:22:00Z",
	"current_snapshot_id": "snap-7h2k9qm4xz7rp3w8",
	"current_snapshot_key": "_system/snapshots/snap-7h2k9qm4xz7rp3w8.json",
	"previous_snapshot_id": "snap-5g43rv2s9pfw8w4d"
}

4.2 _system/snapshots/{id}.json

Written once per mutating operation and never modified. The schema_version field controls how the service reads the structure.

{
	"snapshot_id": "snap-7h2k9qm4xz7rp3w8",
	"schema_version": 1,
	"created_at": "2025-03-05T14:22:00Z",
	"operation": "notebook.create",
	"actor": "user_abc123",
	"projects": [
		{
			"id": "proj-7h2k9qm4xz7rp3w8",
			"name": "Data Science",
			"description": "Exploratory analysis notebooks",
			"owner": "user_abc123",
			"created_at": "2025-01-10T09:00:00Z",
			"updated_at": "2025-03-05T14:22:00Z",
			"notebook_count": 3,
			"security_labels": null,
			"notebooks": [
				{
					"id": "nb-5g43rv2s9pfw8w4d",
					"title": "Revenue Analysis",
					"description": "Monthly revenue breakdown",
					"status": "active",
					"source_type": "local",
					"author": "user_abc123",
					"created_at": "2025-01-15T10:00:00Z",
					"updated_at": "2025-03-05T14:22:00Z",
					"tags": ["finance", "monthly"],
					"last_run_at": "2025-03-04T08:30:00Z",
					"security_labels": null,
					"key_prefix": "projects/proj-7h2k9qm4xz7rp3w8/notebooks/nb-5g43rv2s9pfw8w4d"
				}
			]
		}
	]
}

key_prefix is an internal physical locator. The service removes it from API responses. Clients address notebooks by {project_id, notebook_id}.

Security-label projection. Project and notebook entries carry a tri-state security_labels: an object (labeled), null (known unlabeled), or absent (indeterminate — a legacy entry, or a mutation in flight). List filtering decides labeled/unlabeled entries in one batch and resolves indeterminate ones from the authoritative project.json / meta.json, failing closed on a read error. security_labels_pending: true marks a mutation in flight so routine projection writes (projectCatalogPatch and friends) do not resurrect the pre-mutation labels; only the mutation's own finalizing commit clears it. Both fields are written through CatalogService.mutateSnapshot like every other entry field.

Field	Values	Writer
security_labels	{ classification, compartments }, null, absent	Routine projections (from the authoritative record); parked absent by a label mutation
security_labels_pending	true or absent	Set by ProjectService/NotebookService.setSecurityLabels, cleared by the same mutation's finalize

4.3 projects/{pid}/project.json

{
	"id": "proj-7h2k9qm4xz7rp3w8",
	"name": "Data Science",
	"description": "Exploratory analysis notebooks",
	"owner": "user_abc123",
	"members": [
		{ "user_id": "user_abc123", "role": "admin" },
		{ "user_id": "user_def456", "role": "editor" }
	],
	"created_at": "2025-01-10T09:00:00Z",
	"updated_at": "2025-03-05T14:22:00Z",
	"tags": ["analytics", "internal"],
	"security_labels": { "classification": "SECRET", "compartments": ["element-a"] }
}

Security labels (optional): a classification plus required compartments, normalized on write (compartments deduplicated and sorted). Present only on labeled projects; absent means unlabeled. Written only by ProjectService.setSecurityLabels — an ETag CAS mutateObject on this record, bracketed by two catalog commits (project.security_labels.pending, then project.security_labels) that form the audit trail and park the snapshot projection indeterminate while the record changes (§4.2). Requires super-admin standing (security-labels.raise / security-labels.lower); no project role grants label authority. If-Match is honored (412 on a stale version, checked before the projection is parked).

Member roles: manager has full project control, including member management. editor can change notebooks. viewer has read-only access. admin is reserved for the project owner, deployment super admins, and grandfathered member rows. See §12.

4.4 projects/{pid}/notebooks/{nid}/meta.json

Contains only notebook-level concerns. No code, no paths, no source details — those live in source.json and workspace/. This is what the API returns when a user views a notebook listing.

{
	"id": "nb-5g43rv2s9pfw8w4d",
	"project_id": "proj-7h2k9qm4xz7rp3w8",
	"title": "Revenue Analysis",
	"description": "Monthly revenue breakdown by region",
	"status": "active",
	"author": "user_abc123",
	"created_at": "2025-01-15T10:00:00Z",
	"updated_at": "2025-03-05T14:22:00Z",
	"last_run_at": "2025-03-04T08:30:00Z",
	"tags": ["finance", "monthly"],
	"runtime": {
		"python_version": "3.11",
		"marimo_version": "0.9.0"
	},
	"security_labels": { "classification": "SECRET", "compartments": ["element-a", "element-z"] }
}

Valid status values: draft · active · archived · deleted

Security labels (optional): a notebook-level override enforced in addition to the project labels, so it can only add restrictions. Same write discipline as project.json (§4.3): only NotebookService.setSecurityLabels, ETag CAS, notebook.security_labels.pending / notebook.security_labels catalog commits, super-admin standing, If-Match honored. Lowering or clearing an override is decided against the notebook's current labels, so an admin outside the compartment cannot relax it.

4.5 source.json

The source record selects a local workspace or an immutable Git-synced workspace. A local source uses this shape:

{
	"schema_version": 1,
	"type": "local",
	"current_version_id": "ver_01ARZ3NDEKTSV4RRFFQ69G5FAV"
}

An imported local source also stores entry_notebook, relative to its workspace root. Legacy local sources omit this field and use notebook.py. The live entry file keeps its original path. Version snapshots still use versions/{vid}/notebook.py. The original entry remains protected during workspace moves and deletes.

Folder imports use projects/{pid}/imports/{import-id}/ (paths.project(pid).notebookImport(id)):

Object	Owner and lifetime
snapshot.zip	Immutable folder bytes; expires after 24 hours, deleted after one hour of grace.
preparation.json	Immutable actor and expiry record; deleted 7 days after the import expires.
items/{entry-sha256}.json	NotebookImportService CAS receipt; deleted 7 days after the import expires.

Import IDs are imp- plus 16 random characters. A receipt key is the SHA-256 hex of the entrypoint path, so any valid path fits the 255-byte segment and 1024-byte key limits. The receipt's input.entry_notebook is the source of truth. Folder paths are limited to 896 UTF-8 bytes and 255 bytes per segment. The longest key a path is stored under is projects/{pid}/notebooks/{nid}/versions/{vid}/workspace/{path} (111-byte prefix).

Each receipt binds the actor and submitted notebook settings to one entrypoint. Each attempt gets a separate notebook ID and a ten-minute lease. Retries cannot overwrite another attempt's workspace. Runtime settings are resolved for new attempts only. Completed replays return the stored result, even if deployment options change.

Receipt states: preparing (an attempt holds the lease), publishing (every file exists; publication was authorized), complete (stores the published NotebookMeta), and expired (fenced by expiry, the retry limit, sweep, or a replay that found the notebook deleted).

After every file exists, CAS advances preparing to publishing. Only NotebookImportService calls NotebookService.publishImportNotebook, and only for a receipt it fenced to publishing. Publication is an idempotent catalog append (appendNotebookEntry with ifAbsent). If the notebook was deleted before a replay completes it, the receipt becomes expired and the caller must restart. Maintenance completes interrupted publication and removes abandoned attempt prefixes, including late writes. It preserves the winning workspace and completed receipts. Replays cannot overwrite edits or recreate a deleted notebook. Seven days after the import expires, maintenance logs any receipt still publishing and deletes the whole import prefix.

Imported local workspaces retain supporting files even under MARIMOHUB_PERSIST_WORKSPACE=source. The workspace root is the working directory and is added to PYTHONPATH for nested entrypoints. See Import notebooks for selection, limits, and browser queue behavior.

A Git source stores type: "git", a provider, sync_mode: "push" | "pull", Git coordinates, and sync timestamps. current_version_id is null before the first successful sync.

4.6 Source types

source.json is a discriminated union on type. The current schema supports local and git records. A local record points to the current version in the bucket. A git record stores Git coordinates, its push or pull mode, and the state of the last sync.

In push mode, an external workflow sends the workspace to /api/sync/git/v1 with a notebook-scoped sync token. Pull mode has no token. The hub reads the GitHub branch and creates a credential-free shallow Git directory through the source-control adapter. Both modes store repository files under versions/{vid}/workspace/. Pull mode also stores .git content under the sibling versions/{vid}/git/ prefix. The optional workspace.zip startup artifact duplicates the final sandbox tree and is pruned with the enclosing version. See the sync guide for the API and token lifecycle.

4.7 versions/{vid}/version.json

{
	"version_id": "ver_01ARZ3NDEKTSV4RRFFQ69G5FAV",
	"notebook_id": "nb-5g43rv2s9pfw8w4d",
	"saved_at": "2025-03-05T14:22:00Z",
	"author": "user_abc123",
	"message": "Add regional breakdown by Q",
	"parent_id": "ver_01ARZ3NDEKTSV4RRFFQ69G5FA0",
	"html_snapshot": { "captured_at": "2025-03-05T14:40:00Z", "size_bytes": 524288 },
	"session_snapshot": { "captured_at": "2025-03-05T14:40:00Z", "size_bytes": 81920 }
}

Versions are immutable once written. A local save writes a new version, updates source.current_version_id, and updates the notebook-level workspace. A Git sync writes the complete workspace inside the new version. It then advances the source pointer with CAS.

Optional snapshot descriptors. html_snapshot and session_snapshot record the presence and metadata of a rendered HTML snapshot (notebook.html) and a marimo session state file (session.json) sitting alongside the code in the version folder. They carry captured_at and size_bytes — never a storage path (clients address notebooks by ID, §13). Each field is absent when that artifact wasn't captured; both are written only on session teardown (§8) and only if marimo actually produced the source file, so most versions have neither. Adding these optional fields is forward-tolerant — a reader that doesn't know them ignores them — so it requires no breaking migration of existing version.json objects.

4.8 _system/sessions/{pid}/{sid}.json

A session represents a live marimo kernel instance — a running Python process attached to a notebook. Sessions are mutable records (like catalog.json and the identity directory, §4.10), but they are independent: session writes never touch the snapshot chain.

Records are partitioned by project (_system/sessions/{pid}/{sid}.json) so the interactive reads — listActiveByProject (every notebook-table render) and findReusable (every session create) — list just that project's prefix instead of scanning every session in the deployment. Deployment-wide sweeps (the stale-expiry and terminal-reap passes, per-user session count) still list the whole _system/sessions/ prefix recursively. Because the key now carries the project id, the routing token for proxy exposure mode encodes {pid}.{sid} (see proxyToken.ts) so the forwarder can rebuild the key.

{
	"session_id": "sess-4m8v6k2q9r5t7w3x",
	"notebook_id": "nb-5g43rv2s9pfw8w4d",
	"project_id": "proj-7h2k9qm4xz7rp3w8",
	"user_id": "user_abc123",
	"status": "running",
	"started_at": "2025-03-05T14:30:00Z",
	"last_heartbeat": "2025-03-05T14:35:00Z",
	"authorization_expires_at": "2025-03-05T15:30:00Z",
	"runtime": {
		"python_version": "3.11",
		"marimo_version": "0.9.0"
	},
	"sandbox_id": "sb-3f6h9k2m5p8r4t7v",
	"sandbox_url": "https://sandbox.example.com/sess-4m8v6k2q9r5t7w3x",
	"compute_profile": "small",
	"used_fallback": false
}

When a session goes failed, markFailed may persist an optional sanitized error object so the client polling the record can render why instead of a bare failed. It is omitted on healthy records and never carries secret material:

{ "status": "failed", "error": { "code": "SERVICE_UNAVAILABLE", "message": "Sandbox unreachable" } }

Valid status values: starting · running · terminating · terminated · failed · expired

sandbox_id / sandbox_url link the record to the live kernel runtime (a separate container/compute service); compute_profile records the configured profile name used at launch and is absent when profiles are unset; used_fallback records whether a fallback runtime was used. The optional field is forward-compatible with existing records and requires no migration.

authorization_expires_at records the earliest applicable entitlement or subject-security-context deadline. OIDC group entitlements expire with the browser session or external access token that supplied them. Login-policy entitlements expire with the browser session. Session reuse can shorten this deadline, but the lifecycle never extends it. At expiry, it destroys direct subdomain kernels. HTTP proxy requests fail closed, and the Node proxy closes established WebSockets. This teardown skips the final capture so that the kernel stops promptly. Periodic snapshots limit potential data loss. The session API does not return this internal field.

Session lifecycle:

1. POST /sessions creates a starting record. Successful provisioning changes it to running and records the sandbox location.
2. A heartbeat updates last_heartbeat. The service coalesces bucket writes to one write per minute.
3. DELETE /sessions/{id} changes the record to terminating. Teardown changes it to terminated after the sandbox is gone.
4. The lifecycle sweep changes an overdue live session to expired.
5. The reaper deletes old terminated, failed, and expired records after the retention period. Records with a sandbox_id also require sandbox_reclaimed_at, which confirms sandbox destruction.

On object storage vs. a live state store for sessions: the object store is appropriate for session records — created on open, updated periodically, read for display — and the record carries sandbox_id / sandbox_url pointing at the live kernel runtime (a separate container/compute service). It is not appropriate for sub-second kernel state (output streaming, variable inspection); that lives in the kernel runtime itself, backed by a low-latency state store (in-memory cache / stateful coordinator) if cross-request coordination is needed. For MVP — tracking who has what open, TTL cleanup — the object store is sufficient.

App sessions. A session record may carry mode: "app": the notebook served as a read-only application via marimo run instead of the editor. App pool records route accounts to versioned sandboxes; user_id records the starter for aggregate compute caps. Apps load copy-only and never write back notebook code, outputs, workspace files, or filesystem snapshots. app_pool: true identifies managed pool members; source_version_id pins their committed source. active_connections and connections_checked_at hold the last kernel connection probe. Absent mode means edit.

4.8.1 _system/apps/{pid}/{nid}.json

This legacy singleton claim remains readable during migration. New app admissions use app pool records and never acquire this claim.

{ "session_id": "sess-4m8v6k2q9r5t7w3x", "claimed_at": "2025-03-05T14:30:00Z" }

SessionService.releaseApp CAS-writes the free marker session_id: null only when the claim still names the retiring session. A raw delete could remove a different holder acquired after the read. Notebook or project deletion may remove subordinate claims as cleanup.

Legacy apps are adopted as drain-only pool members. Stop singleton writers before enabling pool writers; see the coordinated rollout requirements.

4.8.2 _system/editors/{pid}/{nid}.json

The editor claim is the persistence fence for edit sandboxes. Exactly one non-ephemeral session holds it. Under shared, every editor attaches to that holder. Under exclusive, only the holder attaches; another editor explicitly chooses a discard-only temporary session or a takeover.

{
	"session_id": "sess-…",
	"sharing": "exclusive",
	"claimed_at": "2026-07-31T12:00:00Z",
	"transfer": {
		"takeover_id": "client-idempotency-key",
		"requested_by": "user_b",
		"phase": "draining",
		"requested_at": "2026-07-31T12:10:00Z",
		"drain_lease_id": "server-attempt-id",
		"drain_lease_expires_at": "2026-07-31T12:20:00Z",
		"drain_lease_stage": "capturing",
		"drain_lease_progress_deadline_at": "2026-07-31T12:40:00Z"
	}
}

Claim acquisition, stale-holder replacement, release, and takeover phase changes use ETag CAS. Snapshot and teardown operations write notebook content only when the session still holds the claim.

A takeover moves through three phases:

1. requested reserves the expected holder and connection state.
2. draining protects ownership while save or sandbox destruction retries. A CAS-acquired lease serializes recovery so concurrent retries cannot save or destroy the old sandbox twice. The holder renews its ten-minute deadline every minute until it atomically moves the claim to ready. An expired lease cannot be renewed by its former holder. Renewal is also capped at 30 minutes without progress. Finishing notebook capture, filesystem snapshotting, or sandbox destruction starts a new 30-minute progress window. Another replica can therefore recover a drain whose worker stopped or whose current operation is permanently stuck.
3. ready permits only the requester to create the replacement session.

The create route completes the transfer after the replacement reaches running. A save or destruction failure does not create the replacement.

4.9 _system/events/{YYYY-MM-DD}/{event-id}.json

The event log is the primary audit trail. Object stores provide no atomic append, and a single shared per-day file would lose events under concurrent writers (a read-modify-write race). So each event is written as its own immutable object, keyed by a monotonic ULID under a per-day prefix. This is write-safe with no locking, and because ULIDs sort lexicographically by time, listing a day's prefix returns events in append order.

Every event carries a schema_version field so the log stays parseable as the event shape evolves. Reading a day lists _system/events/{date}/, excludes _idempotency/, and parses each event object. Operations that need retry-safe appends first create an immutable _idempotency/{key}.json marker that records the event ID and body. A scheduled job may optionally compact a finished day into a single archive file for export/analytics.

// _system/events/2025-03-05/01HXYZ9ABCDEFGHJKMNPQRSTVW.json
{
	"id": "01HXYZ9ABCDEFGHJKMNPQRSTVW",
	"schema_version": 1,
	"ts": "2025-03-05T14:22:00Z",
	"event": "notebook.create",
	"actor": "user_abc123",
	"project_id": "proj-7h2k9qm4xz7rp3w8",
	"notebook_id": "nb-5g43rv2s9pfw8w4d",
	"snapshot_id": "snap-7h2k9qm4xz7rp3w8"
}

Event types (emitted today): every catalog mutation is appended by CatalogService after its winning CAS commit (best-effort: a failed append never fails the mutation, it bumps the events.append_failed metric). The event field is the operation name: project.create · project.update · project.members · project.delete · project.gc · notebook.create · notebook.update · notebook.delete · notebook.gc · notebook.synced.create · notebook.synced.sync. Context fields (project_id, notebook_id, …) ride along per operation. Project managers read one day with GET /api/v1/projects/{pid}/events?date=. Super admins read a paginated deployment stream with GET /api/v1/events. Each deployment query covers at most 30 UTC days.

Planned, not yet emitted: session lifecycle (session.create · session.terminate · session.expired), notebook.run, notebook.snapshot (snapshot capture on teardown, §8), and migration.run.

4.10 _system/identities/{user-id}.json

The identity directory maps a stable user id — the auth sub, which is what every author / user_id / actor / owner field stores — to that user's current human-readable identity. It exists so the UI can render those opaque ids as a person (e.g. "Started by Ada Lovelace", "Created by …") without denormalizing a name/email onto every notebook, session, and snapshot.

// _system/identities/user_abc123.json
{
	"id": "user_abc123",
	"email": "ada@example.com",
	"name": "Ada Lovelace",
	"picture_url": "https://idp.example.com/avatar/ada",
	"suspended_at": "2026-08-11T18:00:00Z",
	"updated_at": "2026-08-11T18:00:00Z"
}

If the identity provider supplies no display name, name uses the email local-part. AuthUser.name is optional. OIDC reads the name claim from the profile scope. Access reads its name claim. The optional picture_url must be an HTTPS URL from the identity provider. An absent suspended_at means the user is active; a timestamp blocks browser sessions and personal access tokens at authentication time.

Why this shape — store the id, refresh the directory. The foreign keys throughout the store (author, session.user_id, snapshot.actor, project.owner/members) keep storing the stable id only, never a name/email copy. A copy denormalized at write time would go stale when a user later changes their display name or email; instead the directory is refreshed on every authenticated request, so resolution always reflects the latest known identity. Reads resolve ids → identities in one batch lookup (GET /api/v1/users?ids=…, §13) rather than touching the snapshot.

Mutability & write semantics. An identity object is mutable and owned by IdentityService. Profile and suspension updates use ETag compare-and-swap with retries. A profile refresh applies the latest IdP fields while preserving suspended_at, including when it races a suspension update on another replica. This record is independent of the catalog.json snapshot chain and never touches it. The key is the URL-encoded user id, so non-subdomain-safe sub values (e.g. auth0|abc) remain addressable.

Suspension cache. Authentication reads status by the requested user key; it never scans the directory. Each process caches active results as fresh for 10 seconds, serves them stale while revalidating until a hard age of 30 seconds, then blocks for storage and fails closed if status cannot be verified. Suspended results are fresh for five minutes and remain denied while stale revalidation runs. Cache loads are single-flighted and bounded to 10,000 least-recently-used entries per process.

Write volume. The upsert runs in the auth middleware on every /api/v1/* request, but is write-coalesced: the service keeps a per-process id → {email,name} signature cache and skips the PUT when the incoming identity is unchanged since this process last wrote it (IdentityService.upsert, packages/core/src/services/identity/IdentityService.ts). So steady-state cost is ~one PUT per user per process lifetime (plus one whenever their email/name actually changes), not one per request. The upsert is best-effort — a failure is logged and never blocks the request.

4.11 _system/tokens/{token-id}.json

A personal access token record is the machine-credential counterpart of a session cookie. The token acts as its issuing user. A scoped record can only reduce the user's current authority.

// _system/tokens/01HXY0S6GWMBASVAG3PZ7Y2K5T.json
{
	"id": "01HXY0S6GWMBASVAG3PZ7Y2K5T",
	"user_id": "user_abc123",
	"name": "ci-deploy",
	"hash": "9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08",
	"created_at": "2026-07-24T14:30:00Z",
	"expires_at": "2026-10-22T14:30:00Z",
	"last_used_at": "2026-07-24T15:00:00Z"
}

This record is a legacy version 1 token. It has no credential_version or grant. It grants full PAT access within the user's current authority.

A scoped token has this version 2 shape:

{
	"id": "01HXY0S6GWMBASVAG3PZ7Y2K5T",
	"user_id": "user_abc123",
	"name": "notebook-runner",
	"hash": "9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08",
	"credential_version": 2,
	"grant": {
		"actions": ["project.read", "integration.read", "integration.use", "session.start"],
		"projects": ["proj-7h2k9qm4xz7rp3w8"]
	},
	"oauth": {
		"client_id": "01HXY0S6GWMBASVAG3PZ7Y2K5V",
		"resource": "https://hub.example.com/mcp",
		"scopes": ["mcp:tools"]
	},
	"created_at": "2026-07-24T14:30:00Z",
	"expires_at": "2026-10-22T14:30:00Z"
}

actions is "*" or a unique canonical action list. projects is "*" or 1 to 100 unique project IDs. Parsing fails for a missing, inconsistent, or unknown credential version. The optional oauth object binds an OAuth token to its client, MCP resource, and scopes. Other fields remain loose so the daily last_used_at rewrite preserves fields from later releases.

Why keyed by token id. The presented bearer is mhub_pat_<tokenId>_<secret>. The embedded ULID enables one-GET verification without a mutable index. Version 1 hashes the secret. Version 2 hashes mhub_pat:v2:<tokenId>:<secret>. This domain separation prevents a legacy verifier from authenticating a scoped token after prefix rewriting. The plaintext is never stored.

Token listing and creation scan the deployment-wide _system/tokens/ namespace. They filter the records by user after the scan. The scan includes expired records. The 20-token live-token cap does not bound this storage cost.

Mutability & write semantics. Creation is a plain PUT to a fresh, unique token-id key. The only rewrite is the last_used_at refresh, and it is conditional — an If-Match on the ETag read at load time — so a token revoked (deleted) between load and the touch is not resurrected by a stale write; the touch is also coalesced to once per UTC day, keeping the request hot path read-only. Revocation is a plain DELETE. Positive verifications are cached per process with a short TTL (TokenService.CACHE_TTL_MS), which also bounds the cross-replica revocation lag.

The Node proxy has a similar per-process cache for successful editor admissions, including VS Code and other surface proxies. An entry lives up to 5 seconds, or less when the credential or entitlements expire sooner. Token signature, sign-in, suspension, and authorization deadlines are checked on every request. So stopping a session, revoking a role, deleting a project, or changing labels can take up to 5 seconds to block an editor connection that was already admitted. App and preview admissions are never cached.

Known limitation: Docker and Podman give each kernel a random local port. When MARIMOHUB_SANDBOX_AUTH is off, a new session could in theory reuse the port of a session stopped within that 5-second window, and a cached admission for the stopped session would then reach the new kernel.

4.11.1 CLI authorization records

A loopback grant binds the user, PAT lifetime, and PKCE challenge to a one-time mhub_cli_… code. A device grant starts without a user. It has an eight-letter, base-20 user code. The create-if-absent user-code claim provides an exact lookup. Browser approval uses CAS to add the user and PAT lifetime. Grants expire after ten minutes. Scoped requests also bind a requested token grant. Approval can remove actions or projects but cannot add them.

The CLI receives the authorization secret. Storage contains only its SHA-256 hash. Device polling must also supply the PKCE verifier before it reveals the grant status. User codes are case-insensitive. Each process limits preview and approval attempts separately to five per user. Polling has a separate limit.

CliAuthorizationService is the only writer. Legacy records use device_pending, pending, and claimed. Scoped records use device_pending_v2, pending_v2, and claimed_v2. The v2 records carry the requested grant. Approved and claimed records also carry the narrowed grant.

Exchange uses CAS to claim a pending record before PAT creation. This prevents replay and concurrent token creation. Exchange then deletes the authorization and user-code claim. Creation prunes at most 100 expired authorization records. Device creation also prunes at most 100 expired user-code claims. Approval, exchange, and polling do not prune records.

Old replicas reject the v2 states. During a mixed rollout, they can reject a scoped authorization or token. They cannot mint or authenticate it as a legacy full-access token.

4.11.2 MCP OAuth records

OAuthClientStore stores immutable public-client registrations under _system/oauth-clients/. Each client uses token_endpoint_auth_method: "none", has permitted redirect URIs, and expires after 90 days. A registration request prunes at most 100 expired records.

OAuthAuthorizationService stores ten-minute records under _system/oauth-authorizations/. begin stores the client, redirect, PKCE challenge, state, scopes, and resource. Approval uses ETag compare-and-swap (CAS) to bind the user, token grant, PAT lifetime, and one-time code hash. The service verifies the bindings, claims the record with CAS, and mints one scoped PAT. This PAT stores the client ID, exact MCP resource, and mcp:tools scope. The MCP endpoint rejects other PATs. A finally block deletes the authorization. Denial uses CAS to mark the pending record as denied before deletion. It then redirects with error=access_denied.

OAuth PATs expire within 90 days. The consent page defaults to 7 days. The authorization server does not issue refresh tokens.

OAuthRateLimitService owns one bounded CAS record per public OAuth endpoint under _system/oauth-rate-limits/. All replicas share these sliding windows. CAS contention fails closed as a rate-limit rejection; storage failures remain server errors.

Dynamic registration is anonymous. Each successful registration emits an oauth_client_registered event with the generated client ID and redirect count. The event omits client-supplied names and URIs.

4.12 projects/{pid}/integrations/{iid}/…

A project integration instance — a named, versioned configuration of a code-registered kind (postgres, iceberg_rest, …) rendered into every session's sandbox as env vars + files. Two records:

// projects/proj-x/integrations/intg-y/integration.json   (head — CAS-managed)
{
	"id": "intg-7h2k9qm4xz7rp3w8",
	"project_id": "proj-7h2k9qm4xz7rp3w8",
	"kind": "postgres",
	"name": "prod",
	"enabled": true,
	"current_version": 3,
	"created_by": "user_abc123",
	"created_at": "2026-07-30T14:30:00Z",
	"updated_at": "2026-07-30T15:00:00Z"
}

// projects/proj-x/integrations/intg-y/versions/000003.json   (immutable)
{
	"schema_version": 1,
	"version": 3,
	"kind": "postgres",
	"kind_schema_version": 1,
	"config": { "host": "db.internal", "password": { "$secret": { "kind": "managed", "envelope": { "kek_id": "…", "alg": "A256GCM", "iv": "…", "ciphertext": "…" } } } },
	"created_by": "user_abc123",
	"created_at": "2026-07-30T15:00:00Z",
	"change_note": "rotate password"
}

Mutability and write semantics. Version records are immutable and use create-if-absent writes. A concurrent writer that loses a version number uses the next number. Every head update uses ETag CAS through mutateObject. ProjectIntegrationsStore is the only writer for a project head.

Two concurrent configuration edits both create history records. The head points to the larger committed version number. Session records pin each rendered integration as { id, name, kind, version }. This pin identifies the exact configuration while the integration exists.

The head and its configuration form one concurrency unit. There is no separate mutable object for an individual secret field. API clients update the complete integration with the head ETag.

Replacing an inline secret writes new ciphertext into a new immutable version. It does not overwrite ciphertext in an older version. Old ciphertext remains until deletion removes the entire integration history. This is an explicit retention tradeoff of versioned integration storage.

Deletion removes the head first. A concurrent head update then fails its CAS and removes the version that it appended. The delete operation then removes the remaining integration objects. Session pins and integration.* events retain the audit identity, but they do not retain deleted secret configuration.

Secret fields contain managed ciphertext envelopes or external reference markers. A reference marker contains only its backend and locator. The resolver fetches its value during a supported connection test or session rendering. A save validates the reference shape and configured backend without fetching the value.

The managed encryption context includes the head path and field path. As a result, a copied envelope cannot decrypt at a different path. A copy decrypts and encrypts each managed value for its destination. It keeps reference markers unchanged. The store rejects secret boxes outside registered secret fields.

The name claim (_names/{name}.json, name URI-encoded) anchors "one instance per name" the same way the app claim anchors "one app per notebook": { integration_id, claimed_at }, acquired via acquireSingletonClaim (create-if-absent, ETag-CAS replace when stale) and released by CAS-ing integration_id to the free marker null, never a bare delete. Writers put the head first, then claim — a holder is live iff its head still exists under that name, so the claim self-heals after a crash between the two writes, and two concurrent creates are arbitrated by the claim key: the loser deletes its own just-written objects. A rename claims the new name (reverting the head on conflict, so a combined rename+config PATCH that loses commits nothing) and then frees the old; delete frees the name last. Sole writer: ProjectIntegrationsStore. _names/ cannot collide with an instance directory (ids are always intg-…), and listings skip it.

The organization tier stores the same head, version, and name-claim records under _system/integrations/. OrgIntegrationsStore is the only writer, and the API limits its management routes to super admins. Organization heads do not contain project_id. Identity validation rejects a record loaded through the wrong tier.

Organization managed values use an HKDF context from the _system/ head path. As a result, an envelope from one tier cannot be decrypted in the other tier.

For a new non-ephemeral session, the project store combines both tiers. It uses an enabled organization integration unless the project has an integration with the same name. The project integration takes precedence even when it is disabled, which supports both overrides and opt-outs. Name claims are unique only within one tier, so both tiers can contain an integration named warehouse.

---

4.13 projects/{pid}/notebooks/{nid}/jobs/{jid}/… and _system/job-runs/

A job is a per-notebook headless-run definition (schedule + policy); a run is the CAS-managed record of one execution until it reaches a terminal status. See docs/jobs.md for the user-facing behavior.

// projects/proj-x/notebooks/nb-y/jobs/job-z/job.json   (head — CAS-managed)
{
	"schema_version": 1,
	"id": "job-7h2k9qm4xz7rp3w8",
	"notebook_id": "nb-3w8h2k9qm4xz7rp3",
	"project_id": "proj-7h2k9qm4xz7rp3w8",
	"name": "Nightly refresh",
	"enabled": true,
	"schedule": { "cron": "0 6 * * *", "timezone": "Europe/Berlin" },
	"parameters": { "region": "eu-west-1" },
	"retry": { "max_retries": 1, "backoff_seconds": 60 },
	"timeout_seconds": 1800,
	"concurrency_policy": "forbid",
	"notifications": { "on": ["failure"] },
	"created_by": "user_abc123",
	"created_at": "2026-09-01T10:00:00Z",
	"updated_at": "2026-09-01T10:00:00Z"
}

// …/jobs/job-z/occurrences/20260902T0400Z.json   (immutable, create-if-absent)
{ "run_id": "run_01JX…", "fired_at": "2026-09-02T04:00:12Z", "outcome": "run" }

// …/notebooks/nb-y/job-index/{created_at}_{job-id}.json   (empty, immutable)

// …/jobs/job-z/run-index/{reverse-ulid}.json   (empty, immutable, create-if-absent)

// …/jobs/job-z/runs/run_01JX…/run.json   (CAS-managed until terminal)
{
	"schema_version": 1,
	"run_id": "run_01JX…",
	"job_id": "job-7h2k9qm4xz7rp3w8",
	"notebook_id": "nb-3w8h2k9qm4xz7rp3",
	"project_id": "proj-7h2k9qm4xz7rp3w8",
	"status": "succeeded",
	"trigger": "schedule",
	"scheduled_for": "2026-09-02T04:00:00Z",
	"source_version_id": "ver_01JX…",
	"parameters": { "region": "eu-west-1" },
	"attempt": 1,
	"sandbox_id": "sb-…",
	"timeout_seconds": 1800,
	"queued_at": "2026-09-02T04:00:12Z",
	"started_at": "2026-09-02T04:00:40Z",
	"finished_at": "2026-09-02T04:03:05Z",
	"deadline_at": "2026-09-02T04:40:40Z",
	"exit_code": 0,
	"output": { "html_bytes": 48211, "logs_bytes": 912 }
}

// _system/job-runs/proj-x/run_01JX….json   (create-once; deleted after finalization)
{ "run_id": "run_01JX…", "continuation_run_id": "run_01JY…", "job_id": "job-…", "notebook_id": "nb-…", "project_id": "proj-…", "created_at": "…" }

Ownership and mutability.

• job.json — mutable, ETag CAS through mutateObject, written only by JobsService, the same discipline as an integration head (§4.12). JobsService also maintains the snapshot's per-notebook jobs index ({ id, enabled, schedule, updated_at }) through CatalogService.mutateSnapshot, so the scheduler enumerates scheduled jobs from two GETs instead of scanning projects/**. The timestamp prevents delayed updates from replacing newer index entries. job.json stays authoritative for the full definition. Offline catalog recovery (CatalogService.recover, writers stopped) is the only other writer. Rebuild mode indexes every surviving job.json head. Snapshot mode keeps the jobs in the selected snapshot's index, validates each against its surviving head, and reports unlisted heads as excluded.
• job-index/{created_at}_{job-id}.json — empty, create-if-absent entries for oldest-first API pagination without materializing every definition.
• run.json — an operational record like a session (§4.8): every status change is a CAS transition through the run state machine, written only by JobRunService. Terminal runs are never rewritten; retention deletes whole runs/{rid}/ prefixes.
• occurrences/{key}.json — create-if-absent, immutable. The key is the UTC minute of the occurrence, so replicas with skewed clocks compute the same key and collide on the conditional PUT: the fire-exactly-once anchor. A crash between the claim and the run record leaves a claim naming a run that does not exist; the next tick re-writes that record idempotently with the claimed run or skip outcome (the claim is the commit point, the record write is retryable).
• run-index/{reverse-ulid}.json — empty, create-if-absent history entries. Complementing each Crockford Base32 digit makes ascending object-key order equal newest-first run order, so a cursor page reads only that page's run records. Retention deletes the entry with its runs/{rid}/ prefix.
• _system/job-runs/{pid}/{rid}.json — the execution/finalization index: written create-if-absent before the run record and retained after the terminal CAS until the scheduler finishes idempotent auditing and retry creation. A marker without a record past the grace window is pruned by maintenance.
• _system/job-operations/{pid}/{nid}/{jid}.json — an expiring singleton claim that serializes scheduling, manual triggers, updates, and deletion for one job. _system/job-deletions/{pid}/{nid}/{jid}.json is the durable deletion fence owned by JobsService and checked before enqueueing or updating. Deletion writes the fence and removes both discovery indexes before cancelling runs; it deletes job.json last so interrupted cleanup can retry.
• Outputs — write-once under a fresh run prefix; no CAS needed. They are not notebook versions and never advance source.json.

Job definitions and job runs have independent schema_version constants and read-upgrade functions. Deploy readers for a new version before enabling its writers. Older replicas reject unknown job or run versions instead of rewriting them. Additive fields within a version are preserved by the loose stored schemas; enum changes require a version bump. Project alert destination kinds are open strings in storage, so an older replica preserves a kind introduced by a newer one, while API inputs accept only the kinds that replica knows.

Deleting a job deletes its whole prefix (after cancelling active runs); deleting a notebook or project reclaims the subtree with everything else.

4.14 Pretty app links

/app/{slug} opens the existing shared app and keeps the slug in the address bar. Project managers and owners manage links through Share notebook → App links. A notebook can have multiple links. Slugs share one deployment-wide namespace. They contain 1–63 characters total: lowercase ASCII letters, digits, or hyphens, with / between segments (for example, /app/team/overview). Each segment must start and end with a letter or digit. Empty segments, dots, backslashes, spaces, and percent escapes are rejected. Prefixes are independent aliases: owning team grants no rights to team/overview.

Browser URLs and storage keys preserve slashes. API clients percent-encode the whole slug as one path parameter, for example /api/v1/deep-links/team%2Foverview. Resolve and release also accept literal slashes because path-prefix proxies can decode %2F before forwarding the request. Both forms use the same slug validation and notebook permissions.

Links retain authentication, project permissions, notebook security labels, and session admission checks. Version 1 accepts only app targets and the inherit access policy. Other target types and access policies require a future change. Existing notebooks need no migration.

Storage

DeepLinkService is the sole writer of these records:

• _system/deep-links/{slug}.json: the authoritative mapping, or a released marker.
• projects/{pid}/notebooks/{nid}/deep-links/{slug}.json: an empty index marker for notebook-local lists. The service writes it before the global registration. Lists check ownership against the global record, so stale markers are harmless.

Registration uses create-if-absent or ETag compare-and-swap (CAS). Release uses CAS to replace the mapping with a marker. The bucket has no conditional delete. Each new registration gets a new ID, so stale removal requests cannot release it. Resolution reads one mapping without a bucket scan, then checks target access. Resolver and notebook link-list responses use Cache-Control: no-store.

Release and deletion

Release makes the slug available immediately. Old shared URLs can then open another notebook. Release does not stop app sessions or change notebook permissions.

Deleted notebooks and projects stop resolving immediately. Cleanup releases their slugs with CAS. Registration can also reclaim a slug whose target is deleted or missing. Storage errors never count as proof of deletion. Purge retries cleanup before it deletes local indexes.

---

5. ID Scheme

Resource IDs (proj-, nb-, snap-, sess-, job-, prev-) are a short prefix plus a 16-character lowercase base32 random body — subdomain-safe and unguessable, but NOT time-sortable (see packages/core/src/ids.ts / schema.ts). Version IDs (ver_) and job run IDs (run_) use uppercase ULIDs because their lexicographic order is load-bearing for version pruning and newest-first run history. The examples below are illustrative; the regex in ids.ts is authoritative.

Do not infer recency from snapshot key order. Because snapshot IDs are random, listing _system/snapshots/ does not return entries in creation order. The current snapshot is always the one named by catalog.json — never the "last" key. Where chronological order is needed (retention, recovery), use each object's storage timestamp (uploaded / LastModified) or the snapshot's created_at field, not the ID.

The current snapshot is always the one named by catalog.json. An interrupted write can momentarily create a snapshot it never commits (the writer deletes it on conflict — see §7.3 / §11). catalog.json is the single source of truth for "current."

ID Format	Usage
proj-{rand}	Project — random 16-char body, for example proj-7h2k9qm4xz7rp3w8
nb-{rand}	Notebook — for example nb-5g43rv2s9pfw8w4d
snap-{rand}	Snapshot — random, not time-sortable, for example snap-…
job-{rand}	Notebook job — random 16-char body, for example job-7h2k9qm4xz7rp3w8
ver_{ulid}	Notebook version — uppercase ULID, time-sortable on purpose
run_{ulid}	Job run — uppercase ULID, time-sortable for newest-first history
sess-{rand}	Session — for example sess-…
opaque string	User/actor — the authentication provider supplies this value
intg-{rand}	Integration — random 16-character body
sb-{rand}	Sandbox — random 16-character body
prev-{rand}	Notebook preview — random 16-character body

---

6. Source Types

A notebook's source.json declares where its code lives. The service reads source.type and selects the correct load path. The snapshot index, meta.json, versioning, and session model do not depend on the source type.

source.type	Code location	Write path
local	Notebook-level workspace/	The API and session teardown save the workspace
git	versions/{vid}/workspace/	An external workflow pushes a complete revision

Both source types keep files in the bucket. A git source has provider: "github" and sync_mode: "push". Its current_version_id selects the active immutable workspace. The hub does not pull from the repository host. See §4.6 and the sync guide.

---

7. CRUD Operations

7.1 Read: List All Projects & Notebooks

Exactly 2 bucket GETs regardless of scale.

GET _system/catalog.json
  → { current_snapshot_key: '_system/snapshots/snap-7h2k9qm4xz7rp3w8.json' }

GET _system/snapshots/snap-7h2k9qm4xz7rp3w8.json
  → Full project + notebook index (metadata only, no code)

7.2 Read: Open a Notebook

// Step 1: Fetch metadata in parallel
GET projects/{pid}/notebooks/{nid}/meta.json
GET projects/{pid}/notebooks/{nid}/README.md
GET projects/{pid}/notebooks/{nid}/source.json

// Step 2: Resolve code based on source.type
if source.type === "local":
  GET projects/{pid}/notebooks/{nid}/workspace/notebook.py
if source.type === "git" and source.current_version_id === null:
  return NOT_FOUND (the synced notebook has not been synced)
if source.type === "git":
  GET projects/{pid}/notebooks/{nid}/versions/{source.current_version_id}/workspace/{source.entry_notebook}

For a git source, source.entry_notebook is the validated notebook path from source.json, relative to source.root_path. A successful push confirms that this path exists in the immutable workspace before it updates source.current_version_id.

7.3 Write: Create or Update a Local Notebook

Every write follows the same atomic sequence. Steps 1–2 are safe to retry independently. Step 5 is the commit.

// Step 1: Write notebook content files
PUT projects/{pid}/notebooks/{nid}/meta.json
PUT projects/{pid}/notebooks/{nid}/README.md
PUT projects/{pid}/notebooks/{nid}/source.json
PUT projects/{pid}/notebooks/{nid}/workspace/notebook.py       // local only
PUT projects/{pid}/notebooks/{nid}/workspace/pyproject.toml

// Step 2: Write immutable version snapshot
PUT projects/{pid}/notebooks/{nid}/versions/{vid}/notebook.py
PUT projects/{pid}/notebooks/{nid}/versions/{vid}/pyproject.toml
PUT projects/{pid}/notebooks/{nid}/versions/{vid}/version.json

// Step 3: Read catalog + current snapshot
GET _system/catalog.json  →  save ETag as current_etag
GET _system/snapshots/{current_snapshot_id}.json

// Step 4: Write new snapshot
PUT _system/snapshots/{new_snapshot_id}.json

// Step 5: Atomic catalog swap (conditional PUT)
PUT _system/catalog.json  If-Match: {current_etag}

// If 412 Precondition Failed:
//   DELETE _system/snapshots/{new_snapshot_id}.json   ← remove the orphan
//   then retry from Step 3

// Step 6: Write event object (best-effort, non-blocking)
PUT _system/events/{today}/{event-id}.json

A Git sync uses a different content write. It stores the fetched or uploaded tree under a new versions/{vid}/workspace/ prefix. Pull mode also stores Git metadata under versions/{vid}/git/. The sync then advances source.current_version_id with ETag CAS. See §4.6.

Conflict Safety: If two writers race on Step 5, one receives 412 Precondition Failed and retries from Step 3. Steps 1–2 are idempotent — re-running them on retry is safe; the content files are already written before the conflict window opens. The loser also deletes the snapshot it wrote in Step 4 before retrying, so a failed attempt never leaves an orphan. Note that the content files in Step 1 are written before and outside the conditional swap: two concurrent saves to the same notebook are last-writer-wins on the live workspace/notebook.py, while the immutable versions/{vid}/ objects remain the authoritative per-version record.

7.4 Delete: Remove a Notebook

// Step 1: Soft-delete — mark status: 'deleted' in new snapshot
//   All objects remain intact for 30-day grace period

// Step 2: Atomic catalog swap (same as 7.3 Steps 3–5)

// Step 3 (deferred, after grace period): Hard delete
DELETE projects/{pid}/notebooks/{nid}/meta.json
DELETE projects/{pid}/notebooks/{nid}/README.md
DELETE projects/{pid}/notebooks/{nid}/source.json
// + all workspace/* objects (notebook.py, pyproject.toml, any runtime files)
// + all versions/{vid}/* objects

---

8. Sessions

Sessions are managed independently of the snapshot chain. A session write never touches catalog.json or any snapshot — it writes directly to _system/sessions/{pid}/{sid}.json.

API routes

Route	Description
GET /projects/{pid}/sessions	List active project sessions with pagination
POST /projects/{pid}/notebooks/{nid}/sessions	Create or reuse a notebook session
GET /projects/{pid}/notebooks/{nid}/sessions/{sid}	Get one session
POST /projects/{pid}/notebooks/{nid}/sessions/{sid}/heartbeat	Update the session heartbeat
DELETE /projects/{pid}/notebooks/{nid}/sessions/{sid}	Save if permitted, destroy, and retire a session
GET /projects/{pid}/notebooks/{nid}/editor-session	Inspect persistent editor ownership
POST /projects/{pid}/notebooks/{nid}/editor-session/takeover	Take over an exclusive persistent editor

TTL cleanup & reaping

A scheduled job (cron, every 5 minutes) performs two passes over _system/sessions/:

1. Expire — any session whose last_heartbeat is older than 5 minutes is flipped to expired.
2. Reap — any terminated, failed, or expired record whose last_heartbeat is older than 24 hours is deleted.

The reap pass prevents unbounded growth under _system/sessions/. The same maintenance cycle also reconciles provider sandboxes and applies retention to snapshots, events, idempotency records, and soft-deleted content.

Heartbeat write volume

Persisting every heartbeat is the dominant session write cost: at 50 concurrent sessions on a 30s interval that is ~50 × 2 × 60 × 24 ≈ 144,000 PUTs/day, each a read-modify-write on a small object. Two levers bound it, in order of preference:

• Don't persist routine heartbeats to the bucket. Liveness belongs in the live kernel runtime; the bucket record only needs to change on create, status transition (starting→running), and terminate. The expiry sweep can then read liveness from the runtime rather than from last_heartbeat. This drops heartbeat writes to ~0.
• If heartbeats must be persisted, widen the interval (e.g. 60s) and/or coalesce — the 5-minute expiry TTL tolerates a coarser cadence.

v1 takes the second lever: heartbeat persistence is coalesced to at most once per 60s per session (SessionService.heartbeat skips the write when the session is already running and its stored heartbeat is younger than the interval). This bounds writes to ~1/min/session regardless of client cadence — roughly halving the figure above at a 30s cadence — and stays well within the 5-minute expiry TTL. Moving liveness off the bucket entirely (the first lever) remains a future option.

Versioning and snapshots on teardown

This write-back path applies to persistent sessions on local notebooks. A user edits the live workspace. The sandbox can mount that workspace or copy it back during teardown. Neither path creates an immutable version during the session.

Teardown creates a version from the final local workspace. It can attach the optional __marimo__/notebook.html and __marimo__/session/{notebook}.py.json artifacts to that version.

Git-synced sessions are ephemeral. They load a copy of the selected immutable workspace and discard all session changes. They do not create teardown versions or snapshots.

On teardown, before destroying the sandbox, the service:

// During SandboxProvisioner.teardown, before sandbox.destroy():

// 1. Always read the final notebook back (not only in the mount-fallback case),
//    so the live workspace/notebook.py is persisted and the session's edits are captured.
//    READ_FAILED / BACKEND_ERROR on notebook.py rejects capture so the caller can retry;
//    NOT_FOUND logs `session_notebook_missing` and continues without code.
read notebook.py, pyproject.toml from the sandbox workspace
PUT projects/{pid}/notebooks/{nid}/workspace/notebook.py       // live code
PUT projects/{pid}/notebooks/{nid}/workspace/pyproject.toml    // live deps

// 2. Cut a new version IF the content changed since current_version_id
//    (skip when unchanged, so a read-only session creates no spurious version).
if content != current version:
  vid = ver_{ulid}
  PUT versions/{vid}/notebook.py
  PUT versions/{vid}/pyproject.toml
  PUT versions/{vid}/version.json        // message e.g. "Session edits", author = session user
  update source.json.current_version_id = vid
  prune to MAX_VERSIONS (§7.3)
else:
  vid = current_version_id

// 3. Copy marimo's optional artifacts into THAT version's folder (best-effort).
if exists __marimo__/notebook.html:
  PUT versions/{vid}/notebook.html   Content-Type: text/html
if exists __marimo__/session/{notebook}.py.json:
  PUT versions/{vid}/session.json    Content-Type: application/json
patch versions/{vid}/version.json with html_snapshot / session_snapshot descriptors (§4.7)

// 4. Capture the rest of the workspace IF PERSIST_WORKSPACE=workspace (best-effort).
//    Steps 1–2 own the root source files (notebook.py / pyproject.toml).
//    Capture excludes those files, regenerable caches at any depth, and .git/hooks/.
//    It includes __marimo__/ and hidden files, subject to path, file-type, and size limits.
//    Budget order: visible files, __marimo__/, other hidden paths, then each .git/ (nested too) all-or-nothing.
//    Step 3 separately saves only the selected HTML/session artifacts in versions/.
if PERSIST_WORKSPACE == "workspace":
  for each eligible runtime file under the working dir (including hidden files and __marimo__/):
    PUT projects/{pid}/notebooks/{nid}/workspace/{path}    // binary-safe
  delete workspace/{path} objects no longer present in the sandbox  // mirror deletes

// 5. Emit notebook.update (version) + best-effort notebook.snapshot (artifacts).

Steps 1–2 (the version cut) own the source files and are the durable record of the session's edits; step 4 captures the rest of the working dir into workspace/ and is covered below. The HTML/session step is a copy, not an export — no marimo export command runs; teardown only persists files marimo already wrote. This extends the SandboxProvisioner.teardown → commitSession flow (which reads notebook.py/pyproject.toml back) by routing the persist through the versioning service so a version is cut.

The version cut (steps 1–2) is the durable record of the session's edits, so a hard failure there should surface as a teardown error. The artifact copy (step 3) is purely additive and best-effort/non-fatal (mirroring version pruning, §7.3): a missing or unreadable __marimo__ file just means that version has no snapshot. Snapshots are therefore sparse — a version has one only if (a) it was cut by a torn-down session and (b) marimo produced the file — and version.json records which (if any) are present.

Because steps 1–3 target the same new version, the saved code, its rendered HTML, and the session outputs are coherent: the notebook.html was rendered from exactly the notebook.py stored beside it. This is the key reason teardown cuts the version before attaching the snapshots, rather than pinning them onto a stale current_version_id.

Workspace persistence & restore

MARIMOHUB_PERSIST_WORKSPACE (source (default) | workspace) controls whether the sandbox's non-source runtime files survive a session. It governs step 4 of teardown above and the restore on the next provision.

Mode	What workspace/ holds	Capture (teardown)	Restore (provision)
source	notebook.py + pyproject.toml only	source files only (via the version cut)	brings back exactly the source
workspace	source files plus runtime files (e.g. data/cars.csv)	source files via the version cut, then mirror the rest of the working dir	brings back source and runtime files

• Restore is unconditional. On provision, the contents of workspace/ are loaded into the sandbox working dir before marimo starts. Because workspace/ reflects whatever was last captured, no mode branch is needed — source mode simply has only the two source files to restore.
• Capture is mode-gated and binary-safe. Under workspace, capture copies runtime files from the working directory into workspace/ without changing their bytes. This includes __marimo__/, hidden directories such as .git/, and dotfiles such as .env and .gitignore. Capture excludes the root source files (notebook.py and pyproject.toml), which the version cut saves separately. It also excludes regenerable directories at any depth (.venv/, __pycache__/, .ipynb_checkpoints/, .mypy_cache/, .pytest_cache/, .ruff_cache/, node_modules/) and .DS_Store files; the list lives in packages/core/src/integrations/workspaceIgnore.ts and proposal capture shares it. Capture never stores .git/hooks/ at any depth, and restore never writes stored hooks back, because hooks are executable code. Mirror deletes remove stored runtime files that no longer exist in the sandbox. They leave the root source files and regenerable paths alone. A change from workspace to source also removes the stored runtime files.
• Caps. Capture is bounded by a max total-bytes and max file-count limit; a file skipped because it would exceed the cap is logged (console.warn), never silently truncated. Selection is ordered so that caches cannot crowd out data: visible files first, then __marimo__/, then other hidden paths, in listing order within each group. .git/ goes last and is all-or-nothing, because a partial repository restores corrupt. If the whole tree does not fit the remaining budget, capture skips it with one warning and keeps the stored .git/ copy instead of mirror-deleting it. A stale but complete repository is safer than none. The same rule applies to each nested repository (pkg/.git/), decided separately after the root .git/.
• workspace/ is never pruned. Version pruning (§11) only reaps versions/{vid}/; the latest-only workspace/ mirror is untouched.

---

9. Migrations

Schema migrations are a first-class concern because there is no database to run ALTER TABLE against. All migration targets — catalog.json, snapshot structure, meta.json, project.json, source.json, and the event log — have different migration strategies.

Prerequisite (satisfied): every independently migrated object carries a schema_version field. catalog.json uses its version field.

Migration types

Target	Strategy
Snapshot structure	Lazy — the service upgrades the current snapshot in memory before it writes the next snapshot. Old snapshots remain as-is.
meta.json / project.json / source.json	Fan-out migration job — reads all affected objects, rewrites with the new schema, emits migration.run. Iterate with pagination and process in chunks so the job stays within the runtime's per-invocation request/time limits (resume via list cursor; idempotent — see below).
_system/events/**	Never migrated — event records are immutable history. New event shapes get a bumped schema_version field. Consumers must handle multiple versions.
catalog.json	Migrated in place during the first write after a deployment that changes the catalog version. Its strict version value is not forward-tolerant.

No schema_version by design: Operational records have no schema_version. This group includes sessions (_system/sessions/**), identities (_system/identities/**), tokens (_system/tokens/**), CLI and OAuth authorizations (_system/cli-authorizations/**, _system/oauth-authorizations/**), and fs_snapshot.json. Writers replace mutable records. Cleanup removes short-lived records. Readers accept compatible changes through optional fields and defaults.

OAuth clients (_system/oauth-clients/**) also omit schema_version, but they are immutable and retained for 90 days. Readers must remain backward-compatible with records written during that window. API response objects are also unversioned. The route path (/api/v1) versions the API contract.

Migration job pseudocode

async function runMigration(s3, fromVersion, toVersion, migrateFn) {
	// s3.list is paginated; iterate the cursor and process in chunks so the job
	// stays within the runtime's per-invocation request/time limits.
	const keys = await s3.list({ prefix: 'projects/' });

	for (const key of keys.filter((k) => k.endsWith('meta.json'))) {
		const obj = await s3.get(key);
		const data = JSON.parse(obj.body);

		if (data.schema_version === fromVersion) {
			const migrated = migrateFn(data);
			migrated.schema_version = toVersion;
			await s3.put(key, JSON.stringify(migrated));
		}
	}

	await appendEvent(s3, {
		event: 'migration.run',
		from_version: fromVersion,
		to_version: toVersion,
		actor: 'system',
	});
}

Fan-out safety: Each migration checks schema_version before it writes. You can safely run an interrupted migration again. Run migrations during a low-traffic period and verify completion in the event log.

---

10. Audit Events and Application Logs

The bucket contains structured audit events. Application logs go to the configured process or platform log destination. The bucket has no _system/logs/ tree.

Path	Purpose
_system/events/{YYYY-MM-DD}/{event-id}.json	Structured audit event. Each event is an immutable object under a UTC-day prefix.
_system/snapshots/	Immutable catalog history. The maintenance job applies the configured snapshot-retention policy.

Append strategy: S3-compatible storage has no native append, and read-modify-write on a shared file loses concurrent events. So each event is a separate immutable object keyed by a monotonic ULID (see §4.9) — naturally concurrency-safe, no locking. Reading a day lists that day's prefix. For high event volume, a buffer (in-memory queue / streaming pipeline) can batch writes upstream, but per-event objects are correct and sufficient at this scale.

---

11. Snapshot Lifecycle & Retention

Topic	Detail
Retention	Keep snapshots for 90 days (with a floor of the N most recent), then delete via the maintenance cron — MaintenanceService.expireSnapshots (packages/core/src/services/catalog/MaintenanceService.ts). Every commit writes a new snapshot, so without this the bucket grows without bound (the Iceberg "snapshot-expiry" problem).
Recovery	Use the offline recovery procedure to select a surviving snapshot or rebuild from records.
Compaction	Not needed at this scale. If snapshot exceeds 1MB, split into per-project manifest files referenced by the snapshot instead of embedded.
Emergency rollback	Snapshot recovery restores catalog membership with current metadata. For local notebooks, use POST /api/v1/projects/{pid}/notebooks/{nid}/versions/{vid}/restore to roll back code. It copies the selected version's code and dependencies into the live workspace and writes a new version.
Snapshot size at full scale	~200KB (200 bytes × 1,000 notebooks)
Version-folder artifacts	The HTML/session snapshots (§8) live inside the immutable versions/{vid}/ folder, not in _system/snapshots/. They need no separate retention job: NotebookService.pruneVersions deletes the whole version prefix when it ages out of MAX_VERSIONS, reaping notebook.html / session.json along with the code.

Snapshot protection. Retention preserves the current and previous snapshots. Normal CAS conflict cleanup deletes only the losing writer's snapshot. If the catalog or its snapshot is missing, follow the recovery runbook. Surviving project objects or snapshots block empty initialization. Unreferenced snapshots require review because their creation time does not prove commit.

---

12. Authentication & Authorization

Authentication identifies the caller. Authorization controls their actions. The API resolves identity before it reaches storage.

Authentication (AuthN)

Identity is resolved per request and controlled by the AUTH_MODE setting:

AUTH_MODE	Use case	Mechanism	Identity
access	Multi-user	Cloudflare Access JWT (CF-Access-JWT-Assertion) verified against the team JWKS	{ id: sub, email } from verified claims
none	Single-user, self-hosted	None — a fixed singleton user from config	{ id: USER_ID, email: USER_EMAIL }

A request that fails authentication receives 401 UNAUTHORIZED. The verified user is attached to the request context and used as the actor recorded in snapshots and events.

Authorization (AuthZ)

All routes use the same effectiveRole calculation:

• Project ownership and super-admin status grant admin.
• Explicit membership overrides defaults. Multiple matching memberships use the highest role.
• Project default_role overrides deployment and OIDC defaults for non-members. none grants no default access.
• An absent project default or inherit uses the higher deployment or OIDC default.

Project-default changes require manager at the API boundary. ProjectService.updateProject persists them through the existing metadata CAS. The stored project.json field default_role is optional; an absent field means inherit. Project responses always include default_role and report inherit when nothing is stored. A change emits the broadcast-only project.default_access_changed project alert.

project.read requires viewer; app.read admits app-user to minimal app metadata. Notebook content changes require editor; notebook administration and project changes require manager. See the role matrix and app restrictions. Normal reads mask inaccessible or soft-deleted resources as 404; insufficient write permissions return 403 FORBIDDEN. Security labels and credential scopes can further restrict access.

Catalog entries carry optional member_ids, member_emails, and member_groups projections, but no roles or default access. A missing member_groups field is empty; it grants no directory access. Directory search builds membership candidates from the snapshot and confirms them against current project heads. A stale projection cannot retain a revoked grant. Listings read permissions from project.json because catalog projections can lag changes. Each listing reuses project reads across role, tag, and label checks. Roles, credential scopes, and labels are filtered before pagination and totals. This preserves the catalog snapshot model (§7.1) without a separate per-user index.

MARIMOHUB_SUPER_ADMINS entries containing @ match login emails case-insensitively; other entries match user IDs exactly. Static super-admin status applies to PATs. OIDC group-derived status applies to browser sessions and enabled external access tokens, but does not transfer to PATs. Super admins cannot demote or remove project owners. Role elevation does not bypass lifecycle checks or token scopes.

Project creation follows MARIMOHUB_PROJECT_CREATION. App-only users always need super-admin status or project-creator, including on open deployments. The server resolves app-only status from current defaults and effective memberships on each request. Creators become project owners with the reserved admin role.

Project members contain exactly one user_id, email, or group, plus a role. Group ids use the auth port's shared validator and match authenticated groups exactly. Equal-role ties report the user-id source before email, and email before group. Group rows cannot carry admin and never become user rows during invite claiming. ProjectService writes group members and their member_groups projection through the existing metadata and catalog CAS paths. These writes are separate; authorization must use the project head after a projection failure. Group mutations use { group } selectors, separate from string selectors for user ids and invite emails.

The schema version remains 1. See group membership rollout before storing group rows or rolling back.

Older versions parse project.json with a stripping schema, so they drop or ignore default_role. A member change or rename on an old replica rewrites the record without it, and a rollback ignores it. Either way the project falls back to the inherited default: the deployment default (editor out of the box) or a higher OIDC group or login-policy default-role grant. Stop every old replica before storing a value other than inherit. Before rollback, reset overrides to inherit. A project that used none then grants nonmembers the inherited default again; explicit memberships do not suppress it. Keep such a project members-only by setting the deployment default to none and removing the OIDC group and login-policy default-role grants that would apply. See default access rollout.

app-user, viewer, editor, and manager are assignable; existing non-owner admin memberships remain valid. Upgrade every replica before assigning a role that older versions cannot parse. Rollback requires removing or explicitly remapping unsupported assignments and configuration. Never silently map app-user to viewer, which grants source access.

Session permissions

App users can start, attach to, and heartbeat shared apps regardless of MARIMOHUB_VIEWER_MODE. They cannot use editor sessions, secondary surfaces, or stop/restart controls. Viewer runtime access depends on MARIMOHUB_VIEWER_MODE:

• static serves saved HTML through GET …/notebooks/{nid}/html, with the same authorization as other content reads.
• applications also permits shared app start, attach, and heartbeat.
• ephemeral-sandbox also permits private editor sessions, accessible only to their owner.

Viewer sandboxes are stamped ephemeral: true and never receive WIF credentials. Every capture or teardown path skips write-back: no versions, HTML/session snapshots, workspace mirrors, or filesystem snapshots. These paths include explicit stop, idle/lifetime reaping, periodic capture, and reconciliation. An app-user downgrade blocks access to existing editor sessions, including previously owned ephemeral sessions. See source protection and revocation for direct sandbox URL limitations.

---

13. API Layer

The API is an authenticated boundary between the frontend and the Bucket port. It runs in the Node and Cloudflare Worker entrypoints. It owns auth, resource IDs, source handling, snapshots, sessions, and audit events.

Clients address notebooks by {project_id, notebook_id}. They never receive storage credentials, raw bucket access, or the stored key_prefix.

The OpenAPI document is the contract

Routes are defined with @hono/zod-openapi: each declares typed request and response schemas, and the live spec is served at GET /api/v1/doc (OpenAPI 3.1). That generated document — not a hand-maintained table — is the source of truth for paths, parameters, request bodies, and response shapes. All routes are mounted under /api/v1, in these groups:

• Auth — current user and personal access token management
• Users — identity resolution and directory search
• Projects — project, member, and audit-event management
• Audit — the super-admin deployment audit stream
• Notebooks — local and Git-synced notebooks, content, versions, restore, duplicate, HTML snapshots, and sync-token rotation
• Sessions — list, create, inspect, heartbeat, stop, editor ownership, and exclusive takeover
• Integrations — kinds, project and organization instances, versions, copy, and connectivity tests
• System — deployment version and capability limits

Error & response model

JSON responses use one envelope. Success:

{
	"success": true,
	"data": {
		/* route-specific */
	}
}

Errors (no data; a typed code plus a human-readable message):

{ "success": false, "error": { "code": "NOT_FOUND", "message": "Notebook nb-… not found" } }

The HTML snapshot route returns raw text/html on success. Its errors use the JSON envelope.

HTTP	code	When
401	UNAUTHORIZED	Authentication missing or invalid (§12)
403	FORBIDDEN	Authorization denied — caller lacks the required role on the project (§12)
404	NOT_FOUND	Resource does not exist
409	CONFLICT	Write conflict — CAS retry budget on catalog.json exhausted
412	PRECONDITION_FAILED	Conditional write mismatch surfaced from storage
422	VALIDATION_ERROR	Request body or params failed schema validation
500	INTERNAL_ERROR	Unexpected server error

API versioning, pagination, idempotency

• Versioning — /api/v1 and the OpenAPI info.version identify the current contract. Put a breaking contract under a new route version.
• Pagination — project, notebook, notebook-version, project-session, integration-instance, and integration-version lists use opaque keyset cursors. Bounded lists, such as project members and API tokens, return arrays. The OpenAPI response schema is authoritative for each route.
• Idempotency — reads, PUT/PATCH, and soft-DELETE are naturally idempotent. POST creates are not by default: a retry after a network failure or a 409 CONFLICT can create a duplicate. Clients that care send an Idempotency-Key header on POST /projects and POST …/notebooks — the server stores key → result under _system/idempotency/ and replays it on retry (24h TTL). POST …/sessions is already idempotent on (user, notebook) via its reuse path. See idempotency.md.

Catalog mutation

All catalog changes use CatalogService.mutateSnapshot. It reads the catalog and current snapshot, applies one mutation, and writes a new immutable snapshot. It then updates catalog.json with an ETag conditional write. If another writer wins, it removes its uncommitted snapshot and retries against the new catalog. Only the successful writer appends the mutation event.

---

14. Scale Analysis

Object count at full scale

Metric	Value
Projects	10
Notebooks per project	100
Objects per notebook (no versions)	5 (meta.json, README.md, source.json, notebook.py, pyproject.toml)
Total notebook objects	10 × 100 × 5 = 5,000
project.json files	10
Snapshots (90-day window @ 20/day)	~1,800
Session records (peak concurrent)	~50
Event objects (90 days @ ~20/day)	~1,800
Total objects (no versions)	~8,660
Snapshot size at full scale	~200KB
Version folders (per notebook)	≤ MAX_VERSIONS (50), each ~3 objects (notebook.py, pyproject.toml, version.json)
Optional snapshot objects	+ ≤ 2 per version that ran a session (notebook.html ~0.5MB, session.json); sparse — only versions with a torn-down session have them

Version + snapshot growth is bounded by MAX_VERSIONS. Each notebook keeps at most 50 version folders; the optional HTML/session snapshots add at most two more objects to the versions that captured them. Both are reaped when the version ages out (pruneVersions, §11), so a notebook's footprint is capped regardless of edit/run frequency. HTML dominates the bytes (~0.5MB each), so a heavily-run notebook approaches ~50 × 0.5MB ≈ 25MB at the ceiling — size it via MAX_VERSIONS if that matters.

Object-store request cost per operation

Operation	Request count
List all projects + notebooks	2 GETs
Open a notebook	4 GETs (meta + README + source + notebook.py)
Create/update notebook	~8 PUTs + 2 GETs + 1 conditional PUT + 1 event PUT
Delete notebook (soft)	2 GETs + 1 PUT + 1 conditional PUT
Session heartbeat	1 PUT

Heartbeat write volume dominates session cost. At 50 concurrent sessions on a 30s interval, persisted heartbeats are ~144,000 PUTs/day — far more than all CRUD traffic combined. §8 covers how to bound this (don't persist routine heartbeats; widen the interval).

---

15. Open Questions

Topic	Detail
Object-store versioning	Enable native object versioning and lifecycle expiry as a backup below the snapshot layer. This can recover overwritten latest-only files and out-of-band deletions. See operations.md §1.
Notebook ordering	Notebook lists use created_at and the random notebook ID as a stable tiebreaker. Manual ordering needs a new explicit field.
Additional source types	local and push-synced git are implemented (§4.6). A future source type must define its write, sync, cache, and credential rules.
Read-side tenant isolation	Project-level isolation is implemented (§12). Listings confirm access from authoritative project heads; snapshot member projections alone do not grant access. Per-notebook ACLs and storage-enforced isolation for untrusted tenants remain open.
Execution logs locality	Run logs currently go to _system/events/. Alternative: projects/{pid}/notebooks/{nid}/runs/{run-id}.log for per-notebook locality. Tradeoff is discoverability vs. centralization.
Live state for real-time sessions	If kernel output streaming or sub-second variable inspection is needed, live session state belongs in a low-latency store (in-memory cache / stateful coordinator) co-located with the kernel runtime. Object storage retains the durable audit record.
Version pruning	Resolved. NotebookService.pruneVersions keeps the most recent MAX_VERSIONS (50) per notebook, pruning on each save (packages/core/src/services/content/NotebookService.ts). Snapshot and event growth are handled separately by MaintenanceService (operations.md §5).
Configurable version count	A variable such as MARIMOHUB_NOTEBOOK_MAX_VERSIONS can let operators trade history depth for storage. The same count can bound the optional HTML and session snapshots (§8).
HTML / session snapshots	Session teardown can capture both files (§8). GET …/notebooks/{nid}/html serves the newest HTML snapshot. The session-state JSON has no read route. API saves do not generate HTML because they have no live kernel.

Notebook thumbnails

ThumbnailService owns projects/{pid}/notebooks/{nid}/thumbnail.json (ETag CAS). The record holds separate custom and automatic image references. Custom images take precedence. Automatic references include the captured source version, HTML hash, renderer revision, and capture time. A deletion tombstone blocks in-flight captures.

Under the same notebook prefix:

• thumbnails/{uuid}.png images are immutable. Maintenance removes unreferenced images after one day.
• Create-once thumbnail-attempts/{sandbox-id}.json markers prevent duplicate shutdown attempts across replicas. Maintenance removes markers after one day.

Soft deletion retains thumbnail objects during the recovery grace period. Hard deletion or garbage collection removes them. Images require notebook authorization and have no public bucket URLs.

App pool records

_system/app-pools/{pid}/{nid}.json is a versioned CAS record owned by AppPoolStore. It stores sandbox reservations, visit assignments, and per-visit leases together. Each assignment is scoped to (user_id, visit_id). Its visit_id remains recorded during reconnect grace. Legacy assignments without visit_id split into one assignment per live visit before routing or expiry. AppPoolService applies the pure router decisions and coordinates session provisioning and retirement. Compute operations run outside CAS retries. Operation tokens fence late provisioning completions. Retiring members remain recorded until sandbox reclamation succeeds. Notebook and project deletion CAS-mark their pools with deleted_at, retire all members, and clear assignments. Maintenance retains each member until reclamation succeeds, even after the notebook or project files are removed. The empty deletion tombstone remains to prevent delayed admissions from recreating the pool.

The legacy _system/apps/{pid}/{nid}.json claim remains owned by SessionService during migration. New pool members do not acquire that singleton claim. See App pools for rollout and presence semantics.

Warm pool records

_system/warm-pools/{backend}.json is a CAS record owned by WarmPoolStore. It stores pool configuration fingerprints, retry deadlines, and sandbox members. Member states are creating, ready, claimed, and retiring.

Creation reservations precede provider calls. Claims record their destination session before session publication. An assigned claim remains until session reclamation, so an interrupted handoff does not lose ownership. Retiring records remain until provider deletion succeeds. A claimed sandbox never returns to ready.

Orphan reconciliation discovers every record under _system/warm-pools/ after its provider snapshot. A failed listing or unreadable warm record prevents orphan deletion during that sweep. Pool cleanup uses recorded sandbox IDs and does not require provider enumeration.

The maintenance replica uses _system/_warm_pool.lock as its advisory lease. Pool CAS transitions remain authoritative when maintenance passes overlap.

Notebook preview records

NotebookPreviewService owns each CAS record at _system/previews/{pid}/{nid}/{preview-id}.json. Records hold the source, prepared revision, runtime ownership ledger, and session admission reservations.

PreviewStore owns these CAS records:

• _system/preview-projects/{pid}.json: bounded membership, recovery intents, and artifact reservations.
• _system/preview-receipts/{pid}.json: bounded idempotency receipts, valid for seven days from creation.
• _system/preview-work.json: rotating project cursor and deployment-wide preparation leases.
• _system/preview-cleanup-cursor.json: independent rotating cleanup cursor.

A one-minute deadline fences each creation attempt, independently of the receipt’s seven-day deduplication window. A rejected capacity reservation can retry with the same key and a fresh attempt deadline. Deleted receipts still reject retries. Expired receipts are pruned on project activity; idle receipt heads hold at most 1,000 entries and do not participate in scheduling.

Creation reserves membership before writing the preview record. Workers can materialize interrupted writes from the stored intent. Listing reads only this bounded membership; it never scans historical preview records or receipts. Deletion closes the receipt immediately. After reclamation and a 15-minute grace period, cleanup marks the membership terminal, deletes the preview record, and removes membership. Terminal intents prevent interrupted cleanup from recreating active records. Empty project CAS heads remain to fence concurrent writers; individual preview records do not remain permanently. Workers page through removable _system/preview-active-projects/{pid}/{work_id}.json markers instead of retained heads. Each activation uses a new work ID. Its marker is written before membership, and project mutations write a fresh revision. Removing a stale marker first CAS-updates the project head to fence pending publications. Interrupted writes leave discoverable orphan markers. Each pass selects up to four active projects and scans at most sixteen marker pages, reclaiming orphans as it goes. Empty historical heads do not add scheduling latency; rotation time depends on projects with outstanding preview work. Runtime access requires live membership and an active preview record.

Preparation runs independently of launches and maintenance. Leases fence publication; deadlines abort provider reads and stop workspace writes. Project reservations bound both revision counts and aggregate workspace bytes before archive download. Cleanup retires compute immediately and retains artifacts, ownership, and reservations through the cleanup grace period. This lets in-flight preparation uploads settle before artifact deletion.

Each revision has a unique internal notebook ID, immutable workspace, and preview ownership marker in preview-runtime.json. Ordinary meta.json is absent, preventing older replicas from rewriting metadata without preview protections. These notebooks never enter catalog snapshots or share pools, source artifacts, or editors with the parent. Metadata reads require a published revision, use current parent labels and configuration, and reject inactive owners. Normal notebook APIs reject internal IDs. Session APIs allow authorized access to running preview sessions.

Reconciliation retains revisions with unreclaimed sessions. Each entry in revisions records its runtime notebook ID, state (preparing, ready, or retiring), and cleanup deadline. Reconciliation marks unused revisions as retiring before deleting their artifacts. Admission reservations and pruning share the preview CAS record, so admitted revisions cannot be pruned before session creation. Committed reservations remain until sandbox reclamation; abandoned reservations without sessions expire after ten minutes. Late admissions fail the ownership check. Cleanup defers fresh starting sessions and retries failed destruction without capturing edits. App pool tombstones remain under their normal owner.

Preview sessions persist immutable origin provenance at creation: parent notebook, preview, prepared version, and commit. The stored session's notebook_id remains the internal workspace identity used by compute, pools, and cleanup. sessionResourceNotebookId resolves the public parent identity; API projections never return the hidden workspace ID. sessionResourcePath provides the notebook or preview navigation target without reading runtime metadata. Session controls accept the public parent notebook ID, then resolve their workspace and pool from the stored session. History reads authorize against current parent labels; live operations additionally validate preview ownership and expiry. Runtime cleanup does not erase provenance from retained sessions or audit events.

Resumable upload records

ResumableUploads owns records under two prefixes:

Scope	Prefix	Caller
Workspace	projects/{pid}/notebooks/{nid}/uploads/	NotebookWorkspaceService
Customer objects	projects/{pid}/object-uploads/	Object-upload routes

Each prefix has a CAS index.json with at most 16 reservations for declared sizes and expiry times. The budget is the configured workspace total or 5 GiB per project for customer objects. Each {id}.json stores the destination, size, owner, expiry, part size, part references and hashes, and publication state. It also stores the native upload ID and completed result when available. Object records also bind the integration ID, bucket, and revision. API requests enforce current authorization and upload ownership.

S3 and R2 workspace uploads create an isolated destination session at start. The upload record reserves each part checksum by CAS before forwarding bytes to the provider. A part appears in progress responses only after its provider receipt is saved. Retries must match its reserved checksum. Completion commits these receipts without storing or reading temporary part objects.

Other uploads keep staged parts at {id}/parts/{n} with create-if-absent writes and exact-size checks. Retries must match the saved hash. Assembly checks each part against its saved size and hash. Existing staged records keep this path when the adapter gains isolated-session support. Completion uses CAS to transition uploading → publishing → complete. An interrupted publishing record never replays its write, because a later writer can change the destination. Workspace completion holds the mutation claim and rechecks capacity and editor state.

Assembly	Adapters
Direct multipart session	S3, R2 workspace uploads
Staged Bucket.multipart	Azure; existing S3/R2 staged uploads
Bucket.putStream	FS, GCS
Buffered, at most 64 MiB	Other adapters

Azure stays staged because other writes to the same blob can discard uncommitted blocks. GCS, filesystem, and object-browser uploads also retain staging. Multipart conditional writes are unsupported. The workspace mutation claim serializes create-only checks. Successful publication survives cleanup failure. Completion retries and maintenance release reservations and remove parts without another write.

Uploads expire after 24 hours. Completed results remain available for seven days past expiry; cancellation preserves completed receipts. A confirmed publication can finish after expiry. A confirmed publication does not revive a receipt already cancelled by maintenance. Notebook/import maintenance aborts known native uploads and removes staged parts. Records remain seven more days so later sweeps can remove parts from delayed requests. Azure expires uncommitted blocks through provider retention. Abort must preserve an existing committed blob. Hard deletion aborts known sessions before removing upload records. Failed aborts leave records for retry. Configure provider lifecycle cleanup for sessions whose creation response or receipt was lost.