From a94e51218fcb9f8e52de87059badb4785f952f48 Mon Sep 17 00:00:00 2001 From: Aleksandr Zimin Date: Thu, 23 Jul 2026 02:06:47 +0300 Subject: [PATCH 1/2] docs: document volume request contracts Signed-off-by: Aleksandr Zimin --- docs/CR.md | 140 +++++++++++++++++++++++- docs/CR.ru.md | 136 ++++++++++++++++++++++- images/controller/docs/TTL_MECHANISM.md | 8 +- 3 files changed, 277 insertions(+), 7 deletions(-) diff --git a/docs/CR.md b/docs/CR.md index 451814c8..6005397b 100644 --- a/docs/CR.md +++ b/docs/CR.md @@ -1,4 +1,140 @@ --- -title: "The storage-foundation module: custom resources" -description: "Custom resources provided by the storage-foundation module: VolumeCaptureRequest, VolumeRestoreRequest, DataExport, and DataImport." +title: "VolumeCaptureRequest and VolumeRestoreRequest" +description: "The current service contract for volume capture and restore requests used by storage controllers." --- + +# Service request resources + +`VolumeCaptureRequest` (VCR) and `VolumeRestoreRequest` (VRR) are one-shot service resources intended +for controllers such as state-snapshotter, not a stable end-user backup UX. Actual create/read access +is determined by cluster RBAC; admission currently does not enforce a controller-only policy. Both +are namespaced in `storage-foundation.deckhouse.io/v1alpha1`; their namespace is also the namespace +of the source or target PVC. + +## VolumeCaptureRequest + +VCR creates a durable data artifact for one PVC. The domain snapshot SDK uses only +`spec.mode: Snapshot`; `Detach` is a separate storage-foundation flow. + +```yaml +apiVersion: storage-foundation.deckhouse.io/v1alpha1 +kind: VolumeCaptureRequest +metadata: + name: nss-vcr-... + namespace: my-app +spec: + mode: Snapshot + target: + uid: "" + apiVersion: v1 + kind: PersistentVolumeClaim + name: data +status: + data: + artifactRef: + apiVersion: snapshot.storage.k8s.io/v1 + kind: VolumeSnapshotContent + name: snapcontent-... + uid: "" + completionTimestamp: "..." + conditions: + - type: Ready + status: "True" + reason: Completed +``` + +Contract: + +- `spec.target` is a single PVC identity; namespace is implicit from the VCR. +- A request is point-in-time. Consumers must not change its target after creation. The SDK rejects + a mismatching existing target fail-closed; the CRD currently does not enforce target transition + immutability with CEL. +- `status.data.artifactRef` points to the durable `VolumeSnapshotContent` or `PersistentVolume`. + The source PVC identity remains in immutable `spec.target` and is not duplicated in status. +- `Ready=True/Completed` is success. +- `Ready=False/TargetsPending` is **non-terminal**: CSI capture is still retrying. In particular, + `VolumeSnapshotContent.status.error` does not by itself make the request terminal. +- Any other `Ready=False` reason is terminal. Current reason constants include `InvalidMode`, + `Incompatible`, `InternalError`, `NotFound`, `RBACDenied`, `InvalidSource`, `PVBound`, and + `RestoreFailed`. `SnapshotCreationFailed` is retained for API compatibility but is no longer + emitted. +- The state-snapshotter core maps terminal VCR failure to its own `VolumeCaptureFailed`; domain + controllers should observe `snapshotsdk.CoreCaptureOutcome`, not interpret VCR conditions + directly. + +## VolumeRestoreRequest + +VRR asks the patched external-provisioner to create and bind one PVC from a +`VolumeSnapshotContent` or `PersistentVolume`. + +```yaml +apiVersion: storage-foundation.deckhouse.io/v1alpha1 +kind: VolumeRestoreRequest +metadata: + name: restore-data + namespace: restore-ns +spec: + sourceRef: + apiVersion: snapshot.storage.k8s.io/v1 + kind: VolumeSnapshotContent + name: snapcontent-... + pvcTemplate: + metadata: + name: data + spec: + accessModes: [ReadWriteOnce] + storageClassName: local + resources: + requests: + storage: 10Gi + fsType: ext4 +status: + pvcRef: + apiVersion: v1 + kind: PersistentVolumeClaim + namespace: restore-ns + name: data + uid: "" + completionTimestamp: "..." + conditions: + - type: Ready + status: "True" + reason: Completed +``` + +Contract: + +- `spec.pvcTemplate.metadata.name` is required. Restore is never cross-namespace. +- Supported source kinds are `VolumeSnapshotContent` and `PersistentVolume`. +- The patched external-provisioner watches VRRs, calls CSI `CreateVolume`, and creates the PV/PVC. + It must not write VRR status. +- `VolumeRestoreRequestController` is the only status writer. It validates the source, observes the + provisioned PVC, and publishes `pvcRef`, `completionTimestamp`, and `Ready`. +- A terminal VRR is one-shot; create a new request for another attempt. + +## Retention and deletion + +Terminal VCR/VRR objects are deleted by cron-driven generic GC using +`status.completionTimestamp`. Defaults: + +| Variable | Default | +|---|---| +| `GC_VCR_TTL` / `GC_VRR_TTL` | `24h` | +| `GC_VCR_SCHEDULE` / `GC_VRR_SCHEDULE` | `0 * * * *` | + +There are no per-object TTL annotations and no module settings for these values. See +`images/controller/docs/TTL_MECHANISM.md` for artifact ownership and DataImport adoption details. +On the successful state-snapshotter path, core may delete a VCR earlier after it has persisted the +data handoff; generic GC is the cleanup path for terminal leftovers. + +## Validation and RBAC + +Validation is performed by CRD schema/CEL plus the controllers; there is no VCR/VRR admission +webhook performing `SelfSubjectAccessReview`. + +Initiating service accounts need only the verbs required by their flow. A snapshot domain normally +creates/reads/patches VCRs and creates/deletes VRRs. The storage-foundation controller owns request +status. The `040-vrr-provisioner-rbac` hook grants patched CSI provisioner service accounts +cluster-wide VRR read/watch and target-PVC creation; those sidecars must not update VRR status. +The current production namespace registry covers `d8-sds-local-volume`; generic cross-driver VRR +support requires extending the shared CSI deployment/RBAC contract. diff --git a/docs/CR.ru.md b/docs/CR.ru.md index 3b2104a4..c7f63b35 100644 --- a/docs/CR.ru.md +++ b/docs/CR.ru.md @@ -1,4 +1,136 @@ --- -title: "Модуль storage-foundation: пользовательские ресурсы" -description: "Пользовательские ресурсы модуля storage-foundation: VolumeCaptureRequest, VolumeRestoreRequest, DataExport и DataImport." +title: "VolumeCaptureRequest и VolumeRestoreRequest" +description: "Текущий служебный контракт запросов захвата и восстановления тома для storage-контроллеров." --- + +# Служебные request-ресурсы + +`VolumeCaptureRequest` (VCR) и `VolumeRestoreRequest` (VRR) — one-shot служебные ресурсы, +предназначенные для контроллеров наподобие state-snapshotter, а не стабильный пользовательский +backup UX. Фактический create/read-доступ определяется RBAC кластера; admission пока не enforce'ит +controller-only policy. Оба ресурса namespaced и находятся в +`storage-foundation.deckhouse.io/v1alpha1`; namespace запроса совпадает с namespace +исходного/целевого PVC. + +## VolumeCaptureRequest + +VCR создаёт durable data artifact для одного PVC. Domain snapshot SDK использует только +`spec.mode: Snapshot`; `Detach` — отдельный flow storage-foundation. + +```yaml +apiVersion: storage-foundation.deckhouse.io/v1alpha1 +kind: VolumeCaptureRequest +metadata: + name: nss-vcr-... + namespace: my-app +spec: + mode: Snapshot + target: + uid: "" + apiVersion: v1 + kind: PersistentVolumeClaim + name: data +status: + data: + artifactRef: + apiVersion: snapshot.storage.k8s.io/v1 + kind: VolumeSnapshotContent + name: snapcontent-... + uid: "" + completionTimestamp: "..." + conditions: + - type: Ready + status: "True" + reason: Completed +``` + +Контракт: + +- `spec.target` — единичная identity PVC; namespace неявно равен namespace VCR. +- Request point-in-time: менять target после создания нельзя. SDK fail-closed отклоняет существующий + request с другим target; CRD пока не enforce'ит transition-иммутабельность target через CEL. +- `status.data.artifactRef` указывает на durable `VolumeSnapshotContent`/`PersistentVolume`. + Identity исходного PVC остаётся в immutable `spec.target` и в status не дублируется. +- `Ready=True/Completed` — успех. +- `Ready=False/TargetsPending` — **нетерминальное** ожидание: CSI capture продолжает retry. + Сам по себе `VolumeSnapshotContent.status.error` request не терминалит. +- Любой другой `Ready=False` терминален. `SnapshotCreationFailed` сохранён для совместимости API, + но контроллер больше его не выставляет. +- Core state-snapshotter сворачивает терминальный VCR в `VolumeCaptureFailed`; доменный контроллер + должен читать `snapshotsdk.CoreCaptureOutcome`, а не интерпретировать VCR conditions напрямую. + +## VolumeRestoreRequest + +VRR просит patched external-provisioner создать и забайндить один PVC из +`VolumeSnapshotContent` или `PersistentVolume`. + +```yaml +apiVersion: storage-foundation.deckhouse.io/v1alpha1 +kind: VolumeRestoreRequest +metadata: + name: restore-data + namespace: restore-ns +spec: + sourceRef: + apiVersion: snapshot.storage.k8s.io/v1 + kind: VolumeSnapshotContent + name: snapcontent-... + pvcTemplate: + metadata: + name: data + spec: + accessModes: [ReadWriteOnce] + storageClassName: local + resources: + requests: + storage: 10Gi + fsType: ext4 +status: + pvcRef: + apiVersion: v1 + kind: PersistentVolumeClaim + namespace: restore-ns + name: data + uid: "" + completionTimestamp: "..." + conditions: + - type: Ready + status: "True" + reason: Completed +``` + +Контракт: + +- `spec.pvcTemplate.metadata.name` обязателен; cross-namespace restore не поддерживается. +- Источник — `VolumeSnapshotContent` или `PersistentVolume`. +- Patched external-provisioner watch'ит VRR, вызывает CSI `CreateVolume` и создаёт PV/PVC, но + **не пишет** VRR status. +- `VolumeRestoreRequestController` — единственный status writer: валидирует source, наблюдает PVC и + публикует `pvcRef`, `completionTimestamp`, `Ready`. +- Терминальный VRR одноразовый; для новой попытки создаётся новый request. + +## Retention и удаление + +Терминальные VCR/VRR удаляет cron-driven generic GC по `status.completionTimestamp`. + +| Переменная | Default | +|---|---| +| `GC_VCR_TTL` / `GC_VRR_TTL` | `24h` | +| `GC_VCR_SCHEDULE` / `GC_VRR_SCHEDULE` | `0 * * * *` | + +Per-object TTL-аннотаций и module settings для этих значений нет. Подробности ownership артефактов +и adoption DataImport — в `images/controller/docs/TTL_MECHANISM.md`. На успешном пути +state-snapshotter может удалить VCR раньше, после durable handoff; generic GC чистит терминальные +остатки. + +## Валидация и RBAC + +Валидация выполняется CRD schema/CEL и контроллерами; admission webhook с +`SelfSubjectAccessReview` для VCR/VRR отсутствует. + +Инициирующим SA нужны только verb'ы конкретного flow. Snapshot-домен обычно создаёт/читает/патчит +VCR и создаёт/удаляет VRR. Status запросов принадлежит controller storage-foundation. +Хук `040-vrr-provisioner-rbac` выдаёт patched CSI provisioner'ам cluster-wide read/watch VRR и +создание целевого PVC; sidecar'ы не должны обновлять VRR status. Текущий production registry +namespace'ов покрывает `d8-sds-local-volume`; generic cross-driver VRR требует расширения общего +CSI deployment/RBAC-контракта. diff --git a/images/controller/docs/TTL_MECHANISM.md b/images/controller/docs/TTL_MECHANISM.md index 654820c8..bca38174 100644 --- a/images/controller/docs/TTL_MECHANISM.md +++ b/images/controller/docs/TTL_MECHANISM.md @@ -7,9 +7,11 @@ reach a terminal state they are no longer useful and are deleted automatically a a cron-triggered garbage-collection controller (one per kind), built on the generic collector in `common/gc`. -**Scope**: garbage collection applies only to the request resources (VCR/VRR). The durable artifacts they -produce (`VolumeSnapshotContent`, `PersistentVolume`) and the `IRetainer` resources have their own, -independent retention policies (`RETENTION_SNAPSHOT_TTL`, `RETENTION_DETACH_TTL`) and are unaffected. +**Scope**: the collector directly deletes only request resources (VCR/VRR), but deleting a VCR also +removes its follow-ObjectKeeper and can therefore Kubernetes-GC an artifact that has no other owner. +Artifacts retained by another bridge/retainer follow that owner's independent retention policy +(`RETENTION_SNAPSHOT_TTL`, `RETENTION_DETACH_TTL`). In other words, request GC never directly selects +artifacts, but it can remove the last ownership edge; see “Deletion effects” below. > Historical note: earlier versions used an informational `storage-foundation.deckhouse.io/ttl` annotation > plus a per-controller background TTL scanner driven by `REQUEST_TTL`. That mechanism has been removed — From ccdc49fe40e79d2d64d6ecba7034ca9020cedc34 Mon Sep 17 00:00:00 2001 From: Aleksandr Zimin Date: Thu, 23 Jul 2026 15:59:00 +0300 Subject: [PATCH 2/2] docs: correct volume request lifecycle contracts Provide executable VCR and VRR examples and document the actual validation, RBAC, identity, and retention behavior used by snapshot consumers. Signed-off-by: Aleksandr Zimin --- docs/CR.md | 127 +++++++++++++++++------- docs/CR.ru.md | 122 +++++++++++++++++------ images/controller/docs/TTL_MECHANISM.md | 51 +++++++--- 3 files changed, 219 insertions(+), 81 deletions(-) diff --git a/docs/CR.md b/docs/CR.md index 6005397b..4654b30b 100644 --- a/docs/CR.md +++ b/docs/CR.md @@ -3,14 +3,18 @@ title: "VolumeCaptureRequest and VolumeRestoreRequest" description: "The current service contract for volume capture and restore requests used by storage controllers." --- -# Service request resources - `VolumeCaptureRequest` (VCR) and `VolumeRestoreRequest` (VRR) are one-shot service resources intended for controllers such as state-snapshotter, not a stable end-user backup UX. Actual create/read access is determined by cluster RBAC; admission currently does not enforce a controller-only policy. Both are namespaced in `storage-foundation.deckhouse.io/v1alpha1`; their namespace is also the namespace of the source or target PVC. +This page summarizes the current generated CRDs and controller/sidecar behavior. The +state-snapshotter domain SDK ADR defines how domain controllers consume these resources, while the +unified-snapshots overview owns the core `Ready` reason mapping. The original +`2025-11-30-volume-capture-and-restore-request.md` ADR is historical rationale, not the current +implementable schema. + ## VolumeCaptureRequest VCR creates a durable data artifact for one PVC. The domain snapshot SDK uses only @@ -20,12 +24,12 @@ VCR creates a durable data artifact for one PVC. The domain snapshot SDK uses on apiVersion: storage-foundation.deckhouse.io/v1alpha1 kind: VolumeCaptureRequest metadata: - name: nss-vcr-... + name: nss-vcr-4f2c8a91d0e3b7c2 namespace: my-app spec: mode: Snapshot target: - uid: "" + uid: "2b4f6c7e-7e1d-4d85-95d0-8b52d61534d8" apiVersion: v1 kind: PersistentVolumeClaim name: data @@ -34,33 +38,41 @@ status: artifactRef: apiVersion: snapshot.storage.k8s.io/v1 kind: VolumeSnapshotContent - name: snapcontent-... - uid: "" - completionTimestamp: "..." + name: snapshot-73a4cda0-4f2c8a91d0e3 + uid: "1148b6bd-9de2-4c86-a814-7688300932eb" + completionTimestamp: "2026-07-23T12:34:56Z" conditions: - type: Ready status: "True" + lastTransitionTime: "2026-07-23T12:34:56Z" reason: Completed + message: "target ready" ``` Contract: -- `spec.target` is a single PVC identity; namespace is implicit from the VCR. -- A request is point-in-time. Consumers must not change its target after creation. The SDK rejects - a mismatching existing target fail-closed; the CRD currently does not enforce target transition - immutability with CEL. +- `spec.target` is one PVC identity; namespace is implicit from the VCR. This is a single-target + request, not a bulk capture API. +- A request is point-in-time, but the enforcement is incomplete. The CRD requires non-empty + `uid`, `apiVersion`, `kind`, and `name` for Snapshot mode, but has no transition-CEL making + `spec` immutable. The SDK detects a changed existing target only when `EnsureVCR` reconciles it + again. The storage-foundation controller currently resolves the live PVC by request namespace and + target name, does not require `apiVersion: v1` or `kind: PersistentVolumeClaim`, and does not compare + the live PVC UID with `spec.target.uid`; the UID is used in deterministic artifact naming. Server-side + immutability/typed drift is backlog item #26 and the remaining UID/GVK/admission hardening is #28. + The CRD only requires `target` for Snapshot mode, although the Detach controller path requires it too. - `status.data.artifactRef` points to the durable `VolumeSnapshotContent` or `PersistentVolume`. - The source PVC identity remains in immutable `spec.target` and is not duplicated in status. + The source PVC identity remains in `spec.target` and is not duplicated in status. - `Ready=True/Completed` is success. - `Ready=False/TargetsPending` is **non-terminal**: CSI capture is still retrying. In particular, `VolumeSnapshotContent.status.error` does not by itself make the request terminal. -- Any other `Ready=False` reason is terminal. Current reason constants include `InvalidMode`, - `Incompatible`, `InternalError`, `NotFound`, `RBACDenied`, `InvalidSource`, `PVBound`, and - `RestoreFailed`. `SnapshotCreationFailed` is retained for API compatibility but is no longer - emitted. -- The state-snapshotter core maps terminal VCR failure to its own `VolumeCaptureFailed`; domain - controllers should observe `snapshotsdk.CoreCaptureOutcome`, not interpret VCR conditions - directly. +- Any other VCR `Ready=False` reason is terminal. +- In the current SDK flow, a domain with a post-capture wait/action calls `MarkPlanned`, then observes + `snapshotsdk.CoreCaptureOutcome`; state-snapshotter maps a terminal VCR failure to + `VolumeCaptureFailed`. The active `namespace-root-mcr-before-planned` plan targets a guarded direct + `Finished` path for a childless domain with a complete plan. That target behavior is not implemented + in this revision and, once implemented, such a leaf will not be required to interpret VCR conditions + itself. ## VolumeRestoreRequest @@ -77,64 +89,107 @@ spec: sourceRef: apiVersion: snapshot.storage.k8s.io/v1 kind: VolumeSnapshotContent - name: snapcontent-... + name: snapshot-73a4cda0-4f2c8a91d0e3 pvcTemplate: metadata: name: data spec: accessModes: [ReadWriteOnce] storageClassName: local - resources: - requests: - storage: 10Gi + volumeMode: Filesystem fsType: ext4 status: pvcRef: - apiVersion: v1 kind: PersistentVolumeClaim namespace: restore-ns name: data - uid: "" - completionTimestamp: "..." + uid: "8c37aa04-af6e-4671-9446-87062a18c189" + completionTimestamp: "2026-07-23T12:36:04Z" conditions: - type: Ready status: "True" + lastTransitionTime: "2026-07-23T12:36:04Z" reason: Completed + message: "PVC restore-ns/data restored successfully" ``` Contract: -- `spec.pvcTemplate.metadata.name` is required. Restore is never cross-namespace. -- Supported source kinds are `VolumeSnapshotContent` and `PersistentVolume`. +- The CRD schema requires `sourceRef.name`, `pvcTemplate`, and + `pvcTemplate.metadata.name`. Restore is never cross-namespace. +- The executor additionally requires non-empty `pvcTemplate.spec.storageClassName` and an exact + `pvcTemplate.spec.volumeMode` of `Block` or `Filesystem`. These fields are optional in the current + schema. A schema-accepted request that omits either field is ignored by the executor with an event; + the controller keeps waiting for a PVC without writing `Ready` or `completionTimestamp`. Such a VRR + is non-terminal and therefore immortal to generic GC. This malformed empty-status gap needs a code + fix; documentation does not make it an accepted execution contract. +- `pvcTemplate` is only partially materialized, and source kinds differ. The executor validates + template storage class and volume mode for every request. For a `VolumeSnapshotContent` source it + uses the template's storage class, volume mode and access modes (default `ReadWriteOnce`), plus root + `fsType` for Filesystem volumes. For a `PersistentVolume` source, the source PV is authoritative + for volume mode, access modes, filesystem type and capacity; corresponding template values are not + the effective restore shape. Neither path copies template labels/annotations or treats + `pvcTemplate.spec.resources.requests.storage` as authoritative. +- Supported source kinds are `VolumeSnapshotContent` and `PersistentVolume`. The current controller + and executor select a source by `kind` and `name`; `sourceRef.apiVersion`, `namespace`, and `uid` are + not enforced against the live source. Backlog item #28 tracks source identity and admission + hardening. - The patched external-provisioner watches VRRs, calls CSI `CreateVolume`, and creates the PV/PVC. It must not write VRR status. - `VolumeRestoreRequestController` is the only status writer. It validates the source, observes the provisioned PVC, and publishes `pvcRef`, `completionTimestamp`, and `Ready`. +- The current `pvcRef` writer publishes `kind`, `namespace`, `name`, and `uid`, but leaves the optional + `apiVersion` empty; the example intentionally matches the writer. Setting it to `v1` and adding a + writer test is active work in the `namespace-root-mcr-before-planned` plan. - A terminal VRR is one-shot; create a new request for another attempt. +## Current conditions and reasons + +| Resource | Reasons written by the current controller | +| --- | --- | +| VCR | `Completed`, `TargetsPending`, `InternalError`, `NotFound`, `RBACDenied`; `InvalidMode` remains a defensive writer path although the CRD enum rejects unknown modes | +| VRR | `Completed`, `InvalidSource`, `InternalError`, `NotFound` | + +`SnapshotCreationFailed` is compatibility-only and is no longer emitted: a VCR-side +`VolumeSnapshotContent.status.error` stays `TargetsPending`. The shared constants `Incompatible`, +`UnsupportedTargetKind`, `PVBound`, and `RestoreFailed` are currently unused by both request +controllers. A VRR source VSC with `status.error` is a different path and is finalized as +`Ready=False/InternalError`. + ## Retention and deletion Terminal VCR/VRR objects are deleted by cron-driven generic GC using `status.completionTimestamp`. Defaults: | Variable | Default | -|---|---| +| --- | --- | | `GC_VCR_TTL` / `GC_VRR_TTL` | `24h` | | `GC_VCR_SCHEDULE` / `GC_VRR_SCHEDULE` | `0 * * * *` | There are no per-object TTL annotations and no module settings for these values. See `images/controller/docs/TTL_MECHANISM.md` for artifact ownership and DataImport adoption details. On the successful state-snapshotter path, core may delete a VCR earlier after it has persisted the -data handoff; generic GC is the cleanup path for terminal leftovers. +data handoff; generic GC is the cleanup path for terminal leftovers. A request without a terminal +`Ready` condition and `completionTimestamp`, including the malformed VRR case above, is not collected. +The current VRR keeper is not connected to the executor-created restore target PVC, so collecting the +VRR does not delete that PVC; see the implementation-specific ownership details in the linked TTL page. ## Validation and RBAC Validation is performed by CRD schema/CEL plus the controllers; there is no VCR/VRR admission webhook performing `SelfSubjectAccessReview`. -Initiating service accounts need only the verbs required by their flow. A snapshot domain normally -creates/reads/patches VCRs and creates/deletes VRRs. The storage-foundation controller owns request -status. The `040-vrr-provisioner-rbac` hook grants patched CSI provisioner service accounts -cluster-wide VRR read/watch and target-PVC creation; those sidecars must not update VRR status. -The current production namespace registry covers `d8-sds-local-volume`; generic cross-driver VRR -support requires extending the shared CSI deployment/RBAC contract. +| Actor | Effective/request-specific access | +| --- | --- | +| Deckhouse `User` / RBAC v2 viewer | Cluster-wide `get`, `list`, `watch` on VCR and VRR | +| Deckhouse `ClusterEditor` / RBAC v2 manager | Effective read access plus `create`, `update`, `patch`, `delete`, `deletecollection`; no request `/status` grant | +| Capture domain using snapshotsdk | Needs VCR `get`/`create`/`patch` (and list/watch if its deployment watches them); it does not write VCR status | +| state-snapshotter core | Its current template grants VCR CRUD/delete and VCR `/status` update/patch, although the service contract assigns VCR status to storage-foundation; core reaps a VCR after durable handoff | +| Restore consumer (current DataExport path) | The data-manager role grants VCR/VRR `create`, `delete`, `list`, `get`, `watch`, `update`; it does not receive request `/status` | +| Patched CSI provisioner executor | Cluster-wide VRR `get`, `list`, `watch` and target-PVC `get`, `list`, `watch`, `create`, `update`, `patch`; no VRR `/status` | +| storage-foundation controller | VCR/VRR CRUD plus both `/status` subresources; it is the request status writer and manages the request-following ObjectKeepers | + +The user-facing grants mean these resources are not effectively controller-only today, despite their +intended service-resource role. The `040-vrr-provisioner-rbac` hook currently binds the executor grant +only to the `csi` ServiceAccount in `d8-sds-local-volume`. Generic cross-driver VRR support requires +extending the shared CSI deployment/RBAC contract. diff --git a/docs/CR.ru.md b/docs/CR.ru.md index c7f63b35..7080f5b8 100644 --- a/docs/CR.ru.md +++ b/docs/CR.ru.md @@ -3,8 +3,6 @@ title: "VolumeCaptureRequest и VolumeRestoreRequest" description: "Текущий служебный контракт запросов захвата и восстановления тома для storage-контроллеров." --- -# Служебные request-ресурсы - `VolumeCaptureRequest` (VCR) и `VolumeRestoreRequest` (VRR) — one-shot служебные ресурсы, предназначенные для контроллеров наподобие state-snapshotter, а не стабильный пользовательский backup UX. Фактический create/read-доступ определяется RBAC кластера; admission пока не enforce'ит @@ -12,6 +10,12 @@ controller-only policy. Оба ресурса namespaced и находятся `storage-foundation.deckhouse.io/v1alpha1`; namespace запроса совпадает с namespace исходного/целевого PVC. +Эта страница резюмирует текущие generated CRD и поведение controller/sidecar. ADR доменного SDK +state-snapshotter задаёт контракт использования этих ресурсов доменными контроллерами, а +unified-snapshots overview владеет core-маппингом `Ready` reasons. Исходный ADR +`2025-11-30-volume-capture-and-restore-request.md` — историческое обоснование, а не текущая +implementable-схема. + ## VolumeCaptureRequest VCR создаёт durable data artifact для одного PVC. Domain snapshot SDK использует только @@ -21,12 +25,12 @@ VCR создаёт durable data artifact для одного PVC. Domain snapsho apiVersion: storage-foundation.deckhouse.io/v1alpha1 kind: VolumeCaptureRequest metadata: - name: nss-vcr-... + name: nss-vcr-4f2c8a91d0e3b7c2 namespace: my-app spec: mode: Snapshot target: - uid: "" + uid: "2b4f6c7e-7e1d-4d85-95d0-8b52d61534d8" apiVersion: v1 kind: PersistentVolumeClaim name: data @@ -35,29 +39,41 @@ status: artifactRef: apiVersion: snapshot.storage.k8s.io/v1 kind: VolumeSnapshotContent - name: snapcontent-... - uid: "" - completionTimestamp: "..." + name: snapshot-73a4cda0-4f2c8a91d0e3 + uid: "1148b6bd-9de2-4c86-a814-7688300932eb" + completionTimestamp: "2026-07-23T12:34:56Z" conditions: - type: Ready status: "True" + lastTransitionTime: "2026-07-23T12:34:56Z" reason: Completed + message: "target ready" ``` Контракт: -- `spec.target` — единичная identity PVC; namespace неявно равен namespace VCR. -- Request point-in-time: менять target после создания нельзя. SDK fail-closed отклоняет существующий - request с другим target; CRD пока не enforce'ит transition-иммутабельность target через CEL. +- `spec.target` — identity одного PVC; namespace неявно равен namespace VCR. Это single-target + request, а не bulk capture API. +- Request point-in-time, но enforcement неполон. Для Snapshot-mode CRD требует непустые + `uid`, `apiVersion`, `kind` и `name`, однако transition-CEL с иммутабельностью `spec` отсутствует. + SDK обнаруживает изменение существующего target только при следующем reconcile `EnsureVCR`. + Controller storage-foundation сейчас резолвит live PVC по namespace запроса и имени target, не + требует точные `apiVersion: v1` и `kind: PersistentVolumeClaim` и не сравнивает UID live PVC с + `spec.target.uid`; UID используется в детерминированном имени артефакта. Server-side + immutability/typed drift вынесены в backlog #26, остальные UID/GVK/admission hardening — в #28. + CRD требует `target` только для Snapshot-mode, хотя controller-path Detach тоже без него не работает. - `status.data.artifactRef` указывает на durable `VolumeSnapshotContent`/`PersistentVolume`. - Identity исходного PVC остаётся в immutable `spec.target` и в status не дублируется. + Identity исходного PVC остаётся в `spec.target` и в status не дублируется. - `Ready=True/Completed` — успех. - `Ready=False/TargetsPending` — **нетерминальное** ожидание: CSI capture продолжает retry. Сам по себе `VolumeSnapshotContent.status.error` request не терминалит. -- Любой другой `Ready=False` терминален. `SnapshotCreationFailed` сохранён для совместимости API, - но контроллер больше его не выставляет. -- Core state-snapshotter сворачивает терминальный VCR в `VolumeCaptureFailed`; доменный контроллер - должен читать `snapshotsdk.CoreCaptureOutcome`, а не интерпретировать VCR conditions напрямую. +- Любой другой VCR `Ready=False` терминален. +- В текущем SDK-flow домен с post-capture wait/action вызывает `MarkPlanned`, затем читает + `snapshotsdk.CoreCaptureOutcome`; state-snapshotter сворачивает терминальный VCR в + `VolumeCaptureFailed`. Активный план `namespace-root-mcr-before-planned` целится в guarded direct + `Finished` для childless-домена с полным планом. В этой ревизии target-поведение ещё не + реализовано; после реализации такой leaf не будет обязан самостоятельно интерпретировать VCR + conditions. ## VolumeRestoreRequest @@ -74,63 +90,107 @@ spec: sourceRef: apiVersion: snapshot.storage.k8s.io/v1 kind: VolumeSnapshotContent - name: snapcontent-... + name: snapshot-73a4cda0-4f2c8a91d0e3 pvcTemplate: metadata: name: data spec: accessModes: [ReadWriteOnce] storageClassName: local - resources: - requests: - storage: 10Gi + volumeMode: Filesystem fsType: ext4 status: pvcRef: - apiVersion: v1 kind: PersistentVolumeClaim namespace: restore-ns name: data - uid: "" - completionTimestamp: "..." + uid: "8c37aa04-af6e-4671-9446-87062a18c189" + completionTimestamp: "2026-07-23T12:36:04Z" conditions: - type: Ready status: "True" + lastTransitionTime: "2026-07-23T12:36:04Z" reason: Completed + message: "PVC restore-ns/data restored successfully" ``` Контракт: -- `spec.pvcTemplate.metadata.name` обязателен; cross-namespace restore не поддерживается. -- Источник — `VolumeSnapshotContent` или `PersistentVolume`. +- CRD schema требует `sourceRef.name`, `pvcTemplate` и `pvcTemplate.metadata.name`. + Cross-namespace restore не поддерживается. +- Executor дополнительно требует непустой `pvcTemplate.spec.storageClassName` и точный + `pvcTemplate.spec.volumeMode` со значением `Block` или `Filesystem`. В текущей schema эти поля + optional. Schema-accepted request без любого из них executor игнорирует с Event, а controller + продолжает ждать PVC без записи `Ready` и `completionTimestamp`. Такой VRR нетерминален и потому + бессмертен для generic GC. Этот malformed empty-status gap требует исправления кода; документация + не объявляет его допустимым execution-контрактом. +- `pvcTemplate` материализуется лишь частично, и source kinds ведут себя по-разному. Executor + валидирует template storage class и volume mode для каждого request. Для источника + `VolumeSnapshotContent` он использует template storage class, volume mode и access modes (default + `ReadWriteOnce`), а также root `fsType` для Filesystem-томов. Для источника `PersistentVolume` + авторитетен source PV: volume mode, access modes, filesystem type и capacity берутся из него, а + соответствующие template-поля не определяют фактическую форму restore. Ни один путь не копирует + template labels/annotations и не считает `pvcTemplate.spec.resources.requests.storage` + авторитетным. +- Поддерживаемые source kinds — `VolumeSnapshotContent` и `PersistentVolume`. Текущие controller и + executor выбирают источник по `kind` и `name`; `sourceRef.apiVersion`, `namespace` и `uid` не + проверяются относительно live source. Hardening source identity и admission отслеживается в + backlog #28. - Patched external-provisioner watch'ит VRR, вызывает CSI `CreateVolume` и создаёт PV/PVC, но **не пишет** VRR status. - `VolumeRestoreRequestController` — единственный status writer: валидирует source, наблюдает PVC и публикует `pvcRef`, `completionTimestamp`, `Ready`. +- Текущий writer `pvcRef` публикует `kind`, `namespace`, `name` и `uid`, но оставляет optional + `apiVersion` пустым; пример намеренно совпадает с writer. Запись `v1` и writer-test — активная + работа плана `namespace-root-mcr-before-planned`. - Терминальный VRR одноразовый; для новой попытки создаётся новый request. +## Текущие conditions и reasons + +| Ресурс | Reasons, которые пишет текущий controller | +| --- | --- | +| VCR | `Completed`, `TargetsPending`, `InternalError`, `NotFound`, `RBACDenied`; `InvalidMode` остаётся defensive writer path, хотя CRD enum отклоняет неизвестный mode | +| VRR | `Completed`, `InvalidSource`, `InternalError`, `NotFound` | + +`SnapshotCreationFailed` оставлен только для совместимости и больше не выставляется: на VCR-пути +`VolumeSnapshotContent.status.error` остаётся `TargetsPending`. Общие константы `Incompatible`, +`UnsupportedTargetKind`, `PVBound` и `RestoreFailed` сейчас не используются ни одним из двух request +controllers. Ошибка `status.error` у VSC-источника VRR — другой путь: он терминалится как +`Ready=False/InternalError`. + ## Retention и удаление Терминальные VCR/VRR удаляет cron-driven generic GC по `status.completionTimestamp`. | Переменная | Default | -|---|---| +| --- | --- | | `GC_VCR_TTL` / `GC_VRR_TTL` | `24h` | | `GC_VCR_SCHEDULE` / `GC_VRR_SCHEDULE` | `0 * * * *` | Per-object TTL-аннотаций и module settings для этих значений нет. Подробности ownership артефактов и adoption DataImport — в `images/controller/docs/TTL_MECHANISM.md`. На успешном пути state-snapshotter может удалить VCR раньше, после durable handoff; generic GC чистит терминальные -остатки. +остатки. Request без терминального `Ready` и `completionTimestamp`, включая malformed VRR выше, +collector не удаляет. +Текущий VRR keeper не связан с созданным executor'ом restore target PVC, поэтому GC запроса не +удаляет этот PVC; implementation-specific детали ownership приведены в linked TTL page. ## Валидация и RBAC Валидация выполняется CRD schema/CEL и контроллерами; admission webhook с `SelfSubjectAccessReview` для VCR/VRR отсутствует. -Инициирующим SA нужны только verb'ы конкретного flow. Snapshot-домен обычно создаёт/читает/патчит -VCR и создаёт/удаляет VRR. Status запросов принадлежит controller storage-foundation. -Хук `040-vrr-provisioner-rbac` выдаёт patched CSI provisioner'ам cluster-wide read/watch VRR и -создание целевого PVC; sidecar'ы не должны обновлять VRR status. Текущий production registry -namespace'ов покрывает `d8-sds-local-volume`; generic cross-driver VRR требует расширения общего +| Actor | Фактический/request-specific доступ | +| --- | --- | +| Deckhouse `User` / RBAC v2 viewer | Cluster-wide `get`, `list`, `watch` VCR и VRR | +| Deckhouse `ClusterEditor` / RBAC v2 manager | Эффективный read-доступ плюс `create`, `update`, `patch`, `delete`, `deletecollection`; grant на request `/status` отсутствует | +| Capture-domain на snapshotsdk | Нужны VCR `get`/`create`/`patch` (и list/watch, если deployment их watch'ит); VCR status домен не пишет | +| Core state-snapshotter | Текущий template выдаёт VCR CRUD/delete и update/patch VCR `/status`, хотя служебный контракт отдаёт VCR status storage-foundation; после durable handoff core реапит VCR | +| Restore-consumer (текущий DataExport path) | Роль data-manager выдаёт VCR/VRR `create`, `delete`, `list`, `get`, `watch`, `update`; request `/status` не выдаётся | +| Patched CSI provisioner executor | Cluster-wide VRR `get`, `list`, `watch` и target-PVC `get`, `list`, `watch`, `create`, `update`, `patch`; VRR `/status` отсутствует | +| Controller storage-foundation | CRUD VCR/VRR плюс оба `/status` subresource; пишет request status и управляет следующими за request ObjectKeeper | + +Из-за user-facing grants эти ресурсы фактически не controller-only, несмотря на предназначение +служебного API. Хук `040-vrr-provisioner-rbac` сейчас bind'ит executor-grant только ServiceAccount +`csi` в `d8-sds-local-volume`. Generic cross-driver VRR требует расширения общего CSI deployment/RBAC-контракта. diff --git a/images/controller/docs/TTL_MECHANISM.md b/images/controller/docs/TTL_MECHANISM.md index bca38174..01150713 100644 --- a/images/controller/docs/TTL_MECHANISM.md +++ b/images/controller/docs/TTL_MECHANISM.md @@ -7,11 +7,18 @@ reach a terminal state they are no longer useful and are deleted automatically a a cron-triggered garbage-collection controller (one per kind), built on the generic collector in `common/gc`. -**Scope**: the collector directly deletes only request resources (VCR/VRR), but deleting a VCR also -removes its follow-ObjectKeeper and can therefore Kubernetes-GC an artifact that has no other owner. -Artifacts retained by another bridge/retainer follow that owner's independent retention policy -(`RETENTION_SNAPSHOT_TTL`, `RETENTION_DETACH_TTL`). In other words, request GC never directly selects -artifacts, but it can remove the last ownership edge; see “Deletion effects” below. +**Scope**: the collector directly deletes only request resources (VCR/VRR). Secondary deletion depends +on the ownership edges that are actually wired: + +- a VCR has a `FollowObject` ObjectKeeper, and that keeper is the controller owner of the produced + `VolumeSnapshotContent` or detached `PersistentVolume`; deleting the VCR removes this owner chain; +- a VRR controller also creates a `FollowObject` ObjectKeeper, but it currently does not put the + `storage-foundation.deckhouse.io/object-keeper-ref` annotation on the VRR. The patched executor only + copies a keeper ownerReference to the restore target PVC when that annotation is present. On the normal + current path the keeper is therefore not connected to the PVC, and request GC does not delete the PVC. + +The loaded legacy `RETENTION_SNAPSHOT_TTL` / `RETENTION_DETACH_TTL` config fields are not used by these +GC managers and do not create or extend an ownership edge. See “Deletion effects” below. > Historical note: earlier versions used an informational `storage-foundation.deckhouse.io/ttl` annotation > plus a per-controller background TTL scanner driven by `REQUEST_TTL`. That mechanism has been removed — @@ -23,14 +30,16 @@ artifacts, but it can remove the last ownership edge; see “Deletion effects” An object is deleted when **all** of the following hold: - it is **terminal** — its `Ready` condition is `True` (completed) or `False` (failed). A VCR that is - `Ready=False` with reason `TargetsPending` (a bulk capture still in progress) is NOT terminal and is - never collected; + `Ready=False` with reason `TargetsPending` (its single-target CSI capture is still in progress) is NOT + terminal and is never collected; - it is not already being deleted (`metadata.deletionTimestamp` is unset); - its retention age has elapsed: `now - status.completionTimestamp > TTL`. The age is measured from `status.completionTimestamp`, which the controller stamps once when the object first becomes terminal — never from `creationTimestamp`. A terminal object without a -`completionTimestamp` is a fail-safe no-op (never collected). +`completionTimestamp` is a fail-safe no-op (never collected). A malformed VRR that the schema accepts but +the executor ignores (for example one without execution-required `storageClassName` or `volumeMode`) can +remain with empty status forever and is likewise not collected. ## Configuration (env-only) @@ -55,8 +64,11 @@ Deletion is a plain `DELETE`: true orphan the cascade cannot reach), via the `common/gc` `PreDeleter` hook — an artifact that still has any owner (its VCR keeper, or a bridging keeper such as a live DataImport's) is left untouched, so GC of the VCR never deletes an artifact another object is still retaining. The reap is best-effort. -- **VRR**: the request is deleted; the export PVC it provisioned through the external-provisioner has its - own lifecycle. +- **VRR**: the request and its otherwise unconnected follow-ObjectKeeper are deleted. The restore target + PVC provisioned by the external-provisioner remains because the current VRR controller does not publish + the keeper-ref annotation consumed by the executor. If that annotation is supplied by another writer, + the executor attaches the keeper as the PVC's controller owner and deleting the VRR can then cascade to + that PVC; this conditional path is not the normal controller wiring. ### Interaction with DataImport artifact adoption @@ -64,10 +76,11 @@ A DataImport captures its scratch volume by creating a VCR; the produced artifac DataImport by an additional (non-controller) ObjectKeeper so it survives past the VCR. That artifact is therefore retained until BOTH bridges fall away: the VCR keeper (dropped when the VCR is GC'd, `GC_VCR_TTL` after the VCR completes) and the DataImport keeper (dropped when the DataImport is GC'd, -`GC_DATA_IMPORT_TTL` after the import completes). The effective artifact-adoption window before an -unadopted artifact can be Kubernetes-GC'd is thus `min(GC_VCR_TTL from VCR completion, GC_DATA_IMPORT_TTL -from import completion)`. Both default to 24h; an operator shortening only `GC_VCR_TTL` also shortens that -window. +`GC_DATA_IMPORT_TTL` after the import completes). If those absolute owner-removal deadlines are `D_vcr` +and `D_import`, the earliest the artifact object can become ownerless is therefore +`max(D_vcr, D_import)`, not the minimum. Both TTLs default to 24h, but their clocks start at different +completion timestamps and each collector adds up to its own sweep delay. Shortening only one TTL shortens +the adoption window only when that owner would otherwise have the later removal deadline. ## Implementation @@ -76,3 +89,13 @@ window. events are dropped), and runs leader-only via the manager. - Per-kind managers and wiring: `internal/controllers/gc.go` (`vcrGCManager`, `vrrGCManager`, `SetupVCRGC`, `SetupVRRGC`), registered from `add_reconcilers.go`. +- Ownership wiring: VCR and VRR keepers are created in + `internal/controllers/volumecapturerequest_controller.go` and + `volumerestorerequest_controller.go`; the DataImport bridge is in + `images/data-manager-controller/internal/controllers/data-import/object_keeper.go`; the conditional + VRR keeper annotation is consumed only by + `images/csi-external-provisioner/patches/v6.2.0/002-vrr-executor.patch`. + +`docs/CR.md` is the current consumer summary. The historical +`2025-11-30-volume-capture-and-restore-request.md` ADR is rationale only; current behavior is determined +by the generated CRDs and the implementation files above.