Check derived labels at exact artifact and memory doors

CHANGELOG.md

@@ -2,6 +2,10 @@
 
 ## Unreleased
 
+- Unreleased (on main, not in v0.20.0): an exact read of a derived artifact, a write that cites a derived memory, explain, memory review, redaction, and an open-loop update use the labels of that row's inputs. v0.20.0 used the label stored on the row, so a public copy of a confidential input could be read. A row whose inputs cannot be checked is refused to a locked key. The owner and an unbound admin still read the stored row. No migration is required.
+- Unreleased (on main, not in v0.20.0): a locked key's consolidation, roll-up, staleness sweep and open-loop review keep only inputs whose scope and floor are both inside the binding. The overlap check stays, and the exact test runs after it. Consolidation and roll-ups compare the group scope (scope united with floor), so a card stored with an empty scope and a floor of two projects is still found and can be accepted. The operator artifact list, given a project, matches that floor as well as the scope. Each new report and aggregate records `derived_from` for the rows it used. A project view that also asks for global rows keeps a row with no Alice project id only when every Alice project id in its floor is in that view. v0.20.0 selected by overlap of the scope alone and wrote no `derived_from`. No migration is required.
+- Unreleased (on main, not in v0.20.0): a memory copied from another row is stored at least as strict as that row. v0.20.0 kept the label the copy was given at insert, so a later reader could see a health or confidential input through a copy stored as `unknown` and `public`. The insert now re-reads the inputs inside the write and raises the domain, the sensitivity, and the project floor. A later metadata write cannot drop the marker that makes the row derived, or put back an older project scope or project floor. Raising a source or a memory walks the rows derived from it and raises those too. A candidate open loop and a generated artifact take the same insert floor. Replacing a SQLite source with a stricter one raises the memories that cite it before they are retired. Moving a source to another project answers with how many derived rows a project-bound key would lose, and waits for confirm_label_hide before it writes. A relabel that cannot finish answers 409, and one that waits more than 3 seconds for the label lock answers 503. No migration is required.
+- Unreleased (on main, not in v0.20.0): the label of a derived row (its domain, its sensitivity, and the projects it requires) is now decided by one pure function. v0.20.0 kept the label a derived row was given when it was made. The function is what the write path calls. A read that does not go through that path still behaves as it does in v0.20.0. A locked key is refused when a caller passes a project floor that is not inside the key's binding. No migration is required.
 - Unreleased (on main, not in v0.20.0): on SQLite, `sources delete`, `sources prune --superseded` and `import-markdown --supersede` now scrub an open loop that names the source only in its metadata: the id, or `source:<id>`, as the text under `source_id`, `source_ids`, `source_ref`, `source_refs`, `source_references` or `selected_source_ids` at any depth. That is the rule the open-loop lookup of a source already used. They blanked only loops whose `source_id` column held the id, so a loop with an empty column kept its title, description and metadata, and they reached a new export. The delete preview now counts the same loops as the receipt. The rule reads the id only as stored or as `source:<id>`: a loop that names the source in any spelling other than those two keeps its text, although the memory rule reads several other spellings. No migration is required.
 - Unreleased (on main, not in v0.20.0): the SQLite derived-label repair now updates and records each row under the id it is stored with (SQLite keeps capitals, missing hyphens, braces and `urn:uuid:` as written) and matches rows by the normalised id only to read the graph. If two stored spellings of one id exist, each is compared with the label its own recorded inputs give it, so neither is left at its old label while the other is relabelled, and neither is lowered. An update that changes no row stops the repair with `DerivedDomainRepairError` before any event or the completion stamp is written, on open and on restore; migration `20261004_0095` refuses the same way. Before, a derived memory stored under such an id kept its label while its relabel event and the stamp were written, and the pair survived export and import. A PostgreSQL uuid column was not affected. A vault that the earlier repair already stamped keeps such a row at its old label until a restore repairs it again. No migration is required.
 - Unreleased (on main, not in v0.20.0): v0.20.0 could store a consolidation report with a sensitivity below the memories it prints. The report took its sensitivity from the near-duplicate clusters only, so a run whose proposals were roll-up cards had no cluster to take it from and was stored as `unknown`. A key whose ceiling is below those memories (a `trusted_local_agent` key for confidential ones, a `read_only_agent` key for private ones) could then read the card topic and the member ids through `GET /v0/vnext/artifacts/{id}`. The report now takes its domain and its sensitivity over every row it names: the cluster members, the members of every proposed roll-up group, the members of the groups that a skip line names by key (`topic:...`, `entity:...`, `semantic:cluster-<id>`), the pending, accepted, expired or held roll-up cards it names by id, and the sources that the `source_refs` of its cluster members name, which the report still prints. The open-loop review is labelled over the sources whose ids it prints as well as over its loops. The run digest of both reports covers those sources, so a source that was reclassified makes a new report instead of returning the earlier one. The first run after the upgrade over loops that link a source, or over cluster members that cite one, makes one new report, and a run that names no source keeps the digest it had. A report whose inputs are all unrestricted is now stored with the label of those inputs, `internal` where it was `unknown`, and every permission profile reads the two the same way. Reports stored by v0.20.0 keep their labels, because the stored-row repair does not change sensitivity. The other report producers (daily brief, weekly synthesis, connection and contradiction reports, project updates, staleness reports) were checked and already label over every row they print. Memories and sources are counted apart when the label is taken, because an id is unique only within its own table: a source that shares a memory's id can no longer replace that memory's label. No migration is required.

apps/api/src/alicebot_api/cli/capture.py

@@ -41,6 +41,7 @@
 from alicebot_api.vnext_embeddings import DeferredMemoryEmbedding
 from alicebot_api.vnext_event_log import append_event
 from alicebot_api.vnext_store import PostgresVNextStore
+from alicebot_api.vnext_label_writes import takes_label_lock as _takes_label_lock
 from .constants import DEFAULT_VNEXT_DEMO_DATASET_PATH, DEMO_SECRET_MARKERS
 from .models import CLIContext
 from .arguments import _object_dict, _object_int, _object_list
@@ -394,6 +395,7 @@ def _demo_tag(dataset_id: str) -> JsonObject:
     return {"demo": True, "demo_dataset_id": dataset_id}
 
 
+@_takes_label_lock
 def _reset_vnext_demo_dataset(store: PostgresVNextStore, *, dataset_id: str) -> JsonObject:
     with store.conn.cursor() as cur:
         cur.execute(
@@ -507,6 +509,7 @@ def _tag_demo_candidate_memories(store: PostgresVNextStore, *, dataset_id: str,
     return updated
 
 
+@_takes_label_lock
 def _tag_demo_artifact(store: PostgresVNextStore, *, artifact_id: str, dataset_id: str) -> None:
     artifact = store.get_artifact(artifact_id)
     if artifact is None:

apps/api/src/alicebot_api/main.py

@@ -779,6 +779,7 @@ async def receive() -> dict[str, object]:
         ("POST", "/v0/vnext/projects/update-candidates/{artifact_id}/review"),
         ("POST", "/v0/vnext/queue/process-next"),
         ("POST", "/v0/vnext/sources/{source_id}/review"),
+        ("POST", "/v0/vnext/sources/{source_id}/regenerate"),
         ("PUT", "/v0/vnext/settings/brain-charter"),
     }
 )

apps/api/src/alicebot_api/mcp/evidence_artifacts.py

@@ -275,18 +275,34 @@ def _authorize_explain_resource(
 ) -> None:
     """Require an unfiltered policy decision for one expanded resource."""
 
+    from alicebot_api.vnext_label_guard import apply_unverified_rule, effective_row_for_fence, policy_labels
+
+    judged: Mapping[str, object] = resource
+    judged_scope = project_scope
+    judged_floor: tuple[str, ...] = ()
+    if target_type in {"memory", "source", "artifact"}:
+        judged = effective_row_for_fence(store, identity, target_type, resource)
+        _domains, _sensitivity, judged_scope, judged_floor = policy_labels(judged)
+        from alicebot_api.vnext_derived_labels import is_derived
+
+        if target_type == "source" or (target_type == "memory" and not is_derived("memory", resource)):
+            # Original legacy memories may keep scope in value. The caller
+            # already resolved that fallback; derived rows use effective labels.
+            judged_scope = project_scope
     _actor_type, _actor_id, decision = _policy_checked(
         store,  # type: ignore[arg-type]
         identity=identity,
         action=EXPLAIN_DISCLOSURE_ACTION,
-        domains=(str(resource.get("domain") or "unknown"),),
-        sensitivity_allowed=(str(resource.get("sensitivity") or "unknown"),),
-        project_scope=project_scope,
+        domains=(str(judged.get("domain") or "unknown"),),
+        sensitivity_allowed=(str(judged.get("sensitivity") or "unknown"),),
+        project_scope=judged_scope,
+        project_floor=judged_floor,
         require_explicit_project_scope=True,
         target_type=target_type,
         target_id=target_id,
         project_view=ProjectView.unscoped(),
     )
+    decision = apply_unverified_rule(decision, judged, identity)
     # ``allowed_with_filtering`` is not sufficient for an explain response:
     # the downstream services expand related rows and do not accept filters.
     if decision.decision != "allowed":
@@ -740,20 +756,25 @@ def _authorize_vnext_artifact_target(
     artifact = store.get_artifact_for_update(artifact_id) if for_update else store.get_artifact(artifact_id)
     if artifact is None:
         raise MCPReferenceNotFoundError(f"artifact {artifact_id} was not found")
+    from alicebot_api.vnext_label_guard import apply_unverified_rule, effective_row_for_fence, policy_labels
 
+    judged = effective_row_for_fence(store, identity, "artifact", artifact)
+    domains, sensitivity_allowed, project_scope, project_floor = policy_labels(judged)
     actor_type, actor_id, raw_decision = _policy_checked(
         store,
         identity=identity,
         action=action,
-        domains=(str(artifact.get("domain") or "unknown"),),
-        sensitivity_allowed=(str(artifact.get("sensitivity") or "unknown"),),
-        project_scope=resource_project_scope(artifact),
+        domains=domains,
+        sensitivity_allowed=sensitivity_allowed,
+        project_scope=project_scope,
+        project_floor=project_floor,
         require_explicit_project_scope=True,
         require_unfiltered_target=True,
         target_type="artifact",
         target_id=artifact_id,
         project_view=ProjectView.unscoped(),
     )
+    raw_decision = apply_unverified_rule(raw_decision, judged, identity)
     return artifact, actor_type, actor_id, raw_decision
 
 
@@ -791,6 +812,10 @@ def _handle_alice_vnext_artifact_review(context: MCPRuntimeContext, arguments: M
     actor_id: str | None = None
     trace_id: str | None = None
     with _vnext_store_context(context) as store:
+        store.lock_graph_mutation()
+        from alicebot_api.vnext_label_writes import acquire_exclusive_label_lock
+
+        acquire_exclusive_label_lock(store)
         _target, actor_type, actor_id, decision = _authorize_vnext_artifact_target(
             store,
             identity=identity,

apps/api/src/alicebot_api/mcp/memories.py

@@ -412,6 +412,9 @@ def redact_memory_flow(
         raise VNextMemoryCommitValidationError("reason is required to redact a memory")
     memory_service = VNextMemoryCommitService(store)
     memory_service.lock_supersession_graph()
+    from alicebot_api.vnext_label_writes import acquire_exclusive_label_lock
+
+    acquire_exclusive_label_lock(store)
     memory = store.get_memory_for_redaction(memory_id)
     if memory is None:
         raise MemoryNotFoundError("memory was not found")
@@ -427,8 +430,11 @@ def redact_memory_flow(
     # before it is raised, and for a deleted row it is raised as a refusal the
     # surface answers "not found" (RefusedOnDeletedMemoryError): a plain not-found
     # error here would roll the audit row back with the call.
+    from alicebot_api.vnext_label_guard import effective_row_for_fence, policy_labels
+
+    judged = effective_row_for_fence(store, identity, "memory", memory)
     try:
-        memory_service.refuse_unauthorized_write(identity=identity, action="memory.redact", memory=memory)
+        memory_service.refuse_unauthorized_write(identity=identity, action="memory.redact", memory=judged)
     except AgentPolicyBlockedError as exc:
         if memory.get("deleted_at") is not None:
             raise RefusedOnDeletedMemoryError(exc.decision) from None
@@ -457,21 +463,26 @@ def redact_memory_flow(
         # row's redaction receipt and writes nothing, so its authorization
         # should not depend on a call made earlier in the function. A test
         # takes the pre-check away and checks the replay is still refused.
+        from alicebot_api.vnext_label_guard import apply_unverified_rule
+
+        domains, sensitivity_allowed, project_scope, project_floor = policy_labels(judged)
         decision = evaluate_agent_policy(
             identity=identity,
             action="memory.redact",
-            domains=(str(memory.get("domain") or "unknown"),),
-            sensitivity_allowed=(str(memory.get("sensitivity") or "unknown"),),
-            project_scope=resource_project_scope(memory),
+            domains=domains,
+            sensitivity_allowed=sensitivity_allowed,
+            project_scope=project_scope,
+            project_floor=project_floor,
             require_explicit_project_scope=True,
         )
+        decision = apply_unverified_rule(decision, judged, identity)
         if decision.decision == "blocked":
             raise AgentPolicyBlockedError(decision)
     else:
         memory_service.authorize_memory_action(
             identity=identity,
             action="memory.redact",
-            memory=memory,
+            memory=judged,
         )
     actor_type = "agent" if identity is not None else "user"
     forgotten_first = False

apps/api/src/alicebot_api/mcp/policy.py

@@ -137,6 +137,7 @@ def _policy_checked(
     domains: tuple[str, ...] = (),
     sensitivity_allowed: tuple[str, ...] = ("public", "internal", "private", "unknown"),
     project_scope: tuple[str, ...] = (),
+    project_floor: tuple[str, ...] = (),
     workflow_type: str | None = None,
     write_policy: str | None = None,
     require_explicit_project_scope: bool = False,
@@ -164,6 +165,7 @@ def _policy_checked(
         domains=domains,
         sensitivity_allowed=sensitivity_allowed,
         project_scope=project_scope,
+        project_floor=project_floor,
         workflow_type=workflow_type,
         write_policy=write_policy,
         require_explicit_project_scope=require_explicit_project_scope,

apps/api/src/alicebot_api/mcp/retrieval.py

@@ -756,13 +756,16 @@ def _handle_alice_open_loops(context: MCPRuntimeContext, arguments: Mapping[str,
         target = store.get_open_loop(loop_id)
         if target is None:
             raise MCPReferenceNotFoundError(f"open loop {loop_id} was not found")
+        from alicebot_api.vnext_label_guard import effective_row_for_fence
+
+        judged = effective_row_for_fence(store, identity, "open_loop", target)
         # Same ceiling block as memory mutations. The policy event names
         # this loop; the previous check logged the decision with no target.
         try:
             VNextMemoryCommitService(store).authorize_memory_action(
                 identity=identity,
                 action="open_loop.update",
-                memory=target,
+                memory=judged,
                 target_type="open_loop",
             )
         except AgentPolicyBlockedError as exc:

apps/api/src/alicebot_api/mcp/retrieval_shared.py

@@ -12,6 +12,7 @@
 from alicebot_api.vnext_agent_control import resource_project_scope
 from alicebot_api.vnext_project_scope import (
     is_global_scope,
+    project_floor_shape,
     project_identifier_identity,
     project_scopes_overlap,
 )
@@ -160,7 +161,12 @@ def _provenance_count(store: SQLiteVNextStore, memory_id: object) -> int:
 def _resource_matches_project_scope(resource: Mapping[str, object], project_scope: tuple[str, ...]) -> bool:
     if not project_scope:
         return True
-    return project_scopes_overlap(resource_project_scope(resource), project_scope)
+    shape, floor = project_floor_shape(resource)
+    return project_scopes_overlap(
+        resource_project_scope(resource),
+        project_scope,
+        floor=floor if shape == "list" else (),
+    )
 
 
 def _resource_is_held_back_global(resource: Mapping[str, object], exclude_global_domains: frozenset[str]) -> bool: