Drop the ks-ch-sync CronJob: adaptive-export mirrors kubescape state, on change

[ConstanzeTU]

Removes the ks-ch-sync CronJob. Adaptive-export 0.16.4-rc1 and later mirror kubescape state into ClickHouse from Go, writing only when the content actually changes.

Why

ks-ch-sync runs a python:3.11-slim pod every 2 minutes that LISTs every ContainerProfile, GETs each one individually, and INSERTs all of them unconditionally. Measured on edge4:

distinct profiles	234
stored rows	624,447
distinct contents	271

About 2,669 stored copies of each profile, and each row carries the container's whole learned behaviour as newline-joined text. That is why kubescape_profiles reached 501 MB for 234 real profiles.

It is not only storage. A single dx/shadow_profiles panel reading that table cost ~1.95 GiB per query, the panel auto-refreshes every 10–40s, and concurrent refreshes crossed the ClickHouse server's 5.40 GiB total budget — the container was OOMKilled (exit 137) on 2026-09-10 at 09:39:19Z. One open browser tab was sufficient. It also cost the KPI series a tick: the single gap in edge4's 220 measured ticks is that outage.

The amplification was never the language. It was a timer with no change detection; a Go rewrite that kept the loop would produce the same table.

What replaces it

internal/ae/kssync in entlein/dx, shipped in adaptive-export:

• Writes a row only when the content differs from the last row written.
• GETs an object only when its resourceVersion has moved, removing the ~235 API calls per run as well.
• Deliberately not an informer. The storage apiserver truncates spec on a LIST (execs/opens/egress come back empty) — which is exactly why the Python script GETs each object. An informer's cache would hold the truncated objects and mirror empty profiles. LIST for metadata, GET for content.
• Leader-only, so one row per change rather than one per node per change. The leader stamps its own node as hostname, which the PEM requires (WHERE hostname = gethostname() is injected onto every clickhouse_dsn read).
• Field mappings transcribed verbatim from sync.py, including its spec→label fallbacks, so the existing views read identical rows.

The three tables (kubescape_profiles, kubescape_rogueartifacts, kubescape_trust) are declared in AE's schema exactly as this script created them — same columns, types, MergeTree engine and ORDER BY. No engine change: event_time leading the sorting key is what makes every re-write distinct, and collapsing it would destroy the profile history, which is currently the only record of how a profile evolved.

Order of operations

This PR must land with or after the adaptive-export roll (k8sstormcenter/pixie#105). If both run at once every changed row is written twice; if neither runs, the mirror stops. ADAPTIVE_KSSYNC_DISABLE=true is available on AE for a cluster that needs to keep the CronJob temporarily.

Verifying after the change

AE logs listed, fetched and written per sync. On a steady cluster the correct reading is non-zero listed, zero written — that is the fix working. Zero listed means it read nothing (a withheld grant or a missing CRD), and the note lines say which.

🤖 Generated with Claude Code

https://claude.ai/code/session_014GDT6HWiFmRxmUaGFSKjPY