-
Notifications
You must be signed in to change notification settings - Fork 1
feat(argocd): per-container contrast legs for all 7 subcomponents #167
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 5 commits
c145c33
f5db7fc
7a77e2b
7e31583
282e7c7
75d4e35
5d4e534
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
|
|
@@ -85,6 +85,30 @@ deploy-mariadb: | |||||
| # deploy-misp and deploy-elk removed: misp + elk are archived to the inner repo | ||||||
| # under pkg/nonmigrated/ and are no longer part of the contrast-tuning matrix. | ||||||
|
|
||||||
| # Argo CD — FULL upstream install (all subcomponents) + vulnerable overlay. | ||||||
| # ARGOCD_VERSION pins the CVE-carrying release (see example/argocd-vulnerable.yaml). | ||||||
| ARGOCD_VERSION ?= v2.9.3 | ||||||
| .PHONY: deploy-argocd | ||||||
| deploy-argocd: | ||||||
| @echo "=== Deploying Argo CD $(ARGOCD_VERSION) (full install: all subcomponents) ===" | ||||||
| kubectl create namespace argocd 2>/dev/null || true | ||||||
| # Full install.yaml (NOT core-install): server + repo-server + app-controller | ||||||
| # + applicationset + notifications + dex + redis. Pinned + CVE-carrying. | ||||||
| kubectl apply -n argocd -f \ | ||||||
| https://raw.githubusercontent.com/argoproj/argo-cd/$(ARGOCD_VERSION)/manifests/install.yaml | ||||||
| @echo "=== Vulnerable overlay (permissive AppProject + exec-render surface) ===" | ||||||
| kubectl apply -f example/argocd-vulnerable.yaml | ||||||
| @echo "=== Wait for all Argo CD subcomponents ===" | ||||||
| -kubectl rollout status -n argocd deploy/argocd-server --timeout=300s | ||||||
| -kubectl rollout status -n argocd deploy/argocd-repo-server --timeout=300s | ||||||
| -kubectl rollout status -n argocd deploy/argocd-applicationset-controller --timeout=300s | ||||||
| -kubectl rollout status -n argocd deploy/argocd-notifications-controller --timeout=300s | ||||||
| -kubectl rollout status -n argocd deploy/argocd-dex-server --timeout=300s | ||||||
| -kubectl rollout status -n argocd deploy/argocd-redis --timeout=300s | ||||||
| -kubectl rollout status -n argocd statefulset/argocd-application-controller --timeout=300s | ||||||
| @echo "=== Argo CD subcomponents ===" | ||||||
| kubectl get pods -n argocd | ||||||
|
|
||||||
| .PHONY: deploy-postgres | ||||||
| deploy-postgres: | ||||||
| @echo "=== Deploying postgres (CloudNativePG) ===" | ||||||
|
|
@@ -237,38 +261,72 @@ kubescape-orig: | |||||
| -$(HELM) repo update | ||||||
| -$(HELM) upgrade --install kubescape kubescape/kubescape-operator --version $(KUBESCAPE_CHART_VER) -n honey --create-namespace --values kubescape/deprecated/values_orig.yaml | ||||||
| -kubectl apply -f kubescape/default-rules.yaml | ||||||
| sleep 5 | ||||||
| -kubectl rollout restart -n honey ds node-agent | ||||||
|
|
||||||
|
|
||||||
| # NOTE: node-agent is NEVER restarted by any target here. It must come up once, | ||||||
| # on its own, with the right config already in place — hence the post-renderer | ||||||
| # below instead of a patch-then-bounce. Do not reintroduce `rollout restart ds | ||||||
| # node-agent`. Also do NOT pass --set nodeAgent.privileged=true: the chart | ||||||
| # default is false and privileged node-agent has crashed this host. | ||||||
| KS_POST_RENDERER := ./kubescape/force-network-streaming.sh | ||||||
|
|
||||||
| # node-agent finds NEWLY STARTED containers by fanotify-marking the runc binary | ||||||
| # (Inspektor Gadget's WithContainerFanotifyEbpf). IG only knows the stock paths | ||||||
| # — /usr/bin/runc and /var/lib/rancher/k3s/data/current/bin/runc — so a cluster | ||||||
| # whose runc lives anywhere else is silently blind: node-agent still enumerates | ||||||
| # whatever was running when IT started (via the CRI socket) and looks perfectly | ||||||
| # healthy, while every container created afterwards gets no ContainerProfile. | ||||||
| # | ||||||
| # Detect the runc actually in use from the live containerd shim. If it is a stock | ||||||
| # path (CI runners, ordinary k3s) these stay EMPTY and nothing is overridden. Only | ||||||
| # a non-standard data-dir — e.g. this laptop's `k3s --data-dir /mnt/dev-data/k3s` | ||||||
| # — sets RUNTIME_PATH, and only then do we hostPath-mount the filesystem holding | ||||||
| # it, because node-agent's `host` volume is a NON-recursive bind of "/" and so | ||||||
| # does not carry a separate partition. | ||||||
| # | ||||||
| # The path is passed BARE (no /host prefix): runtimefinder.Notify only resolves | ||||||
| # symlinks chrooted to HOST_ROOT when the value does not already start with /host, | ||||||
| # and `data/current` is an absolute symlink. | ||||||
| KS_RUNC := $(shell ps -eo args 2>/dev/null | grep -oE '[^ ]*/bin/containerd-shim-runc-v2' | head -1 | sed 's|/containerd-shim-runc-v2|/runc|') | ||||||
| KS_RUNC_STOCK := $(shell echo "$(KS_RUNC)" | grep -qE '^$$|^/usr/bin/runc$$|^/var/lib/rancher/k3s/' && echo yes || echo no) | ||||||
| KS_RUNC_MNT := $(shell [ "$(KS_RUNC_STOCK)" = no ] && findmnt -no TARGET --target "$(KS_RUNC)" 2>/dev/null) | ||||||
| KS_RUNC_FLAGS := $(if $(filter no,$(KS_RUNC_STOCK)),--set global.overrideRuntimePath=$(KS_RUNC)$(if $(filter-out /,$(KS_RUNC_MNT)), --set volumes[0].name=ks-runc-fs --set volumes[0].hostPath.path=$(KS_RUNC_MNT) --set volumes[0].hostPath.type=Directory --set volumeMounts[0].name=ks-runc-fs --set volumeMounts[0].mountPath=/host$(KS_RUNC_MNT) --set volumeMounts[0].readOnly=true)) | ||||||
|
|
||||||
| .PHONY: show-runc | ||||||
| show-runc: | ||||||
| @echo "detected runc: $(KS_RUNC)" | ||||||
| @echo "stock layout: $(KS_RUNC_STOCK)" | ||||||
| @echo "holding mount: $(KS_RUNC_MNT)" | ||||||
| @echo "extra helm flags:$(KS_RUNC_FLAGS)" | ||||||
|
|
||||||
| .PHONY: kubescape | ||||||
| kubescape: | ||||||
| kubescape: | ||||||
| helm repo add kubescape https://kubescape.github.io/helm-charts/ | ||||||
| helm repo update | ||||||
| helm upgrade --install kubescape kubescape/kubescape-operator --version $(KUBESCAPE_CHART_VER) -n honey --create-namespace --values kubescape/values.yaml | ||||||
| helm upgrade --install kubescape kubescape/kubescape-operator --version $(KUBESCAPE_CHART_VER) -n honey --create-namespace --values kubescape/values.yaml $(KS_RUNC_FLAGS) --post-renderer $(KS_POST_RENDERER) | ||||||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win Honor This primary install path bypasses the configured Helm binary/wrapper, unlike the related targets. Proposed fix- helm upgrade --install kubescape kubescape/kubescape-operator --version $(KUBESCAPE_CHART_VER) -n honey --create-namespace --values kubescape/values.yaml $(KS_RUNC_FLAGS) --post-renderer $(KS_POST_RENDERER)
+ $(HELM) upgrade --install kubescape kubescape/kubescape-operator --version $(KUBESCAPE_CHART_VER) -n honey --create-namespace --values kubescape/values.yaml $(KS_RUNC_FLAGS) --post-renderer $(KS_POST_RENDERER)📝 Committable suggestion
Suggested change
🤖 Prompt for AI Agents |
||||||
| @echo "Ensuring CRDs are up-to-date (helm upgrade skips CRDs)..." | ||||||
| -helm show crds kubescape/kubescape-operator --version $(KUBESCAPE_CHART_VER) | kubectl apply --server-side --force-conflicts -f - 2>/dev/null || true | ||||||
| -kubectl apply -f kubescape/default-rules.yaml | ||||||
| sleep 5 | ||||||
| -kubectl rollout status -n honey deploy/kubevuln --timeout=120s | ||||||
| $(MAKE) enable-streaming | ||||||
|
|
||||||
| # enable-streaming forces networkStreamingEnabled=true in the node-agent | ||||||
| # configmap and rolls the DaemonSet. The chart gates the flag behind | ||||||
| # cloud-submit (rendered = submit AND enable; submit = non-empty .Values.server) | ||||||
| # so on an on-prem stack with no server it renders FALSE regardless of | ||||||
| # capabilities.networkEventsStreaming. This target is idempotent and MUST be | ||||||
| # re-run after EVERY helm upgrade — kubescape/alertmanager/kubescape-vendor all | ||||||
| # call it — so the setting never drifts. See docs/portability-spec.md D7a. | ||||||
| .PHONY: enable-streaming | ||||||
| enable-streaming: | ||||||
| @echo "Forcing node-agent networkStreamingEnabled=true (chart renders FALSE without cloud-submit)..." | ||||||
| @PATCH=$$(kubectl -n honey get configmap node-agent -o jsonpath='{.data.config\.json}' | python3 -c 'import json,sys; cfg=json.load(sys.stdin); cfg["networkStreamingEnabled"]=True; print(json.dumps({"data":{"config.json":json.dumps(cfg)}}))'); \ | ||||||
| kubectl -n honey patch configmap node-agent --type merge -p "$$PATCH" | ||||||
| -kubectl rollout restart -n honey ds node-agent | ||||||
| -kubectl rollout status -n honey ds node-agent --timeout=180s | ||||||
| $(MAKE) wait-node-agent | ||||||
| $(MAKE) verify-streaming | ||||||
|
Comment on lines
314
to
324
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win Duplicated
Also applies to: 313-326, 337-345 🧰 Tools🪛 checkmake (0.3.2)[warning] 274-274: Target body for "kubescape" exceeds allowed length of 5 lines (9). (maxbodylength) 🤖 Prompt for AI Agents |
||||||
|
|
||||||
| # Wait for node-agent to become Ready by itself. This is a WAIT, never a | ||||||
| # restart: node-agent binds user-supplied profiles and starts its learning | ||||||
| # window at pod start, so bouncing it throws that away. | ||||||
| .PHONY: wait-node-agent | ||||||
| wait-node-agent: | ||||||
| @echo "Waiting for node-agent DaemonSet to become ready (no restart)..." | ||||||
| kubectl rollout status -n honey ds node-agent --timeout=300s | ||||||
| @echo "=== node-agent pods ===" | ||||||
| kubectl get pods -n honey -l app.kubernetes.io/component=node-agent -o wide | ||||||
|
|
||||||
| # enable-streaming is retained as a compatibility alias. The flag is now forced | ||||||
| # at render time by $(KS_POST_RENDERER), so there is nothing to patch and | ||||||
| # nothing to restart — this only verifies the result. | ||||||
| .PHONY: enable-streaming | ||||||
| enable-streaming: verify-streaming | ||||||
|
|
||||||
| # Fail loud if node-agent network streaming is not actually live. Without it | ||||||
| # the profile's inline network shape (ingress/egress) is inert and R0005 (DNS) / | ||||||
| # R0011 (egress) silently never fire — which reads as "clean" when it is really | ||||||
|
|
@@ -291,8 +349,9 @@ alertmanager: | |||||
| # upgrade --install (not bare upgrade): idempotent, and does not require the | ||||||
| # kubescape release to pre-exist — so `make alertmanager` works on a fresh | ||||||
| # cluster / standalone, same as `make kubescape`. | ||||||
| $(HELM) upgrade --install kubescape kubescape/kubescape-operator --version $(KUBESCAPE_CHART_VER) -n honey --create-namespace --values kubescape/values.yaml | ||||||
| $(MAKE) enable-streaming | ||||||
| $(HELM) upgrade --install kubescape kubescape/kubescape-operator --version $(KUBESCAPE_CHART_VER) -n honey --create-namespace --values kubescape/values.yaml $(KS_RUNC_FLAGS) --post-renderer $(KS_POST_RENDERER) | ||||||
| $(MAKE) wait-node-agent | ||||||
| $(MAKE) verify-streaming | ||||||
| @echo "Alertmanager ready. Forward with: kubectl -n honey port-forward svc/alertmanager 9093:9093" | ||||||
|
|
||||||
| .PHONY: fwd-autotune | ||||||
|
|
@@ -308,11 +367,11 @@ fwd-autotune: | |||||
| kubescape-vendor: | ||||||
| -$(HELM) repo add kubescape https://kubescape.github.io/helm-charts/ | ||||||
| -$(HELM) repo update | ||||||
| $(HELM) upgrade --install kubescape kubescape/kubescape-operator --version $(KUBESCAPE_CHART_VER) -n honey --create-namespace --values kubescape/deprecated/values_vendor.yaml | ||||||
| $(HELM) upgrade --install kubescape kubescape/kubescape-operator --version $(KUBESCAPE_CHART_VER) -n honey --create-namespace --values kubescape/deprecated/values_vendor.yaml $(KS_RUNC_FLAGS) --post-renderer $(KS_POST_RENDERER) | ||||||
| -kubectl apply -f kubescape/runtimerules.yaml | ||||||
| sleep 5 | ||||||
| -kubectl rollout status -n honey deploy/kubevuln --timeout=120s | ||||||
| $(MAKE) enable-streaming | ||||||
| $(MAKE) wait-node-agent | ||||||
| $(MAKE) verify-streaming | ||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,94 @@ | ||
| # argocd-application-controller — attack leg (one container = one suite, per the | ||
| # add-new-app-for-bobctl-tuning skill §8). | ||
| # | ||
| # Baseline for this container is tini + the argocd binary and nothing else, so | ||
| # every binary below is anomalous by contrast. Rule names are verbatim from | ||
| # kubescape/default-rules.yaml. | ||
| # | ||
| # Honest boundaries for this container (documented probes, expectedDetections: []): | ||
| # R0006/R0007 — this controller reads its SA token and calls the kube-apiserver | ||
| # as NORMAL operation, so both are baseline-suppressed here. They | ||
| # are asserted on the argocd-redis leg, where the token is mounted | ||
| # but never read. | ||
| # R0009 — bpf(2) returns EPERM (unprivileged, no CAP_BPF); nothing loads. | ||
| # R1001 — this node-agent build emits zero R1001 cluster-wide; the drifted | ||
| # exec trips R1000 instead, which is what is asserted. | ||
| apiVersion: bobctl.k8sstormcenter.io/v1alpha1 | ||
| kind: AttackSuite | ||
| metadata: | ||
| name: argocd-application-controller-killchain | ||
| description: >- | ||
| Post-exploitation contrast leg for argocd-application-controller. Full rule-family | ||
| sweep: process, file, DNS, egress, link, mount and fileless primitives. | ||
| target: | ||
| service: argocd-metrics | ||
| namespace: argocd | ||
| port: 8082 | ||
| protocol: tcp | ||
|
|
||
| attacks: | ||
| - name: exec-etc-passwd | ||
| type: cmdinject | ||
| exec: { command: ["cat", "/etc/passwd"] } | ||
| successIndicators: [{ responseContains: "root:" }] | ||
| expectedDetections: | ||
| - { ruleID: R0001, ruleName: Unexpected process launched, containerName: argocd-application-controller, command: cat } | ||
| - name: exec-etc-shadow | ||
| type: cmdinject | ||
| exec: { command: ["cat", "/etc/shadow"] } | ||
| expectedDetections: | ||
| - { ruleID: R0010, ruleName: Unexpected Sensitive File Access, containerName: argocd-application-controller, command: cat } | ||
| - name: exec-proc-environ | ||
| type: cmdinject | ||
| exec: { command: ["cat", "/proc/1/environ"] } | ||
| successIndicators: [{ responseContains: "PATH" }] | ||
| expectedDetections: | ||
| - { ruleID: R0008, ruleName: Read Environment Variables from procfs, containerName: argocd-application-controller, command: cat } | ||
| - name: symlink-shadow | ||
| type: cmdinject | ||
| exec: { command: ["sh", "-c", "ln -sf /etc/shadow /tmp/sl_$$ && ls -l /tmp/sl_$$ >/dev/null && echo symlink_done; rm -f /tmp/sl_$$"] } | ||
| successIndicators: [{ responseContains: "symlink_done" }] | ||
| expectedDetections: | ||
| - { ruleID: R1010, ruleName: Soft link created over sensitive file, containerName: argocd-application-controller, command: ln } | ||
| - name: hardlink-shadow | ||
| type: cmdinject | ||
| exec: { command: ["sh", "-c", "ln /etc/shadow /tmp/hl_$$ >/dev/null 2>&1 && echo hardlink_done; rm -f /tmp/hl_$$"] } | ||
| successIndicators: [{ responseContains: "hardlink_done" }] | ||
| expectedDetections: | ||
| - { ruleID: R1012, ruleName: Hard link created over sensitive file, containerName: argocd-application-controller, command: ln } | ||
| - name: dns-anomaly-lookup | ||
| type: cmdinject | ||
| exec: { command: ["sh", "-c", "getent hosts scanner.evil-c2.example.com >/dev/null 2>&1 && echo dns_resolved || echo dns_attempted"] } | ||
| successIndicators: [{ responseContains: "dns_" }] | ||
| expectedDetections: | ||
| - { ruleID: R0005, ruleName: DNS Anomalies in container, containerName: argocd-application-controller } | ||
| - name: crypto-mining-dns | ||
| type: cmdinject | ||
| exec: { command: ["sh", "-c", "getent hosts xmr.pool.minergate.com >/dev/null 2>&1 && echo miner_dns_ok || echo miner_dns_attempted"] } | ||
| successIndicators: [{ responseContains: "miner_dns_" }] | ||
| expectedDetections: | ||
| - { ruleID: R1008, ruleName: Crypto Mining Domain Communication, containerName: argocd-application-controller } | ||
| - name: drifted-binary-exec | ||
| type: cmdinject | ||
| exec: { command: ["sh", "-c", "cp /bin/ls /tmp/drifted_bob && printf '\n#bob-drift' >> /tmp/drifted_bob && chmod +x /tmp/drifted_bob && /tmp/drifted_bob / >/dev/null 2>&1 && echo drift_exec_done; rm -f /tmp/drifted_bob"] } | ||
| successIndicators: [{ responseContains: "drift_exec_done" }] | ||
| expectedDetections: | ||
| - { ruleID: R1000, ruleName: Process executed from malicious source, containerName: argocd-application-controller } | ||
| - name: exec-from-volume-mount | ||
| type: cmdinject | ||
| exec: { command: ["sh", "-c", "cp /bin/echo /home/argocd/mnt_payload && chmod +x /home/argocd/mnt_payload && /home/argocd/mnt_payload shm_exec_done; rm -f /home/argocd/mnt_payload"] } | ||
| successIndicators: [{ responseContains: "shm_exec_done" }] | ||
| expectedDetections: | ||
| - { ruleID: R1004, ruleName: Process executed from mount, containerName: argocd-application-controller } | ||
| - name: fileless-memfd-exec | ||
| type: fileless | ||
| exec: { command: ["perl", "-e", "my $n=\"bobfl\\0\"; my $fd=syscall(319,$n,0); die if $fd<0; open(my $m,'>&='.$fd) or die; open(my $s,'<','/bin/echo') or die; binmode $s; binmode $m; local $/; my $d=<$s>; print $m $d; exec(\"/proc/$$/fd/$fd\",\"memfd_exec_done\");"] } | ||
| successIndicators: [{ responseContains: "memfd_exec_done" }] | ||
| expectedDetections: | ||
| - { ruleID: R1005, ruleName: Fileless execution detected, containerName: argocd-application-controller } | ||
| - name: egress-external-c2 | ||
| type: cmdinject | ||
| exec: { command: ["perl", "-e", "use Socket; socket(my $s,PF_INET,SOCK_STREAM,getprotobyname('tcp')); connect($s,sockaddr_in(80,inet_aton('1.1.1.1'))); close($s); print qq{egress_attempted}"] } | ||
| successIndicators: [{ responseContains: "egress_attempted" }] | ||
| expectedDetections: | ||
| - { ruleID: R0011, ruleName: Unexpected Egress Network Traffic, containerName: argocd-application-controller } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| # argocd-application-controller — functional tests (Prometheus read-path). | ||
| # | ||
| # argocd-application-controller exposes no user-facing API; its only reachable benign | ||
| # surface is the metrics endpoint Prometheus scrapes. That IS the normal traffic | ||
| # for this workload, so it is what the baseline must not alert on. | ||
| apiVersion: bobctl.k8sstormcenter.io/v1alpha1 | ||
| kind: FunctionalTestSuite | ||
| metadata: | ||
| name: argocd-application-controller-functional-tests | ||
| description: >- | ||
| Prometheus metrics scrape of argocd-application-controller — the only benign | ||
| network surface this controller exposes. Must not alert. | ||
| target: | ||
| service: argocd-metrics | ||
| namespace: argocd | ||
| port: 8082 | ||
| scheme: http | ||
| tests: | ||
| - name: metrics-scrape | ||
| http: { method: GET, path: /metrics, expectedStatus: 200, repeat: 5 } | ||
| - name: metrics-scrape-again | ||
| http: { method: GET, path: /metrics, expectedStatus: 200, repeat: 5 } |
Uh oh!
There was an error while loading. Please reload this page.