diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index 23ffcca6207..11fbf0952f5 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -60,6 +60,7 @@ /packages/aws/data_stream/ec2_logs @elastic/obs-ds-hosted-services /packages/aws/data_stream/ec2_metrics @elastic/obs-ds-hosted-services /packages/aws/data_stream/ecs_metrics @elastic/obs-ds-hosted-services +/packages/aws/data_stream/eks_audit @elastic/security-service-integrations /packages/aws/data_stream/elb_logs @elastic/obs-infraobs-integrations /packages/aws/data_stream/elb_metrics @elastic/obs-infraobs-integrations /packages/aws/data_stream/emr_logs @elastic/obs-infraobs-integrations diff --git a/packages/aws/_dev/build/docs/README.md b/packages/aws/_dev/build/docs/README.md index 06941635bc5..a119cd2d4d1 100644 --- a/packages/aws/_dev/build/docs/README.md +++ b/packages/aws/_dev/build/docs/README.md @@ -247,6 +247,7 @@ This is an overview of the type of data you can collect from each AWS service: | EBS | x | | | EC2 | x | x | | ECS | x | | +| EKS | | x | | ELB | x | x | | Fargate | x | | | Kinesis | x | | diff --git a/packages/aws/_dev/build/docs/eks.md b/packages/aws/_dev/build/docs/eks.md new file mode 100644 index 00000000000..6f09de119cd --- /dev/null +++ b/packages/aws/_dev/build/docs/eks.md @@ -0,0 +1,37 @@ +# Amazon EKS + +The Amazon EKS integration collects Kubernetes API audit events from Amazon CloudWatch Logs into the fixed `aws.eks_audit` dataset. + +Enable EKS control-plane audit logging before starting collection. The default discovery prefix is `/aws/eks/`, and the stream filter defaults to `kube-apiserver-audit`. + +## What data does this integration collect? + +The Amazon EKS integration collects Kubernetes API audit logs from the EKS control plane. + +## What do I need to use this integration? + +The AWS principal used by Elastic Agent needs permission to discover and read the selected CloudWatch log groups, including `logs:DescribeLogGroups` and `logs:FilterLogEvents`. When prefix discovery is used across linked accounts, configure the corresponding CloudWatch cross-account access as well. + +## Setup + +Configure either a log group ARN, a log group name, or the `/aws/eks/` log group prefix. The Region setting is required for name and prefix modes, including the default prefix mode. ARN mode ignores the Region setting because the ARN already identifies the Region. Keep the audit stream prefix unless the EKS stream naming in the target account requires a compatible override. + +Do not enable this data stream and `kubernetes.audit_logs` against the same EKS audit log groups. Duplicate collection creates duplicate audit events and can cause duplicate alerts. + +Kubernetes audit request and response objects are retained in document `_source` and can contain sensitive API payloads. For Secret resources, this integration removes `data` and `stringData` from parsed request and response objects. It also removes `data`, `stringData`, and metadata annotations from every item returned by Secret list/watch responses, while retaining each item's metadata name. The `preserve_original_event` option is disabled by default; enabling it retains the unredacted raw audit JSON in `event.original`, including Secret values removed from parsed fields. Unsupported records also retain `event.original` for troubleshooting. Restrict access to `_source` and enable original-event preservation only when its diagnostic value outweighs the exposure and storage costs. + +Authorization decision, authorization reason, and Pod Security audit-violation annotations have explicit searchable mappings. Other string-valued Kubernetes audit annotations are dynamically indexed as keywords after dots in annotation keys are replaced by underscores. + +`event.outcome` is derived from the HTTP response status when `responseStatus.code` is present: codes below 400 are `success` and codes of 400 or above are `failure`. This takes precedence over the `authorization.k8s.io/decision` annotation, because an authorized request can still fail with a 404, 409, or 5xx response. The annotation is used only when no response status is recorded, such as `RequestReceived` stage events. + +Request and response objects are not dynamically mapped. Only the security-relevant `aws.eks.audit.requestObject.*` and `aws.eks.audit.responseObject.*` fields listed in the field reference are indexed and searchable; the rest of each API object is retained in `_source` but cannot be queried or aggregated. This keeps the field count bounded on clusters that use many custom resource definitions, where dynamically mapping arbitrary object bodies would otherwise exhaust the index field limit and cause indexing failures. To query an additional body field, add it to a `logs-aws.eks_audit@custom` component template. + +## Logs reference + +{{event "eks_audit"}} + +**ECS Field Reference** + +Refer to the following [document](https://www.elastic.co/guide/en/ecs/current/ecs-field-reference.html) for detailed information on ECS fields. + +{{fields "eks_audit"}} \ No newline at end of file diff --git a/packages/aws/changelog.yml b/packages/aws/changelog.yml index 713734fd212..fff5b5a46c8 100644 --- a/packages/aws/changelog.yml +++ b/packages/aws/changelog.yml @@ -1,4 +1,9 @@ # newer versions go on top +- version: "8.2.0" + changes: + - description: Add top-level Amazon EKS Kubernetes API audit log collection and parsing from CloudWatch. + type: enhancement + link: https://github.com/elastic/integrations/pull/21051 - version: "8.1.2" changes: - description: Add an `Instance or Pod IAM Role` option to the `Setup Access` selector so Elastic Agent can authenticate with the AWS SDK default credential chain (EC2 instance profile, EKS Pod Identity or IRSA) without entering access keys or a Role ARN. Since 7.0.0 every option required at least one credential field, which made this documented path impossible to save. diff --git a/packages/aws/data_stream/eks_audit/_dev/test/pipeline/test-eks-audit-arn.log b/packages/aws/data_stream/eks_audit/_dev/test/pipeline/test-eks-audit-arn.log new file mode 100644 index 00000000000..eba0a3578ab --- /dev/null +++ b/packages/aws/data_stream/eks_audit/_dev/test/pipeline/test-eks-audit-arn.log @@ -0,0 +1 @@ +{"apiVersion":"audit.k8s.io/v1","kind":"Event","level":"Metadata","stage":"ResponseComplete","requestReceivedTimestamp":"2026-09-02T12:00:00Z","stageTimestamp":"2026-09-02T12:00:01Z","auditID":"arn-log-group","verb":"get","requestURI":"/api/v1/pods","objectRef":{"resource":"pods","namespace":"default","apiVersion":"v1"},"responseStatus":{"code":200}} diff --git a/packages/aws/data_stream/eks_audit/_dev/test/pipeline/test-eks-audit-arn.log-config.yml b/packages/aws/data_stream/eks_audit/_dev/test/pipeline/test-eks-audit-arn.log-config.yml new file mode 100644 index 00000000000..a5cdf33d431 --- /dev/null +++ b/packages/aws/data_stream/eks_audit/_dev/test/pipeline/test-eks-audit-arn.log-config.yml @@ -0,0 +1,9 @@ +fields: + aws: + cloudwatch: + log_group: arn:aws:logs:us-east-1:123456789012:log-group:/aws/eks/prod-cluster/cluster:* + log_stream: kube-apiserver-audit-123 + region: us-east-1 + account_id: "123456789012" + input: + type: aws-cloudwatch diff --git a/packages/aws/data_stream/eks_audit/_dev/test/pipeline/test-eks-audit-arn.log-expected.json b/packages/aws/data_stream/eks_audit/_dev/test/pipeline/test-eks-audit-arn.log-expected.json new file mode 100644 index 00000000000..2a91fc6bfda --- /dev/null +++ b/packages/aws/data_stream/eks_audit/_dev/test/pipeline/test-eks-audit-arn.log-expected.json @@ -0,0 +1,80 @@ +{ + "expected": [ + { + "@timestamp": "2026-09-02T12:00:00.000Z", + "aws": { + "cloudwatch": { + "account_id": "123456789012", + "log_group": "arn:aws:logs:us-east-1:123456789012:log-group:/aws/eks/prod-cluster/cluster:*", + "log_stream": "kube-apiserver-audit-123", + "region": "us-east-1" + }, + "eks": { + "audit": { + "apiVersion": "audit.k8s.io/v1", + "auditID": "arn-log-group", + "kind": "Event", + "level": "Metadata", + "objectRef": { + "apiVersion": "v1", + "namespace": "default", + "resource": "pods" + }, + "requestReceivedTimestamp": "2026-09-02T12:00:00Z", + "requestURI": "/api/v1/pods", + "responseStatus": { + "code": 200 + }, + "stage": "ResponseComplete", + "stageTimestamp": "2026-09-02T12:00:01Z", + "verb": "get" + }, + "cluster": { + "name": "prod-cluster" + }, + "component": "kube-apiserver", + "log_type": "audit" + } + }, + "cloud": { + "account": { + "id": "123456789012" + }, + "provider": "aws", + "region": "us-east-1" + }, + "ecs": { + "version": "8.11.0" + }, + "event": { + "action": "get", + "category": [ + "configuration" + ], + "dataset": "aws.eks_audit", + "end": "2026-09-02T12:00:01.000Z", + "id": "arn-log-group", + "kind": "event", + "original": "{\"apiVersion\":\"audit.k8s.io/v1\",\"kind\":\"Event\",\"level\":\"Metadata\",\"stage\":\"ResponseComplete\",\"requestReceivedTimestamp\":\"2026-09-02T12:00:00Z\",\"stageTimestamp\":\"2026-09-02T12:00:01Z\",\"auditID\":\"arn-log-group\",\"verb\":\"get\",\"requestURI\":\"/api/v1/pods\",\"objectRef\":{\"resource\":\"pods\",\"namespace\":\"default\",\"apiVersion\":\"v1\"},\"responseStatus\":{\"code\":200}}", + "outcome": "success", + "type": [ + "access" + ] + }, + "input": { + "type": "aws-cloudwatch" + }, + "orchestrator": { + "api_version": "v1", + "cluster": { + "name": "prod-cluster" + }, + "namespace": "default", + "resource": { + "type": "pods" + }, + "type": "kubernetes" + } + } + ] +} diff --git a/packages/aws/data_stream/eks_audit/_dev/test/pipeline/test-eks-audit-no-preserve.log b/packages/aws/data_stream/eks_audit/_dev/test/pipeline/test-eks-audit-no-preserve.log new file mode 100644 index 00000000000..95eaf90c81c --- /dev/null +++ b/packages/aws/data_stream/eks_audit/_dev/test/pipeline/test-eks-audit-no-preserve.log @@ -0,0 +1,3 @@ +{"apiVersion":"audit.k8s.io/v1","kind":"Event","level":"RequestResponse","stage":"ResponseComplete","requestReceivedTimestamp":"2026-08-05T08:00:00Z","stageTimestamp":"2026-08-05T08:00:01Z","sourceIPs":["198.51.100.10"],"userAgent":"kubectl/v1.31.0","user":{"username":"arn:aws:iam::123456789012:user/alice","uid":"aws-iam-authenticator:123456789012:alice","groups":["system:authenticated"]},"auditID":"allowed-secret","verb":"get","requestURI":"/api/v1/secrets","objectRef":{"resource":"secrets","namespace":"default","name":"app-secret"},"responseStatus":{"code":200},"annotations":{"authorization.k8s.io/decision":"allow","authorization.k8s.io/reason":"RBAC decision"}} +{"apiVersion":"audit.k8s.io/v1","kind":"Event","level":"RequestResponse","stage":"ResponseComplete","requestReceivedTimestamp":"2026-08-05T08:00:00Z","stageTimestamp":"2026-08-05T08:00:01Z","auditID":"secret-redaction","verb":"create","requestURI":"/api/v1/namespaces/default/secrets/app-secret","objectRef":{"resource":"secrets","namespace":"default","name":"app-secret"},"responseStatus":{"code":201},"requestObject":{"data":{"password":"cmVxdWVzdC1zZWNyZXQ="},"stringData":{"token":"request-token"}},"responseObject":{"data":{"password":"cmVzcG9uc2Utc2VjcmV0"},"stringData":{"token":"response-token"}}} +{"apiVersion":"audit.k8s.io/v1","kind":"Event","level":"RequestResponse","stage":"ResponseComplete","requestReceivedTimestamp":"2026-08-05T08:00:00Z","stageTimestamp":"2026-08-05T08:00:01Z","auditID":"secret-list-redaction","verb":"list","requestURI":"/api/v1/namespaces/default/secrets?watch=true","objectRef":{"resource":"secrets","namespace":"default"},"responseStatus":{"code":200},"responseObject":{"items":[{"metadata":{"name":"first","annotations":{"kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"v1\",\"kind\":\"Secret\",\"metadata\":{\"name\":\"first\"},\"data\":{\"password\":\"c2Vuc2l0aXZl\"}}"}},"data":{"password":"Zmlyc3Q="},"stringData":{"token":"first-token"}},{"metadata":{"name":"second"},"data":{"password":"c2Vjb25k"},"stringData":{"token":"second-token"}}]}} diff --git a/packages/aws/data_stream/eks_audit/_dev/test/pipeline/test-eks-audit-no-preserve.log-config.yml b/packages/aws/data_stream/eks_audit/_dev/test/pipeline/test-eks-audit-no-preserve.log-config.yml new file mode 100644 index 00000000000..90fb25fe63a --- /dev/null +++ b/packages/aws/data_stream/eks_audit/_dev/test/pipeline/test-eks-audit-no-preserve.log-config.yml @@ -0,0 +1,9 @@ +fields: + aws: + cloudwatch: + log_group: /aws/eks/prod-cluster/cluster + log_stream: kube-apiserver-audit-123 + region: us-east-1 + account_id: "123456789012" + input: + type: aws-cloudwatch diff --git a/packages/aws/data_stream/eks_audit/_dev/test/pipeline/test-eks-audit-no-preserve.log-expected.json b/packages/aws/data_stream/eks_audit/_dev/test/pipeline/test-eks-audit-no-preserve.log-expected.json new file mode 100644 index 00000000000..1a4236fbabb --- /dev/null +++ b/packages/aws/data_stream/eks_audit/_dev/test/pipeline/test-eks-audit-no-preserve.log-expected.json @@ -0,0 +1,293 @@ +{ + "expected": [ + { + "@timestamp": "2026-08-05T08:00:00.000Z", + "aws": { + "cloudwatch": { + "account_id": "123456789012", + "log_group": "/aws/eks/prod-cluster/cluster", + "log_stream": "kube-apiserver-audit-123", + "region": "us-east-1" + }, + "eks": { + "audit": { + "annotations": { + "authorization_k8s_io/decision": "allow", + "authorization_k8s_io/reason": "RBAC decision" + }, + "apiVersion": "audit.k8s.io/v1", + "auditID": "allowed-secret", + "kind": "Event", + "level": "RequestResponse", + "objectRef": { + "name": "app-secret", + "namespace": "default", + "resource": "secrets" + }, + "requestReceivedTimestamp": "2026-08-05T08:00:00Z", + "requestURI": "/api/v1/secrets", + "responseStatus": { + "code": 200 + }, + "sourceIPs": [ + "198.51.100.10" + ], + "stage": "ResponseComplete", + "stageTimestamp": "2026-08-05T08:00:01Z", + "user": { + "groups": [ + "system:authenticated" + ], + "uid": "aws-iam-authenticator:123456789012:alice", + "username": "arn:aws:iam::123456789012:user/alice" + }, + "userAgent": "kubectl/v1.31.0", + "verb": "get" + }, + "cluster": { + "name": "prod-cluster" + }, + "component": "kube-apiserver", + "log_type": "audit" + } + }, + "client": { + "ip": [ + "198.51.100.10" + ] + }, + "cloud": { + "account": { + "id": "123456789012" + }, + "provider": "aws", + "region": "us-east-1" + }, + "ecs": { + "version": "8.11.0" + }, + "event": { + "action": "get", + "category": [ + "configuration" + ], + "dataset": "aws.eks_audit", + "end": "2026-08-05T08:00:01.000Z", + "id": "allowed-secret", + "kind": "event", + "original": "{\"apiVersion\":\"audit.k8s.io/v1\",\"kind\":\"Event\",\"level\":\"RequestResponse\",\"stage\":\"ResponseComplete\",\"requestReceivedTimestamp\":\"2026-08-05T08:00:00Z\",\"stageTimestamp\":\"2026-08-05T08:00:01Z\",\"sourceIPs\":[\"198.51.100.10\"],\"userAgent\":\"kubectl/v1.31.0\",\"user\":{\"username\":\"arn:aws:iam::123456789012:user/alice\",\"uid\":\"aws-iam-authenticator:123456789012:alice\",\"groups\":[\"system:authenticated\"]},\"auditID\":\"allowed-secret\",\"verb\":\"get\",\"requestURI\":\"/api/v1/secrets\",\"objectRef\":{\"resource\":\"secrets\",\"namespace\":\"default\",\"name\":\"app-secret\"},\"responseStatus\":{\"code\":200},\"annotations\":{\"authorization.k8s.io/decision\":\"allow\",\"authorization.k8s.io/reason\":\"RBAC decision\"}}", + "outcome": "success", + "type": [ + "access" + ] + }, + "input": { + "type": "aws-cloudwatch" + }, + "orchestrator": { + "api_version": "audit.k8s.io/v1", + "cluster": { + "name": "prod-cluster" + }, + "namespace": "default", + "resource": { + "name": "app-secret", + "type": "secrets" + }, + "type": "kubernetes" + }, + "related": { + "ip": [ + "198.51.100.10" + ], + "user": [ + "aws-iam-authenticator:123456789012:alice", + "arn:aws:iam::123456789012:user/alice" + ] + }, + "source": { + "ip": [ + "198.51.100.10" + ] + }, + "user": { + "group": { + "name": [ + "system:authenticated" + ] + }, + "id": "aws-iam-authenticator:123456789012:alice", + "name": "arn:aws:iam::123456789012:user/alice" + }, + "user_agent": { + "original": "kubectl/v1.31.0" + } + }, + { + "@timestamp": "2026-08-05T08:00:00.000Z", + "aws": { + "cloudwatch": { + "account_id": "123456789012", + "log_group": "/aws/eks/prod-cluster/cluster", + "log_stream": "kube-apiserver-audit-123", + "region": "us-east-1" + }, + "eks": { + "audit": { + "apiVersion": "audit.k8s.io/v1", + "auditID": "secret-redaction", + "kind": "Event", + "level": "RequestResponse", + "objectRef": { + "name": "app-secret", + "namespace": "default", + "resource": "secrets" + }, + "requestReceivedTimestamp": "2026-08-05T08:00:00Z", + "requestURI": "/api/v1/namespaces/default/secrets/app-secret", + "responseStatus": { + "code": 201 + }, + "stage": "ResponseComplete", + "stageTimestamp": "2026-08-05T08:00:01Z", + "verb": "create" + }, + "cluster": { + "name": "prod-cluster" + }, + "component": "kube-apiserver", + "log_type": "audit" + } + }, + "cloud": { + "account": { + "id": "123456789012" + }, + "provider": "aws", + "region": "us-east-1" + }, + "ecs": { + "version": "8.11.0" + }, + "event": { + "action": "create", + "category": [ + "configuration" + ], + "dataset": "aws.eks_audit", + "end": "2026-08-05T08:00:01.000Z", + "id": "secret-redaction", + "kind": "event", + "original": "{\"apiVersion\":\"audit.k8s.io/v1\",\"kind\":\"Event\",\"level\":\"RequestResponse\",\"stage\":\"ResponseComplete\",\"requestReceivedTimestamp\":\"2026-08-05T08:00:00Z\",\"stageTimestamp\":\"2026-08-05T08:00:01Z\",\"auditID\":\"secret-redaction\",\"verb\":\"create\",\"requestURI\":\"/api/v1/namespaces/default/secrets/app-secret\",\"objectRef\":{\"resource\":\"secrets\",\"namespace\":\"default\",\"name\":\"app-secret\"},\"responseStatus\":{\"code\":201},\"requestObject\":{\"data\":{\"password\":\"cmVxdWVzdC1zZWNyZXQ=\"},\"stringData\":{\"token\":\"request-token\"}},\"responseObject\":{\"data\":{\"password\":\"cmVzcG9uc2Utc2VjcmV0\"},\"stringData\":{\"token\":\"response-token\"}}}", + "outcome": "success", + "type": [ + "creation" + ] + }, + "input": { + "type": "aws-cloudwatch" + }, + "orchestrator": { + "api_version": "audit.k8s.io/v1", + "cluster": { + "name": "prod-cluster" + }, + "namespace": "default", + "resource": { + "name": "app-secret", + "type": "secrets" + }, + "type": "kubernetes" + } + }, + { + "@timestamp": "2026-08-05T08:00:00.000Z", + "aws": { + "cloudwatch": { + "account_id": "123456789012", + "log_group": "/aws/eks/prod-cluster/cluster", + "log_stream": "kube-apiserver-audit-123", + "region": "us-east-1" + }, + "eks": { + "audit": { + "apiVersion": "audit.k8s.io/v1", + "auditID": "secret-list-redaction", + "kind": "Event", + "level": "RequestResponse", + "objectRef": { + "namespace": "default", + "resource": "secrets" + }, + "requestReceivedTimestamp": "2026-08-05T08:00:00Z", + "requestURI": "/api/v1/namespaces/default/secrets?watch=true", + "responseObject": { + "items": [ + { + "metadata": { + "name": "first" + } + }, + { + "metadata": { + "name": "second" + } + } + ] + }, + "responseStatus": { + "code": 200 + }, + "stage": "ResponseComplete", + "stageTimestamp": "2026-08-05T08:00:01Z", + "verb": "list" + }, + "cluster": { + "name": "prod-cluster" + }, + "component": "kube-apiserver", + "log_type": "audit" + } + }, + "cloud": { + "account": { + "id": "123456789012" + }, + "provider": "aws", + "region": "us-east-1" + }, + "ecs": { + "version": "8.11.0" + }, + "event": { + "action": "list", + "category": [ + "configuration" + ], + "dataset": "aws.eks_audit", + "end": "2026-08-05T08:00:01.000Z", + "id": "secret-list-redaction", + "kind": "event", + "original": "{\"apiVersion\":\"audit.k8s.io/v1\",\"kind\":\"Event\",\"level\":\"RequestResponse\",\"stage\":\"ResponseComplete\",\"requestReceivedTimestamp\":\"2026-08-05T08:00:00Z\",\"stageTimestamp\":\"2026-08-05T08:00:01Z\",\"auditID\":\"secret-list-redaction\",\"verb\":\"list\",\"requestURI\":\"/api/v1/namespaces/default/secrets?watch=true\",\"objectRef\":{\"resource\":\"secrets\",\"namespace\":\"default\"},\"responseStatus\":{\"code\":200},\"responseObject\":{\"items\":[{\"metadata\":{\"name\":\"first\",\"annotations\":{\"kubectl.kubernetes.io/last-applied-configuration\":\"{\\\"apiVersion\\\":\\\"v1\\\",\\\"kind\\\":\\\"Secret\\\",\\\"metadata\\\":{\\\"name\\\":\\\"first\\\"},\\\"data\\\":{\\\"password\\\":\\\"c2Vuc2l0aXZl\\\"}}\"}},\"data\":{\"password\":\"Zmlyc3Q=\"},\"stringData\":{\"token\":\"first-token\"}},{\"metadata\":{\"name\":\"second\"},\"data\":{\"password\":\"c2Vjb25k\"},\"stringData\":{\"token\":\"second-token\"}}]}}", + "outcome": "success", + "type": [ + "access" + ] + }, + "input": { + "type": "aws-cloudwatch" + }, + "orchestrator": { + "api_version": "audit.k8s.io/v1", + "cluster": { + "name": "prod-cluster" + }, + "namespace": "default", + "resource": { + "type": "secrets" + }, + "type": "kubernetes" + } + } + ] +} diff --git a/packages/aws/data_stream/eks_audit/_dev/test/pipeline/test-eks-audit.log b/packages/aws/data_stream/eks_audit/_dev/test/pipeline/test-eks-audit.log new file mode 100644 index 00000000000..9bf117d1c26 --- /dev/null +++ b/packages/aws/data_stream/eks_audit/_dev/test/pipeline/test-eks-audit.log @@ -0,0 +1,17 @@ +{"apiVersion":"audit.k8s.io/v1","kind":"Event","level":"RequestResponse","stage":"ResponseComplete","requestReceivedTimestamp":"2026-08-05T08:00:00Z","stageTimestamp":"2026-08-05T08:00:01Z","sourceIPs":["198.51.100.10"],"userAgent":"kubectl/v1.31.0","user":{"username":"arn:aws:iam::123456789012:user/alice","uid":"aws-iam-authenticator:123456789012:alice","groups":["system:authenticated"]},"auditID":"allowed-secret","verb":"get","requestURI":"/api/v1/secrets","objectRef":{"resource":"secrets","namespace":"default","name":"app-secret"},"responseStatus":{"code":200},"annotations":{"authorization.k8s.io/decision":"allow","authorization.k8s.io/reason":"RBAC decision"}} +{"apiVersion":"audit.k8s.io/v1","kind":"Event","level":"RequestResponse","stage":"ResponseComplete","requestReceivedTimestamp":"2026-08-05T08:00:00Z","stageTimestamp":"2026-08-05T08:00:01Z","sourceIPs":["198.51.100.10"],"userAgent":"kubectl/v1.31.0","user":{"username":"arn:aws:iam::123456789012:user/alice","uid":"aws-iam-authenticator:123456789012:alice","groups":["system:authenticated"]},"auditID":"forbidden-secret","verb":"get","requestURI":"/api/v1/secrets","objectRef":{"resource":"secrets","namespace":"prod","name":"db-secret"},"responseStatus":{"code":403},"annotations":{"authorization.k8s.io/decision":"forbid","authorization.k8s.io/reason":"RBAC decision"}} +{"apiVersion":"audit.k8s.io/v1","kind":"Event","level":"RequestResponse","stage":"ResponseComplete","requestReceivedTimestamp":"2026-08-05T08:00:00Z","stageTimestamp":"2026-08-05T08:00:01Z","sourceIPs":["198.51.100.10"],"userAgent":"kubectl/v1.31.0","user":{"username":"arn:aws:iam::123456789012:user/alice","uid":"aws-iam-authenticator:123456789012:alice","groups":["system:authenticated"]},"auditID":"impersonation","verb":"get","requestURI":"/api/v1/pods","objectRef":{"resource":"pods"},"responseStatus":{"code":200},"annotations":{"authorization.k8s.io/decision":"allow","authorization.k8s.io/reason":"RBAC decision"},"impersonatedUser":{"username":"system:serviceaccount:default:reader","uid":"impersonated-uid","groups":["system:serviceaccounts"]}} +{"apiVersion":"audit.k8s.io/v1","kind":"Event","level":"RequestResponse","stage":"ResponseComplete","requestReceivedTimestamp":"2026-08-05T08:00:00Z","stageTimestamp":"2026-08-05T08:00:01Z","sourceIPs":["198.51.100.10"],"userAgent":"kubectl/v1.31.0","user":{"username":"arn:aws:iam::123456789012:user/alice","uid":"aws-iam-authenticator:123456789012:alice","groups":["system:authenticated"]},"auditID":"rbac-subjects","verb":"create","requestURI":"/api/v1/rolebindings","objectRef":{"resource":"rolebindings","namespace":"default","name":"deployers"},"responseStatus":{"code":201},"annotations":{"authorization.k8s.io/decision":"allow","authorization.k8s.io/reason":"RBAC decision"},"requestObject":{"subjects":[{"kind":"ServiceAccount","name":"deployer","namespace":"default"},{"kind":"Group","name":"developers"}],"roleRef":{"name":"edit"}},"responseObject":{"subjects":[{"kind":"ServiceAccount","name":"deployer","namespace":"default"},{"kind":"Group","name":"developers"}],"roleRef":{"kind":"ClusterRole","name":"edit"}}} +{"apiVersion":"audit.k8s.io/v1","kind":"Event","level":"RequestResponse","stage":"ResponseComplete","requestReceivedTimestamp":"2026-08-05T08:00:00Z","stageTimestamp":"2026-08-05T08:00:01Z","sourceIPs":["198.51.100.10"],"userAgent":"kubectl/v1.31.0","user":{"username":"arn:aws:iam::123456789012:user/alice","uid":"aws-iam-authenticator:123456789012:alice","groups":["system:authenticated"]},"auditID":"pod-request","verb":"create","requestURI":"/api/v1/pods","objectRef":{"resource":"pods","namespace":"default","name":"privileged"},"responseStatus":{"code":201},"annotations":{"authorization.k8s.io/decision":"allow","authorization.k8s.io/reason":"RBAC decision"},"requestObject":{"spec":{"serviceAccountName":"builder","hostNetwork":true,"hostPID":true,"containers":[{"name":"shell","image":"busybox:latest","command":["sh"],"securityContext":{"privileged":true,"allowPrivilegeEscalation":true,"runAsUser":0,"capabilities":{"add":["SYS_ADMIN"]},"seccompProfile":{"type":"Unconfined"}}}]}}} +{"apiVersion":"audit.k8s.io/v1","kind":"Event","level":"RequestResponse","stage":"ResponseComplete","requestReceivedTimestamp":"2026-08-05T08:00:00Z","stageTimestamp":"2026-08-05T08:00:01Z","sourceIPs":["198.51.100.10"],"userAgent":"kubectl/v1.31.0","user":{"username":"arn:aws:iam::123456789012:user/alice","uid":"aws-iam-authenticator:123456789012:alice","groups":["system:authenticated"]},"auditID":"delete-event","verb":"delete","requestURI":"/api/v1/events","objectRef":{"resource":"events","namespace":"default","name":"suspicious-event"},"responseStatus":{"code":200},"annotations":{"authorization.k8s.io/decision":"allow","authorization.k8s.io/reason":"RBAC decision"}} +{"apiVersion":"audit.k8s.io/v1","kind":"Event","level":"Metadata","stage":"ResponseComplete","requestReceivedTimestamp":"2026-08-05T08:00:00Z","stageTimestamp":"2026-08-05T08:00:01Z","auditID":"denied-status","verb":"update","requestURI":"/apis/apps/v1/namespaces/prod/deployments/api","objectRef":{"resource":"deployments","namespace":"prod","name":"api","apiGroup":"apps","apiVersion":"v1"},"responseStatus":{"code":403,"status":"Failure","reason":"Forbidden","retryAfterSeconds":5,"details":{"group":"apps","kind":"Deployment","name":"api","uid":"deployment-uid","causes":[{"reason":"FieldValueInvalid","message":"invalid value","field":"spec.replicas"}]}}} +{"apiVersion":"audit.k8s.io/v1","kind":"Event","level":"RequestResponse","stage":"ResponseComplete","requestReceivedTimestamp":"2026-08-05T08:00:00Z","stageTimestamp":"2026-08-05T08:00:01Z","auditID":"pods-exec","verb":"create","requestURI":"/api/v1/namespaces/default/pods/debug/exec","objectRef":{"resource":"pods","namespace":"default","name":"debug","subresource":"exec"},"responseStatus":{"code":200},"annotations":{"authorization.k8s.io/decision":"allow","authorization.k8s.io/reason":"RBAC decision"}} +{"apiVersion":"audit.k8s.io/v1","kind":"Event","level":"Metadata","stage":"ResponseComplete","requestReceivedTimestamp":"2026-08-05T08:00:00Z","stageTimestamp":"2026-08-05T08:00:01Z","auditID":"status-overrides-allow","verb":"get","requestURI":"/api/v1/namespaces/default/configmaps/app","objectRef":{"resource":"configmaps","namespace":"default","name":"app"},"responseStatus":{"code":403},"annotations":{"authorization.k8s.io/decision":"allow"}} +{"apiVersion":"audit.k8s.io/v1","kind":"Event","level":"RequestResponse","stage":"ResponseComplete","requestReceivedTimestamp":"2026-08-05T08:00:00Z","stageTimestamp":"2026-08-05T08:00:01Z","auditID":"rbac-rules","verb":"create","requestURI":"/apis/rbac.authorization.k8s.io/v1/clusterroles/auditor","objectRef":{"resource":"clusterroles","name":"auditor","apiGroup":"rbac.authorization.k8s.io","apiVersion":"v1"},"responseStatus":{"code":201},"requestObject":{"rules":[{"apiGroups":["","apps"],"resources":["pods","deployments"],"resourceNames":["api"],"verbs":["get","list"],"nonResourceURLs":["/healthz"]}]},"responseObject":{"rules":[{"apiGroups":[""],"resources":["pods"],"resourceNames":["api"],"verbs":["get"],"nonResourceURLs":["/healthz"]}]}} +{"apiVersion":"audit.k8s.io/v1","kind":"Event","level":"RequestResponse","stage":"ResponseComplete","requestReceivedTimestamp":"2026-08-05T08:00:00Z","stageTimestamp":"2026-08-05T08:00:01Z","auditID":"rbac-core-mixed","verb":"create","requestURI":"/apis/rbac.authorization.k8s.io/v1/roles/mixed","objectRef":{"resource":"roles","namespace":"default","name":"mixed","apiGroup":"rbac.authorization.k8s.io"},"responseStatus":{"code":201},"requestObject":{"rules":[{"apiGroups":["","apps"],"resources":["pods","deployments"],"verbs":["get"]}]}} +{"apiVersion":"audit.k8s.io/v1","kind":"Event","level":"RequestResponse","stage":"ResponseComplete","requestReceivedTimestamp":"2026-08-05T08:00:00Z","stageTimestamp":"2026-08-05T08:00:01Z","auditID":"rbac-core-only","verb":"create","requestURI":"/api/v1/namespaces/default/roles/core","objectRef":{"resource":"roles","namespace":"default","name":"core","apiGroup":""},"responseStatus":{"code":201},"responseObject":{"rules":[{"apiGroups":[""],"resources":["pods"],"verbs":["get","list"]}]}} +{"apiVersion":"audit.k8s.io/v1","kind":"Event","level":"Metadata","stage":"ResponseComplete","requestReceivedTimestamp":"2026-08-05T08:00:00Z","stageTimestamp":"2026-08-05T08:00:01Z","auditID":"user-extra-dotted-keys","verb":"get","requestURI":"/api/v1/pods","objectRef":{"resource":"pods","namespace":"default"},"responseStatus":{"code":200},"user":{"username":"system:node:worker","groups":["system:nodes"],"extra":{"authentication.kubernetes.io/credential-id":["X509SHA256=test"],"sigs.k8s.io/aws-iam-authenticator/principalId":["AIDATEST"]}},"impersonatedUser":{"username":"system:serviceaccount:default:reader","extra":{"authentication.kubernetes.io/pod-name":["reader"]}}} +{"apiVersion":"audit.k8s.io/v1","kind":"Event","level":"RequestResponse","stage":"ResponseComplete","requestReceivedTimestamp":"2026-08-05T08:00:00Z","stageTimestamp":"2026-08-05T08:00:01Z","auditID":"crd-mapped-path-collision","verb":"patch","requestURI":"/apis/example.io/v1/widgets/example","objectRef":{"resource":"widgets","name":"example","apiGroup":"example.io","apiVersion":"v1"},"responseStatus":{"code":200},"requestObject":{"spec":{"hostNetwork":"enabled","containers":"custom"}},"responseObject":{"roleRef":"custom"}} +{"apiVersion":"audit.k8s.io/v1","kind":"Event","level":"RequestResponse","stage":"ResponseComplete","requestReceivedTimestamp":"2026-08-05T08:00:00Z","stageTimestamp":"2026-08-05T08:00:01Z","auditID":"malformed-mapped-shapes","verb":"get","requestURI":"/api/v1/pods","objectRef":"invalid","responseStatus":"invalid","requestObject":"invalid","responseObject":["invalid"],"user":"invalid","impersonatedUser":"invalid","annotations":["invalid"]} +{"apiVersion":"audit.k8s.io/v1","kind":"Event","level":{"unexpected":"object"},"stage":"ResponseComplete","requestReceivedTimestamp":"2026-08-05T08:00:00Z","stageTimestamp":"2026-08-05T08:00:01Z","auditID":"malformed-scalar-shapes","verb":{"unexpected":"object"},"requestURI":{"unexpected":"object"},"userAgent":{"unexpected":"object"},"sourceIPs":{"unexpected":"object"},"objectRef":{"resource":"pods","namespace":"default","name":"app-1","apiVersion":"v1"},"responseStatus":{"code":200},"requestObject":{"apiVersion":"rbac.authorization.k8s.io/v1","kind":"Role","rules":[{"apiGroups":[{"unexpected":"object"}],"resources":[["nested"]],"verbs":["get"]}],"subjects":[{"kind":{"unexpected":"object"},"name":"attacker"}]},"annotations":{"authorization.k8s.io/decision":"allow","authorization.k8s.io/reason":"RBAC"}} +{"apiVersion":"audit.k8s.io/v1","kind":"Event","level":"RequestResponse","stage":"ResponseComplete","requestReceivedTimestamp":"2026-08-05T08:00:00Z","stageTimestamp":"2026-08-05T08:00:01Z","auditID":"subject-access-review","verb":"create","requestURI":"/apis/authorization.k8s.io/v1/subjectaccessreviews","objectRef":{"resource":"subjectaccessreviews","apiGroup":"authorization.k8s.io","apiVersion":"v1"},"responseStatus":{"code":201},"requestObject":{"apiVersion":"authorization.k8s.io/v1","kind":"SubjectAccessReview","spec":{"user":"system:serviceaccount:default:reader","uid":"reader-uid","groups":["system:serviceaccounts"],"extra":{"authentication.kubernetes.io/credential-id":["X509SHA256=test"]},"resourceAttributes":{"group":"","version":"v1","resource":"pods","subresource":"exec","namespace":"default","name":"debug","verb":"create"}}},"responseObject":{"apiVersion":"authorization.k8s.io/v1","kind":"SubjectAccessReview","status":{"allowed":true,"denied":false,"reason":"RBAC allowed"}}} diff --git a/packages/aws/data_stream/eks_audit/_dev/test/pipeline/test-eks-audit.log-config.yml b/packages/aws/data_stream/eks_audit/_dev/test/pipeline/test-eks-audit.log-config.yml new file mode 100644 index 00000000000..85aa3b2173b --- /dev/null +++ b/packages/aws/data_stream/eks_audit/_dev/test/pipeline/test-eks-audit.log-config.yml @@ -0,0 +1,11 @@ +fields: + tags: + - preserve_original_event + aws: + cloudwatch: + log_group: /aws/eks/prod-cluster/cluster + log_stream: kube-apiserver-audit-123 + region: us-east-1 + account_id: "123456789012" + input: + type: aws-cloudwatch diff --git a/packages/aws/data_stream/eks_audit/_dev/test/pipeline/test-eks-audit.log-expected.json b/packages/aws/data_stream/eks_audit/_dev/test/pipeline/test-eks-audit.log-expected.json new file mode 100644 index 00000000000..4d9e4774028 --- /dev/null +++ b/packages/aws/data_stream/eks_audit/_dev/test/pipeline/test-eks-audit.log-expected.json @@ -0,0 +1,1887 @@ +{ + "expected": [ + { + "@timestamp": "2026-08-05T08:00:00.000Z", + "aws": { + "cloudwatch": { + "account_id": "123456789012", + "log_group": "/aws/eks/prod-cluster/cluster", + "log_stream": "kube-apiserver-audit-123", + "region": "us-east-1" + }, + "eks": { + "audit": { + "annotations": { + "authorization_k8s_io/decision": "allow", + "authorization_k8s_io/reason": "RBAC decision" + }, + "apiVersion": "audit.k8s.io/v1", + "auditID": "allowed-secret", + "kind": "Event", + "level": "RequestResponse", + "objectRef": { + "name": "app-secret", + "namespace": "default", + "resource": "secrets" + }, + "requestReceivedTimestamp": "2026-08-05T08:00:00Z", + "requestURI": "/api/v1/secrets", + "responseStatus": { + "code": 200 + }, + "sourceIPs": [ + "198.51.100.10" + ], + "stage": "ResponseComplete", + "stageTimestamp": "2026-08-05T08:00:01Z", + "user": { + "groups": [ + "system:authenticated" + ], + "uid": "aws-iam-authenticator:123456789012:alice", + "username": "arn:aws:iam::123456789012:user/alice" + }, + "userAgent": "kubectl/v1.31.0", + "verb": "get" + }, + "cluster": { + "name": "prod-cluster" + }, + "component": "kube-apiserver", + "log_type": "audit" + } + }, + "client": { + "ip": [ + "198.51.100.10" + ] + }, + "cloud": { + "account": { + "id": "123456789012" + }, + "provider": "aws", + "region": "us-east-1" + }, + "ecs": { + "version": "8.11.0" + }, + "event": { + "action": "get", + "category": [ + "configuration" + ], + "dataset": "aws.eks_audit", + "end": "2026-08-05T08:00:01.000Z", + "id": "allowed-secret", + "kind": "event", + "original": "{\"apiVersion\":\"audit.k8s.io/v1\",\"kind\":\"Event\",\"level\":\"RequestResponse\",\"stage\":\"ResponseComplete\",\"requestReceivedTimestamp\":\"2026-08-05T08:00:00Z\",\"stageTimestamp\":\"2026-08-05T08:00:01Z\",\"sourceIPs\":[\"198.51.100.10\"],\"userAgent\":\"kubectl/v1.31.0\",\"user\":{\"username\":\"arn:aws:iam::123456789012:user/alice\",\"uid\":\"aws-iam-authenticator:123456789012:alice\",\"groups\":[\"system:authenticated\"]},\"auditID\":\"allowed-secret\",\"verb\":\"get\",\"requestURI\":\"/api/v1/secrets\",\"objectRef\":{\"resource\":\"secrets\",\"namespace\":\"default\",\"name\":\"app-secret\"},\"responseStatus\":{\"code\":200},\"annotations\":{\"authorization.k8s.io/decision\":\"allow\",\"authorization.k8s.io/reason\":\"RBAC decision\"}}", + "outcome": "success", + "type": [ + "access" + ] + }, + "input": { + "type": "aws-cloudwatch" + }, + "orchestrator": { + "api_version": "audit.k8s.io/v1", + "cluster": { + "name": "prod-cluster" + }, + "namespace": "default", + "resource": { + "name": "app-secret", + "type": "secrets" + }, + "type": "kubernetes" + }, + "related": { + "ip": [ + "198.51.100.10" + ], + "user": [ + "aws-iam-authenticator:123456789012:alice", + "arn:aws:iam::123456789012:user/alice" + ] + }, + "source": { + "ip": [ + "198.51.100.10" + ] + }, + "tags": [ + "preserve_original_event" + ], + "user": { + "group": { + "name": [ + "system:authenticated" + ] + }, + "id": "aws-iam-authenticator:123456789012:alice", + "name": "arn:aws:iam::123456789012:user/alice" + }, + "user_agent": { + "original": "kubectl/v1.31.0" + } + }, + { + "@timestamp": "2026-08-05T08:00:00.000Z", + "aws": { + "cloudwatch": { + "account_id": "123456789012", + "log_group": "/aws/eks/prod-cluster/cluster", + "log_stream": "kube-apiserver-audit-123", + "region": "us-east-1" + }, + "eks": { + "audit": { + "annotations": { + "authorization_k8s_io/decision": "forbid", + "authorization_k8s_io/reason": "RBAC decision" + }, + "apiVersion": "audit.k8s.io/v1", + "auditID": "forbidden-secret", + "kind": "Event", + "level": "RequestResponse", + "objectRef": { + "name": "db-secret", + "namespace": "prod", + "resource": "secrets" + }, + "requestReceivedTimestamp": "2026-08-05T08:00:00Z", + "requestURI": "/api/v1/secrets", + "responseStatus": { + "code": 403 + }, + "sourceIPs": [ + "198.51.100.10" + ], + "stage": "ResponseComplete", + "stageTimestamp": "2026-08-05T08:00:01Z", + "user": { + "groups": [ + "system:authenticated" + ], + "uid": "aws-iam-authenticator:123456789012:alice", + "username": "arn:aws:iam::123456789012:user/alice" + }, + "userAgent": "kubectl/v1.31.0", + "verb": "get" + }, + "cluster": { + "name": "prod-cluster" + }, + "component": "kube-apiserver", + "log_type": "audit" + } + }, + "client": { + "ip": [ + "198.51.100.10" + ] + }, + "cloud": { + "account": { + "id": "123456789012" + }, + "provider": "aws", + "region": "us-east-1" + }, + "ecs": { + "version": "8.11.0" + }, + "event": { + "action": "get", + "category": [ + "configuration" + ], + "dataset": "aws.eks_audit", + "end": "2026-08-05T08:00:01.000Z", + "id": "forbidden-secret", + "kind": "event", + "original": "{\"apiVersion\":\"audit.k8s.io/v1\",\"kind\":\"Event\",\"level\":\"RequestResponse\",\"stage\":\"ResponseComplete\",\"requestReceivedTimestamp\":\"2026-08-05T08:00:00Z\",\"stageTimestamp\":\"2026-08-05T08:00:01Z\",\"sourceIPs\":[\"198.51.100.10\"],\"userAgent\":\"kubectl/v1.31.0\",\"user\":{\"username\":\"arn:aws:iam::123456789012:user/alice\",\"uid\":\"aws-iam-authenticator:123456789012:alice\",\"groups\":[\"system:authenticated\"]},\"auditID\":\"forbidden-secret\",\"verb\":\"get\",\"requestURI\":\"/api/v1/secrets\",\"objectRef\":{\"resource\":\"secrets\",\"namespace\":\"prod\",\"name\":\"db-secret\"},\"responseStatus\":{\"code\":403},\"annotations\":{\"authorization.k8s.io/decision\":\"forbid\",\"authorization.k8s.io/reason\":\"RBAC decision\"}}", + "outcome": "failure", + "type": [ + "access" + ] + }, + "input": { + "type": "aws-cloudwatch" + }, + "orchestrator": { + "api_version": "audit.k8s.io/v1", + "cluster": { + "name": "prod-cluster" + }, + "namespace": "prod", + "resource": { + "name": "db-secret", + "type": "secrets" + }, + "type": "kubernetes" + }, + "related": { + "ip": [ + "198.51.100.10" + ], + "user": [ + "aws-iam-authenticator:123456789012:alice", + "arn:aws:iam::123456789012:user/alice" + ] + }, + "source": { + "ip": [ + "198.51.100.10" + ] + }, + "tags": [ + "preserve_original_event" + ], + "user": { + "group": { + "name": [ + "system:authenticated" + ] + }, + "id": "aws-iam-authenticator:123456789012:alice", + "name": "arn:aws:iam::123456789012:user/alice" + }, + "user_agent": { + "original": "kubectl/v1.31.0" + } + }, + { + "@timestamp": "2026-08-05T08:00:00.000Z", + "aws": { + "cloudwatch": { + "account_id": "123456789012", + "log_group": "/aws/eks/prod-cluster/cluster", + "log_stream": "kube-apiserver-audit-123", + "region": "us-east-1" + }, + "eks": { + "audit": { + "annotations": { + "authorization_k8s_io/decision": "allow", + "authorization_k8s_io/reason": "RBAC decision" + }, + "apiVersion": "audit.k8s.io/v1", + "auditID": "impersonation", + "impersonatedUser": { + "groups": [ + "system:serviceaccounts" + ], + "uid": "impersonated-uid", + "username": "system:serviceaccount:default:reader" + }, + "kind": "Event", + "level": "RequestResponse", + "objectRef": { + "resource": "pods" + }, + "requestReceivedTimestamp": "2026-08-05T08:00:00Z", + "requestURI": "/api/v1/pods", + "responseStatus": { + "code": 200 + }, + "sourceIPs": [ + "198.51.100.10" + ], + "stage": "ResponseComplete", + "stageTimestamp": "2026-08-05T08:00:01Z", + "user": { + "groups": [ + "system:authenticated" + ], + "uid": "aws-iam-authenticator:123456789012:alice", + "username": "arn:aws:iam::123456789012:user/alice" + }, + "userAgent": "kubectl/v1.31.0", + "verb": "get" + }, + "cluster": { + "name": "prod-cluster" + }, + "component": "kube-apiserver", + "log_type": "audit" + } + }, + "client": { + "ip": [ + "198.51.100.10" + ] + }, + "cloud": { + "account": { + "id": "123456789012" + }, + "provider": "aws", + "region": "us-east-1" + }, + "ecs": { + "version": "8.11.0" + }, + "event": { + "action": "get", + "category": [ + "configuration" + ], + "dataset": "aws.eks_audit", + "end": "2026-08-05T08:00:01.000Z", + "id": "impersonation", + "kind": "event", + "original": "{\"apiVersion\":\"audit.k8s.io/v1\",\"kind\":\"Event\",\"level\":\"RequestResponse\",\"stage\":\"ResponseComplete\",\"requestReceivedTimestamp\":\"2026-08-05T08:00:00Z\",\"stageTimestamp\":\"2026-08-05T08:00:01Z\",\"sourceIPs\":[\"198.51.100.10\"],\"userAgent\":\"kubectl/v1.31.0\",\"user\":{\"username\":\"arn:aws:iam::123456789012:user/alice\",\"uid\":\"aws-iam-authenticator:123456789012:alice\",\"groups\":[\"system:authenticated\"]},\"auditID\":\"impersonation\",\"verb\":\"get\",\"requestURI\":\"/api/v1/pods\",\"objectRef\":{\"resource\":\"pods\"},\"responseStatus\":{\"code\":200},\"annotations\":{\"authorization.k8s.io/decision\":\"allow\",\"authorization.k8s.io/reason\":\"RBAC decision\"},\"impersonatedUser\":{\"username\":\"system:serviceaccount:default:reader\",\"uid\":\"impersonated-uid\",\"groups\":[\"system:serviceaccounts\"]}}", + "outcome": "success", + "type": [ + "access" + ] + }, + "input": { + "type": "aws-cloudwatch" + }, + "orchestrator": { + "api_version": "audit.k8s.io/v1", + "cluster": { + "name": "prod-cluster" + }, + "resource": { + "type": "pods" + }, + "type": "kubernetes" + }, + "related": { + "ip": [ + "198.51.100.10" + ], + "user": [ + "aws-iam-authenticator:123456789012:alice", + "arn:aws:iam::123456789012:user/alice" + ] + }, + "source": { + "ip": [ + "198.51.100.10" + ] + }, + "tags": [ + "preserve_original_event" + ], + "user": { + "effective": { + "group": { + "name": [ + "system:serviceaccounts" + ] + }, + "id": "impersonated-uid", + "name": "system:serviceaccount:default:reader" + }, + "group": { + "name": [ + "system:authenticated" + ] + }, + "id": "aws-iam-authenticator:123456789012:alice", + "name": "arn:aws:iam::123456789012:user/alice" + }, + "user_agent": { + "original": "kubectl/v1.31.0" + } + }, + { + "@timestamp": "2026-08-05T08:00:00.000Z", + "aws": { + "cloudwatch": { + "account_id": "123456789012", + "log_group": "/aws/eks/prod-cluster/cluster", + "log_stream": "kube-apiserver-audit-123", + "region": "us-east-1" + }, + "eks": { + "audit": { + "annotations": { + "authorization_k8s_io/decision": "allow", + "authorization_k8s_io/reason": "RBAC decision" + }, + "apiVersion": "audit.k8s.io/v1", + "auditID": "rbac-subjects", + "kind": "Event", + "level": "RequestResponse", + "objectRef": { + "name": "deployers", + "namespace": "default", + "resource": "rolebindings" + }, + "requestObject": { + "roleRef": { + "name": "edit" + }, + "subjects": [ + { + "kind": "ServiceAccount", + "name": "deployer", + "namespace": "default" + }, + { + "kind": "Group", + "name": "developers" + } + ] + }, + "requestReceivedTimestamp": "2026-08-05T08:00:00Z", + "requestURI": "/api/v1/rolebindings", + "responseObject": { + "roleRef": { + "kind": "ClusterRole", + "name": "edit" + }, + "subjects": [ + { + "kind": "ServiceAccount", + "name": "deployer", + "namespace": "default" + }, + { + "kind": "Group", + "name": "developers" + } + ] + }, + "responseStatus": { + "code": 201 + }, + "sourceIPs": [ + "198.51.100.10" + ], + "stage": "ResponseComplete", + "stageTimestamp": "2026-08-05T08:00:01Z", + "user": { + "groups": [ + "system:authenticated" + ], + "uid": "aws-iam-authenticator:123456789012:alice", + "username": "arn:aws:iam::123456789012:user/alice" + }, + "userAgent": "kubectl/v1.31.0", + "verb": "create" + }, + "cluster": { + "name": "prod-cluster" + }, + "component": "kube-apiserver", + "log_type": "audit" + } + }, + "client": { + "ip": [ + "198.51.100.10" + ] + }, + "cloud": { + "account": { + "id": "123456789012" + }, + "provider": "aws", + "region": "us-east-1" + }, + "ecs": { + "version": "8.11.0" + }, + "event": { + "action": "create", + "category": [ + "configuration" + ], + "dataset": "aws.eks_audit", + "end": "2026-08-05T08:00:01.000Z", + "id": "rbac-subjects", + "kind": "event", + "original": "{\"apiVersion\":\"audit.k8s.io/v1\",\"kind\":\"Event\",\"level\":\"RequestResponse\",\"stage\":\"ResponseComplete\",\"requestReceivedTimestamp\":\"2026-08-05T08:00:00Z\",\"stageTimestamp\":\"2026-08-05T08:00:01Z\",\"sourceIPs\":[\"198.51.100.10\"],\"userAgent\":\"kubectl/v1.31.0\",\"user\":{\"username\":\"arn:aws:iam::123456789012:user/alice\",\"uid\":\"aws-iam-authenticator:123456789012:alice\",\"groups\":[\"system:authenticated\"]},\"auditID\":\"rbac-subjects\",\"verb\":\"create\",\"requestURI\":\"/api/v1/rolebindings\",\"objectRef\":{\"resource\":\"rolebindings\",\"namespace\":\"default\",\"name\":\"deployers\"},\"responseStatus\":{\"code\":201},\"annotations\":{\"authorization.k8s.io/decision\":\"allow\",\"authorization.k8s.io/reason\":\"RBAC decision\"},\"requestObject\":{\"subjects\":[{\"kind\":\"ServiceAccount\",\"name\":\"deployer\",\"namespace\":\"default\"},{\"kind\":\"Group\",\"name\":\"developers\"}],\"roleRef\":{\"name\":\"edit\"}},\"responseObject\":{\"subjects\":[{\"kind\":\"ServiceAccount\",\"name\":\"deployer\",\"namespace\":\"default\"},{\"kind\":\"Group\",\"name\":\"developers\"}],\"roleRef\":{\"kind\":\"ClusterRole\",\"name\":\"edit\"}}}", + "outcome": "success", + "type": [ + "creation" + ] + }, + "input": { + "type": "aws-cloudwatch" + }, + "orchestrator": { + "api_version": "audit.k8s.io/v1", + "cluster": { + "name": "prod-cluster" + }, + "namespace": "default", + "resource": { + "name": "deployers", + "type": "rolebindings" + }, + "type": "kubernetes" + }, + "related": { + "ip": [ + "198.51.100.10" + ], + "user": [ + "aws-iam-authenticator:123456789012:alice", + "arn:aws:iam::123456789012:user/alice" + ] + }, + "source": { + "ip": [ + "198.51.100.10" + ] + }, + "tags": [ + "preserve_original_event" + ], + "user": { + "group": { + "name": [ + "system:authenticated" + ] + }, + "id": "aws-iam-authenticator:123456789012:alice", + "name": "arn:aws:iam::123456789012:user/alice" + }, + "user_agent": { + "original": "kubectl/v1.31.0" + } + }, + { + "@timestamp": "2026-08-05T08:00:00.000Z", + "aws": { + "cloudwatch": { + "account_id": "123456789012", + "log_group": "/aws/eks/prod-cluster/cluster", + "log_stream": "kube-apiserver-audit-123", + "region": "us-east-1" + }, + "eks": { + "audit": { + "annotations": { + "authorization_k8s_io/decision": "allow", + "authorization_k8s_io/reason": "RBAC decision" + }, + "apiVersion": "audit.k8s.io/v1", + "auditID": "pod-request", + "kind": "Event", + "level": "RequestResponse", + "objectRef": { + "name": "privileged", + "namespace": "default", + "resource": "pods" + }, + "requestObject": { + "spec": { + "containers": [ + { + "command": [ + "sh" + ], + "image": "busybox:latest", + "name": "shell", + "securityContext": { + "allowPrivilegeEscalation": true, + "capabilities": { + "add": [ + "SYS_ADMIN" + ] + }, + "privileged": true, + "runAsUser": 0, + "seccompProfile": { + "type": "Unconfined" + } + } + } + ], + "hostNetwork": true, + "hostPID": true, + "serviceAccountName": "builder" + } + }, + "requestReceivedTimestamp": "2026-08-05T08:00:00Z", + "requestURI": "/api/v1/pods", + "responseStatus": { + "code": 201 + }, + "sourceIPs": [ + "198.51.100.10" + ], + "stage": "ResponseComplete", + "stageTimestamp": "2026-08-05T08:00:01Z", + "user": { + "groups": [ + "system:authenticated" + ], + "uid": "aws-iam-authenticator:123456789012:alice", + "username": "arn:aws:iam::123456789012:user/alice" + }, + "userAgent": "kubectl/v1.31.0", + "verb": "create" + }, + "cluster": { + "name": "prod-cluster" + }, + "component": "kube-apiserver", + "log_type": "audit" + } + }, + "client": { + "ip": [ + "198.51.100.10" + ] + }, + "cloud": { + "account": { + "id": "123456789012" + }, + "provider": "aws", + "region": "us-east-1" + }, + "ecs": { + "version": "8.11.0" + }, + "event": { + "action": "create", + "category": [ + "configuration" + ], + "dataset": "aws.eks_audit", + "end": "2026-08-05T08:00:01.000Z", + "id": "pod-request", + "kind": "event", + "original": "{\"apiVersion\":\"audit.k8s.io/v1\",\"kind\":\"Event\",\"level\":\"RequestResponse\",\"stage\":\"ResponseComplete\",\"requestReceivedTimestamp\":\"2026-08-05T08:00:00Z\",\"stageTimestamp\":\"2026-08-05T08:00:01Z\",\"sourceIPs\":[\"198.51.100.10\"],\"userAgent\":\"kubectl/v1.31.0\",\"user\":{\"username\":\"arn:aws:iam::123456789012:user/alice\",\"uid\":\"aws-iam-authenticator:123456789012:alice\",\"groups\":[\"system:authenticated\"]},\"auditID\":\"pod-request\",\"verb\":\"create\",\"requestURI\":\"/api/v1/pods\",\"objectRef\":{\"resource\":\"pods\",\"namespace\":\"default\",\"name\":\"privileged\"},\"responseStatus\":{\"code\":201},\"annotations\":{\"authorization.k8s.io/decision\":\"allow\",\"authorization.k8s.io/reason\":\"RBAC decision\"},\"requestObject\":{\"spec\":{\"serviceAccountName\":\"builder\",\"hostNetwork\":true,\"hostPID\":true,\"containers\":[{\"name\":\"shell\",\"image\":\"busybox:latest\",\"command\":[\"sh\"],\"securityContext\":{\"privileged\":true,\"allowPrivilegeEscalation\":true,\"runAsUser\":0,\"capabilities\":{\"add\":[\"SYS_ADMIN\"]},\"seccompProfile\":{\"type\":\"Unconfined\"}}}]}}}", + "outcome": "success", + "type": [ + "creation" + ] + }, + "input": { + "type": "aws-cloudwatch" + }, + "orchestrator": { + "api_version": "audit.k8s.io/v1", + "cluster": { + "name": "prod-cluster" + }, + "namespace": "default", + "resource": { + "name": "privileged", + "type": "pods" + }, + "type": "kubernetes" + }, + "related": { + "ip": [ + "198.51.100.10" + ], + "user": [ + "aws-iam-authenticator:123456789012:alice", + "arn:aws:iam::123456789012:user/alice" + ] + }, + "source": { + "ip": [ + "198.51.100.10" + ] + }, + "tags": [ + "preserve_original_event" + ], + "user": { + "group": { + "name": [ + "system:authenticated" + ] + }, + "id": "aws-iam-authenticator:123456789012:alice", + "name": "arn:aws:iam::123456789012:user/alice" + }, + "user_agent": { + "original": "kubectl/v1.31.0" + } + }, + { + "@timestamp": "2026-08-05T08:00:00.000Z", + "aws": { + "cloudwatch": { + "account_id": "123456789012", + "log_group": "/aws/eks/prod-cluster/cluster", + "log_stream": "kube-apiserver-audit-123", + "region": "us-east-1" + }, + "eks": { + "audit": { + "annotations": { + "authorization_k8s_io/decision": "allow", + "authorization_k8s_io/reason": "RBAC decision" + }, + "apiVersion": "audit.k8s.io/v1", + "auditID": "delete-event", + "kind": "Event", + "level": "RequestResponse", + "objectRef": { + "name": "suspicious-event", + "namespace": "default", + "resource": "events" + }, + "requestReceivedTimestamp": "2026-08-05T08:00:00Z", + "requestURI": "/api/v1/events", + "responseStatus": { + "code": 200 + }, + "sourceIPs": [ + "198.51.100.10" + ], + "stage": "ResponseComplete", + "stageTimestamp": "2026-08-05T08:00:01Z", + "user": { + "groups": [ + "system:authenticated" + ], + "uid": "aws-iam-authenticator:123456789012:alice", + "username": "arn:aws:iam::123456789012:user/alice" + }, + "userAgent": "kubectl/v1.31.0", + "verb": "delete" + }, + "cluster": { + "name": "prod-cluster" + }, + "component": "kube-apiserver", + "log_type": "audit" + } + }, + "client": { + "ip": [ + "198.51.100.10" + ] + }, + "cloud": { + "account": { + "id": "123456789012" + }, + "provider": "aws", + "region": "us-east-1" + }, + "ecs": { + "version": "8.11.0" + }, + "event": { + "action": "delete", + "category": [ + "configuration" + ], + "dataset": "aws.eks_audit", + "end": "2026-08-05T08:00:01.000Z", + "id": "delete-event", + "kind": "event", + "original": "{\"apiVersion\":\"audit.k8s.io/v1\",\"kind\":\"Event\",\"level\":\"RequestResponse\",\"stage\":\"ResponseComplete\",\"requestReceivedTimestamp\":\"2026-08-05T08:00:00Z\",\"stageTimestamp\":\"2026-08-05T08:00:01Z\",\"sourceIPs\":[\"198.51.100.10\"],\"userAgent\":\"kubectl/v1.31.0\",\"user\":{\"username\":\"arn:aws:iam::123456789012:user/alice\",\"uid\":\"aws-iam-authenticator:123456789012:alice\",\"groups\":[\"system:authenticated\"]},\"auditID\":\"delete-event\",\"verb\":\"delete\",\"requestURI\":\"/api/v1/events\",\"objectRef\":{\"resource\":\"events\",\"namespace\":\"default\",\"name\":\"suspicious-event\"},\"responseStatus\":{\"code\":200},\"annotations\":{\"authorization.k8s.io/decision\":\"allow\",\"authorization.k8s.io/reason\":\"RBAC decision\"}}", + "outcome": "success", + "type": [ + "deletion" + ] + }, + "input": { + "type": "aws-cloudwatch" + }, + "orchestrator": { + "api_version": "audit.k8s.io/v1", + "cluster": { + "name": "prod-cluster" + }, + "namespace": "default", + "resource": { + "name": "suspicious-event", + "type": "events" + }, + "type": "kubernetes" + }, + "related": { + "ip": [ + "198.51.100.10" + ], + "user": [ + "aws-iam-authenticator:123456789012:alice", + "arn:aws:iam::123456789012:user/alice" + ] + }, + "source": { + "ip": [ + "198.51.100.10" + ] + }, + "tags": [ + "preserve_original_event" + ], + "user": { + "group": { + "name": [ + "system:authenticated" + ] + }, + "id": "aws-iam-authenticator:123456789012:alice", + "name": "arn:aws:iam::123456789012:user/alice" + }, + "user_agent": { + "original": "kubectl/v1.31.0" + } + }, + { + "@timestamp": "2026-08-05T08:00:00.000Z", + "aws": { + "cloudwatch": { + "account_id": "123456789012", + "log_group": "/aws/eks/prod-cluster/cluster", + "log_stream": "kube-apiserver-audit-123", + "region": "us-east-1" + }, + "eks": { + "audit": { + "apiVersion": "audit.k8s.io/v1", + "auditID": "denied-status", + "kind": "Event", + "level": "Metadata", + "objectRef": { + "apiGroup": "apps", + "apiVersion": "v1", + "name": "api", + "namespace": "prod", + "resource": "deployments" + }, + "requestReceivedTimestamp": "2026-08-05T08:00:00Z", + "requestURI": "/apis/apps/v1/namespaces/prod/deployments/api", + "responseStatus": { + "code": 403, + "details": { + "causes": [ + { + "field": "spec.replicas", + "message": "invalid value", + "reason": "FieldValueInvalid" + } + ], + "group": "apps", + "kind": "Deployment", + "name": "api", + "uid": "deployment-uid" + }, + "reason": "Forbidden", + "retryAfterSeconds": 5, + "status": "Failure" + }, + "stage": "ResponseComplete", + "stageTimestamp": "2026-08-05T08:00:01Z", + "verb": "update" + }, + "cluster": { + "name": "prod-cluster" + }, + "component": "kube-apiserver", + "log_type": "audit" + } + }, + "cloud": { + "account": { + "id": "123456789012" + }, + "provider": "aws", + "region": "us-east-1" + }, + "ecs": { + "version": "8.11.0" + }, + "event": { + "action": "update", + "category": [ + "configuration" + ], + "dataset": "aws.eks_audit", + "end": "2026-08-05T08:00:01.000Z", + "id": "denied-status", + "kind": "event", + "original": "{\"apiVersion\":\"audit.k8s.io/v1\",\"kind\":\"Event\",\"level\":\"Metadata\",\"stage\":\"ResponseComplete\",\"requestReceivedTimestamp\":\"2026-08-05T08:00:00Z\",\"stageTimestamp\":\"2026-08-05T08:00:01Z\",\"auditID\":\"denied-status\",\"verb\":\"update\",\"requestURI\":\"/apis/apps/v1/namespaces/prod/deployments/api\",\"objectRef\":{\"resource\":\"deployments\",\"namespace\":\"prod\",\"name\":\"api\",\"apiGroup\":\"apps\",\"apiVersion\":\"v1\"},\"responseStatus\":{\"code\":403,\"status\":\"Failure\",\"reason\":\"Forbidden\",\"retryAfterSeconds\":5,\"details\":{\"group\":\"apps\",\"kind\":\"Deployment\",\"name\":\"api\",\"uid\":\"deployment-uid\",\"causes\":[{\"reason\":\"FieldValueInvalid\",\"message\":\"invalid value\",\"field\":\"spec.replicas\"}]}}}", + "outcome": "failure", + "type": [ + "change" + ] + }, + "input": { + "type": "aws-cloudwatch" + }, + "orchestrator": { + "api_version": "v1", + "cluster": { + "name": "prod-cluster" + }, + "namespace": "prod", + "resource": { + "name": "api", + "type": "deployments" + }, + "type": "kubernetes" + }, + "tags": [ + "preserve_original_event" + ] + }, + { + "@timestamp": "2026-08-05T08:00:00.000Z", + "aws": { + "cloudwatch": { + "account_id": "123456789012", + "log_group": "/aws/eks/prod-cluster/cluster", + "log_stream": "kube-apiserver-audit-123", + "region": "us-east-1" + }, + "eks": { + "audit": { + "annotations": { + "authorization_k8s_io/decision": "allow", + "authorization_k8s_io/reason": "RBAC decision" + }, + "apiVersion": "audit.k8s.io/v1", + "auditID": "pods-exec", + "kind": "Event", + "level": "RequestResponse", + "objectRef": { + "name": "debug", + "namespace": "default", + "resource": "pods", + "subresource": "exec" + }, + "requestReceivedTimestamp": "2026-08-05T08:00:00Z", + "requestURI": "/api/v1/namespaces/default/pods/debug/exec", + "responseStatus": { + "code": 200 + }, + "stage": "ResponseComplete", + "stageTimestamp": "2026-08-05T08:00:01Z", + "verb": "create" + }, + "cluster": { + "name": "prod-cluster" + }, + "component": "kube-apiserver", + "log_type": "audit" + } + }, + "cloud": { + "account": { + "id": "123456789012" + }, + "provider": "aws", + "region": "us-east-1" + }, + "ecs": { + "version": "8.11.0" + }, + "event": { + "action": "create", + "category": [ + "configuration" + ], + "dataset": "aws.eks_audit", + "end": "2026-08-05T08:00:01.000Z", + "id": "pods-exec", + "kind": "event", + "original": "{\"apiVersion\":\"audit.k8s.io/v1\",\"kind\":\"Event\",\"level\":\"RequestResponse\",\"stage\":\"ResponseComplete\",\"requestReceivedTimestamp\":\"2026-08-05T08:00:00Z\",\"stageTimestamp\":\"2026-08-05T08:00:01Z\",\"auditID\":\"pods-exec\",\"verb\":\"create\",\"requestURI\":\"/api/v1/namespaces/default/pods/debug/exec\",\"objectRef\":{\"resource\":\"pods\",\"namespace\":\"default\",\"name\":\"debug\",\"subresource\":\"exec\"},\"responseStatus\":{\"code\":200},\"annotations\":{\"authorization.k8s.io/decision\":\"allow\",\"authorization.k8s.io/reason\":\"RBAC decision\"}}", + "outcome": "success", + "type": [ + "access" + ] + }, + "input": { + "type": "aws-cloudwatch" + }, + "orchestrator": { + "api_version": "audit.k8s.io/v1", + "cluster": { + "name": "prod-cluster" + }, + "namespace": "default", + "resource": { + "name": "debug", + "type": "pods" + }, + "type": "kubernetes" + }, + "tags": [ + "preserve_original_event" + ] + }, + { + "@timestamp": "2026-08-05T08:00:00.000Z", + "aws": { + "cloudwatch": { + "account_id": "123456789012", + "log_group": "/aws/eks/prod-cluster/cluster", + "log_stream": "kube-apiserver-audit-123", + "region": "us-east-1" + }, + "eks": { + "audit": { + "annotations": { + "authorization_k8s_io/decision": "allow" + }, + "apiVersion": "audit.k8s.io/v1", + "auditID": "status-overrides-allow", + "kind": "Event", + "level": "Metadata", + "objectRef": { + "name": "app", + "namespace": "default", + "resource": "configmaps" + }, + "requestReceivedTimestamp": "2026-08-05T08:00:00Z", + "requestURI": "/api/v1/namespaces/default/configmaps/app", + "responseStatus": { + "code": 403 + }, + "stage": "ResponseComplete", + "stageTimestamp": "2026-08-05T08:00:01Z", + "verb": "get" + }, + "cluster": { + "name": "prod-cluster" + }, + "component": "kube-apiserver", + "log_type": "audit" + } + }, + "cloud": { + "account": { + "id": "123456789012" + }, + "provider": "aws", + "region": "us-east-1" + }, + "ecs": { + "version": "8.11.0" + }, + "event": { + "action": "get", + "category": [ + "configuration" + ], + "dataset": "aws.eks_audit", + "end": "2026-08-05T08:00:01.000Z", + "id": "status-overrides-allow", + "kind": "event", + "original": "{\"apiVersion\":\"audit.k8s.io/v1\",\"kind\":\"Event\",\"level\":\"Metadata\",\"stage\":\"ResponseComplete\",\"requestReceivedTimestamp\":\"2026-08-05T08:00:00Z\",\"stageTimestamp\":\"2026-08-05T08:00:01Z\",\"auditID\":\"status-overrides-allow\",\"verb\":\"get\",\"requestURI\":\"/api/v1/namespaces/default/configmaps/app\",\"objectRef\":{\"resource\":\"configmaps\",\"namespace\":\"default\",\"name\":\"app\"},\"responseStatus\":{\"code\":403},\"annotations\":{\"authorization.k8s.io/decision\":\"allow\"}}", + "outcome": "failure", + "type": [ + "access" + ] + }, + "input": { + "type": "aws-cloudwatch" + }, + "orchestrator": { + "api_version": "audit.k8s.io/v1", + "cluster": { + "name": "prod-cluster" + }, + "namespace": "default", + "resource": { + "name": "app", + "type": "configmaps" + }, + "type": "kubernetes" + }, + "tags": [ + "preserve_original_event" + ] + }, + { + "@timestamp": "2026-08-05T08:00:00.000Z", + "aws": { + "cloudwatch": { + "account_id": "123456789012", + "log_group": "/aws/eks/prod-cluster/cluster", + "log_stream": "kube-apiserver-audit-123", + "region": "us-east-1" + }, + "eks": { + "audit": { + "apiVersion": "audit.k8s.io/v1", + "auditID": "rbac-rules", + "kind": "Event", + "level": "RequestResponse", + "objectRef": { + "apiGroup": "rbac.authorization.k8s.io", + "apiVersion": "v1", + "name": "auditor", + "resource": "clusterroles" + }, + "requestObject": { + "rules": [ + { + "apiGroups": [ + "core", + "apps" + ], + "nonResourceURLs": [ + "/healthz" + ], + "resourceNames": [ + "api" + ], + "resources": [ + "pods", + "deployments" + ], + "verbs": [ + "get", + "list" + ] + } + ] + }, + "requestReceivedTimestamp": "2026-08-05T08:00:00Z", + "requestURI": "/apis/rbac.authorization.k8s.io/v1/clusterroles/auditor", + "responseObject": { + "rules": [ + { + "apiGroups": [ + "core" + ], + "nonResourceURLs": [ + "/healthz" + ], + "resourceNames": [ + "api" + ], + "resources": [ + "pods" + ], + "verbs": [ + "get" + ] + } + ] + }, + "responseStatus": { + "code": 201 + }, + "stage": "ResponseComplete", + "stageTimestamp": "2026-08-05T08:00:01Z", + "verb": "create" + }, + "cluster": { + "name": "prod-cluster" + }, + "component": "kube-apiserver", + "log_type": "audit" + } + }, + "cloud": { + "account": { + "id": "123456789012" + }, + "provider": "aws", + "region": "us-east-1" + }, + "ecs": { + "version": "8.11.0" + }, + "event": { + "action": "create", + "category": [ + "configuration" + ], + "dataset": "aws.eks_audit", + "end": "2026-08-05T08:00:01.000Z", + "id": "rbac-rules", + "kind": "event", + "original": "{\"apiVersion\":\"audit.k8s.io/v1\",\"kind\":\"Event\",\"level\":\"RequestResponse\",\"stage\":\"ResponseComplete\",\"requestReceivedTimestamp\":\"2026-08-05T08:00:00Z\",\"stageTimestamp\":\"2026-08-05T08:00:01Z\",\"auditID\":\"rbac-rules\",\"verb\":\"create\",\"requestURI\":\"/apis/rbac.authorization.k8s.io/v1/clusterroles/auditor\",\"objectRef\":{\"resource\":\"clusterroles\",\"name\":\"auditor\",\"apiGroup\":\"rbac.authorization.k8s.io\",\"apiVersion\":\"v1\"},\"responseStatus\":{\"code\":201},\"requestObject\":{\"rules\":[{\"apiGroups\":[\"\",\"apps\"],\"resources\":[\"pods\",\"deployments\"],\"resourceNames\":[\"api\"],\"verbs\":[\"get\",\"list\"],\"nonResourceURLs\":[\"/healthz\"]}]},\"responseObject\":{\"rules\":[{\"apiGroups\":[\"\"],\"resources\":[\"pods\"],\"resourceNames\":[\"api\"],\"verbs\":[\"get\"],\"nonResourceURLs\":[\"/healthz\"]}]}}", + "outcome": "success", + "type": [ + "creation" + ] + }, + "input": { + "type": "aws-cloudwatch" + }, + "orchestrator": { + "api_version": "v1", + "cluster": { + "name": "prod-cluster" + }, + "resource": { + "name": "auditor", + "type": "clusterroles" + }, + "type": "kubernetes" + }, + "tags": [ + "preserve_original_event" + ] + }, + { + "@timestamp": "2026-08-05T08:00:00.000Z", + "aws": { + "cloudwatch": { + "account_id": "123456789012", + "log_group": "/aws/eks/prod-cluster/cluster", + "log_stream": "kube-apiserver-audit-123", + "region": "us-east-1" + }, + "eks": { + "audit": { + "apiVersion": "audit.k8s.io/v1", + "auditID": "rbac-core-mixed", + "kind": "Event", + "level": "RequestResponse", + "objectRef": { + "apiGroup": "rbac.authorization.k8s.io", + "name": "mixed", + "namespace": "default", + "resource": "roles" + }, + "requestObject": { + "rules": [ + { + "apiGroups": [ + "core", + "apps" + ], + "resources": [ + "pods", + "deployments" + ], + "verbs": [ + "get" + ] + } + ] + }, + "requestReceivedTimestamp": "2026-08-05T08:00:00Z", + "requestURI": "/apis/rbac.authorization.k8s.io/v1/roles/mixed", + "responseStatus": { + "code": 201 + }, + "stage": "ResponseComplete", + "stageTimestamp": "2026-08-05T08:00:01Z", + "verb": "create" + }, + "cluster": { + "name": "prod-cluster" + }, + "component": "kube-apiserver", + "log_type": "audit" + } + }, + "cloud": { + "account": { + "id": "123456789012" + }, + "provider": "aws", + "region": "us-east-1" + }, + "ecs": { + "version": "8.11.0" + }, + "event": { + "action": "create", + "category": [ + "configuration" + ], + "dataset": "aws.eks_audit", + "end": "2026-08-05T08:00:01.000Z", + "id": "rbac-core-mixed", + "kind": "event", + "original": "{\"apiVersion\":\"audit.k8s.io/v1\",\"kind\":\"Event\",\"level\":\"RequestResponse\",\"stage\":\"ResponseComplete\",\"requestReceivedTimestamp\":\"2026-08-05T08:00:00Z\",\"stageTimestamp\":\"2026-08-05T08:00:01Z\",\"auditID\":\"rbac-core-mixed\",\"verb\":\"create\",\"requestURI\":\"/apis/rbac.authorization.k8s.io/v1/roles/mixed\",\"objectRef\":{\"resource\":\"roles\",\"namespace\":\"default\",\"name\":\"mixed\",\"apiGroup\":\"rbac.authorization.k8s.io\"},\"responseStatus\":{\"code\":201},\"requestObject\":{\"rules\":[{\"apiGroups\":[\"\",\"apps\"],\"resources\":[\"pods\",\"deployments\"],\"verbs\":[\"get\"]}]}}", + "outcome": "success", + "type": [ + "creation" + ] + }, + "input": { + "type": "aws-cloudwatch" + }, + "orchestrator": { + "api_version": "audit.k8s.io/v1", + "cluster": { + "name": "prod-cluster" + }, + "namespace": "default", + "resource": { + "name": "mixed", + "type": "roles" + }, + "type": "kubernetes" + }, + "tags": [ + "preserve_original_event" + ] + }, + { + "@timestamp": "2026-08-05T08:00:00.000Z", + "aws": { + "cloudwatch": { + "account_id": "123456789012", + "log_group": "/aws/eks/prod-cluster/cluster", + "log_stream": "kube-apiserver-audit-123", + "region": "us-east-1" + }, + "eks": { + "audit": { + "apiVersion": "audit.k8s.io/v1", + "auditID": "rbac-core-only", + "kind": "Event", + "level": "RequestResponse", + "objectRef": { + "apiGroup": "core", + "name": "core", + "namespace": "default", + "resource": "roles" + }, + "requestReceivedTimestamp": "2026-08-05T08:00:00Z", + "requestURI": "/api/v1/namespaces/default/roles/core", + "responseObject": { + "rules": [ + { + "apiGroups": [ + "core" + ], + "resources": [ + "pods" + ], + "verbs": [ + "get", + "list" + ] + } + ] + }, + "responseStatus": { + "code": 201 + }, + "stage": "ResponseComplete", + "stageTimestamp": "2026-08-05T08:00:01Z", + "verb": "create" + }, + "cluster": { + "name": "prod-cluster" + }, + "component": "kube-apiserver", + "log_type": "audit" + } + }, + "cloud": { + "account": { + "id": "123456789012" + }, + "provider": "aws", + "region": "us-east-1" + }, + "ecs": { + "version": "8.11.0" + }, + "event": { + "action": "create", + "category": [ + "configuration" + ], + "dataset": "aws.eks_audit", + "end": "2026-08-05T08:00:01.000Z", + "id": "rbac-core-only", + "kind": "event", + "original": "{\"apiVersion\":\"audit.k8s.io/v1\",\"kind\":\"Event\",\"level\":\"RequestResponse\",\"stage\":\"ResponseComplete\",\"requestReceivedTimestamp\":\"2026-08-05T08:00:00Z\",\"stageTimestamp\":\"2026-08-05T08:00:01Z\",\"auditID\":\"rbac-core-only\",\"verb\":\"create\",\"requestURI\":\"/api/v1/namespaces/default/roles/core\",\"objectRef\":{\"resource\":\"roles\",\"namespace\":\"default\",\"name\":\"core\",\"apiGroup\":\"\"},\"responseStatus\":{\"code\":201},\"responseObject\":{\"rules\":[{\"apiGroups\":[\"\"],\"resources\":[\"pods\"],\"verbs\":[\"get\",\"list\"]}]}}", + "outcome": "success", + "type": [ + "creation" + ] + }, + "input": { + "type": "aws-cloudwatch" + }, + "orchestrator": { + "api_version": "audit.k8s.io/v1", + "cluster": { + "name": "prod-cluster" + }, + "namespace": "default", + "resource": { + "name": "core", + "type": "roles" + }, + "type": "kubernetes" + }, + "tags": [ + "preserve_original_event" + ] + }, + { + "@timestamp": "2026-08-05T08:00:00.000Z", + "aws": { + "cloudwatch": { + "account_id": "123456789012", + "log_group": "/aws/eks/prod-cluster/cluster", + "log_stream": "kube-apiserver-audit-123", + "region": "us-east-1" + }, + "eks": { + "audit": { + "apiVersion": "audit.k8s.io/v1", + "auditID": "user-extra-dotted-keys", + "impersonatedUser": { + "extra": { + "authentication_kubernetes_io/pod-name": [ + "reader" + ] + }, + "username": "system:serviceaccount:default:reader" + }, + "kind": "Event", + "level": "Metadata", + "objectRef": { + "namespace": "default", + "resource": "pods" + }, + "requestReceivedTimestamp": "2026-08-05T08:00:00Z", + "requestURI": "/api/v1/pods", + "responseStatus": { + "code": 200 + }, + "stage": "ResponseComplete", + "stageTimestamp": "2026-08-05T08:00:01Z", + "user": { + "extra": { + "authentication_kubernetes_io/credential-id": [ + "X509SHA256=test" + ], + "sigs_k8s_io/aws-iam-authenticator/principalId": [ + "AIDATEST" + ] + }, + "groups": [ + "system:nodes" + ], + "username": "system:node:worker" + }, + "verb": "get" + }, + "cluster": { + "name": "prod-cluster" + }, + "component": "kube-apiserver", + "log_type": "audit" + } + }, + "cloud": { + "account": { + "id": "123456789012" + }, + "provider": "aws", + "region": "us-east-1" + }, + "ecs": { + "version": "8.11.0" + }, + "event": { + "action": "get", + "category": [ + "configuration" + ], + "dataset": "aws.eks_audit", + "end": "2026-08-05T08:00:01.000Z", + "id": "user-extra-dotted-keys", + "kind": "event", + "original": "{\"apiVersion\":\"audit.k8s.io/v1\",\"kind\":\"Event\",\"level\":\"Metadata\",\"stage\":\"ResponseComplete\",\"requestReceivedTimestamp\":\"2026-08-05T08:00:00Z\",\"stageTimestamp\":\"2026-08-05T08:00:01Z\",\"auditID\":\"user-extra-dotted-keys\",\"verb\":\"get\",\"requestURI\":\"/api/v1/pods\",\"objectRef\":{\"resource\":\"pods\",\"namespace\":\"default\"},\"responseStatus\":{\"code\":200},\"user\":{\"username\":\"system:node:worker\",\"groups\":[\"system:nodes\"],\"extra\":{\"authentication.kubernetes.io/credential-id\":[\"X509SHA256=test\"],\"sigs.k8s.io/aws-iam-authenticator/principalId\":[\"AIDATEST\"]}},\"impersonatedUser\":{\"username\":\"system:serviceaccount:default:reader\",\"extra\":{\"authentication.kubernetes.io/pod-name\":[\"reader\"]}}}", + "outcome": "success", + "type": [ + "access" + ] + }, + "input": { + "type": "aws-cloudwatch" + }, + "orchestrator": { + "api_version": "audit.k8s.io/v1", + "cluster": { + "name": "prod-cluster" + }, + "namespace": "default", + "resource": { + "type": "pods" + }, + "type": "kubernetes" + }, + "related": { + "user": [ + "system:node:worker" + ] + }, + "tags": [ + "preserve_original_event" + ], + "user": { + "effective": { + "name": "system:serviceaccount:default:reader" + }, + "group": { + "name": [ + "system:nodes" + ] + }, + "name": "system:node:worker" + } + }, + { + "@timestamp": "2026-08-05T08:00:00.000Z", + "aws": { + "cloudwatch": { + "account_id": "123456789012", + "log_group": "/aws/eks/prod-cluster/cluster", + "log_stream": "kube-apiserver-audit-123", + "region": "us-east-1" + }, + "eks": { + "audit": { + "apiVersion": "audit.k8s.io/v1", + "auditID": "crd-mapped-path-collision", + "kind": "Event", + "level": "RequestResponse", + "objectRef": { + "apiGroup": "example.io", + "apiVersion": "v1", + "name": "example", + "resource": "widgets" + }, + "requestReceivedTimestamp": "2026-08-05T08:00:00Z", + "requestURI": "/apis/example.io/v1/widgets/example", + "responseStatus": { + "code": 200 + }, + "stage": "ResponseComplete", + "stageTimestamp": "2026-08-05T08:00:01Z", + "verb": "patch" + }, + "cluster": { + "name": "prod-cluster" + }, + "component": "kube-apiserver", + "log_type": "audit" + } + }, + "cloud": { + "account": { + "id": "123456789012" + }, + "provider": "aws", + "region": "us-east-1" + }, + "ecs": { + "version": "8.11.0" + }, + "event": { + "action": "patch", + "category": [ + "configuration" + ], + "dataset": "aws.eks_audit", + "end": "2026-08-05T08:00:01.000Z", + "id": "crd-mapped-path-collision", + "kind": "event", + "original": "{\"apiVersion\":\"audit.k8s.io/v1\",\"kind\":\"Event\",\"level\":\"RequestResponse\",\"stage\":\"ResponseComplete\",\"requestReceivedTimestamp\":\"2026-08-05T08:00:00Z\",\"stageTimestamp\":\"2026-08-05T08:00:01Z\",\"auditID\":\"crd-mapped-path-collision\",\"verb\":\"patch\",\"requestURI\":\"/apis/example.io/v1/widgets/example\",\"objectRef\":{\"resource\":\"widgets\",\"name\":\"example\",\"apiGroup\":\"example.io\",\"apiVersion\":\"v1\"},\"responseStatus\":{\"code\":200},\"requestObject\":{\"spec\":{\"hostNetwork\":\"enabled\",\"containers\":\"custom\"}},\"responseObject\":{\"roleRef\":\"custom\"}}", + "outcome": "success", + "type": [ + "change" + ] + }, + "input": { + "type": "aws-cloudwatch" + }, + "orchestrator": { + "api_version": "v1", + "cluster": { + "name": "prod-cluster" + }, + "resource": { + "name": "example", + "type": "widgets" + }, + "type": "kubernetes" + }, + "tags": [ + "preserve_original_event" + ] + }, + { + "@timestamp": "2026-08-05T08:00:00.000Z", + "aws": { + "cloudwatch": { + "account_id": "123456789012", + "log_group": "/aws/eks/prod-cluster/cluster", + "log_stream": "kube-apiserver-audit-123", + "region": "us-east-1" + }, + "eks": { + "audit": { + "apiVersion": "audit.k8s.io/v1", + "auditID": "malformed-mapped-shapes", + "kind": "Event", + "level": "RequestResponse", + "requestReceivedTimestamp": "2026-08-05T08:00:00Z", + "requestURI": "/api/v1/pods", + "stage": "ResponseComplete", + "stageTimestamp": "2026-08-05T08:00:01Z", + "verb": "get" + }, + "cluster": { + "name": "prod-cluster" + }, + "component": "kube-apiserver", + "log_type": "audit" + } + }, + "cloud": { + "account": { + "id": "123456789012" + }, + "provider": "aws", + "region": "us-east-1" + }, + "ecs": { + "version": "8.11.0" + }, + "event": { + "action": "get", + "category": [ + "configuration" + ], + "dataset": "aws.eks_audit", + "end": "2026-08-05T08:00:01.000Z", + "id": "malformed-mapped-shapes", + "kind": "event", + "original": "{\"apiVersion\":\"audit.k8s.io/v1\",\"kind\":\"Event\",\"level\":\"RequestResponse\",\"stage\":\"ResponseComplete\",\"requestReceivedTimestamp\":\"2026-08-05T08:00:00Z\",\"stageTimestamp\":\"2026-08-05T08:00:01Z\",\"auditID\":\"malformed-mapped-shapes\",\"verb\":\"get\",\"requestURI\":\"/api/v1/pods\",\"objectRef\":\"invalid\",\"responseStatus\":\"invalid\",\"requestObject\":\"invalid\",\"responseObject\":[\"invalid\"],\"user\":\"invalid\",\"impersonatedUser\":\"invalid\",\"annotations\":[\"invalid\"]}", + "type": [ + "access" + ] + }, + "input": { + "type": "aws-cloudwatch" + }, + "orchestrator": { + "api_version": "audit.k8s.io/v1", + "cluster": { + "name": "prod-cluster" + }, + "type": "kubernetes" + }, + "tags": [ + "preserve_original_event" + ] + }, + { + "@timestamp": "2026-08-05T08:00:00.000Z", + "aws": { + "cloudwatch": { + "account_id": "123456789012", + "log_group": "/aws/eks/prod-cluster/cluster", + "log_stream": "kube-apiserver-audit-123", + "region": "us-east-1" + }, + "eks": { + "audit": { + "annotations": { + "authorization_k8s_io/decision": "allow", + "authorization_k8s_io/reason": "RBAC" + }, + "apiVersion": "audit.k8s.io/v1", + "auditID": "malformed-scalar-shapes", + "kind": "Event", + "objectRef": { + "apiVersion": "v1", + "name": "app-1", + "namespace": "default", + "resource": "pods" + }, + "requestObject": { + "apiVersion": "rbac.authorization.k8s.io/v1", + "kind": "Role", + "rules": [ + { + "verbs": [ + "get" + ] + } + ], + "subjects": [ + { + "name": "attacker" + } + ] + }, + "requestReceivedTimestamp": "2026-08-05T08:00:00Z", + "responseStatus": { + "code": 200 + }, + "stage": "ResponseComplete", + "stageTimestamp": "2026-08-05T08:00:01Z" + }, + "cluster": { + "name": "prod-cluster" + }, + "component": "kube-apiserver", + "log_type": "audit" + } + }, + "cloud": { + "account": { + "id": "123456789012" + }, + "provider": "aws", + "region": "us-east-1" + }, + "ecs": { + "version": "8.11.0" + }, + "event": { + "category": [ + "configuration" + ], + "dataset": "aws.eks_audit", + "end": "2026-08-05T08:00:01.000Z", + "id": "malformed-scalar-shapes", + "kind": "event", + "original": "{\"apiVersion\":\"audit.k8s.io/v1\",\"kind\":\"Event\",\"level\":{\"unexpected\":\"object\"},\"stage\":\"ResponseComplete\",\"requestReceivedTimestamp\":\"2026-08-05T08:00:00Z\",\"stageTimestamp\":\"2026-08-05T08:00:01Z\",\"auditID\":\"malformed-scalar-shapes\",\"verb\":{\"unexpected\":\"object\"},\"requestURI\":{\"unexpected\":\"object\"},\"userAgent\":{\"unexpected\":\"object\"},\"sourceIPs\":{\"unexpected\":\"object\"},\"objectRef\":{\"resource\":\"pods\",\"namespace\":\"default\",\"name\":\"app-1\",\"apiVersion\":\"v1\"},\"responseStatus\":{\"code\":200},\"requestObject\":{\"apiVersion\":\"rbac.authorization.k8s.io/v1\",\"kind\":\"Role\",\"rules\":[{\"apiGroups\":[{\"unexpected\":\"object\"}],\"resources\":[[\"nested\"]],\"verbs\":[\"get\"]}],\"subjects\":[{\"kind\":{\"unexpected\":\"object\"},\"name\":\"attacker\"}]},\"annotations\":{\"authorization.k8s.io/decision\":\"allow\",\"authorization.k8s.io/reason\":\"RBAC\"}}", + "outcome": "success" + }, + "input": { + "type": "aws-cloudwatch" + }, + "orchestrator": { + "api_version": "v1", + "cluster": { + "name": "prod-cluster" + }, + "namespace": "default", + "resource": { + "name": "app-1", + "type": "pods" + }, + "type": "kubernetes" + }, + "tags": [ + "preserve_original_event" + ] + }, + { + "@timestamp": "2026-08-05T08:00:00.000Z", + "aws": { + "cloudwatch": { + "account_id": "123456789012", + "log_group": "/aws/eks/prod-cluster/cluster", + "log_stream": "kube-apiserver-audit-123", + "region": "us-east-1" + }, + "eks": { + "audit": { + "apiVersion": "audit.k8s.io/v1", + "auditID": "subject-access-review", + "kind": "Event", + "level": "RequestResponse", + "objectRef": { + "apiGroup": "authorization.k8s.io", + "apiVersion": "v1", + "resource": "subjectaccessreviews" + }, + "requestObject": { + "apiVersion": "authorization.k8s.io/v1", + "kind": "SubjectAccessReview", + "spec": { + "extra": { + "authentication_kubernetes_io/credential-id": [ + "X509SHA256=test" + ] + }, + "groups": [ + "system:serviceaccounts" + ], + "resourceAttributes": { + "name": "debug", + "namespace": "default", + "resource": "pods", + "subresource": "exec", + "verb": "create", + "version": "v1" + }, + "uid": "reader-uid", + "user": "system:serviceaccount:default:reader" + } + }, + "requestReceivedTimestamp": "2026-08-05T08:00:00Z", + "requestURI": "/apis/authorization.k8s.io/v1/subjectaccessreviews", + "responseObject": { + "apiVersion": "authorization.k8s.io/v1", + "kind": "SubjectAccessReview", + "status": { + "allowed": true, + "denied": false, + "reason": "RBAC allowed" + } + }, + "responseStatus": { + "code": 201 + }, + "stage": "ResponseComplete", + "stageTimestamp": "2026-08-05T08:00:01Z", + "verb": "create" + }, + "cluster": { + "name": "prod-cluster" + }, + "component": "kube-apiserver", + "log_type": "audit" + } + }, + "cloud": { + "account": { + "id": "123456789012" + }, + "provider": "aws", + "region": "us-east-1" + }, + "ecs": { + "version": "8.11.0" + }, + "event": { + "action": "create", + "category": [ + "configuration" + ], + "dataset": "aws.eks_audit", + "end": "2026-08-05T08:00:01.000Z", + "id": "subject-access-review", + "kind": "event", + "original": "{\"apiVersion\":\"audit.k8s.io/v1\",\"kind\":\"Event\",\"level\":\"RequestResponse\",\"stage\":\"ResponseComplete\",\"requestReceivedTimestamp\":\"2026-08-05T08:00:00Z\",\"stageTimestamp\":\"2026-08-05T08:00:01Z\",\"auditID\":\"subject-access-review\",\"verb\":\"create\",\"requestURI\":\"/apis/authorization.k8s.io/v1/subjectaccessreviews\",\"objectRef\":{\"resource\":\"subjectaccessreviews\",\"apiGroup\":\"authorization.k8s.io\",\"apiVersion\":\"v1\"},\"responseStatus\":{\"code\":201},\"requestObject\":{\"apiVersion\":\"authorization.k8s.io/v1\",\"kind\":\"SubjectAccessReview\",\"spec\":{\"user\":\"system:serviceaccount:default:reader\",\"uid\":\"reader-uid\",\"groups\":[\"system:serviceaccounts\"],\"extra\":{\"authentication.kubernetes.io/credential-id\":[\"X509SHA256=test\"]},\"resourceAttributes\":{\"group\":\"\",\"version\":\"v1\",\"resource\":\"pods\",\"subresource\":\"exec\",\"namespace\":\"default\",\"name\":\"debug\",\"verb\":\"create\"}}},\"responseObject\":{\"apiVersion\":\"authorization.k8s.io/v1\",\"kind\":\"SubjectAccessReview\",\"status\":{\"allowed\":true,\"denied\":false,\"reason\":\"RBAC allowed\"}}}", + "outcome": "success", + "type": [ + "creation" + ] + }, + "input": { + "type": "aws-cloudwatch" + }, + "orchestrator": { + "api_version": "v1", + "cluster": { + "name": "prod-cluster" + }, + "resource": { + "type": "subjectaccessreviews" + }, + "type": "kubernetes" + }, + "tags": [ + "preserve_original_event" + ] + } + ] +} diff --git a/packages/aws/data_stream/eks_audit/_dev/test/policy/test-agentless-cloud-connector.expected b/packages/aws/data_stream/eks_audit/_dev/test/policy/test-agentless-cloud-connector.expected new file mode 100644 index 00000000000..5f36236804f --- /dev/null +++ b/packages/aws/data_stream/eks_audit/_dev/test/policy/test-agentless-cloud-connector.expected @@ -0,0 +1,41 @@ +inputs: + - data_stream: + namespace: ep + meta: + package: + agentVersion: ^9.4.0 + name: aws + name: test-agentless-cloud-connector-aws + streams: + - api_sleep: 200ms + api_timeout: 120s + data_stream: + dataset: aws.eks_audit + log_group_arn: arn:aws:logs:us-east-1:123456789012:log-group:/aws/eks/prod-cluster/cluster:* + log_stream_prefix: kube-apiserver-audit + number_of_workers: 1 + publisher_pipeline.disable_host: true + role_arn: arn:aws:iam::123456789012:role/ElasticFederatedIdentity + scan_frequency: 1m + start_position: beginning + tags: + - forwarded + - aws-eks-audit + use_cloud_connectors: true + type: aws-cloudwatch + use_output: default +output_permissions: + default: + _elastic_agent_checks: + cluster: + - monitor + _elastic_agent_monitoring: + indices: [] + uuid-for-permissions-on-related-indices: + indices: + - names: + - logs-aws.eks_audit-ep + privileges: + - auto_configure + - create_doc +secret_references: [] diff --git a/packages/aws/data_stream/eks_audit/_dev/test/policy/test-agentless-cloud-connector.yml b/packages/aws/data_stream/eks_audit/_dev/test/policy/test-agentless-cloud-connector.yml new file mode 100644 index 00000000000..67f93027c55 --- /dev/null +++ b/packages/aws/data_stream/eks_audit/_dev/test/policy/test-agentless-cloud-connector.yml @@ -0,0 +1,6 @@ +vars: + role_arn: arn:aws:iam::123456789012:role/ElasticFederatedIdentity + supports_identity_federation: true +data_stream: + vars: + log_group_arn: arn:aws:logs:us-east-1:123456789012:log-group:/aws/eks/prod-cluster/cluster:* diff --git a/packages/aws/data_stream/eks_audit/_dev/test/policy/test-arn-precedence.expected b/packages/aws/data_stream/eks_audit/_dev/test/policy/test-arn-precedence.expected new file mode 100644 index 00000000000..bda2a7cf08e --- /dev/null +++ b/packages/aws/data_stream/eks_audit/_dev/test/policy/test-arn-precedence.expected @@ -0,0 +1,39 @@ +inputs: + - data_stream: + namespace: ep + meta: + package: + agentVersion: ^9.4.0 + name: aws + name: test-arn-precedence-aws + streams: + - api_sleep: 200ms + api_timeout: 120s + data_stream: + dataset: aws.eks_audit + log_group_arn: arn:aws:logs:us-east-1:123456789012:log-group:/aws/eks/prod-cluster/cluster:* + log_stream_prefix: kube-apiserver-audit + number_of_workers: 1 + publisher_pipeline.disable_host: true + scan_frequency: 1m + start_position: beginning + tags: + - forwarded + - aws-eks-audit + type: aws-cloudwatch + use_output: default +output_permissions: + default: + _elastic_agent_checks: + cluster: + - monitor + _elastic_agent_monitoring: + indices: [] + uuid-for-permissions-on-related-indices: + indices: + - names: + - logs-aws.eks_audit-ep + privileges: + - auto_configure + - create_doc +secret_references: [] diff --git a/packages/aws/data_stream/eks_audit/_dev/test/policy/test-arn-precedence.yml b/packages/aws/data_stream/eks_audit/_dev/test/policy/test-arn-precedence.yml new file mode 100644 index 00000000000..3e09e26b34b --- /dev/null +++ b/packages/aws/data_stream/eks_audit/_dev/test/policy/test-arn-precedence.yml @@ -0,0 +1,7 @@ +data_stream: + vars: + log_group_arn: arn:aws:logs:us-east-1:123456789012:log-group:/aws/eks/prod-cluster/cluster:* + log_group_name: /aws/eks/ignored-name/cluster + log_group_name_prefix: /aws/eks/ignored-prefix/ + include_linked_accounts_with_prefix: true + region_name: us-east-1 diff --git a/packages/aws/data_stream/eks_audit/_dev/test/policy/test-default-prefix.expected b/packages/aws/data_stream/eks_audit/_dev/test/policy/test-default-prefix.expected new file mode 100644 index 00000000000..783e71a8080 --- /dev/null +++ b/packages/aws/data_stream/eks_audit/_dev/test/policy/test-default-prefix.expected @@ -0,0 +1,40 @@ +inputs: + - data_stream: + namespace: ep + meta: + package: + agentVersion: ^9.4.0 + name: aws + name: test-default-prefix-aws + streams: + - api_sleep: 200ms + api_timeout: 120s + data_stream: + dataset: aws.eks_audit + log_group_name_prefix: /aws/eks/ + log_stream_prefix: kube-apiserver-audit + number_of_workers: 1 + publisher_pipeline.disable_host: true + region_name: us-east-1 + scan_frequency: 1m + start_position: beginning + tags: + - forwarded + - aws-eks-audit + type: aws-cloudwatch + use_output: default +output_permissions: + default: + _elastic_agent_checks: + cluster: + - monitor + _elastic_agent_monitoring: + indices: [] + uuid-for-permissions-on-related-indices: + indices: + - names: + - logs-aws.eks_audit-ep + privileges: + - auto_configure + - create_doc +secret_references: [] diff --git a/packages/aws/data_stream/eks_audit/_dev/test/policy/test-default-prefix.yml b/packages/aws/data_stream/eks_audit/_dev/test/policy/test-default-prefix.yml new file mode 100644 index 00000000000..3fdd5545f00 --- /dev/null +++ b/packages/aws/data_stream/eks_audit/_dev/test/policy/test-default-prefix.yml @@ -0,0 +1,3 @@ +data_stream: + vars: + region_name: us-east-1 diff --git a/packages/aws/data_stream/eks_audit/_dev/test/policy/test-name-precedence.expected b/packages/aws/data_stream/eks_audit/_dev/test/policy/test-name-precedence.expected new file mode 100644 index 00000000000..9fb3ddcb95a --- /dev/null +++ b/packages/aws/data_stream/eks_audit/_dev/test/policy/test-name-precedence.expected @@ -0,0 +1,40 @@ +inputs: + - data_stream: + namespace: ep + meta: + package: + agentVersion: ^9.4.0 + name: aws + name: test-name-precedence-aws + streams: + - api_sleep: 200ms + api_timeout: 120s + data_stream: + dataset: aws.eks_audit + log_group_name: /aws/eks/prod-cluster/cluster + log_stream_prefix: kube-apiserver-audit + number_of_workers: 1 + publisher_pipeline.disable_host: true + region_name: us-east-1 + scan_frequency: 1m + start_position: beginning + tags: + - forwarded + - aws-eks-audit + type: aws-cloudwatch + use_output: default +output_permissions: + default: + _elastic_agent_checks: + cluster: + - monitor + _elastic_agent_monitoring: + indices: [] + uuid-for-permissions-on-related-indices: + indices: + - names: + - logs-aws.eks_audit-ep + privileges: + - auto_configure + - create_doc +secret_references: [] diff --git a/packages/aws/data_stream/eks_audit/_dev/test/policy/test-name-precedence.yml b/packages/aws/data_stream/eks_audit/_dev/test/policy/test-name-precedence.yml new file mode 100644 index 00000000000..19474390f42 --- /dev/null +++ b/packages/aws/data_stream/eks_audit/_dev/test/policy/test-name-precedence.yml @@ -0,0 +1,6 @@ +data_stream: + vars: + log_group_name: /aws/eks/prod-cluster/cluster + log_group_name_prefix: /aws/eks/ignored-prefix/ + include_linked_accounts_with_prefix: true + region_name: us-east-1 diff --git a/packages/aws/data_stream/eks_audit/_dev/test/policy/test-shared-profile-assume-role.expected b/packages/aws/data_stream/eks_audit/_dev/test/policy/test-shared-profile-assume-role.expected new file mode 100644 index 00000000000..d19bea6929e --- /dev/null +++ b/packages/aws/data_stream/eks_audit/_dev/test/policy/test-shared-profile-assume-role.expected @@ -0,0 +1,43 @@ +inputs: + - data_stream: + namespace: ep + meta: + package: + agentVersion: ^9.4.0 + name: aws + name: test-shared-profile-assume-role-aws + streams: + - api_sleep: 200ms + api_timeout: 120s + credential_profile_name: elastic-test + data_stream: + dataset: aws.eks_audit + log_group_name: /aws/eks/prod-cluster/cluster + log_stream_prefix: kube-apiserver-audit + number_of_workers: 1 + publisher_pipeline.disable_host: true + region_name: us-east-1 + role_arn: arn:aws:iam::123456789012:role/ElasticCloudWatchReader + scan_frequency: 1m + shared_credential_file: /tmp/fake-aws-credentials + start_position: beginning + tags: + - forwarded + - aws-eks-audit + type: aws-cloudwatch + use_output: default +output_permissions: + default: + _elastic_agent_checks: + cluster: + - monitor + _elastic_agent_monitoring: + indices: [] + uuid-for-permissions-on-related-indices: + indices: + - names: + - logs-aws.eks_audit-ep + privileges: + - auto_configure + - create_doc +secret_references: [] diff --git a/packages/aws/data_stream/eks_audit/_dev/test/policy/test-shared-profile-assume-role.yml b/packages/aws/data_stream/eks_audit/_dev/test/policy/test-shared-profile-assume-role.yml new file mode 100644 index 00000000000..c21e574ade6 --- /dev/null +++ b/packages/aws/data_stream/eks_audit/_dev/test/policy/test-shared-profile-assume-role.yml @@ -0,0 +1,8 @@ +vars: + credential_profile_name: elastic-test + shared_credential_file: /tmp/fake-aws-credentials + role_arn: arn:aws:iam::123456789012:role/ElasticCloudWatchReader +data_stream: + vars: + log_group_name: /aws/eks/prod-cluster/cluster + region_name: us-east-1 diff --git a/packages/aws/data_stream/eks_audit/_dev/test/policy/test-static-credentials.expected b/packages/aws/data_stream/eks_audit/_dev/test/policy/test-static-credentials.expected new file mode 100644 index 00000000000..3830eac301d --- /dev/null +++ b/packages/aws/data_stream/eks_audit/_dev/test/policy/test-static-credentials.expected @@ -0,0 +1,45 @@ +inputs: + - data_stream: + namespace: ep + meta: + package: + agentVersion: ^9.4.0 + name: aws + name: test-static-credentials-aws + streams: + - access_key_id: ${SECRET_0} + api_sleep: 200ms + api_timeout: 120s + data_stream: + dataset: aws.eks_audit + log_group_arn: arn:aws:logs:us-east-1:123456789012:log-group:/aws/eks/prod-cluster/cluster:* + log_stream_prefix: kube-apiserver-audit + number_of_workers: 1 + publisher_pipeline.disable_host: true + scan_frequency: 1m + secret_access_key: ${SECRET_1} + session_token: ${SECRET_2} + start_position: beginning + tags: + - forwarded + - aws-eks-audit + type: aws-cloudwatch + use_output: default +output_permissions: + default: + _elastic_agent_checks: + cluster: + - monitor + _elastic_agent_monitoring: + indices: [] + uuid-for-permissions-on-related-indices: + indices: + - names: + - logs-aws.eks_audit-ep + privileges: + - auto_configure + - create_doc +secret_references: + - {} + - {} + - {} diff --git a/packages/aws/data_stream/eks_audit/_dev/test/policy/test-static-credentials.yml b/packages/aws/data_stream/eks_audit/_dev/test/policy/test-static-credentials.yml new file mode 100644 index 00000000000..bbefffd2e5e --- /dev/null +++ b/packages/aws/data_stream/eks_audit/_dev/test/policy/test-static-credentials.yml @@ -0,0 +1,7 @@ +vars: + access_key_id: FAKE_AWS_ACCESS_KEY_ID_FOR_TESTS_ONLY + secret_access_key: FAKE_AWS_SECRET_ACCESS_KEY_FOR_TESTS_ONLY + session_token: FAKE_AWS_SESSION_TOKEN_FOR_TESTS_ONLY +data_stream: + vars: + log_group_arn: arn:aws:logs:us-east-1:123456789012:log-group:/aws/eks/prod-cluster/cluster:* diff --git a/packages/aws/data_stream/eks_audit/agent/stream/aws-cloudwatch.yml.hbs b/packages/aws/data_stream/eks_audit/agent/stream/aws-cloudwatch.yml.hbs new file mode 100644 index 00000000000..4935eca5473 --- /dev/null +++ b/packages/aws/data_stream/eks_audit/agent/stream/aws-cloudwatch.yml.hbs @@ -0,0 +1,80 @@ +data_stream: + dataset: aws.eks_audit + +{{#if log_group_arn}} +log_group_arn: {{ log_group_arn }} +{{else}} +{{#if log_group_name}} +log_group_name: {{ log_group_name }} +{{else}} +{{#if log_group_name_prefix}} +log_group_name_prefix: {{ log_group_name_prefix }} +{{#if include_linked_accounts_with_prefix}} +include_linked_accounts_for_prefix_mode: {{ include_linked_accounts_with_prefix }} +{{/if}} +{{/if}} +{{/if}} +{{/if}} +{{#unless log_group_arn}} +region_name: {{ region_name }} +{{/unless}} +log_stream_prefix: {{ log_stream_prefix }} +{{#if number_of_workers }} +number_of_workers: {{ number_of_workers }} +{{/if}} +{{#if start_position }} +start_position: {{ start_position }} +{{/if}} +{{#if scan_frequency }} +scan_frequency: {{ scan_frequency }} +{{/if}} +{{#if api_sleep }} +api_sleep: {{ api_sleep }} +{{/if}} +{{#if latency }} +latency: {{ latency }} +{{/if}} +{{#if credential_profile_name}} +credential_profile_name: {{credential_profile_name}} +{{/if}} +{{#if shared_credential_file}} +shared_credential_file: {{shared_credential_file}} +{{/if}} +{{#if api_timeout}} +api_timeout: {{api_timeout}} +{{/if}} +{{#if default_region}} +default_region: {{default_region}} +{{/if}} +{{#if access_key_id}} +access_key_id: {{access_key_id}} +{{/if}} +{{#if secret_access_key}} +secret_access_key: {{secret_access_key}} +{{/if}} +{{#if session_token}} +session_token: {{session_token}} +{{/if}} +{{#if role_arn}} +role_arn: {{role_arn}} +{{/if}} +{{#if supports_identity_federation}} +use_cloud_connectors: {{supports_identity_federation}} +{{/if}} +{{#if proxy_url }} +proxy_url: {{proxy_url}} +{{/if}} +tags: +{{#if preserve_original_event}} + - preserve_original_event +{{/if}} +{{#each tags as |tag|}} + - {{tag}} +{{/each}} +{{#contains "forwarded" tags}} +publisher_pipeline.disable_host: true +{{/contains}} +{{#if processors}} +processors: +{{processors}} +{{/if}} diff --git a/packages/aws/data_stream/eks_audit/elasticsearch/ingest_pipeline/audit.yml b/packages/aws/data_stream/eks_audit/elasticsearch/ingest_pipeline/audit.yml new file mode 100644 index 00000000000..dceae8ff1ca --- /dev/null +++ b/packages/aws/data_stream/eks_audit/elasticsearch/ingest_pipeline/audit.yml @@ -0,0 +1,655 @@ +--- +description: Parse EKS audit JSON into aws.eks.audit and ECS. +on_failure: +- append: + tag: append_pipeline_error + field: error.message + value: >- + Processor '{{{ _ingest.on_failure_processor_type }}}' + {{{#_ingest.on_failure_processor_tag}}}with tag '{{{ _ingest.on_failure_processor_tag }}}' + {{{/_ingest.on_failure_processor_tag}}}in pipeline '{{{ _ingest.pipeline }}}' failed with message '{{{ _ingest.on_failure_message }}}' +- set: + tag: set_pipeline_error + field: event.kind + value: pipeline_error +- append: + tag: preserve_original_event_on_failure + allow_duplicates: false + field: tags + value: preserve_original_event +processors: +- rename: + tag: rename_message_to_event_original + field: message + if: ctx.event?.original == null + ignore_missing: true + target_field: event.original +- json: + tag: parse_event_original + field: event.original + if: ctx.event?.original != null && ctx.aws?.eks?.audit == null + target_field: aws.eks.audit +- remove: + field: aws.eks.audit + if: ctx.aws?.eks?.audit != null && !(ctx.aws.eks.audit instanceof Map) + ignore_missing: true + tag: discard_non_object_audit_payload +- remove: + tag: remove_message + field: message + if: ctx.event?.original != null + ignore_missing: true +- script: + tag: sanitize_mapped_audit_field_shapes + if: ctx.aws?.eks?.audit instanceof Map + lang: painless + source: | + boolean isScalar(def value) { + return value instanceof String || value instanceof Number || value instanceof Boolean; + } + boolean isScalarOrList(def value) { + if (isScalar(value)) { + return true; + } + if (!(value instanceof List)) { + return false; + } + for (def item : value) { + if (!isScalar(item)) { + return false; + } + } + return true; + } + boolean isMapOrListOfMaps(def value) { + if (value instanceof Map) { + return true; + } + if (!(value instanceof List)) { + return false; + } + for (def item : value) { + if (!(item instanceof Map)) { + return false; + } + } + return true; + } + boolean isBoolean(def value) { + return value instanceof Boolean || (value instanceof String && (value == 'true' || value == 'false')); + } + boolean isInteger(def value) { + if (value instanceof Number) { + return true; + } + if (!(value instanceof String)) { + return false; + } + try { + Long.parseLong(value); + return true; + } catch (Exception e) { + return false; + } + } + void keepMap(Map parent, String key) { + if (parent.containsKey(key) && !(parent.get(key) instanceof Map)) { + parent.remove(key); + } + } + void keepScalar(Map parent, String key) { + if (parent.containsKey(key) && !isScalarOrList(parent.get(key))) { + parent.remove(key); + } + } + void keepBoolean(Map parent, String key) { + if (parent.containsKey(key) && !isBoolean(parent.get(key))) { + parent.remove(key); + } + } + void keepInteger(Map parent, String key) { + if (parent.containsKey(key) && !isInteger(parent.get(key))) { + parent.remove(key); + } + } + void keepObjectCollection(Map parent, String key) { + if (parent.containsKey(key) && !isMapOrListOfMaps(parent.get(key))) { + parent.remove(key); + } + } + void sanitizeScalarMap(def value) { + if (!(value instanceof Map)) { + return; + } + List invalidKeys = new ArrayList(); + for (def entry : value.entrySet()) { + if (!isScalarOrList(entry.getValue())) { + invalidKeys.add(entry.getKey()); + } + } + for (def key : invalidKeys) { + value.remove(key); + } + } + void sanitizeSecurityContext(def value) { + if (!(value instanceof Map)) { + return; + } + Map securityContext = value; + keepBoolean(securityContext, 'privileged'); + keepBoolean(securityContext, 'allowPrivilegeEscalation'); + keepBoolean(securityContext, 'runAsNonRoot'); + keepInteger(securityContext, 'runAsUser'); + keepInteger(securityContext, 'runAsGroup'); + keepScalar(securityContext, 'procMount'); + keepMap(securityContext, 'capabilities'); + keepMap(securityContext, 'seccompProfile'); + } + void sanitizeContainers(def value) { + List containers = value instanceof List ? value : [value]; + for (def item : containers) { + if (!(item instanceof Map)) { + continue; + } + Map container = item; + keepScalar(container, 'image'); + keepScalar(container, 'name'); + keepScalar(container, 'command'); + keepObjectCollection(container, 'volumeMounts'); + keepMap(container, 'securityContext'); + sanitizeSecurityContext(container.get('securityContext')); + } + } + void sanitizeSpec(def value) { + if (!(value instanceof Map)) { + return; + } + Map spec = value; + keepScalar(spec, 'type'); + keepScalar(spec, 'serviceAccountName'); + keepScalar(spec, 'restartPolicy'); + keepScalar(spec, 'groups'); + keepScalar(spec, 'uid'); + keepScalar(spec, 'user'); + keepBoolean(spec, 'hostNetwork'); + keepBoolean(spec, 'hostIPC'); + keepBoolean(spec, 'hostPID'); + keepMap(spec, 'extra'); + sanitizeScalarMap(spec.get('extra')); + keepMap(spec, 'resourceAttributes'); + def resourceAttributes = spec.get('resourceAttributes'); + if (resourceAttributes instanceof Map) { + for (String key : ['group', 'namespace', 'name', 'resource', 'subresource', 'verb', 'version']) { + keepScalar(resourceAttributes, key); + } + } + keepMap(spec, 'securityContext'); + sanitizeSecurityContext(spec.get('securityContext')); + keepObjectCollection(spec, 'containers'); + sanitizeContainers(spec.get('containers')); + keepObjectCollection(spec, 'volumes'); + def volumes = spec.get('volumes'); + if (volumes != null) { + List volumeList = volumes instanceof List ? volumes : [volumes]; + for (def item : volumeList) { + if (item instanceof Map) { + keepObjectCollection(item, 'hostPath'); + } + } + } + } + void sanitizeObjectCollection(def value, List keys) { + if (value == null) { + return; + } + List entries = value instanceof List ? value : [value]; + for (def item : entries) { + if (item instanceof Map) { + for (String key : keys) { + keepScalar(item, key); + } + } + } + } + void sanitizeBody(Map body) { + keepScalar(body, 'apiVersion'); + keepScalar(body, 'kind'); + keepMap(body, 'spec'); + sanitizeSpec(body.get('spec')); + keepObjectCollection(body, 'rules'); + sanitizeObjectCollection( + body.get('rules'), + ['apiGroups', 'resources', 'resourceNames', 'verbs', 'nonResourceURLs'] + ); + keepObjectCollection(body, 'subjects'); + sanitizeObjectCollection(body.get('subjects'), ['kind', 'name', 'namespace']); + keepObjectCollection(body, 'items'); + def items = body.get('items'); + if (items != null) { + List itemList = items instanceof List ? items : [items]; + for (def item : itemList) { + if (item instanceof Map) { + keepMap(item, 'metadata'); + sanitizeObjectCollection(item.get('metadata'), ['name']); + } + } + } + keepMap(body, 'roleRef'); + def roleRef = body.get('roleRef'); + if (roleRef instanceof Map) { + keepScalar(roleRef, 'kind'); + keepScalar(roleRef, 'name'); + } + keepMap(body, 'status'); + def status = body.get('status'); + if (status instanceof Map) { + keepBoolean(status, 'allowed'); + keepBoolean(status, 'denied'); + keepScalar(status, 'reason'); + keepScalar(status, 'evaluationError'); + } + } + Map audit = ctx.aws.eks.audit; + // Scalar-mapped top-level fields reject object values outright, which sends the + // whole document to the failure store instead of the curated error path. + for (String key : ['apiVersion', 'kind', 'level', 'auditID', 'stage', 'requestURI', + 'verb', 'sourceIPs', 'userAgent', 'requestReceivedTimestamp', + 'stageTimestamp']) { + keepScalar(audit, key); + } + for (String key : ['user', 'impersonatedUser', 'objectRef', 'responseStatus', 'annotations']) { + keepMap(audit, key); + } + for (String key : ['user', 'impersonatedUser']) { + def identity = audit.get(key); + if (identity instanceof Map) { + keepScalar(identity, 'groups'); + keepScalar(identity, 'username'); + keepScalar(identity, 'uid'); + keepMap(identity, 'extra'); + sanitizeScalarMap(identity.get('extra')); + } + } + sanitizeScalarMap(audit.get('annotations')); + def objectRef = audit.get('objectRef'); + if (objectRef instanceof Map) { + for (String key : ['resource', 'namespace', 'name', 'uid', 'apiGroup', 'apiVersion', 'resourceVersion', 'subresource']) { + keepScalar(objectRef, key); + } + } + def responseStatus = audit.get('responseStatus'); + if (responseStatus instanceof Map) { + keepInteger(responseStatus, 'code'); + keepInteger(responseStatus, 'retryAfterSeconds'); + for (String key : ['status', 'message', 'reason']) { + keepScalar(responseStatus, key); + } + keepMap(responseStatus, 'details'); + def details = responseStatus.get('details'); + if (details instanceof Map) { + for (String key : ['group', 'kind', 'name', 'uid']) { + keepScalar(details, key); + } + keepObjectCollection(details, 'causes'); + def causes = details.get('causes'); + if (causes != null) { + List causeList = causes instanceof List ? causes : [causes]; + for (def cause : causeList) { + if (cause instanceof Map) { + for (String key : ['reason', 'message', 'field']) { + keepScalar(cause, key); + } + } + } + } + } + } + for (String key : ['requestObject', 'responseObject']) { + def body = audit.get(key); + if (body instanceof Map) { + sanitizeBody(body); + } else if (body != null && !isMapOrListOfMaps(body)) { + audit.remove(key); + } + } +- remove: + tag: remove_response_object_metadata + field: + - aws.eks.audit.responseObject.metadata + if: ctx.aws?.eks?.audit?.responseObject != null + ignore_missing: true +- remove: + tag: remove_request_object_metadata + field: + - aws.eks.audit.requestObject.metadata + if: ctx.aws?.eks?.audit?.requestObject != null + ignore_missing: true +- script: + if: ctx.aws?.eks?.audit instanceof Map + lang: painless + source: | + void normalizeApiGroups(def rules) { + if (!(rules instanceof List)) { + return; + } + for (def rule : rules) { + if (!(rule instanceof Map)) { + continue; + } + def groups = rule.get('apiGroups'); + if (!(groups instanceof List)) { + continue; + } + for (int i = 0; i < groups.size(); i++) { + if (groups.get(i) == '') { + groups.set(i, 'core'); + } + } + } + } + Map audit = ctx.aws.eks.audit; + def objectRef = audit.get('objectRef'); + if (objectRef instanceof Map && objectRef.get('apiGroup') == '') { + objectRef.put('apiGroup', 'core'); + } + def requestObject = audit.get('requestObject'); + if (requestObject instanceof Map) { + normalizeApiGroups(requestObject.get('rules')); + } + def responseObject = audit.get('responseObject'); + if (responseObject instanceof Map) { + normalizeApiGroups(responseObject.get('rules')); + } + tag: normalize_core_api_groups +- remove: + field: + - aws.eks.audit.requestObject.data + - aws.eks.audit.requestObject.stringData + - aws.eks.audit.responseObject.data + - aws.eks.audit.responseObject.stringData + if: ctx.aws?.eks?.audit?.objectRef?.resource == 'secrets' + ignore_missing: true + tag: redact_secret_payload_values +- foreach: + tag: redact_secret_list_payload_values + field: aws.eks.audit.responseObject.items + if: ctx.aws?.eks?.audit?.objectRef?.resource == 'secrets' && ctx.aws?.eks?.audit?.responseObject?.items instanceof List + ignore_missing: true + processor: + remove: + field: + - _ingest._value.data + - _ingest._value.stringData + - _ingest._value.metadata.annotations + ignore_missing: true + tag: redact_secret_list_item_payload_values +- set: + tag: set_event_kind + field: event.kind + value: event +- set: + tag: set_event_action + copy_from: aws.eks.audit.verb + field: event.action + if: ctx.aws?.eks?.audit?.verb != null + ignore_empty_value: true +- append: + tag: set_event_category + field: event.category + value: configuration + if: ctx.aws?.eks?.audit?.apiVersion == 'audit.k8s.io/v1' && ctx.aws?.eks?.audit?.kind == 'Event' && ctx.aws?.eks?.audit?.auditID != null + allow_duplicates: false +- append: + tag: set_event_type_creation + field: event.type + value: creation + if: ctx.aws?.eks?.audit?.apiVersion == 'audit.k8s.io/v1' && ctx.aws?.eks?.audit?.kind == 'Event' && ctx.aws?.eks?.audit?.auditID != null && ctx.aws?.eks?.audit?.verb == 'create' && !['exec', 'attach', 'portforward', 'proxy'].contains(ctx.aws?.eks?.audit?.objectRef?.subresource) + allow_duplicates: false +- append: + tag: set_event_type_deletion + field: event.type + value: deletion + if: ctx.aws?.eks?.audit?.apiVersion == 'audit.k8s.io/v1' && ctx.aws?.eks?.audit?.kind == 'Event' && ctx.aws?.eks?.audit?.auditID != null && ['delete', 'deletecollection'].contains(ctx.aws?.eks?.audit?.verb) + allow_duplicates: false +- append: + tag: set_event_type_change + field: event.type + value: change + if: ctx.aws?.eks?.audit?.apiVersion == 'audit.k8s.io/v1' && ctx.aws?.eks?.audit?.kind == 'Event' && ctx.aws?.eks?.audit?.auditID != null && ['update', 'patch'].contains(ctx.aws?.eks?.audit?.verb) + allow_duplicates: false +- append: + tag: set_event_type_access + field: event.type + value: access + if: ctx.aws?.eks?.audit?.apiVersion == 'audit.k8s.io/v1' && ctx.aws?.eks?.audit?.kind == 'Event' && ctx.aws?.eks?.audit?.auditID != null && (['get', 'list', 'watch'].contains(ctx.aws?.eks?.audit?.verb) || ['exec', 'attach', 'portforward', 'proxy'].contains(ctx.aws?.eks?.audit?.objectRef?.subresource)) + allow_duplicates: false +- append: + tag: set_event_type_info + field: event.type + value: info + if: ctx.aws?.eks?.audit?.apiVersion == 'audit.k8s.io/v1' && ctx.aws?.eks?.audit?.kind == 'Event' && ctx.aws?.eks?.audit?.auditID != null && ctx.aws?.eks?.audit?.verb != null && !['create', 'delete', 'deletecollection', 'update', 'patch', 'get', 'list', 'watch'].contains(ctx.aws?.eks?.audit?.verb) && !['exec', 'attach', 'portforward', 'proxy'].contains(ctx.aws?.eks?.audit?.objectRef?.subresource) + allow_duplicates: false +- script: + if: ctx.aws?.eks?.audit?.user?.extra instanceof Map + lang: painless + source: 'Map extra = ctx.aws.eks.audit.user.extra; Map normalized = new HashMap(); for (def entry : extra.entrySet()) { normalized[entry.getKey().replace(''.'', ''_'')] = entry.getValue(); } ctx.aws.eks.audit.user.extra = normalized;' + tag: normalize_user_extra_keys +- script: + if: ctx.aws?.eks?.audit?.impersonatedUser?.extra instanceof Map + lang: painless + source: 'Map extra = ctx.aws.eks.audit.impersonatedUser.extra; Map normalized = new HashMap(); for (def entry : extra.entrySet()) { normalized[entry.getKey().replace(''.'', ''_'')] = entry.getValue(); } ctx.aws.eks.audit.impersonatedUser.extra = normalized;' + tag: normalize_impersonated_user_extra_keys +- script: + if: ctx.aws?.eks?.audit instanceof Map + lang: painless + source: | + Map audit = ctx.aws.eks.audit; + for (String bodyName : ['requestObject', 'responseObject']) { + def body = audit.get(bodyName); + if (!(body instanceof Map)) { + continue; + } + def spec = body.get('spec'); + if (!(spec instanceof Map) || !(spec.get('extra') instanceof Map)) { + continue; + } + Map normalized = new HashMap(); + for (def entry : spec.extra.entrySet()) { + normalized[entry.getKey().replace('.', '_')] = entry.getValue(); + } + spec.extra = normalized; + } + tag: normalize_authorization_review_extra_keys +- script: + tag: normalize_annotation_keys_and_set_outcome + if: ctx.aws?.eks?.audit?.annotations != null + lang: painless + source: 'Map annotations = ctx[''aws''][''eks''][''audit''][''annotations'']; Map updatedAnnotation = new HashMap(); for (String key: annotations.keySet()) { + updatedAnnotation[key.replace(''.'', ''_'')] = annotations[key]; if (key == ''authorization.k8s.io/decision'') { if (annotations[''authorization.k8s.io/decision''] + == ''allow'') { ctx.event.outcome = ''success''; } else if (annotations[''authorization.k8s.io/decision''] == ''forbid'') { ctx.event.outcome = ''failure''; + } } } ctx[''aws''][''eks''][''audit''][''annotations''] = updatedAnnotation' +- set: + tag: set_success_outcome_from_status + field: event.outcome + value: success + if: ctx.aws?.eks?.audit?.responseStatus?.code != null && ctx.aws.eks.audit.responseStatus.code < 400 +- set: + tag: set_failure_outcome_from_status + field: event.outcome + value: failure + if: ctx.aws?.eks?.audit?.responseStatus?.code != null && ctx.aws.eks.audit.responseStatus.code >= 400 +- set: + tag: set_user_name + copy_from: aws.eks.audit.user.username + field: user.name + if: ctx.aws?.eks?.audit?.user?.username != null + ignore_empty_value: true +- set: + tag: set_user_id + copy_from: aws.eks.audit.user.uid + field: user.id + if: ctx.aws?.eks?.audit?.user?.uid != null + ignore_empty_value: true +- set: + tag: set_user_agent_original + copy_from: aws.eks.audit.userAgent + field: user_agent.original + if: ctx.aws?.eks?.audit?.userAgent != null + ignore_empty_value: true +- convert: + tag: convert_source_ips + field: aws.eks.audit.sourceIPs + if: ctx.aws?.eks?.audit?.sourceIPs != null + ignore_failure: true + ignore_missing: true + target_field: source.ip + type: ip +- set: + tag: set_client_ip + copy_from: source.ip + field: client.ip + if: ctx.source?.ip != null + ignore_empty_value: true +- set: + tag: set_orchestrator_type + field: orchestrator.type + value: kubernetes +- set: + tag: set_orchestrator_api_version + copy_from: aws.eks.audit.objectRef.apiVersion + field: orchestrator.api_version + if: ctx.aws?.eks?.audit?.objectRef?.apiVersion != null + ignore_empty_value: true +- set: + tag: set_orchestrator_api_version_from_audit_envelope + copy_from: aws.eks.audit.apiVersion + field: orchestrator.api_version + if: ctx.orchestrator?.api_version == null && ctx.aws?.eks?.audit?.apiVersion != null + ignore_empty_value: true +- set: + tag: set_orchestrator_resource_type + copy_from: aws.eks.audit.objectRef.resource + field: orchestrator.resource.type + if: ctx.aws?.eks?.audit?.objectRef?.resource != null + ignore_empty_value: true +- set: + tag: set_orchestrator_namespace + copy_from: aws.eks.audit.objectRef.namespace + field: orchestrator.namespace + if: ctx.aws?.eks?.audit?.objectRef?.namespace != null + ignore_empty_value: true +- set: + tag: set_orchestrator_resource_name + copy_from: aws.eks.audit.objectRef.name + field: orchestrator.resource.name + if: ctx.aws?.eks?.audit?.objectRef?.name != null + ignore_empty_value: true +- append: + allow_duplicates: false + field: related.user + if: ctx.aws?.eks?.audit?.user?.uid != null + tag: append_related_user_uid + value: '{{{aws.eks.audit.user.uid}}}' +- append: + allow_duplicates: false + field: related.user + if: ctx.aws?.eks?.audit?.user?.username != null + tag: append_related_user_name + value: '{{{aws.eks.audit.user.username}}}' +- foreach: + tag: append_related_source_ips + field: source.ip + if: ctx.source?.ip != null && ctx.source.ip instanceof List + processor: + append: + allow_duplicates: false + field: related.ip + tag: append_related_source_ip + value: '{{{_ingest._value}}}' +- set: + tag: set_event_id + copy_from: aws.eks.audit.auditID + field: event.id + if: ctx.aws?.eks?.audit?.auditID != null + ignore_empty_value: true +- set: + tag: set_user_group_name + copy_from: aws.eks.audit.user.groups + field: user.group.name + if: ctx.aws?.eks?.audit?.user?.groups != null + ignore_empty_value: true +- set: + tag: set_effective_user_name + copy_from: aws.eks.audit.impersonatedUser.username + field: user.effective.name + if: ctx.aws?.eks?.audit?.impersonatedUser?.username != null + ignore_empty_value: true +- set: + tag: set_effective_user_id + copy_from: aws.eks.audit.impersonatedUser.uid + field: user.effective.id + if: ctx.aws?.eks?.audit?.impersonatedUser?.uid != null + ignore_empty_value: true +- set: + tag: set_effective_user_group_name + copy_from: aws.eks.audit.impersonatedUser.groups + field: user.effective.group.name + if: ctx.aws?.eks?.audit?.impersonatedUser?.groups != null + ignore_empty_value: true +- date: + tag: set_timestamp_from_request_received + field: aws.eks.audit.requestReceivedTimestamp + formats: + - ISO8601 + if: ctx.aws?.eks?.audit?.requestReceivedTimestamp != null + target_field: '@timestamp' +- date: + tag: set_event_end_from_stage_timestamp + field: aws.eks.audit.stageTimestamp + formats: + - ISO8601 + if: ctx.aws?.eks?.audit?.stageTimestamp != null + target_field: event.end +- set: + if: ctx.aws?.eks?.audit?.apiVersion != 'audit.k8s.io/v1' || ctx.aws.eks.audit.kind != 'Event' || ctx.aws.eks.audit.auditID == null + field: event.kind + value: pipeline_error + tag: set_pipeline_error_for_unsupported_audit_record +- append: + if: ctx.event?.kind == 'pipeline_error' + field: tags + value: preserve_original_event + allow_duplicates: false + tag: preserve_unsupported_audit_record +- set: + if: ctx.event?.kind == 'pipeline_error' + field: error.message + value: 'Unsupported Amazon EKS audit record: expected a Kubernetes audit.k8s.io Event with an audit ID.' + tag: report_unsupported_audit_record +- script: + description: Drops null/empty values recursively + ignore_failure: true + lang: painless + source: |- + void handleMap(Map map) { + map.values().removeIf(v -> { + if (v instanceof Map) { + handleMap(v); + } else if (v instanceof List) { + handleList(v); + } + return v == null || v == '' || (v instanceof Map && v.size() == 0) || (v instanceof List && v.size() == 0) + }); + } + void handleList(List list) { + list.removeIf(v -> { + if (v instanceof Map) { + handleMap(v); + } else if (v instanceof List) { + handleList(v); + } + return v == null || v == '' || (v instanceof Map && v.size() == 0) || (v instanceof List && v.size() == 0) + }); + } + handleMap(ctx); + tag: script_to_drop_empty_values diff --git a/packages/aws/data_stream/eks_audit/elasticsearch/ingest_pipeline/default.yml b/packages/aws/data_stream/eks_audit/elasticsearch/ingest_pipeline/default.yml new file mode 100644 index 00000000000..e18c05d0fd0 --- /dev/null +++ b/packages/aws/data_stream/eks_audit/elasticsearch/ingest_pipeline/default.yml @@ -0,0 +1,74 @@ +--- +description: Enrich Amazon EKS CloudWatch metadata and parse Kubernetes audit events. +processors: +- set: + tag: set_ecs_version + field: ecs.version + value: 8.11.0 + override: false +- set: + tag: set_event_dataset + field: event.dataset + value: aws.eks_audit +- set: + tag: set_cloud_provider + field: cloud.provider + value: aws +- set: + tag: set_eks_log_type + field: aws.eks.log_type + value: audit +- set: + tag: set_eks_component + field: aws.eks.component + value: kube-apiserver +- set: + tag: set_cloud_region + field: cloud.region + copy_from: aws.cloudwatch.region + if: ctx.aws?.cloudwatch?.region != null + ignore_empty_value: true +- set: + tag: set_cloud_account_id + field: cloud.account.id + copy_from: aws.cloudwatch.account_id + if: ctx.aws?.cloudwatch?.account_id != null + ignore_empty_value: true +- script: + tag: extract_eks_cluster_name + description: Extract the EKS cluster name from the CloudWatch log group. + lang: painless + if: ctx.aws?.cloudwatch?.log_group instanceof String && ctx.aws.cloudwatch.log_group.contains('/aws/eks/') + source: 'String group = ctx.aws.cloudwatch.log_group; String marker = ''/aws/eks/''; int start = group.indexOf(marker) + marker.length(); int end = group.indexOf(''/'', start); if (end < 0) { end = group.length(); + } if (end > start) { if (ctx.aws.eks == null) { ctx.aws.eks = new HashMap(); } ctx.aws.eks.cluster = [''name'': group.substring(start, end)]; }' +- set: + tag: set_orchestrator_cluster_name + field: orchestrator.cluster.name + copy_from: aws.eks.cluster.name + if: ctx.aws?.eks?.cluster?.name != null + ignore_empty_value: true +- pipeline: + tag: route_to_audit_pipeline + name: '{{ IngestPipeline "audit" }}' +- set: + tag: set_timestamp_from_ingest + field: '@timestamp' + value: '{{{_ingest.timestamp}}}' + override: false +on_failure: +- append: + tag: append_pipeline_error + field: error.message + value: >- + Processor '{{{ _ingest.on_failure_processor_type }}}' + {{{#_ingest.on_failure_processor_tag}}}with tag '{{{ _ingest.on_failure_processor_tag }}}' + {{{/_ingest.on_failure_processor_tag}}}in pipeline '{{{ _ingest.pipeline }}}' failed with message '{{{ _ingest.on_failure_message }}}' +- set: + tag: set_pipeline_error + field: event.kind + value: pipeline_error +- append: + tag: preserve_original_event_on_failure + field: tags + value: preserve_original_event + allow_duplicates: false diff --git a/packages/aws/data_stream/eks_audit/fields/base-fields.yml b/packages/aws/data_stream/eks_audit/fields/base-fields.yml new file mode 100644 index 00000000000..556814fb7fc --- /dev/null +++ b/packages/aws/data_stream/eks_audit/fields/base-fields.yml @@ -0,0 +1,23 @@ +- name: data_stream.type + type: constant_keyword + description: Data stream type. +- name: data_stream.dataset + type: constant_keyword + description: Data stream dataset. +- name: data_stream.namespace + type: constant_keyword + description: Data stream namespace. +- name: '@timestamp' + type: date + description: Event timestamp. +- name: event.module + type: constant_keyword + description: Event module + value: aws +- name: event.dataset + type: constant_keyword + description: Event dataset + value: aws.eks_audit +- name: input.type + type: keyword + description: Type of Filebeat input. diff --git a/packages/aws/data_stream/eks_audit/fields/ecs.yml b/packages/aws/data_stream/eks_audit/fields/ecs.yml new file mode 100644 index 00000000000..261b77a9a42 --- /dev/null +++ b/packages/aws/data_stream/eks_audit/fields/ecs.yml @@ -0,0 +1,76 @@ +- external: ecs + name: ecs.version +- external: ecs + name: agent.name +- external: ecs + name: agent.type +- external: ecs + name: agent.id +- external: ecs + name: agent.ephemeral_id +- external: ecs + name: agent.version +- external: ecs + name: error.message +- external: ecs + name: event.action +- external: ecs + name: event.category +- external: ecs + name: event.end +- external: ecs + name: event.id +- external: ecs + name: event.ingested +- external: ecs + name: event.kind +- external: ecs + name: event.original +- external: ecs + name: event.outcome +- external: ecs + name: event.type +- external: ecs + name: message +- external: ecs + name: source.ip +- external: ecs + name: client.ip +- external: ecs + name: user.id +- external: ecs + name: user.name +- external: ecs + name: user.group.name +- external: ecs + name: user.effective.id +- external: ecs + name: user.effective.name +- external: ecs + name: user.effective.group.name +- external: ecs + name: user_agent.original +- external: ecs + name: related.ip +- external: ecs + name: related.user +- external: ecs + name: orchestrator.api_version +- external: ecs + name: orchestrator.cluster.name +- external: ecs + name: orchestrator.namespace +- external: ecs + name: orchestrator.resource.name +- external: ecs + name: orchestrator.resource.type +- external: ecs + name: orchestrator.type +- external: ecs + name: cloud.account.id +- external: ecs + name: cloud.provider +- external: ecs + name: cloud.region +- external: ecs + name: tags diff --git a/packages/aws/data_stream/eks_audit/fields/fields.yml b/packages/aws/data_stream/eks_audit/fields/fields.yml new file mode 100644 index 00000000000..3041e7dcdd9 --- /dev/null +++ b/packages/aws/data_stream/eks_audit/fields/fields.yml @@ -0,0 +1,458 @@ +- name: aws.eks.audit.apiVersion + type: keyword + description: Kubernetes audit API version of the event. +- name: aws.eks.audit.kind + type: keyword + description: Kubernetes object kind of the audit event. +- name: aws.eks.audit.level + type: keyword + description: Audit policy level at which the event was generated. +- name: aws.eks.audit.auditID + type: keyword + description: Unique audit ID generated for the request. +- name: aws.eks.audit.stage + type: keyword + description: Request-handling stage at which this audit event was generated. +- name: aws.eks.audit.requestURI + type: keyword + description: Request URI sent by the client to the Kubernetes API server. +- name: aws.eks.audit.verb + type: keyword + description: Kubernetes verb associated with the request, or the lowercase HTTP method for a non-resource request. +- name: aws.eks.audit.user.groups + type: keyword + description: Groups to which the authenticated user belongs. +- name: aws.eks.audit.user.username + type: keyword + description: Name that identifies the authenticated user. +- name: aws.eks.audit.user.uid + type: keyword + description: Unique identifier of the authenticated user. +- name: aws.eks.audit.user.extra + type: object + object_type: keyword + description: Additional information supplied by the authenticator for the authenticated user. Dots in keys are replaced with underscores. +- name: aws.eks.audit.impersonatedUser.groups + type: keyword + description: Groups to which the impersonated user belongs. +- name: aws.eks.audit.impersonatedUser.username + type: keyword + description: Name that identifies the impersonated user. +- name: aws.eks.audit.impersonatedUser.uid + type: keyword + description: Unique identifier of the impersonated user. +- name: aws.eks.audit.impersonatedUser.extra + type: object + object_type: keyword + description: Additional information supplied for the impersonated user. Dots in keys are replaced with underscores. +- name: aws.eks.audit.sourceIPs + type: keyword + description: Source IP addresses from which the request originated, including intermediate proxies. +- name: aws.eks.audit.userAgent + type: text + description: User agent string reported by the client. This value is client supplied and must not be trusted. +- name: aws.eks.audit.objectRef.resource + type: keyword + description: Kubernetes resource type targeted by the request. +- name: aws.eks.audit.objectRef.namespace + type: keyword + description: Kubernetes namespace of the targeted object. +- name: aws.eks.audit.objectRef.name + type: keyword + description: Name of the targeted Kubernetes object. +- name: aws.eks.audit.objectRef.uid + type: keyword + description: Unique identifier of the targeted Kubernetes object. +- name: aws.eks.audit.objectRef.apiGroup + type: keyword + description: API group that contains the targeted object. The empty Kubernetes core API group is normalized to `core`. +- name: aws.eks.audit.objectRef.apiVersion + type: keyword + description: Version of the API group that contains the targeted object. +- name: aws.eks.audit.objectRef.resourceVersion + type: keyword + description: Resource version of the targeted Kubernetes object. +- name: aws.eks.audit.objectRef.subresource + type: keyword + description: Kubernetes subresource targeted by the request. +- name: aws.eks.audit.responseStatus.code + type: integer + description: Suggested HTTP response code for the status, or 0 when not set. +- name: aws.eks.audit.responseStatus.status + type: keyword + description: Status of the Kubernetes API operation. +- name: aws.eks.audit.responseStatus.message + type: text + description: Human-readable description of the operation status. +- name: aws.eks.audit.responseStatus.reason + type: keyword + description: Machine-readable reason for a failed operation. +- name: aws.eks.audit.responseStatus.retryAfterSeconds + type: integer + description: Number of seconds to wait before retrying the operation. +- name: aws.eks.audit.responseStatus.details.group + type: keyword + description: API group associated with the status details. +- name: aws.eks.audit.responseStatus.details.kind + type: keyword + description: Resource kind associated with the status details. +- name: aws.eks.audit.responseStatus.details.name + type: keyword + description: Resource name associated with the status details. +- name: aws.eks.audit.responseStatus.details.uid + type: keyword + description: Resource identifier associated with the status details. +- name: aws.eks.audit.responseStatus.details.causes + type: group + description: Causes associated with the Kubernetes API status. + fields: + - name: reason + type: keyword + description: Machine-readable reason for the status cause. + - name: message + type: text + description: Human-readable description of the status cause. + - name: field + type: keyword + description: Field associated with the status cause. +- name: aws.eks.audit.requestObject.spec.type + type: keyword + description: Type specified by the Kubernetes API request object. +- name: aws.eks.audit.requestObject.spec.serviceAccountName + type: keyword + description: Service account assigned to the requested workload. +- name: aws.eks.audit.requestObject.spec.restartPolicy + type: keyword + description: Restart policy specified for the requested workload. +- name: aws.eks.audit.requestObject + type: group + dynamic: false + description: Kubernetes API object supplied in the request. Only the security-relevant fields declared below are indexed; the remaining body is retained in `_source` but not dynamically mapped, which keeps the mapping bounded on clusters with many custom resource definitions. + fields: + - name: apiVersion + type: keyword + description: API version of the Kubernetes object supplied in the request. + - name: kind + type: keyword + description: Kind of the Kubernetes object supplied in the request. +- name: aws.eks.audit.responseObject + type: group + dynamic: false + description: Kubernetes API object returned in the response. Only the security-relevant fields declared below are indexed; the remaining body is retained in `_source` but not dynamically mapped, which keeps the mapping bounded on clusters with many custom resource definitions. + fields: + - name: apiVersion + type: keyword + description: API version of the Kubernetes object returned in the response. + - name: kind + type: keyword + description: Kind of the Kubernetes object returned in the response. +- name: aws.eks.audit.requestObject.spec.hostNetwork + type: boolean + description: Whether the requested workload uses the host network namespace. +- name: aws.eks.audit.requestObject.spec.hostIPC + type: boolean + description: Whether the requested workload uses the host IPC namespace. +- name: aws.eks.audit.requestObject.spec.hostPID + type: boolean + description: Whether the requested workload uses the host PID namespace. +- name: aws.eks.audit.requestObject.spec.volumes.hostPath + type: flattened + description: Host path volume definitions in the Kubernetes API request object. +- name: aws.eks.audit.requestObject.spec.securityContext.runAsUser + type: integer + description: User ID configured for processes in the requested workload. +- name: aws.eks.audit.requestObject.spec.securityContext.runAsGroup + type: integer + description: Primary group ID configured for processes in the requested workload. +- name: aws.eks.audit.requestObject.spec.securityContext.runAsNonRoot + type: boolean + description: Whether the requested workload requires processes to run as a non-root user. +- name: aws.eks.audit.requestObject.spec.containers + type: group + description: Containers specified by the Kubernetes API request object. + fields: + - name: image + type: keyword + description: Container image specified by the request. + - name: name + type: keyword + description: Name of the requested container. + - name: command + type: text + description: Entrypoint command specified for the requested container. + - name: volumeMounts + type: flattened + description: Volume mounts specified for the requested container. + - name: securityContext + type: group + description: Security context specified for the requested container. + fields: + - name: privileged + type: boolean + description: Whether the requested container runs in privileged mode. + - name: allowPrivilegeEscalation + type: boolean + description: Whether the requested container allows privilege escalation. + - name: runAsUser + type: integer + description: User ID configured for processes in the requested container. + - name: runAsGroup + type: integer + description: Primary group ID configured for processes in the requested container. + - name: runAsNonRoot + type: boolean + description: Whether the requested container must run as a non-root user. + - name: procMount + type: keyword + description: Proc filesystem mount type configured for the requested container. + - name: capabilities.add + type: keyword + description: Linux capabilities added to the requested container. + - name: seccompProfile.type + type: keyword + description: Seccomp profile type configured for the requested container. +- name: aws.eks.audit.requestObject.rules + type: group + description: Authorization policy rules supplied in the Kubernetes API request object. + fields: + - name: apiGroups + type: keyword + description: Kubernetes API groups to which the rule applies. The empty Kubernetes core API group is normalized to `core`. Because rule objects are non-nested, arrays flatten across rules and predicates on separate rule fields may match different rule objects. + - name: resources + type: keyword + description: Kubernetes resources to which the rule applies. Because rule objects are non-nested, arrays flatten across rules and predicates on separate rule fields may match different rule objects. + - name: resourceNames + type: keyword + description: Kubernetes resource names to which the rule applies. + - name: verbs + type: keyword + description: Kubernetes verbs allowed by the rule. Because rule objects are non-nested, arrays flatten across rules and predicates on separate rule fields may match different rule objects. + - name: nonResourceURLs + type: keyword + description: Non-resource URLs to which the rule applies. +- name: aws.eks.audit.requestObject.roleRef.name + type: keyword + description: Name of the role referenced by the Kubernetes API request object. +- name: aws.eks.audit.requestObject.spec.extra + type: flattened + description: Additional user information supplied in an authorization review request. Dots in keys are replaced with underscores. +- name: aws.eks.audit.requestObject.spec.groups + type: keyword + description: User groups evaluated by an authorization review request. +- name: aws.eks.audit.requestObject.spec.resourceAttributes.group + type: keyword + description: API group evaluated by an authorization review request. +- name: aws.eks.audit.requestObject.spec.resourceAttributes.namespace + type: keyword + description: Namespace evaluated by an authorization review request. +- name: aws.eks.audit.requestObject.spec.resourceAttributes.name + type: keyword + description: Resource name evaluated by an authorization review request. +- name: aws.eks.audit.requestObject.spec.resourceAttributes.resource + type: keyword + description: Resource type evaluated by an authorization review request. +- name: aws.eks.audit.requestObject.spec.resourceAttributes.subresource + type: keyword + description: Subresource evaluated by an authorization review request. +- name: aws.eks.audit.requestObject.spec.resourceAttributes.verb + type: keyword + description: Kubernetes verb evaluated by an authorization review request. +- name: aws.eks.audit.requestObject.spec.resourceAttributes.version + type: keyword + description: API version evaluated by an authorization review request. +- name: aws.eks.audit.requestObject.spec.uid + type: keyword + description: User identifier evaluated by an authorization review request. +- name: aws.eks.audit.requestObject.spec.user + type: keyword + description: User name evaluated by an authorization review request. +- name: aws.eks.audit.requestObject.status.allowed + type: boolean + description: Whether the authorization review request was allowed. +- name: aws.eks.audit.requestObject.status.denied + type: boolean + description: Whether the authorization review request was denied. +- name: aws.eks.audit.requestObject.status.reason + type: text + description: Reason for the authorization review decision. +- name: aws.eks.audit.requestObject.status.evaluationError + type: text + description: Error encountered while evaluating the authorization review request. +- name: aws.eks.audit.responseObject.spec.hostNetwork + type: boolean + description: Whether the returned workload uses the host network namespace. +- name: aws.eks.audit.responseObject.spec.hostIPC + type: boolean + description: Whether the returned workload uses the host IPC namespace. +- name: aws.eks.audit.responseObject.spec.hostPID + type: boolean + description: Whether the returned workload uses the host PID namespace. +- name: aws.eks.audit.responseObject.spec.volumes.hostPath + type: flattened + description: Host path volume definitions in the Kubernetes API response object. +- name: aws.eks.audit.responseObject.spec.restartPolicy + type: keyword + description: Restart policy in the returned workload specification. +- name: aws.eks.audit.responseObject.spec.containers + type: group + description: Containers returned in the Kubernetes API response object. + fields: + - name: volumeMounts + type: flattened + description: Volume mounts returned for the container. + - name: securityContext + type: group + description: Security context returned for the container. + fields: + - name: privileged + type: boolean + description: Whether the returned container runs in privileged mode. + - name: allowPrivilegeEscalation + type: boolean + description: Whether the returned container allows privilege escalation. + - name: runAsUser + type: integer + description: User ID configured for processes in the returned container. +- name: aws.eks.audit.responseObject.rules + type: group + description: Authorization policy rules returned in the Kubernetes API response object. + fields: + - name: apiGroups + type: keyword + description: Kubernetes API groups to which the returned rule applies. The empty Kubernetes core API group is normalized to `core`. Because rule objects are non-nested, arrays flatten across rules and predicates on separate rule fields may match different rule objects. + - name: resources + type: keyword + description: Kubernetes resources to which the returned rule applies. Because rule objects are non-nested, arrays flatten across rules and predicates on separate rule fields may match different rule objects. + - name: resourceNames + type: keyword + description: Kubernetes resource names to which the returned rule applies. + - name: verbs + type: keyword + description: Kubernetes verbs allowed by the returned rule. Because rule objects are non-nested, arrays flatten across rules and predicates on separate rule fields may match different rule objects. + - name: nonResourceURLs + type: keyword + description: Non-resource URLs to which the returned rule applies. +- name: aws.eks.audit.responseObject.items + type: group + description: Objects returned by a Kubernetes list or watch response. Secret `data`, `stringData`, and metadata annotations are removed from every item. + fields: + - name: metadata.name + type: keyword + description: Name of the returned Kubernetes object. +- name: aws.eks.audit.responseObject.roleRef.kind + type: keyword + description: Kind of role referenced by the Kubernetes API response object. +- name: aws.eks.audit.responseObject.roleRef.name + type: keyword + description: Name of the role referenced by the Kubernetes API response object. +- name: aws.eks.audit.responseObject.spec.extra + type: flattened + description: Additional user information returned in an authorization review response. Dots in keys are replaced with underscores. +- name: aws.eks.audit.responseObject.spec.groups + type: keyword + description: User groups evaluated by an authorization review response. +- name: aws.eks.audit.responseObject.spec.resourceAttributes.group + type: keyword + description: API group evaluated by an authorization review response. +- name: aws.eks.audit.responseObject.spec.resourceAttributes.namespace + type: keyword + description: Namespace evaluated by an authorization review response. +- name: aws.eks.audit.responseObject.spec.resourceAttributes.name + type: keyword + description: Resource name evaluated by an authorization review response. +- name: aws.eks.audit.responseObject.spec.resourceAttributes.resource + type: keyword + description: Resource type evaluated by an authorization review response. +- name: aws.eks.audit.responseObject.spec.resourceAttributes.subresource + type: keyword + description: Subresource evaluated by an authorization review response. +- name: aws.eks.audit.responseObject.spec.resourceAttributes.verb + type: keyword + description: Kubernetes verb evaluated by an authorization review response. +- name: aws.eks.audit.responseObject.spec.resourceAttributes.version + type: keyword + description: API version evaluated by an authorization review response. +- name: aws.eks.audit.responseObject.spec.uid + type: keyword + description: User identifier evaluated by an authorization review response. +- name: aws.eks.audit.responseObject.spec.user + type: keyword + description: User name evaluated by an authorization review response. +- name: aws.eks.audit.responseObject.status.allowed + type: boolean + description: Whether the authorization review response allowed the request. +- name: aws.eks.audit.responseObject.status.denied + type: boolean + description: Whether the authorization review response denied the request. +- name: aws.eks.audit.responseObject.status.reason + type: text + description: Reason for the authorization review decision. +- name: aws.eks.audit.responseObject.status.evaluationError + type: text + description: Error encountered while evaluating the authorization review response. +- name: aws.eks.audit.requestReceivedTimestamp + type: date + description: Time when the request reached the Kubernetes API server. +- name: aws.eks.audit.stageTimestamp + type: date + description: Time when the request reached the audit stage recorded by this event. +- name: aws.eks.audit.annotations.authorization_k8s_io/decision + type: keyword + description: Kubernetes authorization decision for the request, such as allow or forbid. +- name: aws.eks.audit.annotations.authorization_k8s_io/reason + type: text + description: Reason reported by the Kubernetes authorizer for its decision. +- name: aws.eks.audit.annotations.pod-security_kubernetes_io/audit-violations + type: text + description: Pod Security admission violations reported in audit mode. +- name: aws.eks.audit.annotations + type: object + object_type: keyword + description: Additional string-valued Kubernetes audit annotations dynamically indexed as keywords, with dots in annotation keys replaced by underscores. +- name: aws.eks.audit.requestObject.subjects + type: group + description: Subjects bound to a role by the Kubernetes API request object. + fields: + - name: kind + type: keyword + description: Kind of subject in the requested role binding. + - name: name + type: keyword + description: Name of the subject in the requested role binding. + - name: namespace + type: keyword + description: Namespace of the subject in the requested role binding. +- name: aws.eks.audit.responseObject.subjects + type: group + description: Subjects bound to a role in the Kubernetes API response object. + fields: + - name: kind + type: keyword + description: Kind of subject in the returned role binding. + - name: name + type: keyword + description: Name of the subject in the returned role binding. + - name: namespace + type: keyword + description: Namespace of the subject in the returned role binding. +- name: aws.eks.cluster.name + type: keyword + description: Amazon EKS cluster name extracted from the CloudWatch log group. +- name: aws.eks.log_type + type: keyword + description: Amazon EKS control-plane log type. +- name: aws.eks.component + type: keyword + description: Amazon EKS control-plane component. +- name: aws.cloudwatch.log_group + type: keyword + description: Name of the CloudWatch Logs log group from which the event was collected. +- name: aws.cloudwatch.log_stream + type: keyword + description: Name of the CloudWatch Logs log stream from which the event was collected. +- name: aws.cloudwatch.region + type: keyword + description: AWS Region containing the CloudWatch Logs log group. +- name: aws.cloudwatch.account_id + type: keyword + description: AWS account ID that owns the CloudWatch Logs log group. diff --git a/packages/aws/data_stream/eks_audit/manifest.yml b/packages/aws/data_stream/eks_audit/manifest.yml new file mode 100644 index 00000000000..4e3d500ecf5 --- /dev/null +++ b/packages/aws/data_stream/eks_audit/manifest.yml @@ -0,0 +1,127 @@ +title: Amazon EKS audit logs +type: logs +elasticsearch: + index_template: + mappings: + dynamic: false +streams: + - input: aws-cloudwatch + template_path: aws-cloudwatch.yml.hbs + title: Amazon EKS audit logs via CloudWatch + description: Collect and parse Amazon EKS Kubernetes API audit logs from CloudWatch Logs. + vars: + - name: log_group_arn + type: text + title: Log Group ARN + multi: false + required: false + show_user: true + description: ARN of a single EKS control-plane log group. When set, this takes precedence over the log group name and prefix. + - name: log_group_name + type: text + title: Log Group Name + multi: false + required: false + show_user: false + description: Name of a single EKS control-plane log group. Used when no ARN is set and takes precedence over the prefix. Region is required. + - name: log_group_name_prefix + type: text + title: Log Group Name Prefix + multi: false + required: false + show_user: true + default: /aws/eks/ + description: Prefix used to discover EKS control-plane log groups when neither an ARN nor a name is set. + - name: include_linked_accounts_with_prefix + type: bool + title: Include Linked Accounts with Prefix + multi: false + required: false + show_user: false + default: false + description: Include matching log groups from linked source accounts when prefix mode is used. + - name: region_name + type: text + title: Region Name + multi: false + required: false + show_user: true + description: AWS Region containing the EKS control-plane log groups. Required when collection uses the log group name or the log group name prefix, including the default /aws/eks/ prefix. Not used when a log group ARN is set, because the ARN already carries the Region. + - name: number_of_workers + type: integer + title: Number of Workers + default: 1 + required: false + show_user: false + description: Number of workers reading matching CloudWatch log groups. + - name: log_stream_prefix + type: text + title: Log Stream Prefix + multi: false + required: true + show_user: true + default: kube-apiserver-audit + description: Collect only kube-apiserver audit streams. Changing this value may send unsupported records to the audit parser. + - name: start_position + type: text + title: Start Position + multi: false + required: false + default: beginning + show_user: true + description: Start at the beginning, end, or lastSync position of each CloudWatch log stream. + - name: scan_frequency + type: text + title: Scan Frequency + multi: false + required: false + show_user: false + default: 1m + description: How often to check CloudWatch for new log events. + - name: api_timeout + type: text + title: API Timeout + multi: false + required: false + show_user: false + default: 120s + description: Maximum duration of a CloudWatch API request. + - name: api_sleep + type: text + title: API Sleep + multi: false + required: false + show_user: false + default: 200ms + description: Delay between FilterLogEvents calls made during one collection period. + - name: latency + type: text + title: Latency + multi: false + required: false + show_user: false + description: CloudWatch Logs publication delay to include in query windows, for example 5m. + - name: tags + type: text + title: Tags + multi: true + required: true + show_user: true + default: + - forwarded + - aws-eks-audit + - name: processors + type: yaml + title: Processors + multi: false + required: false + show_user: false + description: Agent processors that run before the package ingest pipeline. + - name: preserve_original_event + required: true + show_user: true + title: Preserve original event + description: Enabling this option populates event.original with the raw Kubernetes audit JSON of each record. It roughly doubles the storage each audit event needs, so it is disabled by default. + type: bool + multi: false + default: false diff --git a/packages/aws/data_stream/eks_audit/sample_event.json b/packages/aws/data_stream/eks_audit/sample_event.json new file mode 100644 index 00000000000..b4c313ca61d --- /dev/null +++ b/packages/aws/data_stream/eks_audit/sample_event.json @@ -0,0 +1,132 @@ +{ + "@timestamp": "2026-08-05T08:00:00.000Z", + "aws": { + "cloudwatch": { + "log_group": "/aws/eks/prod-cluster/cluster", + "log_stream": "kube-apiserver-audit-123", + "region": "us-east-1", + "account_id": "123456789012" + }, + "eks": { + "cluster": { + "name": "prod-cluster" + }, + "log_type": "audit", + "component": "kube-apiserver", + "audit": { + "apiVersion": "audit.k8s.io/v1", + "kind": "Event", + "level": "RequestResponse", + "stage": "ResponseComplete", + "requestReceivedTimestamp": "2026-08-05T08:00:00Z", + "stageTimestamp": "2026-08-05T08:00:01Z", + "sourceIPs": [ + "198.51.100.10" + ], + "userAgent": "kubectl/v1.31.0", + "user": { + "username": "arn:aws:iam::123456789012:user/alice", + "uid": "aws-iam-authenticator:123456789012:alice", + "groups": [ + "system:authenticated" + ] + }, + "auditID": "allowed-secret", + "verb": "get", + "requestURI": "/api/v1/secrets", + "objectRef": { + "resource": "secrets", + "namespace": "default", + "name": "app-secret" + }, + "responseStatus": { + "code": 200 + }, + "annotations": { + "authorization_k8s_io/decision": "allow", + "authorization_k8s_io/reason": "RBAC decision" + } + } + } + }, + "client": { + "ip": [ + "198.51.100.10" + ] + }, + "cloud": { + "account": { + "id": "123456789012" + }, + "provider": "aws", + "region": "us-east-1" + }, + "data_stream": { + "dataset": "aws.eks_audit", + "namespace": "default", + "type": "logs" + }, + "ecs": { + "version": "8.11.0" + }, + "event": { + "action": "get", + "category": [ + "configuration" + ], + "dataset": "aws.eks_audit", + "end": "2026-08-05T08:00:01.000Z", + "id": "allowed-secret", + "ingested": "2026-08-05T08:00:02.000Z", + "kind": "event", + "outcome": "success", + "type": [ + "access" + ] + }, + "input": { + "type": "aws-cloudwatch" + }, + "orchestrator": { + "api_version": "audit.k8s.io/v1", + "cluster": { + "name": "prod-cluster" + }, + "namespace": "default", + "resource": { + "name": "app-secret", + "type": "secrets" + }, + "type": "kubernetes" + }, + "related": { + "ip": [ + "198.51.100.10" + ], + "user": [ + "aws-iam-authenticator:123456789012:alice", + "arn:aws:iam::123456789012:user/alice" + ] + }, + "source": { + "ip": [ + "198.51.100.10" + ] + }, + "tags": [ + "forwarded", + "aws-eks-audit" + ], + "user": { + "group": { + "name": [ + "system:authenticated" + ] + }, + "id": "aws-iam-authenticator:123456789012:alice", + "name": "arn:aws:iam::123456789012:user/alice" + }, + "user_agent": { + "original": "kubectl/v1.31.0" + } +} diff --git a/packages/aws/docs/README.md b/packages/aws/docs/README.md index c9c503d957e..045128d8802 100644 --- a/packages/aws/docs/README.md +++ b/packages/aws/docs/README.md @@ -247,6 +247,7 @@ This is an overview of the type of data you can collect from each AWS service: | EBS | x | | | EC2 | x | x | | ECS | x | | +| EKS | | x | | ELB | x | x | | Fargate | x | | | Kinesis | x | | diff --git a/packages/aws/docs/eks.md b/packages/aws/docs/eks.md new file mode 100644 index 00000000000..2b52949de23 --- /dev/null +++ b/packages/aws/docs/eks.md @@ -0,0 +1,358 @@ +# Amazon EKS + +The Amazon EKS integration collects Kubernetes API audit events from Amazon CloudWatch Logs into the fixed `aws.eks_audit` dataset. + +Enable EKS control-plane audit logging before starting collection. The default discovery prefix is `/aws/eks/`, and the stream filter defaults to `kube-apiserver-audit`. + +## What data does this integration collect? + +The Amazon EKS integration collects Kubernetes API audit logs from the EKS control plane. + +## What do I need to use this integration? + +The AWS principal used by Elastic Agent needs permission to discover and read the selected CloudWatch log groups, including `logs:DescribeLogGroups` and `logs:FilterLogEvents`. When prefix discovery is used across linked accounts, configure the corresponding CloudWatch cross-account access as well. + +## Setup + +Configure either a log group ARN, a log group name, or the `/aws/eks/` log group prefix. The Region setting is required for name and prefix modes, including the default prefix mode. ARN mode ignores the Region setting because the ARN already identifies the Region. Keep the audit stream prefix unless the EKS stream naming in the target account requires a compatible override. + +Do not enable this data stream and `kubernetes.audit_logs` against the same EKS audit log groups. Duplicate collection creates duplicate audit events and can cause duplicate alerts. + +Kubernetes audit request and response objects are retained in document `_source` and can contain sensitive API payloads. For Secret resources, this integration removes `data` and `stringData` from parsed request and response objects. It also removes `data`, `stringData`, and metadata annotations from every item returned by Secret list/watch responses, while retaining each item's metadata name. The `preserve_original_event` option is disabled by default; enabling it retains the unredacted raw audit JSON in `event.original`, including Secret values removed from parsed fields. Unsupported records also retain `event.original` for troubleshooting. Restrict access to `_source` and enable original-event preservation only when its diagnostic value outweighs the exposure and storage costs. + +Authorization decision, authorization reason, and Pod Security audit-violation annotations have explicit searchable mappings. Other string-valued Kubernetes audit annotations are dynamically indexed as keywords after dots in annotation keys are replaced by underscores. + +`event.outcome` is derived from the HTTP response status when `responseStatus.code` is present: codes below 400 are `success` and codes of 400 or above are `failure`. This takes precedence over the `authorization.k8s.io/decision` annotation, because an authorized request can still fail with a 404, 409, or 5xx response. The annotation is used only when no response status is recorded, such as `RequestReceived` stage events. + +Request and response objects are not dynamically mapped. Only the security-relevant `aws.eks.audit.requestObject.*` and `aws.eks.audit.responseObject.*` fields listed in the field reference are indexed and searchable; the rest of each API object is retained in `_source` but cannot be queried or aggregated. This keeps the field count bounded on clusters that use many custom resource definitions, where dynamically mapping arbitrary object bodies would otherwise exhaust the index field limit and cause indexing failures. To query an additional body field, add it to a `logs-aws.eks_audit@custom` component template. + +## Logs reference + +An example event for `eks_audit` looks as following: + +```json +{ + "@timestamp": "2026-08-05T08:00:00.000Z", + "aws": { + "cloudwatch": { + "log_group": "/aws/eks/prod-cluster/cluster", + "log_stream": "kube-apiserver-audit-123", + "region": "us-east-1", + "account_id": "123456789012" + }, + "eks": { + "cluster": { + "name": "prod-cluster" + }, + "log_type": "audit", + "component": "kube-apiserver", + "audit": { + "apiVersion": "audit.k8s.io/v1", + "kind": "Event", + "level": "RequestResponse", + "stage": "ResponseComplete", + "requestReceivedTimestamp": "2026-08-05T08:00:00Z", + "stageTimestamp": "2026-08-05T08:00:01Z", + "sourceIPs": [ + "198.51.100.10" + ], + "userAgent": "kubectl/v1.31.0", + "user": { + "username": "arn:aws:iam::123456789012:user/alice", + "uid": "aws-iam-authenticator:123456789012:alice", + "groups": [ + "system:authenticated" + ] + }, + "auditID": "allowed-secret", + "verb": "get", + "requestURI": "/api/v1/secrets", + "objectRef": { + "resource": "secrets", + "namespace": "default", + "name": "app-secret" + }, + "responseStatus": { + "code": 200 + }, + "annotations": { + "authorization_k8s_io/decision": "allow", + "authorization_k8s_io/reason": "RBAC decision" + } + } + } + }, + "client": { + "ip": [ + "198.51.100.10" + ] + }, + "cloud": { + "account": { + "id": "123456789012" + }, + "provider": "aws", + "region": "us-east-1" + }, + "data_stream": { + "dataset": "aws.eks_audit", + "namespace": "default", + "type": "logs" + }, + "ecs": { + "version": "8.11.0" + }, + "event": { + "action": "get", + "category": [ + "configuration" + ], + "dataset": "aws.eks_audit", + "end": "2026-08-05T08:00:01.000Z", + "id": "allowed-secret", + "ingested": "2026-08-05T08:00:02.000Z", + "kind": "event", + "outcome": "success", + "type": [ + "access" + ] + }, + "input": { + "type": "aws-cloudwatch" + }, + "orchestrator": { + "api_version": "audit.k8s.io/v1", + "cluster": { + "name": "prod-cluster" + }, + "namespace": "default", + "resource": { + "name": "app-secret", + "type": "secrets" + }, + "type": "kubernetes" + }, + "related": { + "ip": [ + "198.51.100.10" + ], + "user": [ + "aws-iam-authenticator:123456789012:alice", + "arn:aws:iam::123456789012:user/alice" + ] + }, + "source": { + "ip": [ + "198.51.100.10" + ] + }, + "tags": [ + "forwarded", + "aws-eks-audit" + ], + "user": { + "group": { + "name": [ + "system:authenticated" + ] + }, + "id": "aws-iam-authenticator:123456789012:alice", + "name": "arn:aws:iam::123456789012:user/alice" + }, + "user_agent": { + "original": "kubectl/v1.31.0" + } +} +``` + +**ECS Field Reference** + +Refer to the following [document](https://www.elastic.co/guide/en/ecs/current/ecs-field-reference.html) for detailed information on ECS fields. + +**Exported fields** + +| Field | Description | Type | +|---|---|---| +| @timestamp | Event timestamp. | date | +| agent.ephemeral_id | Ephemeral identifier of this agent (if one exists). This id normally changes across restarts, but `agent.id` does not. | keyword | +| agent.id | Unique identifier of this agent (if one exists). Example: For Beats this would be beat.id. | keyword | +| agent.name | Custom name of the agent. This is a name that can be given to an agent. This can be helpful if for example two Filebeat instances are running on the same host but a human readable separation is needed on which Filebeat instance data is coming from. | keyword | +| agent.type | Type of the agent. The agent type always stays the same and should be given by the agent used. In case of Filebeat the agent would always be Filebeat also if two Filebeat instances are run on the same machine. | keyword | +| agent.version | Version of the agent. | keyword | +| aws.cloudwatch.account_id | AWS account ID that owns the CloudWatch Logs log group. | keyword | +| aws.cloudwatch.log_group | Name of the CloudWatch Logs log group from which the event was collected. | keyword | +| aws.cloudwatch.log_stream | Name of the CloudWatch Logs log stream from which the event was collected. | keyword | +| aws.cloudwatch.region | AWS Region containing the CloudWatch Logs log group. | keyword | +| aws.eks.audit.annotations | Additional string-valued Kubernetes audit annotations dynamically indexed as keywords, with dots in annotation keys replaced by underscores. | object | +| aws.eks.audit.annotations.authorization_k8s_io/decision | Kubernetes authorization decision for the request, such as allow or forbid. | keyword | +| aws.eks.audit.annotations.authorization_k8s_io/reason | Reason reported by the Kubernetes authorizer for its decision. | text | +| aws.eks.audit.annotations.pod-security_kubernetes_io/audit-violations | Pod Security admission violations reported in audit mode. | text | +| aws.eks.audit.apiVersion | Kubernetes audit API version of the event. | keyword | +| aws.eks.audit.auditID | Unique audit ID generated for the request. | keyword | +| aws.eks.audit.impersonatedUser.extra | Additional information supplied for the impersonated user. Dots in keys are replaced with underscores. | object | +| aws.eks.audit.impersonatedUser.groups | Groups to which the impersonated user belongs. | keyword | +| aws.eks.audit.impersonatedUser.uid | Unique identifier of the impersonated user. | keyword | +| aws.eks.audit.impersonatedUser.username | Name that identifies the impersonated user. | keyword | +| aws.eks.audit.kind | Kubernetes object kind of the audit event. | keyword | +| aws.eks.audit.level | Audit policy level at which the event was generated. | keyword | +| aws.eks.audit.objectRef.apiGroup | API group that contains the targeted object. The empty Kubernetes core API group is normalized to `core`. | keyword | +| aws.eks.audit.objectRef.apiVersion | Version of the API group that contains the targeted object. | keyword | +| aws.eks.audit.objectRef.name | Name of the targeted Kubernetes object. | keyword | +| aws.eks.audit.objectRef.namespace | Kubernetes namespace of the targeted object. | keyword | +| aws.eks.audit.objectRef.resource | Kubernetes resource type targeted by the request. | keyword | +| aws.eks.audit.objectRef.resourceVersion | Resource version of the targeted Kubernetes object. | keyword | +| aws.eks.audit.objectRef.subresource | Kubernetes subresource targeted by the request. | keyword | +| aws.eks.audit.objectRef.uid | Unique identifier of the targeted Kubernetes object. | keyword | +| aws.eks.audit.requestObject.apiVersion | API version of the Kubernetes object supplied in the request. | keyword | +| aws.eks.audit.requestObject.kind | Kind of the Kubernetes object supplied in the request. | keyword | +| aws.eks.audit.requestObject.roleRef.name | Name of the role referenced by the Kubernetes API request object. | keyword | +| aws.eks.audit.requestObject.rules.apiGroups | Kubernetes API groups to which the rule applies. The empty Kubernetes core API group is normalized to `core`. Because rule objects are non-nested, arrays flatten across rules and predicates on separate rule fields may match different rule objects. | keyword | +| aws.eks.audit.requestObject.rules.nonResourceURLs | Non-resource URLs to which the rule applies. | keyword | +| aws.eks.audit.requestObject.rules.resourceNames | Kubernetes resource names to which the rule applies. | keyword | +| aws.eks.audit.requestObject.rules.resources | Kubernetes resources to which the rule applies. Because rule objects are non-nested, arrays flatten across rules and predicates on separate rule fields may match different rule objects. | keyword | +| aws.eks.audit.requestObject.rules.verbs | Kubernetes verbs allowed by the rule. Because rule objects are non-nested, arrays flatten across rules and predicates on separate rule fields may match different rule objects. | keyword | +| aws.eks.audit.requestObject.spec.containers.command | Entrypoint command specified for the requested container. | text | +| aws.eks.audit.requestObject.spec.containers.image | Container image specified by the request. | keyword | +| aws.eks.audit.requestObject.spec.containers.name | Name of the requested container. | keyword | +| aws.eks.audit.requestObject.spec.containers.securityContext.allowPrivilegeEscalation | Whether the requested container allows privilege escalation. | boolean | +| aws.eks.audit.requestObject.spec.containers.securityContext.capabilities.add | Linux capabilities added to the requested container. | keyword | +| aws.eks.audit.requestObject.spec.containers.securityContext.privileged | Whether the requested container runs in privileged mode. | boolean | +| aws.eks.audit.requestObject.spec.containers.securityContext.procMount | Proc filesystem mount type configured for the requested container. | keyword | +| aws.eks.audit.requestObject.spec.containers.securityContext.runAsGroup | Primary group ID configured for processes in the requested container. | integer | +| aws.eks.audit.requestObject.spec.containers.securityContext.runAsNonRoot | Whether the requested container must run as a non-root user. | boolean | +| aws.eks.audit.requestObject.spec.containers.securityContext.runAsUser | User ID configured for processes in the requested container. | integer | +| aws.eks.audit.requestObject.spec.containers.securityContext.seccompProfile.type | Seccomp profile type configured for the requested container. | keyword | +| aws.eks.audit.requestObject.spec.containers.volumeMounts | Volume mounts specified for the requested container. | flattened | +| aws.eks.audit.requestObject.spec.extra | Additional user information supplied in an authorization review request. Dots in keys are replaced with underscores. | flattened | +| aws.eks.audit.requestObject.spec.groups | User groups evaluated by an authorization review request. | keyword | +| aws.eks.audit.requestObject.spec.hostIPC | Whether the requested workload uses the host IPC namespace. | boolean | +| aws.eks.audit.requestObject.spec.hostNetwork | Whether the requested workload uses the host network namespace. | boolean | +| aws.eks.audit.requestObject.spec.hostPID | Whether the requested workload uses the host PID namespace. | boolean | +| aws.eks.audit.requestObject.spec.resourceAttributes.group | API group evaluated by an authorization review request. | keyword | +| aws.eks.audit.requestObject.spec.resourceAttributes.name | Resource name evaluated by an authorization review request. | keyword | +| aws.eks.audit.requestObject.spec.resourceAttributes.namespace | Namespace evaluated by an authorization review request. | keyword | +| aws.eks.audit.requestObject.spec.resourceAttributes.resource | Resource type evaluated by an authorization review request. | keyword | +| aws.eks.audit.requestObject.spec.resourceAttributes.subresource | Subresource evaluated by an authorization review request. | keyword | +| aws.eks.audit.requestObject.spec.resourceAttributes.verb | Kubernetes verb evaluated by an authorization review request. | keyword | +| aws.eks.audit.requestObject.spec.resourceAttributes.version | API version evaluated by an authorization review request. | keyword | +| aws.eks.audit.requestObject.spec.restartPolicy | Restart policy specified for the requested workload. | keyword | +| aws.eks.audit.requestObject.spec.securityContext.runAsGroup | Primary group ID configured for processes in the requested workload. | integer | +| aws.eks.audit.requestObject.spec.securityContext.runAsNonRoot | Whether the requested workload requires processes to run as a non-root user. | boolean | +| aws.eks.audit.requestObject.spec.securityContext.runAsUser | User ID configured for processes in the requested workload. | integer | +| aws.eks.audit.requestObject.spec.serviceAccountName | Service account assigned to the requested workload. | keyword | +| aws.eks.audit.requestObject.spec.type | Type specified by the Kubernetes API request object. | keyword | +| aws.eks.audit.requestObject.spec.uid | User identifier evaluated by an authorization review request. | keyword | +| aws.eks.audit.requestObject.spec.user | User name evaluated by an authorization review request. | keyword | +| aws.eks.audit.requestObject.spec.volumes.hostPath | Host path volume definitions in the Kubernetes API request object. | flattened | +| aws.eks.audit.requestObject.status.allowed | Whether the authorization review request was allowed. | boolean | +| aws.eks.audit.requestObject.status.denied | Whether the authorization review request was denied. | boolean | +| aws.eks.audit.requestObject.status.evaluationError | Error encountered while evaluating the authorization review request. | text | +| aws.eks.audit.requestObject.status.reason | Reason for the authorization review decision. | text | +| aws.eks.audit.requestObject.subjects.kind | Kind of subject in the requested role binding. | keyword | +| aws.eks.audit.requestObject.subjects.name | Name of the subject in the requested role binding. | keyword | +| aws.eks.audit.requestObject.subjects.namespace | Namespace of the subject in the requested role binding. | keyword | +| aws.eks.audit.requestReceivedTimestamp | Time when the request reached the Kubernetes API server. | date | +| aws.eks.audit.requestURI | Request URI sent by the client to the Kubernetes API server. | keyword | +| aws.eks.audit.responseObject.apiVersion | API version of the Kubernetes object returned in the response. | keyword | +| aws.eks.audit.responseObject.items.metadata.name | Name of the returned Kubernetes object. | keyword | +| aws.eks.audit.responseObject.kind | Kind of the Kubernetes object returned in the response. | keyword | +| aws.eks.audit.responseObject.roleRef.kind | Kind of role referenced by the Kubernetes API response object. | keyword | +| aws.eks.audit.responseObject.roleRef.name | Name of the role referenced by the Kubernetes API response object. | keyword | +| aws.eks.audit.responseObject.rules.apiGroups | Kubernetes API groups to which the returned rule applies. The empty Kubernetes core API group is normalized to `core`. Because rule objects are non-nested, arrays flatten across rules and predicates on separate rule fields may match different rule objects. | keyword | +| aws.eks.audit.responseObject.rules.nonResourceURLs | Non-resource URLs to which the returned rule applies. | keyword | +| aws.eks.audit.responseObject.rules.resourceNames | Kubernetes resource names to which the returned rule applies. | keyword | +| aws.eks.audit.responseObject.rules.resources | Kubernetes resources to which the returned rule applies. Because rule objects are non-nested, arrays flatten across rules and predicates on separate rule fields may match different rule objects. | keyword | +| aws.eks.audit.responseObject.rules.verbs | Kubernetes verbs allowed by the returned rule. Because rule objects are non-nested, arrays flatten across rules and predicates on separate rule fields may match different rule objects. | keyword | +| aws.eks.audit.responseObject.spec.containers.securityContext.allowPrivilegeEscalation | Whether the returned container allows privilege escalation. | boolean | +| aws.eks.audit.responseObject.spec.containers.securityContext.privileged | Whether the returned container runs in privileged mode. | boolean | +| aws.eks.audit.responseObject.spec.containers.securityContext.runAsUser | User ID configured for processes in the returned container. | integer | +| aws.eks.audit.responseObject.spec.containers.volumeMounts | Volume mounts returned for the container. | flattened | +| aws.eks.audit.responseObject.spec.extra | Additional user information returned in an authorization review response. Dots in keys are replaced with underscores. | flattened | +| aws.eks.audit.responseObject.spec.groups | User groups evaluated by an authorization review response. | keyword | +| aws.eks.audit.responseObject.spec.hostIPC | Whether the returned workload uses the host IPC namespace. | boolean | +| aws.eks.audit.responseObject.spec.hostNetwork | Whether the returned workload uses the host network namespace. | boolean | +| aws.eks.audit.responseObject.spec.hostPID | Whether the returned workload uses the host PID namespace. | boolean | +| aws.eks.audit.responseObject.spec.resourceAttributes.group | API group evaluated by an authorization review response. | keyword | +| aws.eks.audit.responseObject.spec.resourceAttributes.name | Resource name evaluated by an authorization review response. | keyword | +| aws.eks.audit.responseObject.spec.resourceAttributes.namespace | Namespace evaluated by an authorization review response. | keyword | +| aws.eks.audit.responseObject.spec.resourceAttributes.resource | Resource type evaluated by an authorization review response. | keyword | +| aws.eks.audit.responseObject.spec.resourceAttributes.subresource | Subresource evaluated by an authorization review response. | keyword | +| aws.eks.audit.responseObject.spec.resourceAttributes.verb | Kubernetes verb evaluated by an authorization review response. | keyword | +| aws.eks.audit.responseObject.spec.resourceAttributes.version | API version evaluated by an authorization review response. | keyword | +| aws.eks.audit.responseObject.spec.restartPolicy | Restart policy in the returned workload specification. | keyword | +| aws.eks.audit.responseObject.spec.uid | User identifier evaluated by an authorization review response. | keyword | +| aws.eks.audit.responseObject.spec.user | User name evaluated by an authorization review response. | keyword | +| aws.eks.audit.responseObject.spec.volumes.hostPath | Host path volume definitions in the Kubernetes API response object. | flattened | +| aws.eks.audit.responseObject.status.allowed | Whether the authorization review response allowed the request. | boolean | +| aws.eks.audit.responseObject.status.denied | Whether the authorization review response denied the request. | boolean | +| aws.eks.audit.responseObject.status.evaluationError | Error encountered while evaluating the authorization review response. | text | +| aws.eks.audit.responseObject.status.reason | Reason for the authorization review decision. | text | +| aws.eks.audit.responseObject.subjects.kind | Kind of subject in the returned role binding. | keyword | +| aws.eks.audit.responseObject.subjects.name | Name of the subject in the returned role binding. | keyword | +| aws.eks.audit.responseObject.subjects.namespace | Namespace of the subject in the returned role binding. | keyword | +| aws.eks.audit.responseStatus.code | Suggested HTTP response code for the status, or 0 when not set. | integer | +| aws.eks.audit.responseStatus.details.causes.field | Field associated with the status cause. | keyword | +| aws.eks.audit.responseStatus.details.causes.message | Human-readable description of the status cause. | text | +| aws.eks.audit.responseStatus.details.causes.reason | Machine-readable reason for the status cause. | keyword | +| aws.eks.audit.responseStatus.details.group | API group associated with the status details. | keyword | +| aws.eks.audit.responseStatus.details.kind | Resource kind associated with the status details. | keyword | +| aws.eks.audit.responseStatus.details.name | Resource name associated with the status details. | keyword | +| aws.eks.audit.responseStatus.details.uid | Resource identifier associated with the status details. | keyword | +| aws.eks.audit.responseStatus.message | Human-readable description of the operation status. | text | +| aws.eks.audit.responseStatus.reason | Machine-readable reason for a failed operation. | keyword | +| aws.eks.audit.responseStatus.retryAfterSeconds | Number of seconds to wait before retrying the operation. | integer | +| aws.eks.audit.responseStatus.status | Status of the Kubernetes API operation. | keyword | +| aws.eks.audit.sourceIPs | Source IP addresses from which the request originated, including intermediate proxies. | keyword | +| aws.eks.audit.stage | Request-handling stage at which this audit event was generated. | keyword | +| aws.eks.audit.stageTimestamp | Time when the request reached the audit stage recorded by this event. | date | +| aws.eks.audit.user.extra | Additional information supplied by the authenticator for the authenticated user. Dots in keys are replaced with underscores. | object | +| aws.eks.audit.user.groups | Groups to which the authenticated user belongs. | keyword | +| aws.eks.audit.user.uid | Unique identifier of the authenticated user. | keyword | +| aws.eks.audit.user.username | Name that identifies the authenticated user. | keyword | +| aws.eks.audit.userAgent | User agent string reported by the client. This value is client supplied and must not be trusted. | text | +| aws.eks.audit.verb | Kubernetes verb associated with the request, or the lowercase HTTP method for a non-resource request. | keyword | +| aws.eks.cluster.name | Amazon EKS cluster name extracted from the CloudWatch log group. | keyword | +| aws.eks.component | Amazon EKS control-plane component. | keyword | +| aws.eks.log_type | Amazon EKS control-plane log type. | keyword | +| client.ip | IP address of the client (IPv4 or IPv6). | ip | +| cloud.account.id | The cloud account or organization id used to identify different entities in a multi-tenant environment. Examples: AWS account id, Google Cloud ORG Id, or other unique identifier. | keyword | +| cloud.provider | Name of the cloud provider. Example values are aws, azure, gcp, or digitalocean. | keyword | +| cloud.region | Region in which this host, resource, or service is located. | keyword | +| data_stream.dataset | Data stream dataset. | constant_keyword | +| data_stream.namespace | Data stream namespace. | constant_keyword | +| data_stream.type | Data stream type. | constant_keyword | +| ecs.version | ECS version this event conforms to. `ecs.version` is a required field and must exist in all events. When querying across multiple indices -- which may conform to slightly different ECS versions -- this field lets integrations adjust to the schema version of the events. | keyword | +| error.message | Error message. | match_only_text | +| event.action | The action captured by the event. This describes the information in the event. It is more specific than `event.category`. Examples are `group-add`, `process-started`, `file-created`. The value is normally defined by the implementer. | keyword | +| event.category | This is one of four ECS Categorization Fields, and indicates the second level in the ECS category hierarchy. `event.category` represents the "big buckets" of ECS categories. For example, filtering on `event.category:process` yields all events relating to process activity. This field is closely related to `event.type`, which is used as a subcategory. This field is an array. This will allow proper categorization of some events that fall in multiple categories. | keyword | +| event.dataset | Event dataset | constant_keyword | +| event.end | `event.end` contains the date when the event ended or when the activity was last observed. | date | +| event.id | Unique ID to describe the event. | keyword | +| event.ingested | Timestamp when an event arrived in the central data store. This is different from `@timestamp`, which is when the event originally occurred. It's also different from `event.created`, which is meant to capture the first time an agent saw the event. In normal conditions, assuming no tampering, the timestamps should chronologically look like this: `@timestamp` \< `event.created` \< `event.ingested`. | date | +| event.kind | This is one of four ECS Categorization Fields, and indicates the highest level in the ECS category hierarchy. `event.kind` gives high-level information about what type of information the event contains, without being specific to the contents of the event. For example, values of this field distinguish alert events from metric events. The value of this field can be used to inform how these kinds of events should be handled. They may warrant different retention, different access control, it may also help understand whether the data is coming in at a regular interval or not. | keyword | +| event.module | Event module | constant_keyword | +| event.original | Raw text message of entire event. Used to demonstrate log integrity or where the full log message (before splitting it up in multiple parts) may be required, e.g. for reindex. This field is not indexed and doc_values are disabled. It cannot be searched, but it can be retrieved from `_source`. If users wish to override this and index this field, please see `Field data types` in the `Elasticsearch Reference`. | keyword | +| event.outcome | This is one of four ECS Categorization Fields, and indicates the lowest level in the ECS category hierarchy. `event.outcome` simply denotes whether the event represents a success or a failure from the perspective of the entity that produced the event. Note that when a single transaction is described in multiple events, each event may populate different values of `event.outcome`, according to their perspective. Also note that in the case of a compound event (a single event that contains multiple logical events), this field should be populated with the value that best captures the overall success or failure from the perspective of the event producer. Further note that not all events will have an associated outcome. For example, this field is generally not populated for metric events, events with `event.type:info`, or any events for which an outcome does not make logical sense. | keyword | +| event.type | This is one of four ECS Categorization Fields, and indicates the third level in the ECS category hierarchy. `event.type` represents a categorization "sub-bucket" that, when used along with the `event.category` field values, enables filtering events down to a level appropriate for single visualization. This field is an array. This will allow proper categorization of some events that fall in multiple event types. | keyword | +| input.type | Type of Filebeat input. | keyword | +| message | For log events the message field contains the log message, optimized for viewing in a log viewer. For structured logs without an original message field, other fields can be concatenated to form a human-readable summary of the event. If multiple messages exist, they can be combined into one message. | match_only_text | +| orchestrator.api_version | API version being used to carry out the action | keyword | +| orchestrator.cluster.name | Name of the cluster. | keyword | +| orchestrator.namespace | Namespace in which the action is taking place. | keyword | +| orchestrator.resource.name | Name of the resource being acted upon. | keyword | +| orchestrator.resource.type | Type of resource being acted upon. | keyword | +| orchestrator.type | Orchestrator cluster type (e.g. kubernetes, nomad or cloudfoundry). | keyword | +| related.ip | All of the IPs seen on your event. | ip | +| related.user | All the user names or other user identifiers seen on the event. | keyword | +| source.ip | IP address of the source (IPv4 or IPv6). | ip | +| tags | List of keywords used to tag each event. | keyword | +| user.effective.group.name | Name of the group. | keyword | +| user.effective.id | Unique identifier of the user. | keyword | +| user.effective.name | Short name or login of the user. | keyword | +| user.effective.name.text | Multi-field of `user.effective.name`. | match_only_text | +| user.group.name | Name of the group. | keyword | +| user.id | Unique identifier of the user. | keyword | +| user.name | Short name or login of the user. | keyword | +| user.name.text | Multi-field of `user.name`. | match_only_text | +| user_agent.original | Unparsed user_agent string. | keyword | +| user_agent.original.text | Multi-field of `user_agent.original`. | match_only_text | diff --git a/packages/aws/manifest.yml b/packages/aws/manifest.yml index 47296ff079e..0ffa1bacf8a 100644 --- a/packages/aws/manifest.yml +++ b/packages/aws/manifest.yml @@ -1,7 +1,7 @@ format_version: 3.6.4 name: aws title: AWS -version: 8.1.2 +version: 8.2.0 description: Collect logs and metrics from Amazon Web Services (AWS) with Elastic Agent. type: integration categories: @@ -427,6 +427,33 @@ policy_templates: title: AWS ECS logo size: 32x32 type: image/svg+xml + - name: eks + title: Amazon EKS + description: Collect Amazon EKS Kubernetes API audit logs from CloudWatch with Elastic Agent. + data_streams: + - eks_audit + categories: + - security + - containers + inputs: + - type: aws-cloudwatch + title: Collect Amazon EKS audit logs from CloudWatch + description: Collect EKS kube-apiserver audit logs using the aws-cloudwatch input. + input_group: logs + deployment_modes: + default: + enabled: true + agentless: + enabled: true + release: beta + organization: security + division: engineering + team: security-service-integrations + icons: + - src: /img/logo_aws.svg + title: AWS logo + size: 32x32 + type: image/svg+xml - name: elb title: AWS ELB description: Collect logs and metrics for Amazon Elastic Load Balancing service with Elastic Agent