Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/CODEOWNERS
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@
/packages/aws/data_stream/ec2_logs @elastic/obs-ds-hosted-services
/packages/aws/data_stream/ec2_metrics @elastic/obs-ds-hosted-services
/packages/aws/data_stream/ecs_metrics @elastic/obs-ds-hosted-services
/packages/aws/data_stream/eks_audit @elastic/security-service-integrations
/packages/aws/data_stream/elb_logs @elastic/obs-infraobs-integrations
/packages/aws/data_stream/elb_metrics @elastic/obs-infraobs-integrations
/packages/aws/data_stream/emr_logs @elastic/obs-infraobs-integrations
Expand Down
1 change: 1 addition & 0 deletions packages/aws/_dev/build/docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -247,6 +247,7 @@ This is an overview of the type of data you can collect from each AWS service:
| EBS | x | |
| EC2 | x | x |
| ECS | x | |
| EKS | | x |
| ELB | x | x |
| Fargate | x | |
| Kinesis | x | |
Expand Down
37 changes: 37 additions & 0 deletions packages/aws/_dev/build/docs/eks.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# Amazon EKS

The Amazon EKS integration collects Kubernetes API audit events from Amazon CloudWatch Logs into the fixed `aws.eks_audit` dataset.

Enable EKS control-plane audit logging before starting collection. The default discovery prefix is `/aws/eks/`, and the stream filter defaults to `kube-apiserver-audit`.

## What data does this integration collect?

The Amazon EKS integration collects Kubernetes API audit logs from the EKS control plane.

## What do I need to use this integration?

The AWS principal used by Elastic Agent needs permission to discover and read the selected CloudWatch log groups, including `logs:DescribeLogGroups` and `logs:FilterLogEvents`. When prefix discovery is used across linked accounts, configure the corresponding CloudWatch cross-account access as well.

## Setup

Configure either a log group ARN, a log group name, or the `/aws/eks/` log group prefix. The Region setting is required for name and prefix modes, including the default prefix mode. ARN mode ignores the Region setting because the ARN already identifies the Region. Keep the audit stream prefix unless the EKS stream naming in the target account requires a compatible override.

Do not enable this data stream and `kubernetes.audit_logs` against the same EKS audit log groups. Duplicate collection creates duplicate audit events and can cause duplicate alerts.

Kubernetes audit request and response objects are retained in document `_source` and can contain sensitive API payloads. For Secret resources, this integration removes `data` and `stringData` from parsed request and response objects. It also removes `data`, `stringData`, and metadata annotations from every item returned by Secret list/watch responses, while retaining each item's metadata name. The `preserve_original_event` option is disabled by default; enabling it retains the unredacted raw audit JSON in `event.original`, including Secret values removed from parsed fields. Unsupported records also retain `event.original` for troubleshooting. Restrict access to `_source` and enable original-event preservation only when its diagnostic value outweighs the exposure and storage costs.

Authorization decision, authorization reason, and Pod Security audit-violation annotations have explicit searchable mappings. Other string-valued Kubernetes audit annotations are dynamically indexed as keywords after dots in annotation keys are replaced by underscores.

`event.outcome` is derived from the HTTP response status when `responseStatus.code` is present: codes below 400 are `success` and codes of 400 or above are `failure`. This takes precedence over the `authorization.k8s.io/decision` annotation, because an authorized request can still fail with a 404, 409, or 5xx response. The annotation is used only when no response status is recorded, such as `RequestReceived` stage events.

Request and response objects are not dynamically mapped. Only the security-relevant `aws.eks.audit.requestObject.*` and `aws.eks.audit.responseObject.*` fields listed in the field reference are indexed and searchable; the rest of each API object is retained in `_source` but cannot be queried or aggregated. This keeps the field count bounded on clusters that use many custom resource definitions, where dynamically mapping arbitrary object bodies would otherwise exhaust the index field limit and cause indexing failures. To query an additional body field, add it to a `logs-aws.eks_audit@custom` component template.

## Logs reference

{{event "eks_audit"}}

**ECS Field Reference**

Refer to the following [document](https://www.elastic.co/guide/en/ecs/current/ecs-field-reference.html) for detailed information on ECS fields.

{{fields "eks_audit"}}
5 changes: 5 additions & 0 deletions packages/aws/changelog.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,9 @@
# newer versions go on top
- version: "8.2.0"
changes:
- description: Add top-level Amazon EKS Kubernetes API audit log collection and parsing from CloudWatch.
type: enhancement
link: https://github.com/elastic/integrations/pull/21051
- version: "8.1.2"
changes:
- description: Add an `Instance or Pod IAM Role` option to the `Setup Access` selector so Elastic Agent can authenticate with the AWS SDK default credential chain (EC2 instance profile, EKS Pod Identity or IRSA) without entering access keys or a Role ARN. Since 7.0.0 every option required at least one credential field, which made this documented path impossible to save.
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"apiVersion":"audit.k8s.io/v1","kind":"Event","level":"Metadata","stage":"ResponseComplete","requestReceivedTimestamp":"2026-09-02T12:00:00Z","stageTimestamp":"2026-09-02T12:00:01Z","auditID":"arn-log-group","verb":"get","requestURI":"/api/v1/pods","objectRef":{"resource":"pods","namespace":"default","apiVersion":"v1"},"responseStatus":{"code":200}}
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
fields:
aws:
cloudwatch:
log_group: arn:aws:logs:us-east-1:123456789012:log-group:/aws/eks/prod-cluster/cluster:*
log_stream: kube-apiserver-audit-123
region: us-east-1
account_id: "123456789012"
input:
type: aws-cloudwatch
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
{
"expected": [
{
"@timestamp": "2026-09-02T12:00:00.000Z",
"aws": {
"cloudwatch": {
"account_id": "123456789012",
"log_group": "arn:aws:logs:us-east-1:123456789012:log-group:/aws/eks/prod-cluster/cluster:*",
"log_stream": "kube-apiserver-audit-123",
"region": "us-east-1"
},
"eks": {
"audit": {
"apiVersion": "audit.k8s.io/v1",
"auditID": "arn-log-group",
"kind": "Event",
"level": "Metadata",
"objectRef": {
"apiVersion": "v1",
"namespace": "default",
"resource": "pods"
},
"requestReceivedTimestamp": "2026-09-02T12:00:00Z",
"requestURI": "/api/v1/pods",
"responseStatus": {
"code": 200
},
"stage": "ResponseComplete",
"stageTimestamp": "2026-09-02T12:00:01Z",
"verb": "get"
},
"cluster": {
"name": "prod-cluster"
},
"component": "kube-apiserver",
"log_type": "audit"
}
},
"cloud": {
"account": {
"id": "123456789012"
},
"provider": "aws",
"region": "us-east-1"
},
"ecs": {
"version": "8.11.0"
},
"event": {
"action": "get",
"category": [
"configuration"
],
"dataset": "aws.eks_audit",
"end": "2026-09-02T12:00:01.000Z",
"id": "arn-log-group",
"kind": "event",
"original": "{\"apiVersion\":\"audit.k8s.io/v1\",\"kind\":\"Event\",\"level\":\"Metadata\",\"stage\":\"ResponseComplete\",\"requestReceivedTimestamp\":\"2026-09-02T12:00:00Z\",\"stageTimestamp\":\"2026-09-02T12:00:01Z\",\"auditID\":\"arn-log-group\",\"verb\":\"get\",\"requestURI\":\"/api/v1/pods\",\"objectRef\":{\"resource\":\"pods\",\"namespace\":\"default\",\"apiVersion\":\"v1\"},\"responseStatus\":{\"code\":200}}",
"outcome": "success",
"type": [
"access"
]
},
"input": {
"type": "aws-cloudwatch"
},
"orchestrator": {
"api_version": "v1",
"cluster": {
"name": "prod-cluster"
},
"namespace": "default",
"resource": {
"type": "pods"
},
"type": "kubernetes"
}
}
]
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{"apiVersion":"audit.k8s.io/v1","kind":"Event","level":"RequestResponse","stage":"ResponseComplete","requestReceivedTimestamp":"2026-08-05T08:00:00Z","stageTimestamp":"2026-08-05T08:00:01Z","sourceIPs":["198.51.100.10"],"userAgent":"kubectl/v1.31.0","user":{"username":"arn:aws:iam::123456789012:user/alice","uid":"aws-iam-authenticator:123456789012:alice","groups":["system:authenticated"]},"auditID":"allowed-secret","verb":"get","requestURI":"/api/v1/secrets","objectRef":{"resource":"secrets","namespace":"default","name":"app-secret"},"responseStatus":{"code":200},"annotations":{"authorization.k8s.io/decision":"allow","authorization.k8s.io/reason":"RBAC decision"}}
{"apiVersion":"audit.k8s.io/v1","kind":"Event","level":"RequestResponse","stage":"ResponseComplete","requestReceivedTimestamp":"2026-08-05T08:00:00Z","stageTimestamp":"2026-08-05T08:00:01Z","auditID":"secret-redaction","verb":"create","requestURI":"/api/v1/namespaces/default/secrets/app-secret","objectRef":{"resource":"secrets","namespace":"default","name":"app-secret"},"responseStatus":{"code":201},"requestObject":{"data":{"password":"cmVxdWVzdC1zZWNyZXQ="},"stringData":{"token":"request-token"}},"responseObject":{"data":{"password":"cmVzcG9uc2Utc2VjcmV0"},"stringData":{"token":"response-token"}}}
{"apiVersion":"audit.k8s.io/v1","kind":"Event","level":"RequestResponse","stage":"ResponseComplete","requestReceivedTimestamp":"2026-08-05T08:00:00Z","stageTimestamp":"2026-08-05T08:00:01Z","auditID":"secret-list-redaction","verb":"list","requestURI":"/api/v1/namespaces/default/secrets?watch=true","objectRef":{"resource":"secrets","namespace":"default"},"responseStatus":{"code":200},"responseObject":{"items":[{"metadata":{"name":"first","annotations":{"kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"v1\",\"kind\":\"Secret\",\"metadata\":{\"name\":\"first\"},\"data\":{\"password\":\"c2Vuc2l0aXZl\"}}"}},"data":{"password":"Zmlyc3Q="},"stringData":{"token":"first-token"}},{"metadata":{"name":"second"},"data":{"password":"c2Vjb25k"},"stringData":{"token":"second-token"}}]}}
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
fields:
aws:
cloudwatch:
log_group: /aws/eks/prod-cluster/cluster
log_stream: kube-apiserver-audit-123
region: us-east-1
account_id: "123456789012"
input:
type: aws-cloudwatch
Loading
Loading