[Elasticsearch] The Audit ingest pipeline wrongfully populates success events with failure in event.outcome.

[leandrojmp]

Hello,

We recently configured audit log in our cluster and the respective Elastic Agent integration, while doing some tests to create custom dashboards and alerts we saw that some actions that are performed with success in Kibana are being populate with event.outcome: failure by the Audit Ingest pipeline.

Looking at the ingests pipeline it seems that just a couple of events will ever be populated with event.outcome: success.

  - script:
      lang: painless
      source: >-
        def successEvents = ['authentication_success', 'access_granted', 'run_as_granted', 'connection_granted'];
        if (ctx?.elasticsearch?.audit?.event_type != null && successEvents.contains(ctx.elasticsearch.audit.event_type)) {
          ctx.event.outcome = 'success';
        } else {
          ctx.event.outcome = 'failure';
        }
        if (ctx?.event.action != null && successEvents.contains(ctx.event.action)) {
          ctx.event.outcome = 'success';
        } else {
          ctx.event.outcome = 'failure';
        }

But for example, doing a put_role with success, will be shown in kibana with event.outcome: failure, which is misleading and makes pretty hard to create any security rule on this data.

I'm not sure if this can be solved on the integration, it seems that the audit logs from elasticsearch lacks a lot of basic information about the events.

[leandrojmp]

As an example on how this can be confusing.

If you delete a native realm user, you seem to get two logs, one from the transport audit layer and other from the security_config_change audit layer.

The one from the transport audit layer will match the conditionals in the script processar as the event action is access_granted, so it will populate event.outcome with success.

But the one from the security_config_change will populate event.outcome with failure, but the user was deleted with no issues, so this failure becomes a false negative.

[leandrojmp]

Hello, any update on this?

We want to create some security rules based on those audit logs, but we cannot trust the data, almost everything is marked as an failure, it is not clear if the actions failed or not because of this.

[consulthys]

@leandrojmp sorry for the wait, we're trying to improve the way event.outcome is being emitted at the source together with the ES Security team. In the meantime, I'll check how we could improve this and report back.

[consulthys]

@leandrojmp I've created #21041 to fix the event.outcome not getting the right value. Feel free to review it