[Wiz] Set CDR transform destination indices to 3 primary shards - #21016
[Wiz] Set CDR transform destination indices to 3 primary shards#21016moxarth-rathod wants to merge 1 commit into
Conversation
|
Pinging @elastic/security-service-integrations (Team:Security-Service Integrations) |
✅ Elastic Docs Style Checker (Vale)No issues found on modified lines! The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale. |
|
Changelog link mismatch — expected
Tip If expected, add the |
💔 Build Failed
Failed CI StepsHistory |
| changes: | ||
| - description: Set the `latest_cdr_vulnerabilities` and `latest_cdr_misconfigurations` transform destination indices to 3 primary shards to avoid shard-size warnings at scale. The destination indices are re-created (`-v3` and `-v4` respectively). | ||
| type: enhancement | ||
| link: https://github.com/elastic/integrations/pull/1 |
There was a problem hiding this comment.
Severity: 🟡 Medium confidence: high path: packages/wiz/changelog.yml:6
Changelog links a different PR number
Details
This changelog entry's link: points at pull/1, but it was added in PR #21016. It is likely a leftover template placeholder or a copy from another PR.
Recommendation:
Point each added changelog entry's link at this PR:
link: https://github.com/elastic/integrations/pull/21016🤖 AI-Generated Review | Vera Review Bot - v0.3.0 | 📚 Knowledge base: integration-skills
⚠️ Automated review — verify suggestions before applying.
Review summaryIssues found across the latest commits 5b43cee — 1 medium
🤖 AI-Generated Review | Vera Review Bot - v0.3.0 | 📚 Knowledge base: integration-skills
|
Proposed commit message
The
latest_cdr_vulnerabilitiesandlatest_cdr_misconfigurationsWiz transforms write to destination indices that are created with a single primary shard, because neither transform shipped amanifest.ymlwith adestination_index_template. At scale this trips Elasticsearch shard-size warnings.This standardises both CDR "latest" transforms on
number_of_shards: 3, a reasonable static middle-ground (~25 GB/shard at the reported 76 GB) that clears the warning for realistic index sizes without over-sharding small deployments.Because
number_of_shardsis an immutable, create-time index setting, adding adestination_index_templateand bumpingfleet_transform_versionalone is not enough: on upgrade Fleet reinstalls the transform but keeps the existing destination index, so an already-created 1-shard index would not be re-sharded. To make the new shard count take effect for existing customers, the destination index-vNsuffix is bumped so a fresh index is created from the updated template, the transform repopulates it, and themove_on_creation: truealias moves onto the new index.Checklist
changelog.ymlfile.Author's Checklist
-v3/-v4destination indices are created with 3 primary shards.security_solution-wiz.{vulnerability,misconfiguration}_latestaliases point at the new indices after install/upgrade.How to test this PR locally
elastic-package checkthenelastic-package install(frompackages/wiz).GET security_solution-wiz.vulnerability_latest-v3/_settings→index.number_of_shards: 3GET security_solution-wiz.misconfiguration_latest-v4/_settings→index.number_of_shards: 3GET security_solution-wiz.vulnerability_latest/_aliasGET security_solution-wiz.misconfiguration_latest/_aliasRelated issues