ART-14091: Add support for ignorable repos in scan-sources

[locriandev]

Summary

This PR adds support for marking repos as ignorable in scan-sources to prevent mass rebuilds from routine RHEL updates.

Config

Needs to go along with ocp-build-data changes to group.yml in the repos stanza, e.g.

   rhel-9-baseos-rpms:
     scan_sources:
       ignorable: true

Testing

https://art-jenkins.apps.prod-stable-spoke1-dc-iad2.itup.redhat.com/job/aos-cd-builds/job/build%252Focp4-scan-konflux/51918/

Summary by CodeRabbit

• New Features

  • Added support for specifying per-repository scan sources in configuration, including an ignorable flag to control whether a repository participates in scan comparisons.

• Bug Fixes

  • Outdated RPM detection now ignores repositories marked as ignorable in both image and build/architecture inspection paths.
  • When all enabled repositories are ignorable, RPM checks now return empty results immediately to avoid unnecessary repodata scanning.

• Tests

  • Updated related tests to reflect the new scan_sources/ignorable handling behavior.

[openshift-ci-robot]

@locriandev: This pull request references ART-14091 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "5.0.0" version, but no target version was set.

Details

In response to this:

Summary

This PR adds support for marking repos as ignorable in scan-sources to prevent mass rebuilds from routine RHEL updates.

Changes

1. Add Pydantic model support for scan_sources:

• Add ScanSources Pydantic model with ignorable field to artcommonlib/config/repo.py
• Add scan_sources field to the Repo Pydantic model
• Convert scan_sources from Pydantic to dict in Repo.from_repo_config()

2. Add filtering logic in build_info.py:

• Filter out repos marked with scan_sources.ignorable: true during RPM change detection
• Apply filtering in both BrewBuildRecordInspector and KonfluxBuildRecordInspector

Problem

The scan_sources.ignorable configuration was being silently dropped when repos were loaded from group.yml because the Pydantic Repo model didn't include this field. This caused the filtering logic to never execute.

Testing

Tested with ocp-build-data branch ignorable-repos-4.19 which marks rhel-9-baseos-rpms, rhel-9-appstream-rpms, rhel-96-baseos, and rhel-96-appstream as ignorable.

Related

• ocp-build-data PR: (to be created)
• JIRA: ART-14091

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign locriandev for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

Walkthrough

Adds optional scan_sources repo config, carries it through repo construction, and filters ignorable repos from non-latest RPM checks in Brew, Konflux, and RHCOS paths.

Changes

Ignorable scan sources flow

Layer / File(s)	Summary
Repo config and parsing artcommon/artcommonlib/config/repo.py, doozer/doozerlib/runtime.py, doozer/doozerlib/repos.py	Adds ScanSources, makes Repo.scan_sources optional, and propagates validated scan_sources through new-style and legacy repo construction.
Brew and Konflux filtering doozer/doozerlib/build_info.py	Filters enabled repos marked ignorable, logs skipped repos, and returns empty results when no non-ignorable repos remain.
RHCOS filtering and test setup doozer/doozerlib/rhcos.py, doozer/tests/test_rhcos_optimized.py, pyartcd/tests/pipelines/test_ocp4_scan_konflux.py	Filters ignorable enabled repos before RHEL partitioning in RHCOS, and updates tests and pipeline setup to exercise the scan path.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Suggested labels

approved, lgtm

Possibly related PRs

• openshift-eng/art-tools#2201: Touches the same repo config parsing and repo construction path in doozer/doozerlib/runtime.py and doozer/doozerlib/repos.py.

Suggested reviewers

• joepvd

🚥 Pre-merge checks | ✅ 10 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 37.50% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (10 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly matches the main change: adding ignorable repos support in scan-sources.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
No-Weak-Crypto	✅ Passed	Touched files only add scan_sources plumbing/filtering; no MD5/SHA1/DES/RC4/3DES/Blowfish/ECB, custom crypto, or secret/token comparisons were added.
Container-Privileges	✅ Passed	The PR only touches Python/test files; no container/K8s manifests were changed and no flagged privilege settings appear in the modified files.
No-Sensitive-Data-In-Logs	✅ Passed	New logs only report ignorable repo names/build IDs and skip status; no added logging emits credentials, PII, hostnames, or customer data.
No-Hardcoded-Secrets	✅ Passed	Added only scan_sources config/filtering and test tweaks; no hardcoded secrets, credentialed URLs, or literal api_key/secret/token/password assignments appear in the changes.
No-Injection-Vectors	✅ Passed	No unsafe injection primitives appear in touched code; searches found no shell=True, yaml.load, pickle.loads, os.system, or untrusted eval/exec uses.
Ai-Attribution	✅ Passed	No AI-tool use is mentioned in the PR or commit, and no Assisted-by/Generated-by or Co-Authored-By trailers appear; attribution isn’t required.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[locriandev]

Added RHCOS scanning support in commit f73d1fc. The ignorable repo filtering now applies to both:

1. Regular image RPM scanning (BrewBuildRecordInspector and KonfluxBuildRecordInspector)
2. RHCOS RPM scanning (rhcos.py)

This ensures that routine RHEL updates in repos marked as scan_sources.ignorable: true won't trigger unnecessary rebuilds for either regular images or RHCOS.

[locriandev]

✅ All unit tests passing (commit 0816af5)

Fixed test failures:

• Added skip_rpms parameter to Ocp4ScanPipeline test instantiations
• Fixed RHCOS test mocking to properly handle scan_sources checks
• Fixed Repos object access in rhcos.py

Test results: 2910 tests passed across all packages (artcommon, doozer, elliott, pyartcd, ocp-build-data-validator)

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@doozer/doozerlib/runtime.py`:
- Around line 368-370: The optional scan_sources handling in runtime.py is too
strict because Repo.scan_sources can be null, but the current scan_sources setup
always calls ScanSources.model_validate whenever the key exists. Update the
repo_data parsing around scan_sources in the runtime logic so that a
present-but-None value is treated the same as a missing field and skipped before
validation, while still validating only non-null values through
ScanSources.model_validate.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Repository: openshift-eng/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 068dc8cf-930f-490b-9d27-4e8b30f31376

📥 Commits

Reviewing files that changed from the base of the PR and between 79517b4 and 69fffc0.

📒 Files selected for processing (1)

• doozer/doozerlib/runtime.py

[coderabbitai]

🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win

Preserve null for optional scan_sources.

Line 369 validates whenever the key exists, so old-style group.yml now rejects scan_sources: null even though Repo.scan_sources is optional. Treat None the same as an omitted field before calling model_validate.

Suggested fix

-                scan_sources = None
-                if 'scan_sources' in repo_data:
-                    scan_sources = ScanSources.model_validate(repo_data['scan_sources'])
+                scan_sources = None
+                scan_sources_data = repo_data.get('scan_sources')
+                if scan_sources_data not in (None, Missing):
+                    scan_sources = ScanSources.model_validate(scan_sources_data)

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@doozer/doozerlib/runtime.py` around lines 368 - 370, The optional
scan_sources handling in runtime.py is too strict because Repo.scan_sources can
be null, but the current scan_sources setup always calls
ScanSources.model_validate whenever the key exists. Update the repo_data parsing
around scan_sources in the runtime logic so that a present-but-None value is
treated the same as a missing field and skipped before validation, while still
validating only non-null values through ScanSources.model_validate.

[openshift-ci]

@locriandev: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/security	5a44b0a	link	false	/test security

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.