Skip to content

fix: generalize JUnit XML injection for all early-exit failures#5646

Open
rnetser wants to merge 1 commit into
RedHatQE:mainfrom
rnetser:fix/sanity-junit-injection
Open

fix: generalize JUnit XML injection for all early-exit failures#5646
rnetser wants to merge 1 commit into
RedHatQE:mainfrom
rnetser:fix/sanity-junit-injection

Conversation

@rnetser

@rnetser rnetser commented Jul 15, 2026

Copy link
Copy Markdown
Collaborator
What this PR does / why we need it:

Adds JUnit XML injection for early pytest exit failures from exit_pytest_execution.

When sanity checks or other pre-test verifications fail and exit_pytest_execution aborts the session, the JUnit XML report may contain zero testcases. CI systems (Jenkins, Prow) ignore empty reports, making these failures invisible. This PR adds a synthetic error testcase to the JUnit XML so failures are always reported.

Key changes:

  • Add inject_failure_junit() in utilities/pytest_utils.py — re-opens the XML file after LogXML writes it and appends a synthetic error testcase
  • Store failure info (message, log_message, return_code) from exit_pytest_execution in a module-level TypedDict-annotated dict
  • Derive dynamic testcase names from the failure message (e.g., cluster_sanity_failed, storage_class_failure)
  • Sanitize XML-illegal control characters to prevent corrupted JUnit reports
  • Collapse consecutive underscores in sanitized names
  • Call inject_failure_junit from pytest_sessionfinish in root conftest

Removed private API dependency: replaced _pytest.junitxml.bin_xml_escape with inline control character sanitization using stdlib re.

Co-authored-by: Claude noreply@anthropic.com

Which issue(s) this PR fixes:
Special notes for reviewer:
  • The function injects for ALL recorded failures regardless of existing test count — session fixtures can fail after earlier tests populated the XML
  • Uses TypedDict under TYPE_CHECKING for type safety without runtime overhead
  • 11 unit tests cover: no-op paths, empty/non-empty suites, underscore collapsing, special char fallback, XML escaping, control char sanitization, and pytest.exit() call verification
jira-ticket:

Summary by CodeRabbit

  • Bug Fixes
    • Improved JUnit XML handling for early pytest termination by capturing the exit failure details and, when enabled, appending a synthetic failing testcase to the existing report.
    • Added robust sanitization and escaping for the generated testcase name and error text, and ensured report counters are updated consistently even during teardown issues.
  • Tests
    • Expanded unit tests to validate injection/no-op behavior, missing or malformed report structure handling, payload sanitization/escaping, preservation on write failures, and execution during teardown exceptions.

@coderabbitai

coderabbitai Bot commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 9b1efd17-df38-42ce-a3bf-613b49780f11

📥 Commits

Reviewing files that changed from the base of the PR and between f9c6d71 and 8e71bd3.

📒 Files selected for processing (3)
  • conftest.py
  • utilities/pytest_utils.py
  • utilities/unittests/test_pytest_utils.py
🔗 Linked repositories identified

CodeRabbit considers these linked repositories for cross-repo context during reviews:

  • RedHatQE/openshift-virtualization-tests-design-docs (manual)
📜 Recent review details
⏰ Context from checks skipped due to timeout. (29)
  • GitHub Check: can-be-merged
  • GitHub Check: can-be-merged
  • GitHub Check: can-be-merged
  • GitHub Check: can-be-merged
  • GitHub Check: can-be-merged
  • GitHub Check: can-be-merged
  • GitHub Check: can-be-merged
  • GitHub Check: can-be-merged
  • GitHub Check: can-be-merged
  • GitHub Check: can-be-merged
  • GitHub Check: can-be-merged
  • GitHub Check: can-be-merged
  • GitHub Check: can-be-merged
  • GitHub Check: can-be-merged
  • GitHub Check: can-be-merged
  • GitHub Check: can-be-merged
  • GitHub Check: can-be-merged
  • GitHub Check: can-be-merged
  • GitHub Check: can-be-merged
  • GitHub Check: can-be-merged
  • GitHub Check: verify-bugs-are-open
  • GitHub Check: build-container
  • GitHub Check: tox
  • GitHub Check: can-be-merged
  • GitHub Check: can-be-merged
  • GitHub Check: security-committer-identity
  • GitHub Check: security-suspicious-paths
  • GitHub Check: can-be-merged
  • GitHub Check: can-be-merged
⚠️ CI failures not shown inline (24)

GitHub Check: can-be-merged: Check if can be merged

Conclusion: failure

View job details

Some check runs not started: cnv-smoke/validation, verified
Missing approved from approvers: myakove, dshchedr, vsibirsk, rnetser
Missing lgtm from reviewers. Minimum 2 required, (0 given). Reviewers: myakove, geetikakay, dshchedr, RoniKishner, vsibirsk.

GitHub Check: can-be-merged: Check if can be merged

Conclusion: failure

View job details

Some check runs not started: cnv-smoke/validation, verified
Missing approved from approvers: rnetser, myakove, vsibirsk, dshchedr
Missing lgtm from reviewers. Minimum 2 required, (0 given). Reviewers: geetikakay, RoniKishner, myakove, dshchedr, vsibirsk.

GitHub Check: can-be-merged: Check if can be merged

Conclusion: failure

View job details

Some check runs not started: cnv-smoke/validation, verified
Missing approved from approvers: rnetser, vsibirsk, dshchedr, myakove
Missing lgtm from reviewers. Minimum 2 required, (0 given). Reviewers: vsibirsk, myakove, geetikakay, dshchedr, RoniKishner.

GitHub Check: can-be-merged: Check if can be merged

Conclusion: failure

View job details

Some check runs not started: cnv-smoke/validation, verified
Missing approved from approvers: myakove, dshchedr, vsibirsk, rnetser
Missing lgtm from reviewers. Minimum 2 required, (0 given). Reviewers: RoniKishner, myakove, geetikakay, vsibirsk, dshchedr.

GitHub Check: can-be-merged: Check if can be merged

Conclusion: failure

View job details

Some check runs not started: cnv-smoke/validation, verified
Missing approved from approvers: vsibirsk, myakove, dshchedr, rnetser
Missing lgtm from reviewers. Minimum 2 required, (0 given). Reviewers: geetikakay, vsibirsk, RoniKishner, dshchedr, myakove.

GitHub Check: can-be-merged: Check if can be merged

Conclusion: failure

View job details

Some check runs not started: cnv-smoke/validation, verified
Missing approved from approvers: dshchedr, myakove, rnetser, vsibirsk
Missing lgtm from reviewers. Minimum 2 required, (0 given). Reviewers: geetikakay, RoniKishner, myakove, dshchedr, vsibirsk.

GitHub Check: can-be-merged: Check if can be merged

Conclusion: failure

View job details

Some check runs not started: cnv-smoke/validation, verified
Missing approved from approvers: myakove, dshchedr, vsibirsk, rnetser
Missing lgtm from reviewers. Minimum 2 required, (0 given). Reviewers: dshchedr, geetikakay, vsibirsk, myakove, RoniKishner.

GitHub Check: can-be-merged: Check if can be merged

Conclusion: failure

View job details

Some check runs not started: cnv-smoke/validation, verified
Missing approved from approvers: vsibirsk, rnetser, dshchedr, myakove
Missing lgtm from reviewers. Minimum 2 required, (0 given). Reviewers: RoniKishner, dshchedr, myakove, vsibirsk, geetikakay.

GitHub Check: can-be-merged: Check if can be merged

Conclusion: failure

View job details

Some check runs not started: cnv-smoke/validation, verified
Missing approved from approvers: myakove, vsibirsk, dshchedr, rnetser
Missing lgtm from reviewers. Minimum 2 required, (0 given). Reviewers: myakove, vsibirsk, dshchedr, RoniKishner, geetikakay.

GitHub Check: can-be-merged: Check if can be merged

Conclusion: failure

View job details

Some check runs not started: cnv-smoke/validation, verified
Missing approved from approvers: dshchedr, rnetser, vsibirsk, myakove
Missing lgtm from reviewers. Minimum 2 required, (0 given). Reviewers: geetikakay, dshchedr, myakove, RoniKishner, vsibirsk.

GitHub Check: can-be-merged: Check if can be merged

Conclusion: failure

View job details

Some check runs not started: cnv-smoke/validation, verified
Missing approved from approvers: myakove, dshchedr, vsibirsk, rnetser
Missing lgtm from reviewers. Minimum 2 required, (0 given). Reviewers: dshchedr, geetikakay, vsibirsk, myakove, RoniKishner.

GitHub Check: can-be-merged: Check if can be merged

Conclusion: failure

View job details

Some required check runs in progress tox
Some check runs not started: cnv-smoke/validation, tox, verified
Missing approved from approvers: vsibirsk, rnetser, dshchedr, myakove
Missing lgtm from reviewers. Minimum 2 required, (0 given). Reviewers: RoniKishner, dshchedr, myakove, vsibirsk, geetikakay.

GitHub Check: can-be-merged: Check if can be merged

Conclusion: failure

View job details

Some required check runs in progress build-container, tox
Some check runs not started: cnv-smoke/validation, build-container, tox, verified
Missing approved from approvers: rnetser, myakove, dshchedr, vsibirsk
Missing lgtm from reviewers. Minimum 2 required, (0 given). Reviewers: myakove, dshchedr, vsibirsk, geetikakay, RoniKishner.

GitHub Check: can-be-merged: Check if can be merged

Conclusion: failure

View job details

Some required check runs in progress build-container, tox
Some check runs not started: cnv-smoke/validation, build-container, tox, verified
Missing approved from approvers: vsibirsk, myakove, dshchedr, rnetser
Missing lgtm from reviewers. Minimum 2 required, (0 given). Reviewers: geetikakay, vsibirsk, RoniKishner, dshchedr, myakove.

GitHub Check: can-be-merged: Check if can be merged

Conclusion: failure

View job details

Some required check runs in progress build-container, tox
Some check runs not started: cnv-smoke/validation, build-container, tox, verified
Missing approved from approvers: myakove, dshchedr, vsibirsk, rnetser
Missing lgtm from reviewers. Minimum 2 required, (0 given). Reviewers: dshchedr, geetikakay, vsibirsk, myakove, RoniKishner.

GitHub Check: can-be-merged: Check if can be merged

Conclusion: failure

View job details

Some required check runs in progress build-container, tox
Some check runs not started: cnv-smoke/validation, build-container, tox, verified
Missing approved from approvers: myakove, dshchedr, vsibirsk, rnetser
Missing lgtm from reviewers. Minimum 2 required, (0 given). Reviewers: RoniKishner, myakove, geetikakay, vsibirsk, dshchedr.

GitHub Check: can-be-merged: Check if can be merged

Conclusion: failure

View job details

Some required check runs in progress verify-bugs-are-open, build-container, tox
Some check runs not started: cnv-smoke/validation, verify-bugs-are-open, build-container, tox, verified
Missing approved from approvers: myakove, dshchedr, rnetser, vsibirsk
Missing lgtm from reviewers. Minimum 2 required, (0 given). Reviewers: dshchedr, myakove, geetikakay, RoniKishner, vsibirsk.

GitHub Check: can-be-merged: Check if can be merged

Conclusion: failure

View job details

Some required check runs in progress verify-bugs-are-open, build-container, tox
Some check runs not started: cnv-smoke/validation, verify-bugs-are-open, build-container, tox, verified
Missing approved from approvers: vsibirsk, rnetser, dshchedr, myakove
Missing lgtm from reviewers. Minimum 2 required, (0 given). Reviewers: RoniKishner, dshchedr, myakove, vsibirsk, geetikakay.

GitHub Check: can-be-merged: Check if can be merged

Conclusion: failure

View job details

Some required check runs in progress security-committer-identity, security-suspicious-paths
Some check runs not started: cnv-smoke/validation, security-committer-identity, security-suspicious-paths, verified, verify-bugs-are-open, build-container, tox
Missing approved from approvers: vsibirsk, myakove, dshchedr, rnetser
Missing lgtm from reviewers. Minimum 2 required, (0 given). Reviewers: geetikakay, vsibirsk, RoniKishner, dshchedr, myakove.

GitHub Check: can-be-merged: Check if can be merged

Conclusion: failure

View job details

Some required check runs in progress security-committer-identity, security-suspicious-paths
Some check runs not started: cnv-smoke/validation, security-committer-identity, security-suspicious-paths, verified, verify-bugs-are-open, build-container, tox
Missing required labels: tox:verify-tc-requirement-polarion:passed
Missing approved from approvers: vsibirsk, myakove, dshchedr, rnetser
Missing lgtm from reviewers. Minimum 2 required, (0 given). Reviewers: geetikakay, vsibirsk, RoniKishner, dshchedr, myakove.

GitHub Check: can-be-merged: Check if can be merged

Conclusion: failure

View job details

Some check runs not started: verified, verify-bugs-are-open, tox, security-committer-identity, cnv-smoke/validation, security-suspicious-paths, build-container
Missing required labels: tox:verify-tc-requirement-polarion:passed
Missing approved from approvers: vsibirsk, rnetser, dshchedr, myakove
Missing lgtm from reviewers. Minimum 2 required, (0 given). Reviewers: RoniKishner, dshchedr, myakove, vsibirsk, geetikakay.

GitHub Check: can-be-merged: Check if can be merged

Conclusion: failure

View job details

Some required check runs in progress verify-bugs-are-open, build-container, tox
Some check runs not started: cnv-smoke/validation, verify-bugs-are-open, build-container, tox, verified
Missing required labels: tox:verify-tc-requirement-polarion:passed
Missing approved from approvers: rnetser, myakove, vsibirsk, dshchedr
Missing lgtm from reviewers. Minimum 2 required, (0 given). Reviewers: geetikakay, RoniKishner, myakove, dshchedr, vsibirsk.

GitHub Check: can-be-merged: Check if can be merged

Conclusion: failure

View job details

Some required check runs in progress verify-bugs-are-open, build-container, tox
Some check runs not started: cnv-smoke/validation, verify-bugs-are-open, build-container, tox, verified
Missing required labels: tox:verify-tc-requirement-polarion:passed
Missing approved from approvers: myakove, vsibirsk, dshchedr, rnetser
Missing lgtm from reviewers. Minimum 2 required, (0 given). Reviewers: myakove, vsibirsk, dshchedr, RoniKishner, geetikakay.

GitHub Check: can-be-merged: Check if can be merged

Conclusion: failure

View job details

Some check runs not started: cnv-smoke/validation, verify-bugs-are-open, verified, security-committer-identity, security-suspicious-paths, build-container, tox
Missing required labels: tox:verify-tc-requirement-polarion:passed
Missing approved from approvers: myakove, rnetser, vsibirsk, dshchedr
Missing lgtm from reviewers. Minimum 2 required, (0 given). Reviewers: geetikakay, myakove, dshchedr, RoniKishner, vsibirsk.
🧰 Additional context used
📓 Path-based instructions (5)
**/*.py

📄 CodeRabbit inference engine (AGENTS.md)

**/*.py: Never add linter suppressions such as # noqa, # type: ignore, or # pylint: disable; fix the underlying code instead.
Before adding code, search for existing implementations in the repository, especially utilities/, libs/, tests/, and project dependencies; reuse shared logic and never duplicate it.
Use TYPE_CHECKING for imports needed only by type annotations.
Use Google-format docstrings for public functions with non-obvious return values or side effects.
Use absolute imports, keep imports at module scope, and prefer specific from module import name imports; avoid bare import module unless retaining the module name materially improves readability.
Use named arguments for function calls with more than one argument and avoid single-letter variable names.
Do not add dead code, and prefer direct attribute access unless storing an attribute improves reuse or clarity.
Do not define helper functions, utility functions, or classes in conftest.py or test_*.py; place them in dedicated utility modules. conftest.py files should contain fixtures only, except the root file, which is reserved for hooks and plugin registration.
Use context managers for files, connections, and locks; re-raise exceptions with context, catch specific exception types, never silently swallow exceptions, and provide specific error messages.
Do not use async or async/await; the codebase uses synchronous patterns.
Log transitions, resource operations, configuration changes, responses, and state at INFO; unusual or skipped operations at WARNING; contextual exceptions at ERROR. Never use DEBUG or log secrets, tokens, passwords, or PII, and use f-string formatting.
Do not add defensive checks for guarantees established by types, constructors, schemas, or pinned dependencies; allowed exceptions are cleanup, explicit optional values, lazy initialization, platform constants, and unstable external libraries. Use isinstance rather than hasattr for type discrimination.
Funct...

Files:

  • conftest.py
  • utilities/pytest_utils.py
  • utilities/unittests/test_pytest_utils.py
**/*.{py,pyi}

📄 CodeRabbit inference engine (AGENTS.md)

Use type hints for all new public functions under utilities/; libs/ must satisfy mypy strict mode.

Files:

  • conftest.py
  • utilities/pytest_utils.py
  • utilities/unittests/test_pytest_utils.py
**

⚙️ CodeRabbit configuration file

**: ## PR Template Validation
Check the PR description for required sections from .github/pull_request_template.md.
Required sections (must be present, even if empty):

  • ##### What this PR does / why we need it: — MUST be present AND have meaningful content.
    Flag as HIGH if the section is missing, empty, whitespace-only, contains only HTML comments,
    or contains only placeholder tokens such as TBD, TBA, N/A, -, , none, or ..
  • ##### Which issue(s) this PR fixes: — must be present (may be empty)
  • ##### Special notes for reviewer: — must be present (may be empty)
  • ##### jira-ticket: — must be present (may be empty)
    If any required section is absent, or What this PR does / why we need it: has no content,
    flag it as HIGH severity and ask the author to restore the missing template section(s).

Approval Policy

You may approve the PR when ALL of the following are true:

  • All your review comments have been addressed with either:
    • a code/doc change that fixes the issue, or
    • a substantive author response that justifies no code change.
      Thread "resolved" state alone is not sufficient.
      OR you had no review comments.
  • If you posted a test execution plan comment requesting tests, and the PR author replied
    with a comment explaining why the requested tests are not needed or were already covered,
    treat that as an acceptable response — do not block approval on the test plan alone.
  • The author's explanation must be reasonable and specific (not just "N/A" or "not needed").
    Accept explanations like: "these tests were already run in CI", "this change is docs-only",
    "the affected tests are quarantined", or "verified manually on cluster X".

Files:

  • conftest.py
  • utilities/pytest_utils.py
  • utilities/unittests/test_pytest_utils.py
utilities/**/*.py

📄 CodeRabbit inference engine (AGENTS.md)

Place utility functions in the domain-matching module under utilities/; create a domain module only when no suitable module exists.

Files:

  • utilities/pytest_utils.py
  • utilities/unittests/test_pytest_utils.py
utilities/unittests/**

⚙️ CodeRabbit configuration file

utilities/unittests/**: ## Unit Tests — Relaxed Rules
Files under utilities/unittests/ are unit tests for shared utilities.
Do NOT enforce:

  • STD docstring format (Preconditions/Steps/Expected sections)
  • STP link requirements
  • RFE/Jira link requirements
  • Coverage tracking rules
    These rules apply only to feature/integration tests under tests/.
    DO still enforce: code quality, type hints, naming conventions, and all other coding standards.

Files:

  • utilities/unittests/test_pytest_utils.py
🧠 Learnings (40)
📚 Learning: 2026-01-12T11:24:13.825Z
Learnt from: servolkov
Repo: RedHatQE/openshift-virtualization-tests PR: 3387
File: tests/network/provider_migration/libprovider.py:50-52
Timestamp: 2026-01-12T11:24:13.825Z
Learning: In the RedHatQE/openshift-virtualization-tests repository, when catching exceptions in Python, use LOGGER.error before re-raising and do not replace it with LOGGER.exception in except blocks. This follows the established pattern across the codebase.

Applied to files:

  • conftest.py
  • utilities/pytest_utils.py
  • utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-01-12T14:25:05.723Z
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 3366
File: tests/storage/cdi_clone/test_clone.py:5-9
Timestamp: 2026-01-12T14:25:05.723Z
Learning: In Python tests and utility code across the repository, bitmath.parse_string_unsafe correctly parses Kubernetes quantities (e.g., '4Gi', '512Mi', PVC storage requests) without supplying system=bitmath.NIST. There are 30+ usages indicating this is the standard behavior. Reviewers should verify that code that builds or compares quantity strings does not pass the NIST parameter, and if a new test relies on quantity parsing, assume no NIST parameter is required unless explicitly documented.

Applied to files:

  • conftest.py
  • utilities/pytest_utils.py
  • utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-01-20T01:03:13.139Z
Learnt from: servolkov
Repo: RedHatQE/openshift-virtualization-tests PR: 3387
File: tests/network/provider_migration/libprovider.py:1-8
Timestamp: 2026-01-20T01:03:13.139Z
Learning: In the openshift-virtualization-tests repository, Python imports should consistently use module-level imports for the logging module (i.e., import logging) rather than from logging import ... The established pattern spans 270+ files and should not be flagged for refactoring. Apply this guideline to Python files across the repo (e.g., tests/network/provider_migration/libprovider.py).

Applied to files:

  • conftest.py
  • utilities/pytest_utils.py
  • utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-01-21T21:26:41.805Z
Learnt from: geetikakay
Repo: RedHatQE/openshift-virtualization-tests PR: 3559
File: utilities/infra.py:251-254
Timestamp: 2026-01-21T21:26:41.805Z
Learning: In the RedHatQE/openshift-virtualization-tests repository, when reviewing Python code, recognize that with Python 3.14 the syntax 'except ValueError, TypeError:' is valid if there is no 'as' clause, and should not be flagged as Python 2 syntax. If you use an 'as' binding (e.g., 'except (ValueError, TypeError) as e:'), parentheses are required. Ensure this pattern is version-consistent and not flagged as Python 2 syntax when 'as' is absent.

Applied to files:

  • conftest.py
  • utilities/pytest_utils.py
  • utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-01-25T13:18:21.675Z
Learnt from: jpeimer
Repo: RedHatQE/openshift-virtualization-tests PR: 3571
File: tests/storage/storage_migration/utils.py:158-167
Timestamp: 2026-01-25T13:18:21.675Z
Learning: In reviews of the openshift-virtualization-tests repo (and similar Python code), avoid suggesting minor stylistic changes that require extra verification (e.g., removing dict.keys() checks for membership) unless the change has clear correctness or maintainability impact. Focus on fixes with observable behavior, security, performance, or maintainability benefits; defer low-impact style tweaks that are costly to verify.

Applied to files:

  • conftest.py
  • utilities/pytest_utils.py
  • utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-02-18T06:35:39.536Z
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 3847
File: utilities/virt.py:2449-2453
Timestamp: 2026-02-18T06:35:39.536Z
Learning: In Python code, a function named clearly and self-descriptively can be deemed not to require a docstring. However, treat this as a context-specific guideline and not a universal rule. For public APIs or functions with side effects, prefer concise docstrings explaining behavior, inputs, outputs, and side effects. This guidance is based on the example in utilities/virt.py from RedHatQE/openshift-virtualization-tests where validate_libvirt_persistent_domain(vm, admin_client) was considered self-documenting.

Applied to files:

  • conftest.py
  • utilities/pytest_utils.py
  • utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-02-23T16:33:22.070Z
Learnt from: vsibirsk
Repo: RedHatQE/openshift-virtualization-tests PR: 3883
File: utilities/pytest_utils.py:441-463
Timestamp: 2026-02-23T16:33:22.070Z
Learning: In Python code reviews, the guideline to always use named arguments for multi-argument calls does not apply to built-ins or methods that have positional-only parameters (those defined with a / in their signature). Do not flag or require named arguments for calls like dict.get(key, default=None, /), list.pop(), str.split(sep, maxsplit) and similar built-ins that cannot accept keyword arguments. Apply the named-argument rule only to functions/methods that explicitly accept keyword arguments.

Applied to files:

  • conftest.py
  • utilities/pytest_utils.py
  • utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-03-17T01:32:02.617Z
Learnt from: dshchedr
Repo: RedHatQE/openshift-virtualization-tests PR: 4118
File: utilities/database.py:0-0
Timestamp: 2026-03-17T01:32:02.617Z
Learning: In RedHatQE/openshift-virtualization-tests, when reviewing Python files, post targeted inline comments on the Files changed tab at the exact location (file and line) of the issue rather than opening a single discussion thread for multiple issues. This should be done for each applicable location to improve traceability and clarity. If multiple issues exist in the same file, address them with separate inline comments pointing to the specific lines.

Applied to files:

  • conftest.py
  • utilities/pytest_utils.py
  • utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-03-17T01:32:02.617Z
Learnt from: dshchedr
Repo: RedHatQE/openshift-virtualization-tests PR: 4118
File: utilities/database.py:0-0
Timestamp: 2026-03-17T01:32:02.617Z
Learning: In the RedHatQE/openshift-virtualization-tests repository, CodeRabbit should post targeted inline comments at each applicable location in the Files Changed tab, rather than aggregating multiple issues into a single PR discussion thread reply. This guideline applies to all Python files (any file ending in .py) changed in a PR; for non-Python files, follow the same inline-comment-at-location principle if relevant.

Applied to files:

  • conftest.py
  • utilities/pytest_utils.py
  • utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-05-04T13:45:29.122Z
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 4725
File: utilities/console.py:54-59
Timestamp: 2026-05-04T13:45:29.122Z
Learning: During review of RedHatQE/openshift-virtualization-tests “lint-cleanup” PRs (e.g., changes targeting lint issues like stale noqa/utf-8 headers), do not flag existing `# type: ignore` directives that were already present before the PR and were not introduced or modified by the PR. Only raise findings for `# type: ignore` suppressions that the PR itself adds, changes, or otherwise makes newly effective (i.e., they appear in the diff as additions/edits).

Applied to files:

  • conftest.py
  • utilities/pytest_utils.py
  • utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-05-04T13:45:33.892Z
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 4725
File: tests/virt/cluster/common_templates/centos/test_centos_os_support.py:78-83
Timestamp: 2026-05-04T13:45:33.892Z
Learning: When reviewing lint-cleanup or formatting-only pull requests in this repo (e.g., changes like removing/updating `# noqa` comments or UTF-8 headers), do not raise findings for code patterns that already existed before the PR. Specifically, if a problematic construct such as `.is_connective(tcp_timeout=120)` was present in the base branch, suppress that finding and only raise issues when the PR itself introduces or modifies that construct (i.e., the diff adds/changes the call or its arguments). Apply this rule across all Python files (`**/*.py`).

Applied to files:

  • conftest.py
  • utilities/pytest_utils.py
  • utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-05-05T17:01:15.294Z
Learnt from: dshchedr
Repo: RedHatQE/openshift-virtualization-tests PR: 4739
File: tests/virt/node/descheduler/conftest.py:2-2
Timestamp: 2026-05-05T17:01:15.294Z
Learning: In this repo’s Python code, it’s acceptable (and preferred by convention) to build `run_command` inputs using `shlex.split(f"<command> {arg}")` rather than converting to direct list literals like `['oc', 'adm', 'uncordon', name]`. During code review, generally don’t flag `shlex.split(...)` usage for `run_command` calls and don’t suggest replacing it with list literals; the string-form pattern is used to keep commands readable and consistent with how they’re typed in a terminal.

Applied to files:

  • conftest.py
  • utilities/pytest_utils.py
  • utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-05-08T12:49:20.694Z
Learnt from: geetikakay
Repo: RedHatQE/openshift-virtualization-tests PR: 4788
File: utilities/os_utils.py:257-262
Timestamp: 2026-05-08T12:49:20.694Z
Learning: In RedHatQE/openshift-virtualization-tests, the Ruff flake8-boolean-trap rules FBT001/FBT002 are intentionally not enabled (pyproject.toml does not select the FBT rules; confirmed via `ruff check --show-settings`). Therefore, do not flag boolean positional parameters as FBT001/FBT002 violations in this repository. If Ruff configuration changes and starts selecting FBT rules, this exception should be reconsidered.

Applied to files:

  • conftest.py
  • utilities/pytest_utils.py
  • utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-05-12T05:10:24.601Z
Learnt from: acinko-rh
Repo: RedHatQE/openshift-virtualization-tests PR: 4780
File: tests/storage/utils.py:568-572
Timestamp: 2026-05-12T05:10:24.601Z
Learning: In this repository, Ruff rule UP043 ("unnecessary default type arguments") is enforced. When annotating `collections.abc.Generator` return types, prefer the single-parameter form `Generator[YieldType]` rather than `Generator[YieldType, None, None]`. Explicit `None, None` for the SendType and ReturnType are unnecessary defaults (per PEP 696) and will trigger UP043. Apply this consistently across all Python files.

Applied to files:

  • conftest.py
  • utilities/pytest_utils.py
  • utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-05-13T19:23:09.603Z
Learnt from: Anatw
Repo: RedHatQE/openshift-virtualization-tests PR: 4833
File: tests/network/localnet/migration_stuntime/libstuntime.py:25-25
Timestamp: 2026-05-13T19:23:09.603Z
Learning: In this repository, do not recommend adding `from __future__ import annotations` to fix forward-reference type annotation issues (e.g., Ruff UP037). Follow the established convention: use quoted string type annotations for forward references when the referenced class/type is defined later in the same file (e.g., `"ContinuousPing"`), and prefer `typing.Self` for self-referential return types.

Applied to files:

  • conftest.py
  • utilities/pytest_utils.py
  • utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-05-18T06:30:56.781Z
Learnt from: EdDev
Repo: RedHatQE/openshift-virtualization-tests PR: 4819
File: utilities/unittests/test_bitwarden.py:207-207
Timestamp: 2026-05-18T06:30:56.781Z
Learning: During Ruff/lint rule-enablement PRs in this repository (e.g., when introducing a new rule like PLC0415), it’s acceptable to keep CI green by adding per-line, targeted suppressions for pre-existing violations: add only `# noqa: <single-ruff-rule-id>` at the end of the specific violating line. In this PR context, reviewers should NOT flag these targeted `# noqa: PLC0415` comments as policy violations, assuming the suppression is for a pre-existing issue and is documented in the PR description as a candidate for follow-up cleanup. Do not allow blanket `# noqa` (without a specific rule) or `per-file-ignores`; those remain disallowed.

Applied to files:

  • conftest.py
  • utilities/pytest_utils.py
  • utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-05-18T06:31:12.015Z
Learnt from: EdDev
Repo: RedHatQE/openshift-virtualization-tests PR: 4819
File: utilities/unittests/test_pytest_utils.py:270-270
Timestamp: 2026-05-18T06:31:12.015Z
Learning: In RedHatQE/openshift-virtualization-tests, if a PR is a Ruff rule-enforcement PR and its “Special notes for reviewer” documents that pre-existing Ruff violations are being temporarily handled via per-line suppressions (e.g., `# noqa: PLC0415`) to keep CI green, reviewers should treat those specific `# noqa: <rule>` comments as an agreed, temporary mechanism. Do not flag them as code-quality issues and do not recommend removing, consolidating, or refactoring those suppressions within the same PR; cleanup/remediation is expected to happen in dedicated follow-up PRs instead.

Applied to files:

  • conftest.py
  • utilities/pytest_utils.py
  • utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-05-18T06:31:15.083Z
Learnt from: EdDev
Repo: RedHatQE/openshift-virtualization-tests PR: 4819
File: utilities/unittests/test_data_collector.py:304-304
Timestamp: 2026-05-18T06:31:15.083Z
Learning: When reviewing Python code in this repository for Ruff/linter rule rollouts, do not treat temporary suppression comments as violations in the specific migration scenario where a PR enables a new Ruff rule (e.g., PLC0415) and the PR description explicitly documents that all *pre-existing* violations are being annotated with `# noqa: <RULE>` as a short-lived measure. In that case, only flag `# noqa: <RULE>` suppressions that are newly introduced on code that did not previously violate the rule—i.e., verify via the PR diff against the prior state (and/or prior Ruff findings) that the suppressed line was already violating before the rule was enabled. Ignore suppressions that are covering violations that existed before the new rule rollout and were intentionally bulk-added for cleanup in follow-up PRs.

Applied to files:

  • conftest.py
  • utilities/pytest_utils.py
  • utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-05-18T06:31:20.848Z
Learnt from: EdDev
Repo: RedHatQE/openshift-virtualization-tests PR: 4819
File: utilities/unittests/test_hco.py:501-501
Timestamp: 2026-05-18T06:31:20.848Z
Learning: When reviewing Python code in RedHatQE/openshift-virtualization-tests, avoid flagging Ruff `# noqa: <RULE>` suppressions as issues if they were intentionally added as a temporary measure to keep CI green after a PR enables a new Ruff/lint rule (e.g., PLC0415) and the PR description documents this under "Special notes for reviewer". Treat these suppressions as deferred technical debt. Only flag `# noqa: PLC0415` (and similar rule-specific suppressions) when they are newly introduced without an accompanying documented intent in the PR (and thus appear to be masking a new violation rather than a pre-existing one).

Applied to files:

  • conftest.py
  • utilities/pytest_utils.py
  • utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-05-18T09:09:09.479Z
Learnt from: EdDev
Repo: RedHatQE/openshift-virtualization-tests PR: 4878
File: utilities/unittests/test_pytest_utils.py:2194-2197
Timestamp: 2026-05-18T09:09:09.479Z
Learning: In this repository (RedHatQE/openshift-virtualization-tests), do not flag missing return type annotations or missing argument type annotations as Ruff “ANN” rule violations (e.g., ANN001/ANN002/ANN201/ANN202). The repo’s Ruff configuration does not enable ANN rules and only uses `extend-select = ["PLC0415"]`, so missing type annotations should not be treated as ANN lint failures during code review.

Applied to files:

  • conftest.py
  • utilities/pytest_utils.py
  • utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-05-19T07:48:17.119Z
Learnt from: azhivovk
Repo: RedHatQE/openshift-virtualization-tests PR: 4784
File: libs/vm/affinity.py:104-104
Timestamp: 2026-05-19T07:48:17.119Z
Learning: When using Kubernetes API models like `NodeSelectorRequirement` or `LabelSelectorRequirement` with operators `Exists` or `DoesNotExist`, the `values` field must not be non-empty. It is valid for `values` to be omitted / left as `None` (Python) / passed as `null`—Kubernetes rejects non-empty `values` for these operators, but does not require the field to be present or explicitly set to an empty list. In code reviews, do not treat missing `values=[]` for `Exists`/`DoesNotExist` as a validation issue; only flag cases where `values` is provided with actual elements.

Applied to files:

  • conftest.py
  • utilities/pytest_utils.py
  • utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-05-19T07:48:17.119Z
Learnt from: azhivovk
Repo: RedHatQE/openshift-virtualization-tests PR: 4784
File: libs/vm/affinity.py:104-104
Timestamp: 2026-05-19T07:48:17.119Z
Learning: When constructing Kubernetes `NodeSelectorRequirement` (or `LabelSelectorRequirement`) objects in code, do not treat `values` being omitted, `None`, or an empty list as an API-validation problem when the requirement’s operator is `Exists` or `DoesNotExist`. Per the Kubernetes API spec, these operators only require that the `values` array is not non-empty (i.e., it must be empty); they do not require the field to be explicitly present as `[]`. Therefore, reviewers should not flag `values=None`/missing `values` for `Exists`/`DoesNotExist`.

Applied to files:

  • conftest.py
  • utilities/pytest_utils.py
  • utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-05-26T15:52:31.613Z
Learnt from: rlobillo
Repo: RedHatQE/openshift-virtualization-tests PR: 4983
File: utilities/hco.py:376-378
Timestamp: 2026-05-26T15:52:31.613Z
Learning: For Python files in this repo, don’t raise review findings for missing type hints or missing/Google-style docstrings on an existing function when the PR’s only functional change is adding one or more new parameters to that function and the PR does not otherwise refactor or substantially rewrite its body/signature. Treat type-annotation/docstring improvements as out of scope for focused parameter-add PRs and defer them to a follow-up. Only raise missing type-hint or docstring issues when the PR introduces an entirely new function or substantially rewrites an existing one.

Applied to files:

  • conftest.py
  • utilities/pytest_utils.py
  • utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-06-21T20:28:07.727Z
Learnt from: Anatw
Repo: RedHatQE/openshift-virtualization-tests PR: 5283
File: tests/network/libs/service.py:14-16
Timestamp: 2026-06-21T20:28:07.727Z
Learning: When reviewing Python code in this repository targeting Python 3.14 with PEP 649 (deferred annotation evaluation using descriptors) enabled by default, do not flag annotations that reference types imported only under `if TYPE_CHECKING:` (e.g., `client: DynamicClient | None = None`) as potential runtime `NameError` problems. With PEP 649 enabled, annotations are not evaluated at function definition time, so these patterns are valid without `from __future__ import annotations`.

Applied to files:

  • conftest.py
  • utilities/pytest_utils.py
  • utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-07-09T12:58:50.301Z
Learnt from: yossisegev
Repo: RedHatQE/openshift-virtualization-tests PR: 5578
File: tests/network/l2_bridge/libl2bridge.py:9-9
Timestamp: 2026-07-09T12:58:50.301Z
Learning: In this repo, follow the established convention for type-only imports: do not require moving `ResourceField` imports from `kubernetes.dynamic.client` behind an `if TYPE_CHECKING:` guard when the imported symbol is used only for type annotations (e.g., in function/class annotations) and not referenced at runtime. Only raise/flag this if `ResourceField` is actually used in executable code (runtime values/calls), since then the import may need to be handled differently.

Applied to files:

  • conftest.py
  • utilities/pytest_utils.py
  • utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-01-26T10:11:23.629Z
Learnt from: OhadRevah
Repo: RedHatQE/openshift-virtualization-tests PR: 2348
File: tests/observability/upgrade/conftest.py:1-7
Timestamp: 2026-01-26T10:11:23.629Z
Learning: In conftest.py files that define session-scoped fixtures which provision resources (e.g., VMs, namespaces) for upgrade testing, logging imports and structured INFO logs for setup are optional. Prefer quieter fixture setup to reduce log noise unless debugging is required. Apply this guideline across similar conftest.py fixtures that perform straightforward resource provisioning.

Applied to files:

  • conftest.py
📚 Learning: 2026-01-29T15:01:46.322Z
Learnt from: EdDev
Repo: RedHatQE/openshift-virtualization-tests PR: 3649
File: tests/network/user_defined_network/ip_specification/conftest.py:32-48
Timestamp: 2026-01-29T15:01:46.322Z
Learning: In test fixtures (e.g., tests/network/user_defined_network/ip_specification/conftest.py and other VM-related conftest.py files), after vm.start(), prefer calling vm.wait_for_agent_connected() as the validation step. Do not add vm.wait_for_ready_status(status=True) beforehand, since wait_for_agent_connected() ensures the guest OS is actually running (not just powered on) and provides a stronger, more reliable readiness check for test initialization.

Applied to files:

  • conftest.py
📚 Learning: 2026-03-03T12:01:13.888Z
Learnt from: jpeimer
Repo: RedHatQE/openshift-virtualization-tests PR: 4053
File: tests/storage/cross_cluster_live_migration/conftest.py:47-55
Timestamp: 2026-03-03T12:01:13.888Z
Learning: In test configuration files (for example tests/.../conftest.py), log the absence of optional CLI arguments or configurations at INFO level. Use WARNING only when missing configuration could cause a problem or be unusual. For parameters that may be present or absent by design, prefer INFO to reflect normality.

Applied to files:

  • conftest.py
📚 Learning: 2026-02-18T06:34:38.042Z
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 3847
File: tests/virt/cluster/common_templates/utils.py:58-58
Timestamp: 2026-02-18T06:34:38.042Z
Learning: In RedHatQE/openshift-virtualization-tests, treat 'public' functions as those defined in any Python files under libs/ or utilities/ (any depth). Functions inside nested test directories (e.g., tests/virt/cluster/common_templates/, tests/virt/node/, etc.) are test helpers and do not require Google-format docstrings unless explicitly requested. Use this rule during reviews to decide whether to enforce docstrings on public API functions in libs/utilities.

Applied to files:

  • utilities/pytest_utils.py
  • utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-05-03T15:38:09.624Z
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 4701
File: tests/virt/node/general/test_windows_vtpm_bitlocker.py:50-52
Timestamp: 2026-05-03T15:38:09.624Z
Learning: During review of PRs that are lint cleanups or tooling/version bumps, do not flag code-quality issues for patterns that pre-existed before the PR. Specifically, if the diff does not introduce/modify constructs such as nested `if` blocks or unnecessary list comprehensions, treat them as known/deferred and leave them for dedicated follow-up cleanup PRs. Only raise issues when the PR itself adds, changes, or refactors the problematic code.

Applied to files:

  • utilities/pytest_utils.py
  • utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-05-15T18:42:02.504Z
Learnt from: geetikakay
Repo: RedHatQE/openshift-virtualization-tests PR: 4860
File: utilities/hco.py:385-389
Timestamp: 2026-05-15T18:42:02.504Z
Learning: In this repository, under the Python `utilities/` directory (utility/helper modules, not tests), do not flag bare `assert` statements as correctness or style issues. The codebase conventionally uses `assert` in these utilities (e.g., `utilities/virt.py`, `utilities/infra.py`, etc.) and does not enforce running Python with `-O`/`PYTHONOPTIMIZE`, so the usual “asserts may be stripped” concern should not be treated as a review blocker here.

Applied to files:

  • utilities/pytest_utils.py
  • utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-06-15T10:56:21.758Z
Learnt from: vsibirsk
Repo: RedHatQE/openshift-virtualization-tests PR: 4994
File: tests/network/user_defined_network/ip_specification/test_ip_specification.py:123-127
Timestamp: 2026-06-15T10:56:21.758Z
Learning: In this repository, do not require or flag missing `-> None` return type annotations on pytest test functions/methods (i.e., functions named `test_*`) located under `tests/**`. Return type annotations for `-> None` on these test functions are optional and should not be treated as inconsistent. Separately, in `utilities/**` and `libs/**`, enforce return type annotations for non-test public functions (e.g., functions that are not internal/private such as those not starting with `_`).

Applied to files:

  • utilities/pytest_utils.py
  • utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-01-07T11:51:57.253Z
Learnt from: qwang1
Repo: RedHatQE/openshift-virtualization-tests PR: 3301
File: utilities/unittests/test_oadp.py:5-5
Timestamp: 2026-01-07T11:51:57.253Z
Learning: In utilities/unittests/test_oadp.py and similar test files in this repo, include a # flake8: noqa: E402 directive to suppress E402 warnings caused by pre-commit checks. This is needed because mock setup must precede imports to avoid circular dependencies, and Ruff may report unused imports. Apply this directive to all similar test files that require imports to guard against circular import issues.

Applied to files:

  • utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-01-19T07:53:55.780Z
Learnt from: qwang1
Repo: RedHatQE/openshift-virtualization-tests PR: 3301
File: utilities/unittests/test_oadp.py:48-57
Timestamp: 2026-01-19T07:53:55.780Z
Learning: Enforce that pytest markers (e.g., marker requirement for integration tests) apply only to integration/functional tests under the tests/ directory. Unit tests located in utilities/unittests/ must not use pytest markers, following repository convention. For all unit test files under utilities/unittests (and subdirectories), do not require or mandate markers; for integration tests under tests/ and its subdirectories, ensure markers are used as per the project rule.

Applied to files:

  • utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-02-23T21:16:51.920Z
Learnt from: vsibirsk
Repo: RedHatQE/openshift-virtualization-tests PR: 3883
File: utilities/unittests/test_os_utils.py:358-371
Timestamp: 2026-02-23T21:16:51.920Z
Learning: In the openshift-virtualization-tests repository, the guideline to require named arguments for function calls with more than one argument does not apply to unit tests under utilities/unittests/. For these tests, allow standard pytest patterns like pytest.raises(ExceptionType, match="pattern") without requiring named parameters. This exception applies specifically to files under utilities/unittests/ and does not generalize beyond that directory.

Applied to files:

  • utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-05-03T15:38:22.954Z
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 4701
File: utilities/unittests/test_database.py:12-12
Timestamp: 2026-05-03T15:38:22.954Z
Learning: In RedHatQE/openshift-virtualization-tests unit tests under `utilities/unittests/`, it’s intentional/pre-existing to use `sys.path.insert(0, ...)` in cases where tests need to import local modules. In lint-cleanup PRs, reviewers should not flag this `sys.path` manipulation as a violation. Also, don’t re-add `# noqa: E402` or other import-order suppressions if `ruff/flake8` no longer reports E402 for that import—handle any broader refactor (to remove the `sys.path` hack) in a dedicated follow-up.

Applied to files:

  • utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-05-18T06:31:30.830Z
Learnt from: EdDev
Repo: RedHatQE/openshift-virtualization-tests PR: 4819
File: utilities/unittests/test_sanity.py:22-22
Timestamp: 2026-05-18T06:31:30.830Z
Learning: In this repo’s unit test files under `utilities/unittests/`, treat `# noqa: PLC0415` (import-outside-top-level) suppressions as intentional and do not flag them as violations when enabling Ruff’s PLC0415 rule incrementally. These suppressions are expected as temporary measures and are documented in the PR description; they should only be targeted for later cleanup follow-ups.
Also, allow imports placed inside unit test methods under `utilities/unittests/` when they are intentionally positioned to run after `patch` decorators are applied—especially when the imported module has import-time side effects.

Applied to files:

  • utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-07-01T15:15:37.516Z
Learnt from: jpeimer
Repo: RedHatQE/openshift-virtualization-tests PR: 5443
File: utilities/unittests/test_storage.py:14-19
Timestamp: 2026-07-01T15:15:37.516Z
Learning: In this repository, within `utilities/unittests/**`, it is acceptable to use the established convention `module_obj.submodule = value  # type: ignore[attr-defined]` (e.g., `utilities.storage = utilities.storage`, `utilities.hco = mock_hco`, `utilities.virt = mock_virt`) when reassigning/reloading submodules as attributes on the `utilities` package for test mocking/reload purposes. These assignments are intentionally suppressed for mypy’s `attr-defined` warning, and equivalent rebinding via local variables does not provide the required reload/rebind semantics for these tests. Code review should not treat this specific `# type: ignore[attr-defined]` pattern in `utilities/unittests/**/*.py` as a new linter/type-suppression violation.

Applied to files:

  • utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-05-18T10:38:33.820Z
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 4884
File: .coderabbit.yaml:79-80
Timestamp: 2026-05-18T10:38:33.820Z
Learning: In this repo, pytest tests are expected to live only in files named `test_*.py`. When applying CodeRabbit STP case B (newly added `def test_*` inside an existing test file), scope the check to existing files that match `test_*.py` and do not broaden it to newly added `def test_*` found in other non-`test_*.py` Python files.

Applied to files:

  • utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-06-01T06:43:07.271Z
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 4908
File: utilities/unittests/test_pytest_utils.py:374-374
Timestamp: 2026-06-01T06:43:07.271Z
Learning: In RedHatQE/openshift-virtualization-tests, do not require STP/RFE/Jira traceability links in unit test function docstrings for any `test_*.py` file under `utilities/unittests/` (including subdirectories). Only integration/functional tests under the repository’s `tests/` directory should be checked for STP/RFE/Jira traceability links; therefore, missing STP/RFE/Jira docstring links in `utilities/unittests/**/test_*.py` should not be flagged.

Applied to files:

  • utilities/unittests/test_pytest_utils.py
🪛 ast-grep (0.44.1)
utilities/pytest_utils.py

[warning] 102-102: Potential XXE attack
Context: ElementTree.parse(xml_path)
Note: [CWE-611] Improper Restriction of XML External Entity Reference.

(xxe-injection)

utilities/unittests/test_pytest_utils.py

[info] 389-389: Do not hardcode temporary file or directory names
Context: "/tmp"
Note: [CWE-377] Insecure Temporary File.

(hardcoded-tmp-file)


[info] 420-420: Do not hardcode temporary file or directory names
Context: "/tmp"
Note: [CWE-377] Insecure Temporary File.

(hardcoded-tmp-file)


[info] 451-451: Do not hardcode temporary file or directory names
Context: "/tmp"
Note: [CWE-377] Insecure Temporary File.

(hardcoded-tmp-file)


[info] 2949-2949: Do not hardcode temporary file or directory names
Context: "/tmp/test"
Note: [CWE-377] Insecure Temporary File.

(hardcoded-tmp-file)


[info] 2969-2969: Do not hardcode temporary file or directory names
Context: "/tmp/test"
Note: [CWE-377] Insecure Temporary File.

(hardcoded-tmp-file)


[warning] 2866-2866: Potential XXE attack
Context: ElementTree.parse(xml_path)
Note: [CWE-611] Improper Restriction of XML External Entity Reference.

(xxe-injection)


[warning] 2891-2891: Potential XXE attack
Context: ElementTree.parse(xml_path)
Note: [CWE-611] Improper Restriction of XML External Entity Reference.

(xxe-injection)


[warning] 2933-2933: Potential XXE attack
Context: ElementTree.parse(xml_path)
Note: [CWE-611] Improper Restriction of XML External Entity Reference.

(xxe-injection)


[warning] 3005-3005: Potential XXE attack
Context: ElementTree.parse(xml_path)
Note: [CWE-611] Improper Restriction of XML External Entity Reference.

(xxe-injection)


[warning] 3032-3032: Potential XXE attack
Context: ElementTree.parse(xml_path)
Note: [CWE-611] Improper Restriction of XML External Entity Reference.

(xxe-injection)


[warning] 3059-3059: Potential XXE attack
Context: ElementTree.parse(xml_path)
Note: [CWE-611] Improper Restriction of XML External Entity Reference.

(xxe-injection)


[warning] 3091-3091: Potential XXE attack
Context: ElementTree.parse(xml_path)
Note: [CWE-611] Improper Restriction of XML External Entity Reference.

(xxe-injection)


[warning] 3134-3134: Potential XXE attack
Context: ElementTree.parse(xml_path)
Note: [CWE-611] Improper Restriction of XML External Entity Reference.

(xxe-injection)

🪛 Ruff (0.15.21)
conftest.py

[warning] 886-886: Logging statement uses f-string

(G004)


[warning] 891-891: Loop control variable files not used within loop body

Rename unused files to _files

(B007)

utilities/pytest_utils.py

[error] 103-103: Using xml to parse untrusted data is known to be vulnerable to XML attacks; use defusedxml equivalents

(S314)


[warning] 139-139: Logging statement uses f-string

(G004)


[warning] 220-223: Avoid specifying long messages outside the exception class

(TRY003)


[warning] 227-230: Avoid specifying long messages outside the exception class

(TRY003)


[warning] 236-239: Avoid specifying long messages outside the exception class

(TRY003)


[warning] 539-539: Using the global statement to update _failure_info is discouraged

(PLW0603)

utilities/unittests/test_pytest_utils.py

[error] 390-390: Probable insecure usage of temporary file or directory: "/tmp"

(S108)


[warning] 393-393: Unused method argument: mock_get_base_dir

(ARG002)


[warning] 397-400: Lambda may be unnecessary; consider inlining inner function

Inline function call

(PLW0108)


[error] 421-421: Probable insecure usage of temporary file or directory: "/tmp"

(S108)


[warning] 424-424: Unused method argument: mock_get_base_dir

(ARG002)


[warning] 428-431: Lambda may be unnecessary; consider inlining inner function

Inline function call

(PLW0108)


[error] 452-452: Probable insecure usage of temporary file or directory: "/tmp"

(S108)


[warning] 455-455: Unused method argument: mock_get_base_dir

(ARG002)


[warning] 459-462: Lambda may be unnecessary; consider inlining inner function

Inline function call

(PLW0108)


[warning] 485-488: Lambda may be unnecessary; consider inlining inner function

Inline function call

(PLW0108)


[warning] 508-511: Lambda may be unnecessary; consider inlining inner function

Inline function call

(PLW0108)


[error] 2867-2867: Using xml to parse untrusted data is known to be vulnerable to XML attacks; use defusedxml equivalents

(S314)


[error] 2892-2892: Using xml to parse untrusted data is known to be vulnerable to XML attacks; use defusedxml equivalents

(S314)


[error] 2934-2934: Using xml to parse untrusted data is known to be vulnerable to XML attacks; use defusedxml equivalents

(S314)


[error] 2950-2950: Probable insecure usage of temporary file or directory: "/tmp/test"

(S108)


[error] 2970-2970: Probable insecure usage of temporary file or directory: "/tmp/test"

(S108)


[error] 3006-3006: Using xml to parse untrusted data is known to be vulnerable to XML attacks; use defusedxml equivalents

(S314)


[error] 3033-3033: Using xml to parse untrusted data is known to be vulnerable to XML attacks; use defusedxml equivalents

(S314)


[error] 3060-3060: Using xml to parse untrusted data is known to be vulnerable to XML attacks; use defusedxml equivalents

(S314)


[error] 3092-3092: Using xml to parse untrusted data is known to be vulnerable to XML attacks; use defusedxml equivalents

(S314)


[warning] 3129-3133: pytest.raises() block should contain a single simple statement

(PT012)


[warning] 3131-3131: Avoid specifying long messages outside the exception class

(TRY003)


[error] 3135-3135: Using xml to parse untrusted data is known to be vulnerable to XML attacks; use defusedxml equivalents

(S314)


[warning] 3164-3165: Use a single with statement with multiple contexts instead of nested with statements

(SIM117)

🔇 Additional comments (4)
utilities/unittests/test_pytest_utils.py (2)

3104-3140: HIGH: Exercise the real pytest_sessionfinish hook.

Lines 3104-3140 only prove local finally behavior. Invoke conftest.py::pytest_sessionfinish, force an earlier teardown operation to raise, and assert _inject_failure_junit(session=mock_session) still runs. Otherwise, moving or removing the production call would not fail this test.

with patch("conftest.shutil.rmtree", side_effect=RuntimeError("teardown failed")):
    with pytest.raises(RuntimeError, match="teardown failed"):
        pytest_sessionfinish(session=mock_session, exitstatus=1)

mock_inject.assert_called_once_with(session=mock_session)

6-13: LGTM!

Also applies to: 27-27, 276-635, 2819-3102, 3142-3171

utilities/pytest_utils.py (1)

11-24: LGTM!

Also applies to: 67-141, 202-241, 253-289, 514-518, 538-545

conftest.py (1)

54-54: LGTM!

Also applies to: 874-914


📝 Walkthrough

Walkthrough

Early pytest exits now record failure details and inject a sanitized synthetic JUnit XML error testcase during pytest_sessionfinish. Unit tests cover no-op conditions, counter updates, fallback messages, XML escaping, illegal control-character removal, teardown ordering, and atomic-write preservation.

Changes

Early-exit JUnit reporting

Layer / File(s) Summary
Failure state and JUnit injection
utilities/pytest_utils.py
Stores early-exit details, sanitizes testcase names and error text, appends a synthetic <error> testcase, updates suite counters, and atomically rewrites the configured JUnit XML.
Pytest teardown wiring and validation
conftest.py, utilities/unittests/test_pytest_utils.py
Runs failure injection from teardown regardless of earlier cleanup outcomes and tests missing configuration, XML updates, sanitization, teardown ordering, and write-failure preservation.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related PRs

Suggested labels: new-tests

Suggested reviewers: geetikakay, dshchedr, vsibirsk, ronikishner

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

@openshift-virtualization-qe-bot-2

Copy link
Copy Markdown
Contributor

Report bugs in Issues

Welcome! 🎉

This pull request will be automatically processed with the following features:

🔄 Automatic Actions

  • Reviewer Assignment: Reviewers are automatically assigned based on the OWNERS file in the repository root
  • Size Labeling: PR size labels (XS, S, M, L, XL, XXL) are automatically applied based on changes
  • Issue Creation: A tracking issue is created for this PR and will be closed when the PR is merged or closed
  • Branch Labeling: Branch-specific labels are applied to track the target branch
  • Auto-verification: Auto-verified users have their PRs automatically marked as verified
  • Labels: Enabled categories: branch, can-be-merged, cherry-pick, has-conflicts, hold, needs-rebase, size, verified, wip

📋 Available Commands

PR Status Management

  • /wip - Mark PR as work in progress (adds WIP: prefix to title)
  • /wip cancel - Remove work in progress status
  • /hold - Block PR merging (approvers only)
  • /hold cancel - Unblock PR merging
  • /verified - Mark PR as verified
  • /verified cancel - Remove verification status
  • /reprocess - Trigger complete PR workflow reprocessing (useful if webhook failed or configuration changed)
  • /regenerate-welcome - Regenerate this welcome message
  • /security-override - Set security check runs to pass (maintainers only)
  • /security-override cancel - Re-run security checks

Review & Approval

  • /lgtm - Approve changes (looks good to me)
  • /approve - Approve PR (approvers only)
  • /assign-reviewers - Assign reviewers based on OWNERS file
  • /assign-reviewer @username - Assign specific reviewer
  • /check-can-merge - Check if PR meets merge requirements

Testing & Validation

  • /retest tox - Run Python test suite with tox
  • /retest build-container - Rebuild and test container image
  • /retest verify-bugs-are-open - verify-bugs-are-open
  • /retest all - Run all available tests

Container Operations

  • /build-and-push-container - Build and push container image (tagged with PR number)
    • Supports additional build arguments: /build-and-push-container --build-arg KEY=value

Cherry-pick Operations

  • /cherry-pick <branch> - Schedule cherry-pick to target branch when PR is merged
    • Multiple branches: /cherry-pick branch1 branch2 branch3
  • /cherry-pick-retry <branch> - Retry a failed cherry-pick (merged PRs only)

Branch Management

  • /rebase - Rebase this PR branch onto its base branch

Label Management

  • /<label-name> - Add a label to the PR
  • /<label-name> cancel - Remove a label from the PR

✅ Merge Requirements

This PR will be automatically approved when the following conditions are met:

  1. Approval: /approve from at least one approver
  2. LGTM Count: Minimum 2 /lgtm from reviewers
  3. Status Checks: All required status checks must pass
  4. No Blockers: No wip, hold, has-conflicts labels and PR must be mergeable (no conflicts)
  5. Verified: PR must be marked as verified

📊 Review Process

Approvers and Reviewers

Approvers:

  • dshchedr
  • myakove
  • rnetser
  • vsibirsk

Reviewers:

  • RoniKishner
  • dshchedr
  • geetikakay
  • rnetser
  • vsibirsk
Available Labels
  • hold
  • verified
  • wip
  • lgtm
  • approve
AI Features
  • Cherry-Pick Conflict Resolution: Enabled (claude/claude-opus-4-6-1m)
Security Checks
  • Suspicious Path Detection: Monitors paths: .claude/, .vscode/, .cursor/, .devcontainer/, .pi/, .github/workflows/, .github/actions/
  • Committer Identity Check: Verifies last committer matches PR author
  • Mandatory: Security checks block merge (use /security-override to bypass — maintainers only)

💡 Tips

  • WIP Status: Use /wip when your PR is not ready for review
  • Verification: The verified label is removed on new commits unless the push is detected as a clean rebase
  • Cherry-picking: Cherry-pick labels are processed when the PR is merged
  • Container Builds: Container images are automatically tagged with the PR number
  • Permission Levels: Some commands require approver permissions
  • Auto-verified Users: Certain users have automatic verification and merge privileges

📌 Additional Information

Custom Commands:

  • /test-plan — Triggers CodeRabbit to analyze the PR's changed files and post a test execution plan
  • /rerun-smoke — Adds retest-smoke label on the PR to trigger smoke tests execution

For more information, please refer to the project documentation or contact the maintainers.

@rnetser

rnetser commented Jul 15, 2026

Copy link
Copy Markdown
Collaborator Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jul 15, 2026

Copy link
Copy Markdown
Contributor
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@rnetser

rnetser commented Jul 15, 2026

Copy link
Copy Markdown
Collaborator Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jul 15, 2026

Copy link
Copy Markdown
Contributor
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@rnetser

rnetser commented Jul 15, 2026

Copy link
Copy Markdown
Collaborator Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jul 15, 2026

Copy link
Copy Markdown
Contributor
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

coderabbitai[bot]
coderabbitai Bot previously approved these changes Jul 15, 2026
@openshift-virtualization-qe-bot-3

Copy link
Copy Markdown
Contributor

/retest all

Auto-triggered: Files in this PR were modified by merged PR #4571.

Overlapping files

conftest.py

@codecov

codecov Bot commented Jul 15, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.89%. Comparing base (a2dc143) to head (8e71bd3).
⚠️ Report is 252 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #5646      +/-   ##
==========================================
+ Coverage   98.67%   98.89%   +0.21%     
==========================================
  Files          25       42      +17     
  Lines        2487     2617     +130     
==========================================
+ Hits         2454     2588     +134     
+ Misses         33       29       -4     
Flag Coverage Δ
utilities 98.89% <100.00%> (+0.21%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

coderabbitai[bot]
coderabbitai Bot previously approved these changes Jul 16, 2026
@openshift-virtualization-qe-bot

Copy link
Copy Markdown

D/S test tox -e verify-tc-requirement-polarion failed: cnv-tests-tox-executor/30636

@rnetser

rnetser commented Jul 16, 2026

Copy link
Copy Markdown
Collaborator Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

When exit_pytest_execution aborts the session, the exit reason may not
appear in the JUnit XML report. Add _inject_failure_junit to create a
synthetic error testcase in the XML, ensuring CI systems always see the
failure.

- Record failure details in _failure_info before pytest.exit()
- Inject synthetic testcase into JUnit XML during pytest_sessionfinish
- Use atomic write (tempfile + os.replace) to preserve XML on failure
- Wrap in try/finally to guarantee injection despite teardown errors

Signed-off-by: Ruth Netser <rnetser@redhat.com>

Co-authored-by: PI (claude-opus-4-6-1m) <noreply@pi.dev>
Signed-off-by: rnetser <rnetser@redhat.com>
@rnetser

rnetser commented Jul 16, 2026

Copy link
Copy Markdown
Collaborator Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@openshift-virtualization-qe-bot

Copy link
Copy Markdown

/build-and-push-container

@openshift-virtualization-qe-bot-6

Copy link
Copy Markdown

New container for quay.io/openshift-cnv/openshift-virtualization-tests:pr-5646 published

@rnetser

rnetser commented Jul 16, 2026

Copy link
Copy Markdown
Collaborator Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@openshift-virtualization-qe-bot

Copy link
Copy Markdown

Verification failed for PR #5646.
Result: FAILURE
Job: openshift-virtualization-tests-runner #5943

Execution details
pytest -s -o log_cli=true --jira tests/network -m sriov
Image: openshift-virtualization-tests:pr-5646

@rnetser

rnetser commented Jul 16, 2026

Copy link
Copy Markdown
Collaborator Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@rnetser

rnetser commented Jul 16, 2026

Copy link
Copy Markdown
Collaborator Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@rnetser

rnetser commented Jul 16, 2026

Copy link
Copy Markdown
Collaborator Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@rnetser

rnetser commented Jul 16, 2026

Copy link
Copy Markdown
Collaborator Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@rnetser

rnetser commented Jul 16, 2026

Copy link
Copy Markdown
Collaborator Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants