Add post-test KubeVirtDeprecatedAPIRequested check#5232
Conversation
openshift-virtualization-qe-bot
added
the
tox:verify-tc-requirement-polarion:running
📝 WalkthroughWalkthroughAdds a post-test Prometheus check for ChangesPost-test critical alert checker
Estimated code review effort: 2 (Simple) | ~10 minutes Suggested labels: Suggested reviewers: Caution Pre-merge checks failedPlease resolve all errors before merging. Addressing warnings is optional.
❌ Failed checks (1 error)
✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
openshift-virtualization-qe-bot
removed
the
tox:verify-tc-requirement-polarion:running
|
D/S test |
openshift-virtualization-qe-bot
added
the
tox:verify-tc-requirement-polarion:failed
|
Report bugs in Issues Welcome! 🎉This pull request will be automatically processed with the following features: 🔄 Automatic Actions
📋 Available CommandsPR Status Management
Review & Approval
Testing & Validation
Container Operations
Cherry-pick Operations
Branch Management
Label Management
✅ Merge RequirementsThis PR will be automatically approved when the following conditions are met:
📊 Review ProcessApprovers and ReviewersApprovers:
Reviewers:
Available Labels
AI Features
Security Checks
💡 Tips
For more information, please refer to the project documentation or contact the maintainers. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@tests/post_test_alerts/test_post_test_alerts.py`:
- Around line 1-7: The module docstring in test_post_test_alerts.py contains
only the Jira key "CNV-80353" instead of a full Jira URL as required by coding
guidelines for traceability. Replace the line "Jira: CNV-80353" with a complete
URL in the format "Jira: https://issues.redhat.com/browse/CNV-80353" to provide
a proper link for coverage tracking and follow the requirement that RFE/Jira
links must be included as actual links in module, class, or test docstrings.
- Around line 48-49: The condition at line 48 only inspects the first element of
alerts_by_name using alerts_by_name[0]["state"], which means if a firing alert
exists in later entries of the list, it will be missed causing false negatives.
Fix this by using a loop or the any() function to check if ANY element in the
alerts_by_name list has state equal to "firing", rather than only checking the
first element at index 0. This ensures all alert samples are inspected to
determine if the alert is actively firing.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 2e876e14-e533-4540-9a89-ce379cad38d7
📒 Files selected for processing (3)
conftest.pytests/post_test_alerts/__init__.pytests/post_test_alerts/test_post_test_alerts.py
Qodo reviews are paused for this user.Troubleshooting steps vary by plan Learn more → On a Teams plan? Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center? |
openshift-virtualization-qe-bot
added
tox:verify-tc-requirement-polarion:running
|
/wip cancel |
|
/build-and-push-container |
|
/verified 4.22 |
Test execution plan request detailsCRITICAL: You MUST post an inline review comment on the first changed line of the first file. As an expert software testing engineer, analyze all modified files in this PR and create a targeted test execution plan. Analysis Requirements:
Output rules: Your deliverable: Test Execution Plan
Use these formats:
Real test commands (MANDATORY when changes affect session/runtime code): When the affected code runs at session/collection time (conftest fixtures, pytest plugins,
Example output for a session-startup fix: Guidelines:
Hardware-Related Checks (SR-IOV, GPU, DPDK): When PR modifies fixtures for hardware-specific resources:
CRITICAL WORKFLOW COMPLETION RULES: When responding to this test execution plan request, you MUST follow these rules EXACTLY:
ABSOLUTE PROHIBITIONS (violating these creates empty/meaningless reviews):
Remember: The pull request review is visible to users. Additional comments are redundant noise. CRITICAL — Verification rules for smoke and gating decisions: |
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
REPO="RedHatQE/openshift-virtualization-tests"
PR_NUMBER="5232"
echo "== Fetch PR files in order =="
gh api "repos/$REPO/pulls/$PR_NUMBER/files?per_page=100" > /tmp/pr_files.json
jq -r '.[].filename' /tmp/pr_files.json
FIRST_FILE=$(jq -r '.[0].filename' /tmp/pr_files.json)
echo "FIRST_FILE=$FIRST_FILE"
echo
echo "== Determine first added line in first file patch =="
PATCH=$(jq -r '.[0].patch // ""' /tmp/pr_files.json)
python - <<'PY' > /tmp/first_line.txt
import json, re, sys
from pathlib import Path
data = json.loads(Path('/tmp/pr_files.json').read_text())
patch = data[0].get('patch') or ''
current_new = None
first_added = None
for raw in patch.splitlines():
if raw.startswith('@@'):
m = re.match(r'^@@ -\d+(?:,\d+)? \+(\d+)(?:,\d+)? @@', raw)
if m:
current_new = int(m.group(1))
continue
if current_new is None:
continue
if raw.startswith('+') and not raw.startswith('+++'):
first_added = current_new
break
if raw.startswith('-') and not raw.startswith('---'):
continue
current_new += 1
if first_added is None:
raise SystemExit('Could not determine first added line in first file patch')
print(first_added)
PY
FIRST_LINE=$(cat /tmp/first_line.txt)
echo "FIRST_LINE=$FIRST_LINE"
echo
echo "== Latest head commit =="
COMMIT_SHA=$(gh pr view "$PR_NUMBER" --repo "$REPO" --json headRefOid --jq '.headRefOid')
echo "$COMMIT_SHA"
echo
echo "== Outline and inspect changed files =="
ast-grep outline "$FIRST_FILE" --view expanded || true
echo
SECOND_FILE=$(jq -r '.[1].filename // empty' /tmp/pr_files.json)
if [ -n "$SECOND_FILE" ]; then
ast-grep outline "$SECOND_FILE" --view expanded || true
fi
echo
echo "== Targeted conftest evidence =="
rg -n -C3 'post_test_alerts|skip-post-test-alerts|deprecated_api|pytest_configure|pytest_collection_modifyitems|TEAM_MARKERS|file_or_dir|_test_execution_start_time' conftest.py
echo
echo "== New test evidence =="
sed -n '1,220p' tests/post_test_alerts/test_post_test_alerts.py
echo
echo "== Smoke-marked tests (verify actual file paths) =="
rg -n --glob 'tests/**/*.py' '`@pytest`\.mark\.smoke' tests | head -100 || true
echo
echo "== Gating-marked tests (verify actual file paths) =="
rg -n --glob 'tests/**/*.py' '`@pytest`\.mark\.gating' tests | head -100 || true
echo
echo "== Search for direct references to post_test_alerts marker and new CLI option =="
rg -n -C2 'post_test_alerts|skip-post-test-alerts' tests conftest.py || true
COMMENT_FILE=/tmp/test_execution_plan.md
cat > "$COMMENT_FILE" <<'EOF'
## Test Execution Plan
**Run smoke tests: False**
No verified dependency path from any `@pytest.mark.smoke` test to the changed code. The changes are limited to shared collection/session behavior in `conftest.py` plus the new post-test alert check, and no smoke-marked test was traced to this new lane.
**Run gating tests: False**
No verified dependency path from any `@pytest.mark.gating` test to the changed code, and no gating-marked test was traced to the new post-test alert lane.
**Affected tests to run**
- `tests/post_test_alerts/test_post_test_alerts.py::test_no_critical_alerts_after_tests`
- `tests/observability/metrics/test_vms_metrics.py` — representative path to verify the auto-inclusion behavior when running a specific test path
- `tests/post_test_alerts/test_post_test_alerts.py` with `--skip-post-test-alerts` disabled/enabled to verify collection filtering in shared `conftest.py`
**Real tests (cluster required)**
Error/fix path:
```bash
pytest tests/observability/metrics/test_vms_metrics.py --skip-post-test-alerts --collect-onlyExpected: Happy path (regression): pytest tests/observability/metrics/test_vms_metrics.py --collect-onlyExpected: the requested metrics test(s) are collected and End-to-end alert verification: pytest -s -o log_cli=true -m tier2 --jira tests/observability/metrics/test_vms_metrics.pyExpected: workload tests run normally and the post-test alert check runs last, failing only if one of the configured critical CNV alerts fired during the test window. echo echo "$RESP" if [ $STATUS -ne 0 ]; then if [ $STATUS2 -ne 0 ]; then Expected: Happy path (regression): pytest tests/observability/metrics/test_vms_metrics.py --collect-onlyExpected: the requested metrics test(s) are collected and End-to-end alert verification: pytest -s -o log_cli=true -m tier2 --jira tests/observability/metrics/test_vms_metrics.pyExpected: workload tests run normally and the post-test alert check runs last, failing only if one of the configured critical CNV alerts fired during the test window. |
|
/lgtm |
|
/retest all Auto-triggered: Files in this PR were modified by merged PR #5440. Overlapping filesconftest.py |
|
/retest all Auto-triggered: Files in this PR were modified by merged PR #5124. Overlapping filesconftest.py |
| return items | ||
|
|
||
|
|
||
| def filter_post_test_alerts_tests(items: list[Item], config: Config) -> list[Item]: |
There was a problem hiding this comment.
| "VirtControllerRESTErrorsHigh", | ||
| "VirtHandlerRESTErrorsHigh", | ||
| "HCOOperatorConditionsUnhealthy", | ||
| ] |
There was a problem hiding this comment.
the request for this test was raised bacuse we saw KubeVirtDeprecatedAPIRequested alert
can you please check with each one of the sigs if they have more alerts like these one?
There was a problem hiding this comment.
ive reduced the alerts checked to just KubeVirtDeprecatedAPIRequested. after negative tests in other lanes are revised, we can add back these alerts or others
|
/retest all Auto-triggered: Files in this PR were modified by merged PR #5532. Overlapping filesconftest.py |
albarker-rh
changed the title
openshift-virtualization-qe-bot
added
tox:verify-tc-requirement-polarion:running
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@conftest.py`:
- Around line 415-417: In the configuration validation block, combine the `not
config.getoption("cnv_image")` check with the existing `upgrade_option == "eus"
or config.getoption("cnv_source") != "production"` condition into one
conditional while preserving the current `ValueError("Missing --cnv-image")`
behavior.
- Line 571: Update the type annotations in filter_post_test_alerts_tests to use
fully qualified pytest.Item and pytest.Config references, and ensure pytest is
imported as the module. Remove reliance on bare Item and Config names while
preserving the function’s behavior.
In `@tests/post_test_alerts/test_post_test_alerts.py`:
- Line 6: Append the required # <skip-jira-utils-check> marker to the Jira
traceability line in the test docstring, keeping it on the same line as the Jira
URL.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 70ee3989-d012-4e4e-8b92-70b47585dae5
📒 Files selected for processing (3)
conftest.pytests/post_test_alerts/__init__.pytests/post_test_alerts/test_post_test_alerts.py
🔗 Linked repositories identified
CodeRabbit considers these linked repositories for cross-repo context during reviews:
RedHatQE/openshift-virtualization-tests-design-docs(manual)
There was a problem hiding this comment.
Caution
Inline review comments failed to post. This is likely due to GitHub's internal server error or limits when posting large numbers of comments. If you are seeing this consistently it is likely a permissions issue. Please check "Moderation" -> "Code review limits" under your organization settings.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@conftest.py`:
- Around line 415-417: In the configuration validation block, combine the `not
config.getoption("cnv_image")` check with the existing `upgrade_option == "eus"
or config.getoption("cnv_source") != "production"` condition into one
conditional while preserving the current `ValueError("Missing --cnv-image")`
behavior.
- Line 571: Update the type annotations in filter_post_test_alerts_tests to use
fully qualified pytest.Item and pytest.Config references, and ensure pytest is
imported as the module. Remove reliance on bare Item and Config names while
preserving the function’s behavior.
In `@tests/post_test_alerts/test_post_test_alerts.py`:
- Line 6: Append the required # <skip-jira-utils-check> marker to the Jira
traceability line in the test docstring, keeping it on the same line as the Jira
URL.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 70ee3989-d012-4e4e-8b92-70b47585dae5
📒 Files selected for processing (3)
conftest.pytests/post_test_alerts/__init__.pytests/post_test_alerts/test_post_test_alerts.py
🔗 Linked repositories identified
CodeRabbit considers these linked repositories for cross-repo context during reviews:
RedHatQE/openshift-virtualization-tests-design-docs(manual)
🛑 Comments failed to post (3)
conftest.py (2)
415-417: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value
MEDIUM: Combine nested
ifstatements.Combine the nested
ifstatements into a single condition to improve readability and reduce nesting depth.🛠️ Proposed fix
- if not config.getoption("cnv_image"): - if upgrade_option == "eus" or config.getoption("cnv_source") != "production": - raise ValueError("Missing --cnv-image") + if not config.getoption("cnv_image") and ( + upgrade_option == "eus" or config.getoption("cnv_source") != "production" + ): + raise ValueError("Missing --cnv-image")📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.if not config.getoption("cnv_image") and ( upgrade_option == "eus" or config.getoption("cnv_source") != "production" ): raise ValueError("Missing --cnv-image")🧰 Tools
🪛 Ruff (0.15.21)
[warning] 415-416: Use a single
ifstatement instead of nestedifstatements(SIM102)
[warning] 417-417: Avoid specifying long messages outside the exception class
(TRY003)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@conftest.py` around lines 415 - 417, In the configuration validation block, combine the `not config.getoption("cnv_image")` check with the existing `upgrade_option == "eus" or config.getoption("cnv_source") != "production"` condition into one conditional while preserving the current `ValueError("Missing --cnv-image")` behavior.Source: Linters/SAST tools
571-571: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
HIGH: Use fully qualified
pytest.Itemandpytest.Configtypes.The
ItemandConfigtypes are referenced without thepytest.namespace. Since the repository convention dictates importingpytestdirectly rather than specific components, bareItemandConfigreferences will cause a runtimeNameErrorduring type hint evaluation.🛠️ Proposed fix
-def filter_post_test_alerts_tests(items: list[Item], config: Config) -> list[Item]: +def filter_post_test_alerts_tests(items: list[pytest.Item], config: pytest.Config) -> list[pytest.Item]:📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.def filter_post_test_alerts_tests(items: list[pytest.Item], config: pytest.Config) -> list[pytest.Item]:🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@conftest.py` at line 571, Update the type annotations in filter_post_test_alerts_tests to use fully qualified pytest.Item and pytest.Config references, and ensure pytest is imported as the module. Remove reliance on bare Item and Config names while preserving the function’s behavior.Source: Learnings
tests/post_test_alerts/test_post_test_alerts.py (1)
6-6: 📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win
HIGH: Add the required
# <skip-jira-utils-check>inline comment.The traceability guideline requires
Jira:lines in test docstrings to include# <skip-jira-utils-check>on the same line to bypass strict Jira utility validations. As per path instructions, this marker must be appended for Jira URLs.🛠️ Proposed fix
-Jira: https://redhat.atlassian.net/browse/CNV-80353 +Jira: https://redhat.atlassian.net/browse/CNV-80353 # <skip-jira-utils-check>📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.Jira: https://redhat.atlassian.net/browse/CNV-80353 # <skip-jira-utils-check>🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/post_test_alerts/test_post_test_alerts.py` at line 6, Append the required # <skip-jira-utils-check> marker to the Jira traceability line in the test docstring, keeping it on the same line as the Jira URL.Source: Path instructions
|
changed the scope of the pr to only check KubeVirtDeprecatedAPIRequested. negative tests in other lanes will fail this test by forcing alerts to be triggered. until those tests are amended, we'll just focus on KubeVirtDeprecatedAPIRequested |
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: albarker-rh <albarker@redhat.com>
Signed-off-by: albarker-rh <albarker@redhat.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #5232 +/- ##
==========================================
+ Coverage 98.67% 98.87% +0.19%
==========================================
Files 25 42 +17
Lines 2487 2567 +80
==========================================
+ Hits 2454 2538 +84
+ Misses 33 29 -4
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
D/S test |
|
/retest all |
|
/retest all Auto-triggered: Files in this PR were modified by merged PR #4571. Overlapping filesconftest.py |
Signed-off-by: albarker-rh <albarker@redhat.com> Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> # Conflicts: # utilities/unittests/test_pytest_utils.py
for more information, see https://pre-commit.ci
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@utilities/unittests/test_pytest_utils.py`:
- Around line 2556-2564: Update test_no_filtering_when_no_flags_set to configure
config.getoption with a direct constant return value instead of a side-effect
lambda; remove the unused flag parameter while preserving False for every option
lookup.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: a55af1ea-1b84-4bf4-8b40-89f9d202ca57
📒 Files selected for processing (1)
utilities/unittests/test_pytest_utils.py
🔗 Linked repositories identified
CodeRabbit considers these linked repositories for cross-repo context during reviews:
RedHatQE/openshift-virtualization-tests-design-docs(manual)
📜 Review details
⏰ Context from checks skipped due to timeout. (2)
- GitHub Check: can-be-merged
- GitHub Check: can-be-merged
⚠️ CI failures not shown inline (2)
GitHub Check: can-be-merged: Check if can be merged
Conclusion: failure
Some check runs not started: tox, security-committer-identity, security-suspicious-paths, cnv-smoke/validation, verify-bugs-are-open, verified, build-container
Missing required labels: tox:verify-tc-requirement-polarion:passed
PR has 4 unresolved review conversation(s):
- tests/post_test_alerts/test_post_test_alerts.py:None (outdated) (https://github.com/RedHatQE/openshift-virtualization-tests/pull/5232#discussion_r3432889844)
- tests/post_test_alerts/test_post_test_alerts.py:23 (https://github.com/RedHatQE/openshift-virtualization-tests/pull/5232#discussion_r3432900071)
- conftest.py:None (outdated) (https://github.com/RedHatQE/openshift-virtualization-tests/pull/5232#discussion_r3565776838)
- tests/post_test_alerts/test_post_test_alerts.py:None (outdated) (https://github.com/RedHatQE/openshift-virtualization-tests/pull/5232#discussion_r3565780331)
Missing approved from approvers: myakove, dshchedr, vsibirsk, rnetser
Missing lgtm from reviewers. Minimum 2 required, (0 given). Reviewers: geetikakay, vsibirsk, rnetser, myakove, RoniKishner, dshchedr.
GitHub Check: can-be-merged: Check if can be merged
Conclusion: failure
Some check runs not started: verify-bugs-are-open, build-container, security-committer-identity, cnv-smoke/validation, verified, security-suspicious-paths, tox
PR has 4 unresolved review conversation(s):
- tests/post_test_alerts/test_post_test_alerts.py:None (outdated) (https://github.com/RedHatQE/openshift-virtualization-tests/pull/5232#discussion_r3432889844)
- tests/post_test_alerts/test_post_test_alerts.py:23 (https://github.com/RedHatQE/openshift-virtualization-tests/pull/5232#discussion_r3432900071)
- conftest.py:None (outdated) (https://github.com/RedHatQE/openshift-virtualization-tests/pull/5232#discussion_r3565776838)
- tests/post_test_alerts/test_post_test_alerts.py:None (outdated) (https://github.com/RedHatQE/openshift-virtualization-tests/pull/5232#discussion_r3565780331)
Missing approved from approvers: dshchedr, rnetser, vsibirsk, myakove
Missing lgtm from reviewers. Minimum 2 required, (0 given). Reviewers: rnetser, geetikakay, dshchedr, myakove, RoniKishner, vsibirsk.
🧰 Additional context used
📓 Path-based instructions (5)
**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
**/*.py: Never add linter suppressions such as# noqa,# type: ignore, or# pylint: disable; fix the underlying code instead.
Before adding code, search for existing implementations in the repository, especiallyutilities/,libs/,tests/, and project dependencies; reuse shared logic and never duplicate it.
UseTYPE_CHECKINGfor imports needed only by type annotations.
Use Google-format docstrings for public functions with non-obvious return values or side effects.
Use absolute imports, keep imports at module scope, and prefer specificfrom module import nameimports; avoid bareimport moduleunless retaining the module name materially improves readability.
Use named arguments for function calls with more than one argument and avoid single-letter variable names.
Do not add dead code, and prefer direct attribute access unless storing an attribute improves reuse or clarity.
Do not define helper functions, utility functions, or classes inconftest.pyortest_*.py; place them in dedicated utility modules.conftest.pyfiles should contain fixtures only, except the root file, which is reserved for hooks and plugin registration.
Use context managers for files, connections, and locks; re-raise exceptions with context, catch specific exception types, never silently swallow exceptions, and provide specific error messages.
Do not useasyncor async/await; the codebase uses synchronous patterns.
Log transitions, resource operations, configuration changes, responses, and state at INFO; unusual or skipped operations at WARNING; contextual exceptions at ERROR. Never use DEBUG or log secrets, tokens, passwords, or PII, and use f-string formatting.
Do not add defensive checks for guarantees established by types, constructors, schemas, or pinned dependencies; allowed exceptions are cleanup, explicit optional values, lazy initialization, platform constants, and unstable external libraries. Useisinstancerather thanhasattrfor type discrimination.
Funct...
Files:
utilities/unittests/test_pytest_utils.py
**/*.{py,pyi}
📄 CodeRabbit inference engine (AGENTS.md)
Use type hints for all new public functions under
utilities/;libs/must satisfy mypy strict mode.
Files:
utilities/unittests/test_pytest_utils.py
utilities/**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
Place utility functions in the domain-matching module under
utilities/; create a domain module only when no suitable module exists.
Files:
utilities/unittests/test_pytest_utils.py
utilities/unittests/**
⚙️ CodeRabbit configuration file
utilities/unittests/**: ## Unit Tests — Relaxed Rules
Files underutilities/unittests/are unit tests for shared utilities.
Do NOT enforce:
- STD docstring format (Preconditions/Steps/Expected sections)
- STP link requirements
- RFE/Jira link requirements
- Coverage tracking rules
These rules apply only to feature/integration tests undertests/.
DO still enforce: code quality, type hints, naming conventions, and all other coding standards.
Files:
utilities/unittests/test_pytest_utils.py
**
⚙️ CodeRabbit configuration file
**: ## PR Template Validation
Check the PR description for required sections from.github/pull_request_template.md.
Required sections (must be present, even if empty):
##### What this PR does / why we need it:— MUST be present AND have meaningful content.
Flag as HIGH if the section is missing, empty, whitespace-only, contains only HTML comments,
or contains only placeholder tokens such asTBD,TBA,N/A,-,—,none, or..##### Which issue(s) this PR fixes:— must be present (may be empty)##### Special notes for reviewer:— must be present (may be empty)##### jira-ticket:— must be present (may be empty)
If any required section is absent, orWhat this PR does / why we need it:has no content,
flag it as HIGH severity and ask the author to restore the missing template section(s).Approval Policy
You may approve the PR when ALL of the following are true:
- All your review comments have been addressed with either:
- a code/doc change that fixes the issue, or
- a substantive author response that justifies no code change.
Thread "resolved" state alone is not sufficient.
OR you had no review comments.- If you posted a test execution plan comment requesting tests, and the PR author replied
with a comment explaining why the requested tests are not needed or were already covered,
treat that as an acceptable response — do not block approval on the test plan alone.- The author's explanation must be reasonable and specific (not just "N/A" or "not needed").
Accept explanations like: "these tests were already run in CI", "this change is docs-only",
"the affected tests are quarantined", or "verified manually on cluster X".
Files:
utilities/unittests/test_pytest_utils.py
🧠 Learnings (37)
📚 Learning: 2026-01-07T11:51:57.253Z
Learnt from: qwang1
Repo: RedHatQE/openshift-virtualization-tests PR: 3301
File: utilities/unittests/test_oadp.py:5-5
Timestamp: 2026-01-07T11:51:57.253Z
Learning: In utilities/unittests/test_oadp.py and similar test files in this repo, include a # flake8: noqa: E402 directive to suppress E402 warnings caused by pre-commit checks. This is needed because mock setup must precede imports to avoid circular dependencies, and Ruff may report unused imports. Apply this directive to all similar test files that require imports to guard against circular import issues.
Applied to files:
utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-01-12T11:24:13.825Z
Learnt from: servolkov
Repo: RedHatQE/openshift-virtualization-tests PR: 3387
File: tests/network/provider_migration/libprovider.py:50-52
Timestamp: 2026-01-12T11:24:13.825Z
Learning: In the RedHatQE/openshift-virtualization-tests repository, when catching exceptions in Python, use LOGGER.error before re-raising and do not replace it with LOGGER.exception in except blocks. This follows the established pattern across the codebase.
Applied to files:
utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-01-12T14:25:05.723Z
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 3366
File: tests/storage/cdi_clone/test_clone.py:5-9
Timestamp: 2026-01-12T14:25:05.723Z
Learning: In Python tests and utility code across the repository, bitmath.parse_string_unsafe correctly parses Kubernetes quantities (e.g., '4Gi', '512Mi', PVC storage requests) without supplying system=bitmath.NIST. There are 30+ usages indicating this is the standard behavior. Reviewers should verify that code that builds or compares quantity strings does not pass the NIST parameter, and if a new test relies on quantity parsing, assume no NIST parameter is required unless explicitly documented.
Applied to files:
utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-01-20T01:03:13.139Z
Learnt from: servolkov
Repo: RedHatQE/openshift-virtualization-tests PR: 3387
File: tests/network/provider_migration/libprovider.py:1-8
Timestamp: 2026-01-20T01:03:13.139Z
Learning: In the openshift-virtualization-tests repository, Python imports should consistently use module-level imports for the logging module (i.e., import logging) rather than from logging import ... The established pattern spans 270+ files and should not be flagged for refactoring. Apply this guideline to Python files across the repo (e.g., tests/network/provider_migration/libprovider.py).
Applied to files:
utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-01-21T21:26:41.805Z
Learnt from: geetikakay
Repo: RedHatQE/openshift-virtualization-tests PR: 3559
File: utilities/infra.py:251-254
Timestamp: 2026-01-21T21:26:41.805Z
Learning: In the RedHatQE/openshift-virtualization-tests repository, when reviewing Python code, recognize that with Python 3.14 the syntax 'except ValueError, TypeError:' is valid if there is no 'as' clause, and should not be flagged as Python 2 syntax. If you use an 'as' binding (e.g., 'except (ValueError, TypeError) as e:'), parentheses are required. Ensure this pattern is version-consistent and not flagged as Python 2 syntax when 'as' is absent.
Applied to files:
utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-01-25T13:18:21.675Z
Learnt from: jpeimer
Repo: RedHatQE/openshift-virtualization-tests PR: 3571
File: tests/storage/storage_migration/utils.py:158-167
Timestamp: 2026-01-25T13:18:21.675Z
Learning: In reviews of the openshift-virtualization-tests repo (and similar Python code), avoid suggesting minor stylistic changes that require extra verification (e.g., removing dict.keys() checks for membership) unless the change has clear correctness or maintainability impact. Focus on fixes with observable behavior, security, performance, or maintainability benefits; defer low-impact style tweaks that are costly to verify.
Applied to files:
utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-02-18T06:35:39.536Z
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 3847
File: utilities/virt.py:2449-2453
Timestamp: 2026-02-18T06:35:39.536Z
Learning: In Python code, a function named clearly and self-descriptively can be deemed not to require a docstring. However, treat this as a context-specific guideline and not a universal rule. For public APIs or functions with side effects, prefer concise docstrings explaining behavior, inputs, outputs, and side effects. This guidance is based on the example in utilities/virt.py from RedHatQE/openshift-virtualization-tests where validate_libvirt_persistent_domain(vm, admin_client) was considered self-documenting.
Applied to files:
utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-02-23T16:33:22.070Z
Learnt from: vsibirsk
Repo: RedHatQE/openshift-virtualization-tests PR: 3883
File: utilities/pytest_utils.py:441-463
Timestamp: 2026-02-23T16:33:22.070Z
Learning: In Python code reviews, the guideline to always use named arguments for multi-argument calls does not apply to built-ins or methods that have positional-only parameters (those defined with a / in their signature). Do not flag or require named arguments for calls like dict.get(key, default=None, /), list.pop(), str.split(sep, maxsplit) and similar built-ins that cannot accept keyword arguments. Apply the named-argument rule only to functions/methods that explicitly accept keyword arguments.
Applied to files:
utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-03-17T01:32:02.617Z
Learnt from: dshchedr
Repo: RedHatQE/openshift-virtualization-tests PR: 4118
File: utilities/database.py:0-0
Timestamp: 2026-03-17T01:32:02.617Z
Learning: In RedHatQE/openshift-virtualization-tests, when reviewing Python files, post targeted inline comments on the Files changed tab at the exact location (file and line) of the issue rather than opening a single discussion thread for multiple issues. This should be done for each applicable location to improve traceability and clarity. If multiple issues exist in the same file, address them with separate inline comments pointing to the specific lines.
Applied to files:
utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-03-17T01:32:02.617Z
Learnt from: dshchedr
Repo: RedHatQE/openshift-virtualization-tests PR: 4118
File: utilities/database.py:0-0
Timestamp: 2026-03-17T01:32:02.617Z
Learning: In the RedHatQE/openshift-virtualization-tests repository, CodeRabbit should post targeted inline comments at each applicable location in the Files Changed tab, rather than aggregating multiple issues into a single PR discussion thread reply. This guideline applies to all Python files (any file ending in .py) changed in a PR; for non-Python files, follow the same inline-comment-at-location principle if relevant.
Applied to files:
utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-05-04T13:45:29.122Z
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 4725
File: utilities/console.py:54-59
Timestamp: 2026-05-04T13:45:29.122Z
Learning: During review of RedHatQE/openshift-virtualization-tests “lint-cleanup” PRs (e.g., changes targeting lint issues like stale noqa/utf-8 headers), do not flag existing `# type: ignore` directives that were already present before the PR and were not introduced or modified by the PR. Only raise findings for `# type: ignore` suppressions that the PR itself adds, changes, or otherwise makes newly effective (i.e., they appear in the diff as additions/edits).
Applied to files:
utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-05-04T13:45:33.892Z
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 4725
File: tests/virt/cluster/common_templates/centos/test_centos_os_support.py:78-83
Timestamp: 2026-05-04T13:45:33.892Z
Learning: When reviewing lint-cleanup or formatting-only pull requests in this repo (e.g., changes like removing/updating `# noqa` comments or UTF-8 headers), do not raise findings for code patterns that already existed before the PR. Specifically, if a problematic construct such as `.is_connective(tcp_timeout=120)` was present in the base branch, suppress that finding and only raise issues when the PR itself introduces or modifies that construct (i.e., the diff adds/changes the call or its arguments). Apply this rule across all Python files (`**/*.py`).
Applied to files:
utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-05-05T17:01:15.294Z
Learnt from: dshchedr
Repo: RedHatQE/openshift-virtualization-tests PR: 4739
File: tests/virt/node/descheduler/conftest.py:2-2
Timestamp: 2026-05-05T17:01:15.294Z
Learning: In this repo’s Python code, it’s acceptable (and preferred by convention) to build `run_command` inputs using `shlex.split(f"<command> {arg}")` rather than converting to direct list literals like `['oc', 'adm', 'uncordon', name]`. During code review, generally don’t flag `shlex.split(...)` usage for `run_command` calls and don’t suggest replacing it with list literals; the string-form pattern is used to keep commands readable and consistent with how they’re typed in a terminal.
Applied to files:
utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-05-08T12:49:20.694Z
Learnt from: geetikakay
Repo: RedHatQE/openshift-virtualization-tests PR: 4788
File: utilities/os_utils.py:257-262
Timestamp: 2026-05-08T12:49:20.694Z
Learning: In RedHatQE/openshift-virtualization-tests, the Ruff flake8-boolean-trap rules FBT001/FBT002 are intentionally not enabled (pyproject.toml does not select the FBT rules; confirmed via `ruff check --show-settings`). Therefore, do not flag boolean positional parameters as FBT001/FBT002 violations in this repository. If Ruff configuration changes and starts selecting FBT rules, this exception should be reconsidered.
Applied to files:
utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-05-12T05:10:24.601Z
Learnt from: acinko-rh
Repo: RedHatQE/openshift-virtualization-tests PR: 4780
File: tests/storage/utils.py:568-572
Timestamp: 2026-05-12T05:10:24.601Z
Learning: In this repository, Ruff rule UP043 ("unnecessary default type arguments") is enforced. When annotating `collections.abc.Generator` return types, prefer the single-parameter form `Generator[YieldType]` rather than `Generator[YieldType, None, None]`. Explicit `None, None` for the SendType and ReturnType are unnecessary defaults (per PEP 696) and will trigger UP043. Apply this consistently across all Python files.
Applied to files:
utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-05-13T19:23:09.603Z
Learnt from: Anatw
Repo: RedHatQE/openshift-virtualization-tests PR: 4833
File: tests/network/localnet/migration_stuntime/libstuntime.py:25-25
Timestamp: 2026-05-13T19:23:09.603Z
Learning: In this repository, do not recommend adding `from __future__ import annotations` to fix forward-reference type annotation issues (e.g., Ruff UP037). Follow the established convention: use quoted string type annotations for forward references when the referenced class/type is defined later in the same file (e.g., `"ContinuousPing"`), and prefer `typing.Self` for self-referential return types.
Applied to files:
utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-05-18T06:30:56.781Z
Learnt from: EdDev
Repo: RedHatQE/openshift-virtualization-tests PR: 4819
File: utilities/unittests/test_bitwarden.py:207-207
Timestamp: 2026-05-18T06:30:56.781Z
Learning: During Ruff/lint rule-enablement PRs in this repository (e.g., when introducing a new rule like PLC0415), it’s acceptable to keep CI green by adding per-line, targeted suppressions for pre-existing violations: add only `# noqa: <single-ruff-rule-id>` at the end of the specific violating line. In this PR context, reviewers should NOT flag these targeted `# noqa: PLC0415` comments as policy violations, assuming the suppression is for a pre-existing issue and is documented in the PR description as a candidate for follow-up cleanup. Do not allow blanket `# noqa` (without a specific rule) or `per-file-ignores`; those remain disallowed.
Applied to files:
utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-05-18T06:31:12.015Z
Learnt from: EdDev
Repo: RedHatQE/openshift-virtualization-tests PR: 4819
File: utilities/unittests/test_pytest_utils.py:270-270
Timestamp: 2026-05-18T06:31:12.015Z
Learning: In RedHatQE/openshift-virtualization-tests, if a PR is a Ruff rule-enforcement PR and its “Special notes for reviewer” documents that pre-existing Ruff violations are being temporarily handled via per-line suppressions (e.g., `# noqa: PLC0415`) to keep CI green, reviewers should treat those specific `# noqa: <rule>` comments as an agreed, temporary mechanism. Do not flag them as code-quality issues and do not recommend removing, consolidating, or refactoring those suppressions within the same PR; cleanup/remediation is expected to happen in dedicated follow-up PRs instead.
Applied to files:
utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-05-18T06:31:15.083Z
Learnt from: EdDev
Repo: RedHatQE/openshift-virtualization-tests PR: 4819
File: utilities/unittests/test_data_collector.py:304-304
Timestamp: 2026-05-18T06:31:15.083Z
Learning: When reviewing Python code in this repository for Ruff/linter rule rollouts, do not treat temporary suppression comments as violations in the specific migration scenario where a PR enables a new Ruff rule (e.g., PLC0415) and the PR description explicitly documents that all *pre-existing* violations are being annotated with `# noqa: <RULE>` as a short-lived measure. In that case, only flag `# noqa: <RULE>` suppressions that are newly introduced on code that did not previously violate the rule—i.e., verify via the PR diff against the prior state (and/or prior Ruff findings) that the suppressed line was already violating before the rule was enabled. Ignore suppressions that are covering violations that existed before the new rule rollout and were intentionally bulk-added for cleanup in follow-up PRs.
Applied to files:
utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-05-18T06:31:20.848Z
Learnt from: EdDev
Repo: RedHatQE/openshift-virtualization-tests PR: 4819
File: utilities/unittests/test_hco.py:501-501
Timestamp: 2026-05-18T06:31:20.848Z
Learning: When reviewing Python code in RedHatQE/openshift-virtualization-tests, avoid flagging Ruff `# noqa: <RULE>` suppressions as issues if they were intentionally added as a temporary measure to keep CI green after a PR enables a new Ruff/lint rule (e.g., PLC0415) and the PR description documents this under "Special notes for reviewer". Treat these suppressions as deferred technical debt. Only flag `# noqa: PLC0415` (and similar rule-specific suppressions) when they are newly introduced without an accompanying documented intent in the PR (and thus appear to be masking a new violation rather than a pre-existing one).
Applied to files:
utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-05-18T09:09:09.479Z
Learnt from: EdDev
Repo: RedHatQE/openshift-virtualization-tests PR: 4878
File: utilities/unittests/test_pytest_utils.py:2194-2197
Timestamp: 2026-05-18T09:09:09.479Z
Learning: In this repository (RedHatQE/openshift-virtualization-tests), do not flag missing return type annotations or missing argument type annotations as Ruff “ANN” rule violations (e.g., ANN001/ANN002/ANN201/ANN202). The repo’s Ruff configuration does not enable ANN rules and only uses `extend-select = ["PLC0415"]`, so missing type annotations should not be treated as ANN lint failures during code review.
Applied to files:
utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-05-19T07:48:17.119Z
Learnt from: azhivovk
Repo: RedHatQE/openshift-virtualization-tests PR: 4784
File: libs/vm/affinity.py:104-104
Timestamp: 2026-05-19T07:48:17.119Z
Learning: When using Kubernetes API models like `NodeSelectorRequirement` or `LabelSelectorRequirement` with operators `Exists` or `DoesNotExist`, the `values` field must not be non-empty. It is valid for `values` to be omitted / left as `None` (Python) / passed as `null`—Kubernetes rejects non-empty `values` for these operators, but does not require the field to be present or explicitly set to an empty list. In code reviews, do not treat missing `values=[]` for `Exists`/`DoesNotExist` as a validation issue; only flag cases where `values` is provided with actual elements.
Applied to files:
utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-05-19T07:48:17.119Z
Learnt from: azhivovk
Repo: RedHatQE/openshift-virtualization-tests PR: 4784
File: libs/vm/affinity.py:104-104
Timestamp: 2026-05-19T07:48:17.119Z
Learning: When constructing Kubernetes `NodeSelectorRequirement` (or `LabelSelectorRequirement`) objects in code, do not treat `values` being omitted, `None`, or an empty list as an API-validation problem when the requirement’s operator is `Exists` or `DoesNotExist`. Per the Kubernetes API spec, these operators only require that the `values` array is not non-empty (i.e., it must be empty); they do not require the field to be explicitly present as `[]`. Therefore, reviewers should not flag `values=None`/missing `values` for `Exists`/`DoesNotExist`.
Applied to files:
utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-05-26T15:52:31.613Z
Learnt from: rlobillo
Repo: RedHatQE/openshift-virtualization-tests PR: 4983
File: utilities/hco.py:376-378
Timestamp: 2026-05-26T15:52:31.613Z
Learning: For Python files in this repo, don’t raise review findings for missing type hints or missing/Google-style docstrings on an existing function when the PR’s only functional change is adding one or more new parameters to that function and the PR does not otherwise refactor or substantially rewrite its body/signature. Treat type-annotation/docstring improvements as out of scope for focused parameter-add PRs and defer them to a follow-up. Only raise missing type-hint or docstring issues when the PR introduces an entirely new function or substantially rewrites an existing one.
Applied to files:
utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-06-21T20:28:07.727Z
Learnt from: Anatw
Repo: RedHatQE/openshift-virtualization-tests PR: 5283
File: tests/network/libs/service.py:14-16
Timestamp: 2026-06-21T20:28:07.727Z
Learning: When reviewing Python code in this repository targeting Python 3.14 with PEP 649 (deferred annotation evaluation using descriptors) enabled by default, do not flag annotations that reference types imported only under `if TYPE_CHECKING:` (e.g., `client: DynamicClient | None = None`) as potential runtime `NameError` problems. With PEP 649 enabled, annotations are not evaluated at function definition time, so these patterns are valid without `from __future__ import annotations`.
Applied to files:
utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-07-09T12:58:50.301Z
Learnt from: yossisegev
Repo: RedHatQE/openshift-virtualization-tests PR: 5578
File: tests/network/l2_bridge/libl2bridge.py:9-9
Timestamp: 2026-07-09T12:58:50.301Z
Learning: In this repo, follow the established convention for type-only imports: do not require moving `ResourceField` imports from `kubernetes.dynamic.client` behind an `if TYPE_CHECKING:` guard when the imported symbol is used only for type annotations (e.g., in function/class annotations) and not referenced at runtime. Only raise/flag this if `ResourceField` is actually used in executable code (runtime values/calls), since then the import may need to be handled differently.
Applied to files:
utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-01-19T07:53:55.780Z
Learnt from: qwang1
Repo: RedHatQE/openshift-virtualization-tests PR: 3301
File: utilities/unittests/test_oadp.py:48-57
Timestamp: 2026-01-19T07:53:55.780Z
Learning: Enforce that pytest markers (e.g., marker requirement for integration tests) apply only to integration/functional tests under the tests/ directory. Unit tests located in utilities/unittests/ must not use pytest markers, following repository convention. For all unit test files under utilities/unittests (and subdirectories), do not require or mandate markers; for integration tests under tests/ and its subdirectories, ensure markers are used as per the project rule.
Applied to files:
utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-02-18T06:34:38.042Z
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 3847
File: tests/virt/cluster/common_templates/utils.py:58-58
Timestamp: 2026-02-18T06:34:38.042Z
Learning: In RedHatQE/openshift-virtualization-tests, treat 'public' functions as those defined in any Python files under libs/ or utilities/ (any depth). Functions inside nested test directories (e.g., tests/virt/cluster/common_templates/, tests/virt/node/, etc.) are test helpers and do not require Google-format docstrings unless explicitly requested. Use this rule during reviews to decide whether to enforce docstrings on public API functions in libs/utilities.
Applied to files:
utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-02-23T21:16:51.920Z
Learnt from: vsibirsk
Repo: RedHatQE/openshift-virtualization-tests PR: 3883
File: utilities/unittests/test_os_utils.py:358-371
Timestamp: 2026-02-23T21:16:51.920Z
Learning: In the openshift-virtualization-tests repository, the guideline to require named arguments for function calls with more than one argument does not apply to unit tests under utilities/unittests/. For these tests, allow standard pytest patterns like pytest.raises(ExceptionType, match="pattern") without requiring named parameters. This exception applies specifically to files under utilities/unittests/ and does not generalize beyond that directory.
Applied to files:
utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-05-03T15:38:22.954Z
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 4701
File: utilities/unittests/test_database.py:12-12
Timestamp: 2026-05-03T15:38:22.954Z
Learning: In RedHatQE/openshift-virtualization-tests unit tests under `utilities/unittests/`, it’s intentional/pre-existing to use `sys.path.insert(0, ...)` in cases where tests need to import local modules. In lint-cleanup PRs, reviewers should not flag this `sys.path` manipulation as a violation. Also, don’t re-add `# noqa: E402` or other import-order suppressions if `ruff/flake8` no longer reports E402 for that import—handle any broader refactor (to remove the `sys.path` hack) in a dedicated follow-up.
Applied to files:
utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-05-18T06:31:30.830Z
Learnt from: EdDev
Repo: RedHatQE/openshift-virtualization-tests PR: 4819
File: utilities/unittests/test_sanity.py:22-22
Timestamp: 2026-05-18T06:31:30.830Z
Learning: In this repo’s unit test files under `utilities/unittests/`, treat `# noqa: PLC0415` (import-outside-top-level) suppressions as intentional and do not flag them as violations when enabling Ruff’s PLC0415 rule incrementally. These suppressions are expected as temporary measures and are documented in the PR description; they should only be targeted for later cleanup follow-ups.
Also, allow imports placed inside unit test methods under `utilities/unittests/` when they are intentionally positioned to run after `patch` decorators are applied—especially when the imported module has import-time side effects.
Applied to files:
utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-07-01T15:15:37.516Z
Learnt from: jpeimer
Repo: RedHatQE/openshift-virtualization-tests PR: 5443
File: utilities/unittests/test_storage.py:14-19
Timestamp: 2026-07-01T15:15:37.516Z
Learning: In this repository, within `utilities/unittests/**`, it is acceptable to use the established convention `module_obj.submodule = value # type: ignore[attr-defined]` (e.g., `utilities.storage = utilities.storage`, `utilities.hco = mock_hco`, `utilities.virt = mock_virt`) when reassigning/reloading submodules as attributes on the `utilities` package for test mocking/reload purposes. These assignments are intentionally suppressed for mypy’s `attr-defined` warning, and equivalent rebinding via local variables does not provide the required reload/rebind semantics for these tests. Code review should not treat this specific `# type: ignore[attr-defined]` pattern in `utilities/unittests/**/*.py` as a new linter/type-suppression violation.
Applied to files:
utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-05-03T15:38:09.624Z
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 4701
File: tests/virt/node/general/test_windows_vtpm_bitlocker.py:50-52
Timestamp: 2026-05-03T15:38:09.624Z
Learning: During review of PRs that are lint cleanups or tooling/version bumps, do not flag code-quality issues for patterns that pre-existed before the PR. Specifically, if the diff does not introduce/modify constructs such as nested `if` blocks or unnecessary list comprehensions, treat them as known/deferred and leave them for dedicated follow-up cleanup PRs. Only raise issues when the PR itself adds, changes, or refactors the problematic code.
Applied to files:
utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-05-15T18:42:02.504Z
Learnt from: geetikakay
Repo: RedHatQE/openshift-virtualization-tests PR: 4860
File: utilities/hco.py:385-389
Timestamp: 2026-05-15T18:42:02.504Z
Learning: In this repository, under the Python `utilities/` directory (utility/helper modules, not tests), do not flag bare `assert` statements as correctness or style issues. The codebase conventionally uses `assert` in these utilities (e.g., `utilities/virt.py`, `utilities/infra.py`, etc.) and does not enforce running Python with `-O`/`PYTHONOPTIMIZE`, so the usual “asserts may be stripped” concern should not be treated as a review blocker here.
Applied to files:
utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-05-18T10:38:33.820Z
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 4884
File: .coderabbit.yaml:79-80
Timestamp: 2026-05-18T10:38:33.820Z
Learning: In this repo, pytest tests are expected to live only in files named `test_*.py`. When applying CodeRabbit STP case B (newly added `def test_*` inside an existing test file), scope the check to existing files that match `test_*.py` and do not broaden it to newly added `def test_*` found in other non-`test_*.py` Python files.
Applied to files:
utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-06-01T06:43:07.271Z
Learnt from: rnetser
Repo: RedHatQE/openshift-virtualization-tests PR: 4908
File: utilities/unittests/test_pytest_utils.py:374-374
Timestamp: 2026-06-01T06:43:07.271Z
Learning: In RedHatQE/openshift-virtualization-tests, do not require STP/RFE/Jira traceability links in unit test function docstrings for any `test_*.py` file under `utilities/unittests/` (including subdirectories). Only integration/functional tests under the repository’s `tests/` directory should be checked for STP/RFE/Jira traceability links; therefore, missing STP/RFE/Jira docstring links in `utilities/unittests/**/test_*.py` should not be flagged.
Applied to files:
utilities/unittests/test_pytest_utils.py
📚 Learning: 2026-06-15T10:56:21.758Z
Learnt from: vsibirsk
Repo: RedHatQE/openshift-virtualization-tests PR: 4994
File: tests/network/user_defined_network/ip_specification/test_ip_specification.py:123-127
Timestamp: 2026-06-15T10:56:21.758Z
Learning: In this repository, do not require or flag missing `-> None` return type annotations on pytest test functions/methods (i.e., functions named `test_*`) located under `tests/**`. Return type annotations for `-> None` on these test functions are optional and should not be treated as inconsistent. Separately, in `utilities/**` and `libs/**`, enforce return type annotations for non-test public functions (e.g., functions that are not internal/private such as those not starting with `_`).
Applied to files:
utilities/unittests/test_pytest_utils.py
🪛 Ruff (0.15.21)
utilities/unittests/test_pytest_utils.py
[warning] 2564-2564: Unused lambda argument: flag
(ARG005)
🔇 Additional comments (3)
utilities/unittests/test_pytest_utils.py (3)
32-32: LGTM!
2497-2554: LGTM!
2566-2569: LGTM!
| def test_no_filtering_when_no_flags_set(self): | ||
| """All items are returned unchanged when no filtering flags are set.""" | ||
| item_post_test_alerts = MagicMock() | ||
| item_post_test_alerts.keywords = {"post_test_alerts": True} | ||
| item_other = MagicMock() | ||
| item_other.keywords = {"other_test": True} | ||
| items = [item_post_test_alerts, item_other] | ||
| config = MagicMock() | ||
| config.getoption.side_effect = lambda flag: False |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Avoid the unused lambda parameter.
Ruff reports ARG005 for flag. Configure the mock’s return value directly instead of using a lambda.
Proposed fix
- config.getoption.side_effect = lambda flag: False
+ config.getoption.return_value = False📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| def test_no_filtering_when_no_flags_set(self): | |
| """All items are returned unchanged when no filtering flags are set.""" | |
| item_post_test_alerts = MagicMock() | |
| item_post_test_alerts.keywords = {"post_test_alerts": True} | |
| item_other = MagicMock() | |
| item_other.keywords = {"other_test": True} | |
| items = [item_post_test_alerts, item_other] | |
| config = MagicMock() | |
| config.getoption.side_effect = lambda flag: False | |
| def test_no_filtering_when_no_flags_set(self): | |
| """All items are returned unchanged when no filtering flags are set.""" | |
| item_post_test_alerts = MagicMock() | |
| item_post_test_alerts.keywords = {"post_test_alerts": True} | |
| item_other = MagicMock() | |
| item_other.keywords = {"other_test": True} | |
| items = [item_post_test_alerts, item_other] | |
| config = MagicMock() | |
| config.getoption.return_value = False |
🧰 Tools
🪛 Ruff (0.15.21)
[warning] 2564-2564: Unused lambda argument: flag
(ARG005)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@utilities/unittests/test_pytest_utils.py` around lines 2556 - 2564, Update
test_no_filtering_when_no_flags_set to configure config.getoption with a direct
constant return value instead of a side-effect lambda; remove the unused flag
parameter while preserving False for every option lookup.
Source: Linters/SAST tools
|
|
||
|
|
||
| def filter_post_test_alerts_tests(items: list[Item], config: Config) -> list[Item]: | ||
| # filter out post test alerts tests, if explicitly asked or if running upgrade/install tests |
There was a problem hiding this comment.
please use google docstring formatting
| or config.getoption("--upgrade") | ||
| or config.getoption("--upgrade_custom") |
There was a problem hiding this comment.
@hmeir you mentioned that upgrade tests should not be skipped, right?
|
/retest all Auto-triggered: Files in this PR were modified by merged PR #4908. Overlapping filesutilities/pytest_utils.py |
10 participants
What this PR does / why we need it:
Verifies that the KubeVirtDeprecatedAPIRequested alert was not triggered during test execution.
Which issue(s) this PR fixes:
We don't check for the above critical alerts firing during test execution
Special notes for reviewer:
jira-ticket:
https://redhat.atlassian.net/browse/CNV-80353
Summary by CodeRabbit
New Features
post_test_alertstest lane and a--skip-post-test-alertsflag to omit these checks.Test Improvements
KubeVirtDeprecatedAPIRequestedis still firing after tests complete.Test Utilities
post_test_alertsitems when the skip/install/upgrade modes are active.