Skip to content

fix(kserve): use llmisvc_core marker for e2e test selection#82043

Merged
openshift-merge-bot[bot] merged 2 commits into
openshift:mainfrom
jlost:fix-kserve-autoscaling-marker-v2
Jul 17, 2026
Merged

fix(kserve): use llmisvc_core marker for e2e test selection#82043
openshift-merge-bot[bot] merged 2 commits into
openshift:mainfrom
jlost:fix-kserve-autoscaling-marker-v2

Conversation

@jlost

@jlost jlost commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Switch kserve e2e-llm-inference-service Prow job from negative not autoscaling filter to positive llmisvc_core marker
  • The upstream marker refactor (kserve#5771) replaced @pytest.mark.autoscaling with auto-assigned autoscaling_hpa/autoscaling_keda from filenames, so not autoscaling no longer excludes them
  • llmisvc_core is auto-assigned by conftest to non-autoscaling, non-tracing files, correctly scoping the job
  • Applied to both master and release-v0.17 branches

Test plan

  • Prow rehearsal e2e-llm-inference-service no longer runs autoscaling tests
  • Core llmisvc tests still selected and pass

Summary by CodeRabbit

Updates KServe’s e2e-llm-inference-service Prow jobs to change the run-e2e-tests.sh test-selection filter on both master and release-v0.17. The jobs now run:

  • "llmisvc_core and cluster_cpu and not pvc_storage" (count: 2, suite label: llm-d)

This switches selection to the positive llmisvc_core marker (which targets core/non-autoscaling/non-tracing LLM inference service tests) while continuing to explicitly exclude pvc_storage tests.

The upstream marker refactor (kserve#5771) replaced @pytest.mark.autoscaling
with auto-assigned llmisvc_autoscaling from the filename. The old
"not autoscaling" negative filter no longer excludes autoscaling tests,
causing them to run without WVA installed and fail.

Switch to the positive llmisvc_core marker which conftest auto-assigns
to non-autoscaling, non-tracing test files. This correctly excludes
autoscaling_hpa, autoscaling_keda, and tracing tests. pvc_storage still
needs explicit exclusion as those tests live inside llmisvc_core files.

Signed-off-by: James Ostrander <jostrand@redhat.com>
@coderabbitai

coderabbitai Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 5c7406c9-15d2-4630-85b7-431b34676443

📥 Commits

Reviewing files that changed from the base of the PR and between 4a2a9e0 and 49c0da5.

📒 Files selected for processing (2)
  • ci-operator/config/opendatahub-io/kserve/opendatahub-io-kserve-master.yaml
  • ci-operator/config/opendatahub-io/kserve/opendatahub-io-kserve-release-v0.17.yaml
🚧 Files skipped from review as they are similar to previous changes (2)
  • ci-operator/config/opendatahub-io/kserve/opendatahub-io-kserve-release-v0.17.yaml
  • ci-operator/config/opendatahub-io/kserve/opendatahub-io-kserve-master.yaml

Walkthrough

KServe CI configurations now select llmisvc_core E2E tests for LLM inference services while retaining the pvc_storage exclusion. The release configuration continues to include the cluster_cpu filter.

Changes

KServe E2E test filters

Layer / File(s) Summary
Update LLM inference test selection
ci-operator/config/opendatahub-io/kserve/opendatahub-io-kserve-master.yaml, ci-operator/config/opendatahub-io/kserve/opendatahub-io-kserve-release-v0.17.yaml
The e2e-llm-inference-service commands now target llmisvc_core; the release configuration also includes cluster_cpu, and both retain the not pvc_storage exclusion.

Estimated code review effort: 1 (Trivial) | ~5 minutes

Suggested reviewers: bartoszmajsak, israel-hdez

🚥 Pre-merge checks | ✅ 15
✅ Passed checks (15 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly matches the main change: switching KServe e2e test selection to the llmisvc_core marker.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed The PR only edits ci-operator YAML job selectors; it adds no Ginkgo It/Describe/Context/When titles, so no unstable test names are introduced.
Test Structure And Quality ✅ Passed PR only updates ci-operator YAML test filters; no Ginkgo test code, setup, timeouts, or assertions were changed, so this quality review is not applicable.
Microshift Test Compatibility ✅ Passed PASS: PR only updates CI job test-selection strings in YAML; it does not add or modify any Ginkgo e2e tests, so MicroShift compatibility is not implicated.
Single Node Openshift (Sno) Test Compatibility ✅ Passed PASS: The PR only rewraps an existing e2e selector in CI config; it adds no new Ginkgo tests or SNO-sensitive assertions.
Topology-Aware Scheduling Compatibility ✅ Passed Only ci-operator test filters changed; no manifests, operators, or controllers were modified, so no new scheduling constraints were introduced.
Ote Binary Stdout Contract ✅ Passed Only ci-operator YAML changed; no process-level code or stdout logging behavior was modified, so the OTE stdout contract isn’t affected.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed Only CI job filter strings changed in two YAMLs; no Ginkgo test sources or new It/Describe specs were added, so the IPv6/disconnected check is not applicable.
No-Weak-Crypto ✅ Passed Only CI test-selector strings changed in two YAMLs; no MD5/SHA1/DES/RC4/3DES/Blowfish/ECB, custom crypto, or secret comparisons were added.
Container-Privileges ✅ Passed The only changes are e2e test filter strings in ci-operator YAML; no privileged, hostPID/Network/IPC, SYS_ADMIN, root, or allowPrivilegeEscalation settings were added.
No-Sensitive-Data-In-Logs ✅ Passed The only edits change e2e test selectors; no new logging or sensitive-data exposure appears in the touched YAML blocks.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 16, 2026
Signed-off-by: James Ostrander <jostrand@redhat.com>
@openshift-ci

openshift-ci Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

@jlost: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@jlost

jlost commented Jul 16, 2026

Copy link
Copy Markdown
Contributor Author

/pj-rehearse

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@jlost: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@jlost

jlost commented Jul 16, 2026

Copy link
Copy Markdown
Contributor Author

/pj-rehearse help

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@jlost: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@jlost

jlost commented Jul 17, 2026

Copy link
Copy Markdown
Contributor Author

/pj-rehearse auto-ack

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@jlost: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@vivekk16

Copy link
Copy Markdown
Member

@jlost Are the marker changes already synced to midstream?

@vivekk16 vivekk16 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ensure the changes from upstream are already synced to midstream to avoid failing CI in midstream.

/lgtm
/approve

@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Jul 17, 2026
@openshift-ci

openshift-ci Bot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jlost, vivekk16

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@jlost

jlost commented Jul 17, 2026

Copy link
Copy Markdown
Contributor Author

/pj-rehearse skip

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@jlost: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@openshift-merge-bot openshift-merge-bot Bot added the rehearsals-ack Signifies that rehearsal jobs have been acknowledged label Jul 17, 2026
@openshift-merge-bot
openshift-merge-bot Bot merged commit 9a4ed46 into openshift:main Jul 17, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged. rehearsals-ack Signifies that rehearsal jobs have been acknowledged

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants