Skip to content

ROSAENG-62387 | fix(test): align OCP-45509 proxy assertion and skip on expired AWS creds#3417

Draft
amandahla wants to merge 1 commit into
openshift:masterfrom
amandahla:ROSAENG-62387-day1-negative-45509
Draft

ROSAENG-62387 | fix(test): align OCP-45509 proxy assertion and skip on expired AWS creds#3417
amandahla wants to merge 1 commit into
openshift:masterfrom
amandahla:ROSAENG-62387-day1-negative-45509

Conversation

@amandahla

@amandahla amandahla commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

PR Summary

Align day1-negative OCP-45509 with current CLI/OCM proxy-without-subnets error, and skip AWS-heavy day1-negative specs when STS credentials are expired so infra noise is not reported as product failures.

Detailed Description of the Issue

periodic-ci-openshift-rosa-master-e2e-rosa-day1-negative-f7 has been failing consistently on OCP-45509. The first step calls rosa create cluster --dry-run with HTTP/HTTPS proxy flags and no --subnet-ids. It still expected the old local message number of subnets for a 'single AZ' 'cluster' should be '2', instead received: '0'.

With --yes (already set by CreateDryRun), the CLI accepts the managed-VPC prompt path and reaches OCM, which rejects the request with cluster_wide_proxy is only supported if subnetIDs exist. That is the real validation for proxy without subnet IDs today, not a missing-VPC flake.

Separately, mid-suite STS expiry (InvalidClientTokenId / related AWS auth errors) caused rotating product assertion failures on other day1-negative cases.

Related Issues and PRs

  • Jira: ROSAENG-62387
  • Fixes: N/A
  • Related PR(s): N/A
  • Related design/docs: N/A

Type of Change

  • feat - adds a new user-facing capability.
  • fix - resolves an incorrect behavior or bug.
  • docs - updates documentation only.
  • style - formatting or naming changes with no logic impact.
  • refactor - code restructuring with no behavior change.
  • test - adds or updates tests only.
  • chore - maintenance work (tooling, housekeeping, non-product code).
  • build - changes build system, packaging, or dependencies for build output.
  • ci - changes CI pipelines, jobs, or automation workflows.
  • perf - improves performance without changing intended behavior.

Previous Behavior

  • OCP-45509 failed every run by asserting an obsolete local subnet-count error string.
  • Expired AWS/STS credentials mid-job failed subsequent Its as mismatched product error expectations.
  • Changing tests/e2e sources caused coverage-changed-files to attempt running the full e2e suite locally.

Behavior After This Change

  • OCP-45509 first step asserts cluster_wide_proxy is only supported if subnetIDs exist (later VPC/proxy steps unchanged).
  • AWS-heavy day1-negative BeforeEach hooks call helper.SkipIfAWSCredentialsInvalid() and Skip with an infra message on known credential failures.
  • hack/coverage-changed-files.sh excludes tests/e2e/ from unit coverage collection.

How to Test (Step-by-Step)

Preconditions

  • Local clone with hooks installed (make install-hooks).
  • Optional: AWS credentials for a live dry-run of OCP-45509 (not required for unit checks).

Test Steps

  1. make fmt-check
  2. make coverage-changed-files (or ./hack/coverage-changed-files.sh with staged/committed changes)
  3. make rosa
  4. make lint
  5. make test GO_TEST_FLAGS='-count=1'
  6. go test ./tests/utils/helper/ -count=1 -run 'TestIsAWSCredentialError|TestAWSCredentialsInvalid|TestSkipIfAWSCredentialsInvalid'
  7. Optional: run day1-negative focus for OCP-45509 against stage with valid AWS/OCM creds.

Expected Results

  • Unit/helper tests and repo gates pass.
  • OCP-45509 first dry-run failure output contains cluster_wide_proxy is only supported if subnetIDs exist.
  • With invalid/expired STS tokens, AWS-heavy day1-negative specs Skip instead of failing product assertions.

Proof of the Fix

  • Screenshots: N/A
  • Videos: N/A
  • Logs/CLI output: Local make fmt-check, coverage (88.9% delta on helper), make rosa, make lint, make test, helper package tests, and CodeRabbit (review --agent --uncommitted) with 0 findings. Pre-push checks passed on push.
  • Other artifacts: Prow investigation for periodic-ci-openshift-rosa-master-e2e-rosa-day1-negative-f7 builds (OCP-45509 consistent; STS failures rotating).

Breaking Changes

  • No breaking changes
  • Yes, this PR introduces a breaking change (describe impact and migration plan below)

Breaking Change Details / Migration Plan

N/A

Developer Verification Checklist

  • Commit subject/title follows [JIRA-TICKET] | [TYPE]: <MESSAGE>.
  • PR description clearly explains both what changed and why.
  • Relevant Jira/GitHub issues and related PRs are linked.
  • make install-hooks has been run in this clone.
  • Tests were added/updated where appropriate.
  • I manually tested the change.
  • make test passes.
  • make lint passes.
  • make rosa passes.
  • Documentation or repo-local agent guidance was added/updated where appropriate.
  • Any risk, limitation, or follow-up work is documented.

Made with Cursor

Summary by CodeRabbit

  • Bug Fixes

    • Improved end-to-end test stability by skipping suites when AWS credentials are invalid.
    • Updated proxy negative-test assertions to match the latest expected error text.
    • Hardened end-to-end cleanup to avoid failing when setup didn’t complete.
  • Tests

    • Added AWS credential validation helpers and unit tests for detection and conditional skipping.
  • Chores

    • Excluded end-to-end tests from local and differential coverage computations.

@coderabbitai

coderabbitai Bot commented Jul 23, 2026

Copy link
Copy Markdown

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Pro Plus

Run ID: 32d20920-75d7-4b89-a3e4-5e859e45bad2

📥 Commits

Reviewing files that changed from the base of the PR and between ecac803 and 8177a8a.

📒 Files selected for processing (5)
  • hack/coverage-changed-files.sh
  • tests/e2e/test_rosacli_cluster.go
  • tests/utils/helper/aws.go
  • tests/utils/helper/aws_test.go
  • tests/utils/helper/helper_suite_test.go
🚧 Files skipped from review as they are similar to previous changes (3)
  • tests/utils/helper/aws.go
  • hack/coverage-changed-files.sh
  • tests/e2e/test_rosacli_cluster.go

📝 Walkthrough

Walkthrough

The coverage script excludes tests/e2e paths from local package selection and gocovdiff calculations. A new AWS helper detects known credential failures and skips selected end-to-end setup blocks through configurable Ginkgo functions. Unit tests cover error matching, credential outcomes, and skip behavior. Several ROSA CLI cluster suites invoke the guard, and one proxy-related error assertion is updated.

Suggested reviewers: jerichokeyne, marcolan018

🚥 Pre-merge checks | ✅ 14 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Test Structure And Quality ⚠️ Warning New helper tests use bare assertions with no diagnostic messages, and the 45509 proxy case bundles multiple scenarios into one It, hurting diagnosability. Split the proxy validations into per-scenario It/DescribeTable cases and add context to key Expect() messages in the new helper tests.
✅ Passed checks (14 passed)
Check name Status Explanation
Title check ✅ Passed Matches the main changes and follows the required ticket/type format.
Description check ✅ Passed Includes all required sections and gives context, related issue, testing, and verification details.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed The touched Ginkgo titles are static strings; no dynamic, runtime-derived, or changing values appear in the new/modified tests.
Microshift Test Compatibility ✅ Passed No new e2e test bodies were added; the PR only adds AWS-credential skip guards and an error-string update, with no MicroShift-specific APIs or assumptions introduced.
Single Node Openshift (Sno) Test Compatibility ✅ Passed The changed e2e specs only validate CLI/OCM errors and AWS-credential skips via dry-run/API calls; no node-count, rescheduling, or multi-node topology assumptions were added.
Topology-Aware Scheduling Compatibility ✅ Passed Only test/helper scripts changed; no deployment manifests, controllers, replicas, affinity, node selectors, or other scheduling constraints were introduced.
Ote Binary Stdout Contract ✅ Passed The new process-level entrypoint only calls RegisterFailHandler/RunSpecs; no stdout writes or klog/log redirection issues were added.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed The PR only adds AWS-credential preflight and updates a proxy error assertion; the modified e2e tests add no new IPv4 literals, IPv4-only parsing, or public-internet fetches.
No-Weak-Crypto ✅ Passed No weak crypto, custom crypto, or secret/token comparisons were introduced; the new code only matches AWS error substrings and test scaffolding.
Container-Privileges ✅ Passed Diff only changes test Go files and a shell script; no container/K8s manifests or privilege flags (privileged, hostPID/Network/IPC, SYS_ADMIN, allowPrivilegeEscalation) appear.
No-Sensitive-Data-In-Logs ✅ Passed No new log paths expose secrets/PII; added output is a fixed AWS-credential skip message, and the only interpolated data is the generic AWS error text.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@openshift-ci

openshift-ci Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: amandahla

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 23, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/e2e/test_rosacli_cluster.go`:
- Line 1275: Make teardown safe when BeforeEach skips setup: in
tests/e2e/test_rosacli_cluster.go at lines 1275-1275, 2659-2659, and 3191-3191,
guard clusterHandler.Destroy() so it runs only when cleanup state is
initialized; at line 2386-2386, reset accountRolePrefixToClean before the guard
and perform cleanup only when rosaClient is initialized.

In `@tests/utils/helper/aws_test.go`:
- Around line 3-150: Convert the tests in TestIsAWSCredentialError,
TestAWSCredentialsInvalid, and TestSkipIfAWSCredentialsInvalid from testing.T
subtests and Fatalf checks to the repository’s Ginkgo v2 Describe/It structure
with Gomega assertions. Preserve all existing scenarios, setup/cleanup behavior,
and expected outcomes while using the package’s established Ginkgo/Gomega
conventions.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Pro Plus

Run ID: aaf1acff-d56a-4bed-82b0-fe2cb23ad602

📥 Commits

Reviewing files that changed from the base of the PR and between 16fe535 and ecac803.

📒 Files selected for processing (4)
  • hack/coverage-changed-files.sh
  • tests/e2e/test_rosacli_cluster.go
  • tests/utils/helper/aws.go
  • tests/utils/helper/aws_test.go

Comment thread tests/e2e/test_rosacli_cluster.go
Comment thread tests/utils/helper/aws_test.go Outdated
…n expired AWS creds

Day1-negative OCP-45509 expected an obsolete local subnet-count error;
with --yes the CLI reaches OCM and returns cluster_wide_proxy requires
subnetIDs. Also skip AWS-heavy day1-negative Describes when STS tokens
are expired so infra failures are not recorded as product regressions.
Exclude tests/e2e from changed-files coverage so e2e edits do not run
the full suite locally.

Signed-off-by: Amanda Hager Lopes de Andrade Katz <amanda.katz@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@amandahla
amandahla force-pushed the ROSAENG-62387-day1-negative-45509 branch from ecac803 to 8177a8a Compare July 23, 2026 19:24
BeforeEach(func() {
var err error

helper.SkipIfAWSCredentialsInvalid()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would rather we not add anything to skip on invalid credentials. If we do that and our CI credentials do expire ever, we won't know immediately since all the tests would be skipped and the CI jobs would be marked as passed then. Also, the error I think you're trying to fix here should be fixed already by #3409

"--prefix", accountRolePrefixToClean,
"-y")
Expect(err).To(BeNil())
prefix := accountRolePrefixToClean

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the point of this section of code? I don't really see what's being changed other than checking if rosaClient is nil

@openshift-ci

openshift-ci Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

@amandahla: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/govulncheck 8177a8a link false /test govulncheck
ci/prow/security 8177a8a link false /test security

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@amandahla

Copy link
Copy Markdown
Contributor Author

Moving to draft to investigate better and also considering Jericho's reviews.

@amandahla
amandahla marked this pull request as draft July 24, 2026 15:52
@openshift-ci openshift-ci Bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jul 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. dco-signoff: yes do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants