test(ci): validate the batched Pulp delivery under load [do not merge]#6305
Closed
kduret wants to merge 16 commits into
Closed
test(ci): validate the batched Pulp delivery under load [do not merge]#6305kduret wants to merge 16 commits into
kduret wants to merge 16 commits into
Conversation
TEMP test branch (do not merge). In plugins.yml: - disable the Artifactory deliver-packages job (if: false) - force the Pulp deliver to stability: unstable, drop the stability gates (a PR runs with canary stability) and decouple the twin from test-plugins - add a marker file in the os::linux::snmp plugin packaging so get-plugins packages something to deliver All changes tagged TEMP(test-pulp-unstable). (cherry picked from commit 9174df1)
TEMP(test-pulp-unstable): reproduce the volume of the failed develop run (~30 packages per distrib) so the batched task wait is validated under a realistic load.
TEMP(test-pulp-unstable): unit-tests and test-plugins bring nothing to the batched-delivery validation and would consume up to 25 runners for the 31 marked plugins; fatpacker/package tolerate the skipped dependency.
TEMP(test-pulp-unstable): replace the 31 packaging markers with a touch of the common packaging template (changes_common=true), so get-plugins lists all 675 plugins and the batched Pulp delivery is exercised at full nightly scale (~675 packages per distrib across the 8 parallel deliver jobs).
github-actions
Bot
added
skip-workflow-plugins
and removed
skip-workflow-plugins
labels
… into MON-200796-plugins-test-batch-wait
… into MON-200796-plugins-test-batch-wait
TEMP(test-pulp-unstable): one rpm and one deb distrib are enough to measure the delivery with the autoscaled workers (675 packages each).
… into MON-200796-plugins-test-batch-wait
Contributor
Author
|
Validation complete: full matrix (8 distributions, no concurrency cap), delivery to testing + promotion to stable, |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Functional + load validation of the batched Pulp task wait (#6304): deliver plugins to Pulp
unstableonly, with a volume comparable to the develop run where all 8 deliver jobs timed out (~30 packages per distrib).What it does (on top of #6304)
deliver-packagesdisabled (if: false),deliver-packages-pulpforced tostability: unstable, stability gates dropped (PR = canary) and decoupled fromtest-plugins.get-pluginspackages ~30 packages per distrib — exercisingwait_taskson a real batch across the 8 parallel deliver jobs.What to check
Waiting for N upload task(s) to complete, thenPackages are available.did not complete in timefailure as long as the pulp task queue keeps draining.All changes tagged
# TEMP(test-pulp-unstable). Refs: MON-200796