Repository navigation
WPT results: count subtests gained and lost in each direction - #1161
Merged
Merged
Conversation
Contributor
|
I'll fix CI failures and address comments from users with write access that start with 'Devin'.
|
nicoburns
enabled auto-merge (squash)
October 9, 2026 16:24
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The WPT results section in PR descriptions undercounted when subtests of one test moved in both directions. The headline summed each test's net pass-count change, and
wpt diff(without--verbose) omits a test whose status and pass count are both unchanged. So a test that gains 40 subtests and loses 40 was missing from the headline and the full diff, while the by-area table (which compares subtests by name) counted it.Example from #1046: headline "25 newly passing, 0 newly failing", by-area table
+25 (+65 / -40). The difference iscss/css-sizing/stretch/block-height-008.html, 80/160 before and after with 40 subtests fixed and 40 regressed.Changes:
wpt.yml: the JSON diff is generated withwpt diff --verbose, which joins subtests by name and reports tests whose individual subtests changed.wpt_diff_to_pr.py: gained/lost are counted per subtest from each entry'ssubtestslist, falling back to the net change when the list is empty (tests without subtests, or a diff produced without--verbose).!with+G/-Linstead of the net:! FAIL => FAIL [80/160] +40/-40 css/css-sizing/stretch/block-height-008.htmlFAILtoTIMEOUT) with the same test status and pass count are left out, as before.--verbosewould otherwise add them to the list.wpt-post-results.ymlruns the script frommainagainst the artifact of the PR's run, so until a PR is rebased past this its diff has no subtest lists and renders as it does today.Link to Devin session: https://dioxus.staging.devinenterprise.com/sessions/3a06d93089f2434ba3519a7813dea1e8
Open in Devin Desktop: https://dioxus.staging.devinenterprise.com/desktop/session/3a06d93089f2434ba3519a7813dea1e8?variant=devin-insiders
Requested by: @nicoburns
WPT results
No changes in test results compared to
main.Generated by the WPT workflow.