test: verify exact pixel output for every codec + fail CI on silently-skipped suites#72
Acknowledged Regression: -1.76%
⚠️ Different runtime environments detected
Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.
A heads-up, this is a breaking change and it might affect your current performance baseline a bit. But here's the exciting part - it's packed with new, cool features and promises improved result stability 🥳!
Curious about what's new? Visit our releases page to delve into all the awesome details about this new version.
⚡ 1 improved benchmark
❌ 1 (👁 1) regressed benchmark
✅ 35 untouched benchmarks
🆕 69 new benchmarks
⏩ 13 skipped benchmarks1
Performance Changes
ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.
Tip
Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.
Comparing test-pixel-correctness (aaa5154) with main (04c3e87)
Footnotes
-
13 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩