Skip to content

Propagate release downstream resolver failures#1446

Draft
ChrisRackauckas-Claude wants to merge 2 commits into
SciML:masterfrom
ChrisRackauckas-Claude:agent/propagate-release-resolver-failures
Draft

Propagate release downstream resolver failures#1446
ChrisRackauckas-Claude wants to merge 2 commits into
SciML:masterfrom
ChrisRackauckas-Claude:agent/propagate-release-resolver-failures

Conversation

@ChrisRackauckas-Claude

Copy link
Copy Markdown
Member

Summary

  • remove the ResolverError catch from the release downstream job
  • let dependency-resolution and Catalyst test failures propagate to GitHub Actions
  • retain the existing local SciMLBase development and Catalyst test sequence

This is stacked on #1445; the extra concurrency-context commit will disappear from this diff when that PR merges.

Ignore this PR until reviewed by @ChrisRackauckas.

Local verification

  • /home/crackauc/.juliaup/bin/julialauncher +1.12 --startup-file=no -m Runic --check .
  • reproduced the workflow's downstream environment locally with the branch version of SciMLBase and ran Pkg.test("Catalyst"; coverage=true)
  • the command ran Catalyst's real test suite and exited 1 at the already-investigated UndefKeywordError: keyword argument parent_sys not assigned failure (20 passed, 1 errored in Symbolic Stoichiometry), rather than converting it to success

The Catalyst failure is pre-existing on the unmodified base and is tracked separately; this PR intentionally exposes it.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>

Copy link
Copy Markdown
Member Author

The newly visible LinearSolve.jl/All failure is another self-hosted runner loss, not a resolver or test assertion. Exact comparison: clean default-branch SHA db9ea39e job 87286414203 failed on self-hosted-4vcpu-8gb, while PR #1445 job 87305228663 used the same SciMLBase/LinearSolve source state on GitHub-hosted infrastructure and ended Testing LinearSolve tests passed. Five other recent 8 GB self-hosted LinearSolve jobs have the same runner-lost annotation, while three recent GitHub-hosted jobs passed. The workflow requests ubuntu-latest; routing is external to this PR. Evidence is being added to the existing infrastructure tracker SciML/.github#116.

Copy link
Copy Markdown
Member Author

Two additional failures in the current run have the same self-hosted-runner disconnect signature as the already-classified LinearSolve job:

In both jobs the main downstream test step was still recorded as in_progress when the runner stopped communicating; neither job reported a Julia assertion, exception, or package-test failure. I am tracking this shared runner-loss pattern in SciML/.github#116 rather than treating these as downstream-matrix regressions.

Copy link
Copy Markdown
Member Author

One more job has now ended with the same infrastructure signature: DownstreamAD (Julia 1) lost self-hosted runner czdb2 while test step 12 was still in_progress. It also emitted no Julia test failure. This brings the current run's unambiguous self-hosted disconnects to LinearSolve plus Downstream, Downstream pre, and DownstreamAD.

Copy link
Copy Markdown
Member Author

Local clean-base verification is now terminal: at clean SciMLBase commit db9ea39e, Julia +1, GROUP=Downstream, JULIA_NUM_THREADS=11, and a clean depot, the full Downstream group exited 0. The final blocks included Initialization 80/80, Table Traits 10/10, SplitODE cache 1/1, and Scalar RODE errors 17/17, followed by Testing SciMLBase tests passed.

That direct run, together with the CI step metadata showing the test step left in_progress, confirms the failed Julia 1/pre Downstream cells are runner losses rather than source failures.

Copy link
Copy Markdown
Member Author

The matching LinearSolve downstream control is also terminal: a clean current SciMLBase/LinearSolve reproduction using four parallel Julia test tasks completed with exit 0 and Testing LinearSolve tests passed. The CI failure therefore remains classified as runner loss. The cross-repository job evidence, local controls, and hosted/self-hosted routing split are consolidated in SciML/.github#116.

Copy link
Copy Markdown
Member Author

A late-finishing fourth same-workflow disconnect is now classified: SymbolicIndexingInterface (Julia 1) lost self-hosted runner xq7hr while main test step 12 remained in_progress. It emitted no Julia failure. I have forwarded it for inclusion in the infrastructure rollup.

Copy link
Copy Markdown
Member Author

Terminal CI classification for this workflow-audit PR: 55 checks succeeded and 8 failed.

Five failures are runner disconnects, not Julia test failures:

  • LinearSolve
  • Downstream / Julia 1
  • Downstream / prerelease
  • DownstreamAD / Julia 1
  • SymbolicIndexingInterface / Julia 1

In each of those logs, the test step was still marked in progress when the hosted runner stopped responding.

The other three checks reached and ran substantive downstream tests:

  • Catalyst's internal and reusable-workflow jobs both report the same Symbolic Stoichiometry error: UndefKeywordError: keyword argument parent_sys not assigned at test/integrator_interface.jl:235.
  • NonlinearSolve's Core group reports one failing generalized-Rosenbrock Broyden case (norm(u - u_true) = 4.4 > 0.001; summary: 94 passed, 1 failed, 20 broken).

Those are genuine downstream test failures, so they are deliberately not being hidden or converted to warnings. The workflow change is doing its intended job: each remaining non-infrastructure failure actually entered a package test group.

Local controls also completed successfully on clean current bases:

  • SciMLBase GROUP=Downstream with Julia release: package test exited 0 (including Init 80/80, Table 10/10, Split 1/1, and RODE 17/17).
  • LinearSolve GROUP=Core with four parallel test tasks: package test exited 0.

The hosted-runner disconnect pattern is tracked in SciML/.github#116.

Copy link
Copy Markdown
Member Author

Follow-up on the three substantive downstream failures:

  • Catalyst's duplicated parent_sys failure is addressed by focused draft Test symbolic stoichiometry through public API Catalyst.jl#1511. It replaces the test's private ModelingToolkitBase AffectSystem inspection with a public end-to-end JumpProblem solve; the changed symbolic-stoichiometry file passed locally 30/30.
  • There is also an open NonlinearSolve draft, Update bad Broyden robustness expectations NonlinearSolve.jl#1069, for the Broyden case. That draft adds the failure to the suite's broken_tests list. I am not treating that as an acceptable resolution for this audit: the repository instructions explicitly prohibit marking a newly failing case broken to make the suite pass. A separate underlying-cause reproduction/bisect is queued instead.

This does not change the classification above: all three #1446 failures ran substantive tests rather than silently no-oping.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants