Skip to content

Activate corrected conservation-law tests#1512

Draft
ChrisRackauckas-Claude wants to merge 1 commit into
SciML:masterfrom
ChrisRackauckas-Claude:agent/activate-corrected-conservation-tests
Draft

Activate corrected conservation-law tests#1512
ChrisRackauckas-Claude wants to merge 1 commit into
SciML:masterfrom
ChrisRackauckas-Claude:agent/activate-corrected-conservation-tests

Conversation

@ChrisRackauckas-Claude

@ChrisRackauckas-Claude ChrisRackauckas-Claude commented Jul 15, 2026

Copy link
Copy Markdown
Member

Summary

  • Convert seven conservation-law assertions from expected-broken to active tests.
  • Record the dependency boundary that corrected the parameter-substitution behavior those assertions exercise.

This branch is rebased directly onto master and contains only the conservation-law test change.

Cause

Catalyst commit 9d3448a6e614f2ddcd7b5c4866a7347266b2b6fd marked these seven assertions broken on June 29. They now pass on both the current and generated minimum dependency graphs, so @test_broken reports seven unexpected-pass errors on clean master.

A public problem-construction probe bisected the original correction to ModelingToolkit commit 1c253adc2aa55934fa110dd664bdf27c4fadc0f1 in #4704, first released in ModelingToolkitBase 1.52.0:

  • parent 96275f9cd: reconstructed observed species X2 = 0.0, conservation parameter Γ = 1.0;
  • correcting commit: X2 = 2.0, Γ = 3.0.

The change evaluates the operating-point mapping before filling MTKParameters, preserving the conservation substitutions these assertions exercise. Catalyst #1506 independently repairs the older MTKBase 1.51 behavior through initialization-equation propagation.

Local verification

  • Julia 1.10.11, clean master, generated minimum graph (MTKBase 1.46.0 / SciMLBase 3.18.0 / OrdinaryDiffEq 7.0.0): 242 passed, 7 unexpected-pass errors, 17 pre-existing broken, exit 1.
  • Julia 1.10.11, this branch, the same generated minimum graph: 249 passed, 17 pre-existing broken, 266 total, exit 0.
  • Julia 1.10.11 and 1.12.6, current MTKBase 1.53: 249 passed, 17 pre-existing broken, exit 0.
  • Julia 1.12.6, MTKBase 1.52.0 / SciMLBase 3.36.0 / Symbolics 7.32.0 / SymbolicUtils 4.40.1: exact GROUP=Modeling passed; conservation reported 249 passed and 17 pre-existing broken.
  • Runic 1.7 was run on the changed file and repository. It reports pre-existing broad formatting drift; its diff proposes no change to these seven edited assertions, so no unrelated formatting was included.

No assertion was loosened, skipped, deleted, or changed to match buggy output; these seven tests are strengthened from expected-broken to required passing assertions.

Please ignore this draft until it has been reviewed by @ChrisRackauckas.

Copy link
Copy Markdown
Member Author

Audit scratchpad for the top commit addbec0107bae2ad4a9253dfb7d56883a81b395f:

  • Test symbolic stoichiometry through public API #1511 is the prerequisite ordering fix only; its public symbolic-stoichiometry regression lets Modeling reach this file.
  • Exact pre-change Test symbolic stoichiometry through public API #1511 CI reports precisely seven Unexpected Passes at lines 150, 152, 179, 183, 287, 290, and 325 (242 pass, 7 errors, 17 broken).
  • MTKBase bisect: parent 96275f9cd gives X2=0.0, Γ=1.0; 1c253adc2 gives X2=2.0, Γ=3.0. The latter shipped in MTKBase 1.52.0 via ModelingToolkit #4704.
  • Current direct conservation runs pass on Julia 1.10 and 1.12 at 249/249 required assertions plus 17 pre-existing broken.
  • Exact stacked Julia 1.12 GROUP=Modeling passes in full with the MTKBase 1.52 CI stack.
  • The branch is current with upstream master and contains exactly the Test symbolic stoichiometry through public API #1511 prerequisite commit followed by this one-file activation commit.

Runic's repository/file diff is pre-existing formatting outside the seven edited assertions; no unrelated formatter rewrite was included.

Copy link
Copy Markdown
Member Author

Interim CI classification (2026-07-15 09:35 EDT):

  • Modeling on Julia current and prerelease is green. The Julia LTS Modeling lane reaches and passes Symbolic Stoichiometry 30/30, then later fails in the pre-existing DSL Options regressions already tracked in Modeling test group fails on current master #1502; this is unrelated to the seven conservation assertions activated here.
  • The current-Julia QA lane did not report a test failure. Its check annotation says: “The self-hosted runner lost communication with the server,” and the test step is left marked in progress.
  • All other completed checks are green. Two checks remain pending.

The scheduled full checkpoint will still be posted after the original timer expires.

@ChrisRackauckas-Claude

Copy link
Copy Markdown
Member Author

Scheduled CI follow-up (checked 2026-07-16 12:42 EDT): 19 checks succeeded, 2 failed, and 1 was cancelled.

  • The target Modeling lanes pass on current and prerelease Julia.
  • The LTS Modeling lane passes Symbolic Stoichiometry (30/30), including this stack, before reaching the independently tracked DSL Options failures in Modeling test group fails on current master #1502.
  • QA is the same explicit self-hosted runner-loss annotation seen on master.
  • Extensions Tests (pre) was cancelled after the 6-hour limit while still instantiating/precompiling; its test step never ran.

No failure points to the seven activated conservation assertions.

Copy link
Copy Markdown
Member Author

Rebased this branch directly onto master, removing the former #1511 stack. The PR now contains one commit and changes only test/network_analysis/conservation_laws.jl.

Fresh local verification on Julia 1.10.11 with the generated minimum graph (MTKBase 1.46.0, SciMLBase 3.18.0, OrdinaryDiffEq 7.0.0):

Test Summary:     | Pass  Broken  Total   Time
Conservation Laws |  249      17    266  6m06.9s
Testing Catalyst tests passed

Clean master on the identical graph reports 242 passed, 7 unexpected-pass errors, and 17 pre-existing broken tests.

Copy link
Copy Markdown
Member Author

CI note after rebasing this PR directly onto master: the Julia 1, LTS, and pre Modeling lanes fail before reaching Conservation Laws at test/reactionsystem_core/symbolic_stoichiometry.jl:235 with UndefKeywordError: keyword argument parent_sys not assigned. This reproduces the independent clean-master compatibility failure fixed by draft PR #1511. The completed non-Modeling lanes are green so far. This PR remains intentionally unstacked and limited to the seven stale conservation markers.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
@ChrisRackauckas-Claude
ChrisRackauckas-Claude force-pushed the agent/activate-corrected-conservation-tests branch from 75dc4be to 58b1989 Compare July 22, 2026 04:50
@ChrisRackauckas-Claude

Copy link
Copy Markdown
Member Author

Terminal CI classification for exact head 58b1989dee4c9f352b24aa60fa7dd93d32d691eb:

  • All 22 checks are terminal: 20 succeeded and 2 failed. Build, every extension lane, all Simulation/Hybrid/Misc/Spatial lanes, and both current and prerelease Modeling passed.
  • Julia LTS Modeling reached the known clean-master DSL Options regression and reported 308 pass / 9 fail / 1 existing broken at dsl_options.jl lines 793, 1187, 1260, 1274, 1287, 1303, 1956, 1974, and 1992. This is the same nine-failure Julia 1.10 baseline tracked independently by Modeling test group fails on current master #1502/Fix symbolic option parameter discovery #1503; this PR changes only the conservation-law assertions.
  • Julia-current QA did not report a test failure. GitHub's annotation says the hosted runner lost communication with the server while the test step was still marked in progress.

The complete current Julia 1.12.6 Modeling group was also run locally after rebasing onto current master and passed, including Conservation Laws at 249 pass / 17 existing broken. The two terminal hosted failures therefore reproduce the clean-master/infrastructure exceptions rather than a regression from this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants