Skip to content

Add time-step-exit and aes-graupel savepoint validation - #1363

Closed
jcanton wants to merge 5 commits into
physics_driver_l2from
physics_driver_fix_muphys
Closed

Add time-step-exit and aes-graupel savepoint validation#1363
jcanton wants to merge 5 commits into
physics_driver_l2from
physics_driver_fix_muphys

Conversation

@jcanton

@jcanton jcanton commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Readers for the three new savepoints generated by the instrumented ICON run (icon-nwp add_exp_ape_aes):

  • IconTimeStepExitSavepoint: end-of-timestep prognostic state, written after integrate_nh returns (all physics applied, time levels swapped)
  • IconAesGraupelInit/ExitSavepoint: state + prm_tend accumulators around the mig block (cloud_mig = satad + graupel + satad), plus aggregated surface precip (rsfl, ssfl, pr, ufcs)

Replace test_standalone_driver_moist_physics: instead of only checking the fields muphys does not touch against diffusion/nonhydro exits, the full prognostic state (vn, w, rho, exner, theta_v, all six tracers) is now validated against the single time-step-exit savepoint. Provisional tolerances document the known phy2dyn fidelity gaps (linearized exner update, no theta_v update, no negative-tracer clipping, full-column vs jks_cloudy) until they are ported.

Add test_aes_graupel_granule: first validation of the muphys granule against Fortran ICON (previous coverage was netCDF references from the C++ reference implementation only). Compares component tendencies and surface precip against the aes-graupel-init/exit pair, restricted to the jks_cloudy..nlev levels ICON computes, and checks the granule produces (near-)zero tendencies above.

Tolerances are provisional until measured on the regenerated exclaim_ape_aesPhys v05 archive (overwritten in place, no version bump).

sync with https://gitlab.dkrz.de/icon/icon-nwp/-/merge_requests/2200

jcanton and others added 4 commits July 8, 2026 02:27
Readers for the three new savepoints generated by the instrumented ICON
run (icon-nwp add_exp_ape_aes):
- IconTimeStepExitSavepoint: end-of-timestep prognostic state, written
  after integrate_nh returns (all physics applied, time levels swapped)
- IconAesGraupelInit/ExitSavepoint: state + prm_tend accumulators around
  the mig block (cloud_mig = satad + graupel + satad), plus aggregated
  surface precip (rsfl, ssfl, pr, ufcs)

Replace test_standalone_driver_moist_physics: instead of only checking
the fields muphys does not touch against diffusion/nonhydro exits, the
full prognostic state (vn, w, rho, exner, theta_v, all six tracers) is
now validated against the single time-step-exit savepoint. Provisional
tolerances document the known phy2dyn fidelity gaps (linearized exner
update, no theta_v update, no negative-tracer clipping, full-column vs
jks_cloudy) until they are ported.

Add test_aes_graupel_granule: first validation of the muphys granule
against Fortran ICON (previous coverage was netCDF references from the
C++ reference implementation only). Compares component tendencies and
surface precip against the aes-graupel-init/exit pair, restricted to
the jks_cloudy..nlev levels ICON computes, and checks the granule
produces (near-)zero tendencies above.

Tolerances are provisional until measured on the regenerated
exclaim_ape_aesPhys v05 archive (overwritten in place, no version bump).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Reader classes, provider methods, test file and function names now use the
muphys nomenclature; the serialbox savepoint names (aes-graupel-init/exit)
and the Fortran sources keep the aes_graupel naming so it stays clear the
data interface is the icon-nwp Fortran mig block, not the icon-mpim Kokkos
port. Also drop the qnc-units bug commentary obsoleted by the MuphysConfig
fix (qnc now 50.0e6 m^-3, matching Fortran cloud_num) and read qnc from the
config again in the datatest.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The muphys port followed the older muphys C++ reference implementation,
while icon-nwp carries the newer MPIM rain-microphysics revisions. This
adds the icon-nwp formulation (mo_aes_graupel.f90) as a second scheme,
selected via MuphysConfig.scheme / a use_icon_nwp flag threaded through
graupel_run and muphys_run:

- cloud_to_rain: rho-dependent polynomial accretion kernel in
  log(clamped rho*qr) instead of the constant 5.25
- rain_to_vapor: exp-polynomial evaporation instead of the power law
- cloud_to_snow: riming tuning factor 3.0
- snow_number/snow_lambda: operate on the snow mass density rho*qs with
  a lower clamp instead of the additive (qs + 2e-6) offset, and branch
  on rho*qs > qmin
- fall speeds (vm): rain becomes a log-polynomial (with the constant
  term not density-corrected, faithfully reproducing the Fortran
  operator precedence), ice prefactor 0.80, snow prefactor 115.6 with
  snow_number evaluated at the clamped falling-mass density and the
  previous level's temperature (now carried in the scan state), and a
  two-sided clamp of the hydrometeor density for all species
- ice_dep is computed from the vapor_x_ice deposition before the
  nucleation contribution is added, matching the Fortran ordering

Defaults: setup_graupel/setup_muphys/MuphysComponent keep
CPP_REFERENCE, so the existing netCDF-based tests are unchanged
(verified: graupel_only/full_muphys/component datatests pass at
rtol 1e-14 after the refactor); MuphysConfig defaults to ICON_NWP, so
the standalone driver and the aes-graupel serialbox datatest use the
ported scheme matching the Fortran that generates the reference data.

Also fix the fused muphys_run to return the temperature updated by the
final saturation adjustment (as the Fortran does), consistent with the
adjusted qv/qc; the separate-programs path already did this.

New stencil tests validate every ported formula against independent
numpy references (pass on gtfn_cpu); an execution smoke run confirms
the ICON_NWP path compiles, stays finite, and differs from
CPP_REFERENCE by physically plausible magnitudes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
yiluchen1066 added a commit that referenced this pull request Jul 8, 2026
Scheme names now say what each formulation is: KOKKOS_MUPHYS is the
muphys C++/Kokkos reference implementation, AES_GRAUPEL is the icon-nwp
mo_aes_graupel.f90 formulation. All _icon_nwp suffixes (field operators,
programs, constants class, stencil tests, the use_icon_nwp flag) become
_aes_graupel accordingly.

The Python-level scheme dispatch in setup_graupel/setup_muphys uses
match/case with an explicit case per scheme. Inside the GT4Py operators
the branches stay scalar-bool if/else with a labeled else: the DSL has
no match statement, and an if/elif pair on a bool is rejected by its
definedness analysis (it cannot prove the two conditions exhaustive, so
names assigned in the branches count as possibly undefined).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown

Mandatory Tests

Before merging, run the merge pipeline with cscs-ci run merge. Merging is blocked unless this pipeline passes.

When developing, you can test your changes on CSCS CI before merge with the default pipeline: cscs-ci run default. This will run a default subset of tests.

You can pass options to override pipeline variables, for example:

  • cscs-ci run default;BACKENDS=gtfn_cpu;LEVELS=unit
  • cscs-ci run default;MODEL_SUBPACKAGES=common:driver;SESSIONS=model
    Avoid running the pipeline for all tests when you are developing.

Available options are:

  • SESSIONS: model, model_mpi, or tools (correspond to nox sessions)
  • MODEL_SUBSETS: datatest, basic, or stencils (correspond to nox session selections)
  • MODEL_SUBPACKAGES: subpackages for non-MPI tests (last component, e.g. diffusion, standalone_driver)
  • MODEL_MPI_SUBPACKAGES: subpackages for MPI tests (as above)
  • BACKENDS: backends
  • GRIDS: grids for stencil tests (simple, icon_regional, or icon_global)
  • LEVELS: testing level for non-stencil tests (unit or integration)

For each option, all can be used as a shorthand for all possible values of that variable, e.g. LEVELS=all.

See scripts/python/generate_ci_pipeline.py and noxfile.py for available values for each option.

The all pipeline can be run with cscs-ci run all. This will run all icon4py tests in CSCS CI which can be expensive. This pipeline runs on a schedule on main, and can be run when extensive validation is needed (e.g. before releases).

Optional Tests

To run benchmarks you can use:

  • cscs-ci run benchmark-bencher

For more detailed information please look at CI in the EXCLAIM universe.

@jcanton

jcanton commented Jul 13, 2026

Copy link
Copy Markdown
Contributor Author

already merged into #1301

@jcanton jcanton closed this Jul 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant