Skip to content
Open
Show file tree
Hide file tree
Changes from 75 commits
Commits
Show all changes
91 commits
Select commit Hold shift + click to select a range
717b3e2
first commit: implement physics interface (WIP)
yiluchen1066 Jun 4, 2026
e0bfe9e
pre commit: adding licence
yiluchen1066 Jun 4, 2026
c38a902
Merge branch 'main' into physics_driver_l2
yiluchen1066 Jun 4, 2026
1466c21
add physics state (MuphysState)
yiluchen1066 Jun 8, 2026
a3bb29a
add the wrapper for muphys
yiluchen1066 Jun 9, 2026
66617a4
rename the muphys granule
yiluchen1066 Jun 9, 2026
60c9c6a
up + add the datatest for muphys granule wrapper
yiluchen1066 Jun 9, 2026
9f193b9
fix the test
yiluchen1066 Jun 9, 2026
47f0aa7
some small fixes and renaming
yiluchen1066 Jun 10, 2026
b6d166a
move the muphysState to muphys
yiluchen1066 Jun 10, 2026
c325c5f
move the physics_driver to a physics_interface package
yiluchen1066 Jun 10, 2026
b035382
move the physics_driver to a physics_interface package (more)
yiluchen1066 Jun 10, 2026
730893f
small fix
yiluchen1066 Jun 10, 2026
f74672b
some small clean ups
yiluchen1066 Jun 11, 2026
2c61eae
Merge branch 'main' into physics_driver_l2
yiluchen1066 Jun 11, 2026
e720c40
solve the inplace update in muphys, input_properties and some mypy stuff
yiluchen1066 Jun 12, 2026
4722175
Merge branch 'main' into physics_driver_l2
yiluchen1066 Jun 12, 2026
fea8cd8
WIP (resolving comments)
yiluchen1066 Jun 16, 2026
42ddc80
apply tracers pr
yiluchen1066 Jun 17, 2026
bb30857
merge from main and resolve conflict
yiluchen1066 Jun 17, 2026
044b2c6
merge from main and resolve conflict
yiluchen1066 Jun 23, 2026
596286b
fix the Any type in the protocol
yiluchen1066 Jun 23, 2026
98be0a6
add the per-process state to the physics driver
yiluchen1066 Jun 23, 2026
5ef96df
small fix
yiluchen1066 Jun 24, 2026
5f714e2
Merge branch 'main' into physics_driver_l2
yiluchen1066 Jun 24, 2026
eac8adb
more fix on muphys component
yiluchen1066 Jun 24, 2026
47d8dab
plug in muphys to standalone driver
yiluchen1066 Jun 24, 2026
acac3fd
Merge branch 'main' into physics_driver_l2
yiluchen1066 Jun 24, 2026
dd3bc13
bump version
jcanton Jun 24, 2026
c7e780f
add experiment description
jcanton Jun 24, 2026
6aa8056
more clean up and renaming
yiluchen1066 Jun 25, 2026
acf8a20
rename PhysicsStateProtocol to PhysicsState
yiluchen1066 Jun 25, 2026
701686a
solve the KeyError
yiluchen1066 Jun 25, 2026
7569596
Merge branch 'main' into add_exp_ape_aes
yiluchen1066 Jun 26, 2026
41cc4ae
Merge branch 'main' into physics_driver_l2
yiluchen1066 Jun 26, 2026
8aa91e8
small fix on comment
yiluchen1066 Jun 26, 2026
f20e891
removing the duplication on the definition of spiecies
yiluchen1066 Jun 26, 2026
dd3dd17
porting wip
jcanton Jun 26, 2026
302b573
move constants to constants.py
jcanton Jun 26, 2026
516bc7e
add thermo helpers
jcanton Jun 26, 2026
ffdb42b
move this
jcanton Jun 26, 2026
46c5d68
pass global reductions
jcanton Jun 26, 2026
cb028d0
add tests
jcanton Jun 26, 2026
ddae38d
this should work
jcanton Jun 26, 2026
7520535
mostly refactor
yiluchen1066 Jun 26, 2026
f011e9b
refactoring
jcanton Jun 26, 2026
1266ac2
somebody thought this was a great idea.... and then asked a friend to…
jcanton Jun 26, 2026
568d35e
bugfix, now passing
jcanton Jun 26, 2026
c001e14
cleanup
jcanton Jun 26, 2026
c40475b
pre-commit
jcanton Jun 26, 2026
09b8924
Merge branch 'main' into add_exp_ape_aes
jcanton Jun 29, 2026
4b5b5fb
if it's always present we should not use .get
jcanton Jun 29, 2026
895ed66
add comments on confusing return
jcanton Jun 29, 2026
79f1ec0
tiny lookup save
jcanton Jun 29, 2026
ba77f68
kwargs
jcanton Jun 29, 2026
2437069
renaming for clarity is always nice
jcanton Jun 29, 2026
2482ce0
explaining is always nice
jcanton Jun 29, 2026
3f4d9e6
fix regex to be more robust
jcanton Jun 29, 2026
27ac411
it should never be missing
jcanton Jun 29, 2026
731fa38
Merge branch 'main' into add_exp_ape_aes
jcanton Jun 29, 2026
c571a20
Merge branch 'main' into physics_driver_l2
yiluchen1066 Jul 2, 2026
dee4edd
merge PR #1347
yiluchen1066 Jul 3, 2026
59cae8d
adding the datatest for standalone_driver+muphys run
yiluchen1066 Jul 7, 2026
55f036c
Move ProcessTimeControl to its own file; split forcing_mode to Physic…
yiluchen1066 Jul 7, 2026
89b4b17
make the precip_diagnostic property for IO/plotting
yiluchen1066 Jul 7, 2026
59ed7a6
Add time-step-exit and aes-graupel savepoint validation
jcanton Jul 7, 2026
b347a62
tiny fix
jcanton Jul 8, 2026
25c2854
Rename aes_graupel to muphys on the icon4py side
jcanton Jul 8, 2026
fee4c76
Port the icon-nwp graupel formulation as a selectable MuphysScheme
jcanton Jul 8, 2026
21b9bfb
update on the datatest and rebase with #1363
yiluchen1066 Jul 8, 2026
8119c66
small up
yiluchen1066 Jul 8, 2026
ad485d1
adding new stencils for updating the exner and theta_v
yiluchen1066 Jul 8, 2026
2890d83
Merge branch 'main' into physics_driver_l2
yiluchen1066 Jul 9, 2026
9524a28
some small updates and refactors
yiluchen1066 Jul 9, 2026
42b46fd
Rename muphys schemes to KOKKOS_MUPHYS / AES_GRAUPEL
yiluchen1066 Jul 10, 2026
85dd678
fix the NaN in the test
yiluchen1066 Jul 22, 2026
bb80c24
resolve one small commit
yiluchen1066 Jul 22, 2026
246f033
merge main
yiluchen1066 Jul 22, 2026
f02be4a
chaneg the relativeTime for time_step in component
yiluchen1066 Jul 22, 2026
dd3e382
rename everywhere consistently to physicsdriver including the package…
yiluchen1066 Jul 22, 2026
8f3352c
small ups
yiluchen1066 Jul 23, 2026
336cdda
small clean up on ZERO/ONE...
yiluchen1066 Jul 23, 2026
874ac28
cleanups
yiluchen1066 Jul 23, 2026
d36d592
pass in grid instead of ncells and nlevs to muphys component
yiluchen1066 Jul 24, 2026
6108160
more cleanups
yiluchen1066 Jul 24, 2026
08399a3
small clean up on the docs mostly
yiluchen1066 Jul 27, 2026
a67a489
merge with main
yiluchen1066 Jul 27, 2026
d763a33
small clean up and added physics_driver to the workflow matrix
yiluchen1066 Jul 27, 2026
5a95578
Merge branch 'main' into physics_driver_l2
yiluchen1066 Jul 27, 2026
241d6a0
fix the pre-commit
yiluchen1066 Jul 27, 2026
8d2eb57
Merge branch 'main' into physics_driver_l2
yiluchen1066 Jul 27, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,208 @@
# ICON4Py - ICON inspired code in Python and GT4Py
#
# Copyright (c) 2022-2024, ETH Zurich and MeteoSwiss
# All rights reserved.
#
# Please, refer to the LICENSE file in the root directory.
# SPDX-License-Identifier: BSD-3-Clause

from __future__ import annotations

import datetime
import types
from collections.abc import Callable
from typing import TYPE_CHECKING, Any, cast

import gt4py.next as gtx

from icon4py.model.atmosphere.subgrid_scale_physics.muphys import (
config as muphys_config,
data as muphys_data,
)
from icon4py.model.atmosphere.subgrid_scale_physics.muphys.core.definitions import SPECIES, Q
from icon4py.model.atmosphere.subgrid_scale_physics.muphys.driver.run_full_muphys import (
setup_muphys,
)
from icon4py.model.common import (
dimension as dims,
field_type_aliases as fa,
model_backends,
model_options,
type_alias as ta,
)
from icon4py.model.common.diagnostic_calculations.stencils import calculate_tendency
from icon4py.model.common.math.stencils import generic_math_operations


if TYPE_CHECKING:
import gt4py.next.typing as gtx_typing

from icon4py.model.common.states import model


class MuphysComponent:
"""The muphys Granule: a per-process adapter wrapping the muphys microphysics program."""
Comment thread
yiluchen1066 marked this conversation as resolved.
Outdated

# TODO (Yilu): inherit the Component protocol once it is formalized (deferred to a separate PR).

inputs_properties = muphys_data.INPUTS_PROPERTIES
outputs_properties = muphys_data.OUTPUTS_PROPERTIES

def __init__(
self,
ncells: int,
nlev: int,
Comment thread
yiluchen1066 marked this conversation as resolved.
Outdated
dtime: datetime.timedelta,
qnc: float,
backend: gtx_typing.Backend | None = None,
*,
scheme: muphys_config.MuphysScheme = muphys_config.MuphysScheme.KOKKOS_MUPHYS,
step: Callable[..., Any] | None = None,
Comment thread
yiluchen1066 marked this conversation as resolved.
) -> None:
self._ncells = ncells
self._nlev = nlev
self._dt_seconds = dtime.total_seconds()
self._qnc = qnc
self._backend = model_options.customize_backend(program=None, backend=backend)

horizontal_sizes = {
"horizontal_start": gtx.int32(0),
"horizontal_end": gtx.int32(self._ncells),
Comment thread
yiluchen1066 marked this conversation as resolved.
}
vertical_sizes = {"vertical_start": gtx.int32(0), "vertical_end": gtx.int32(self._nlev)}

@OngChia OngChia Jul 21, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

usually in ICON or other models, there is k level above which microphysics is turned off (usually for manual performance tuning, may be for stability issues in some rare cases). In ICON, it is kstart_moist. How about passing in vertical params? Could you check whether we can remove this manual performance tuning so that we always run the microphysics for all levels?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will check this

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think kstart_moist is only used in nwp physics, not aes

self._calculate_tendency = model_options.setup_program(
program=calculate_tendency.calculate_cell_kdim_field_tendency,
backend=self._backend,
horizontal_sizes=horizontal_sizes,
vertical_sizes=vertical_sizes,
offset_provider={},
)
self._copy_field = model_options.setup_program(
program=generic_math_operations.copy_field_on_cell_k,
backend=self._backend,
horizontal_sizes=horizontal_sizes,
vertical_sizes=vertical_sizes,
offset_provider={},
)

allocator = model_backends.get_allocator(backend)

if step is None:
sizes = types.SimpleNamespace(ncells=ncells, nlev=nlev)
step = setup_muphys(
inp=sizes, # type: ignore[arg-type] # only .ncells/.nlev are read
dt=self._dt_seconds,
qnc=qnc,
backend=backend,
single_program=False,
scheme=scheme,
)
self._step = step

cell_k_domain = gtx.domain({dims.CellDim: ncells, dims.KDim: nlev})
self._t_out = gtx.zeros(cell_k_domain, dtype=ta.wpfloat, allocator=allocator)
self._q_out = Q(
v=gtx.zeros(cell_k_domain, dtype=ta.wpfloat, allocator=allocator),
c=gtx.zeros(cell_k_domain, dtype=ta.wpfloat, allocator=allocator),
r=gtx.zeros(cell_k_domain, dtype=ta.wpfloat, allocator=allocator),
s=gtx.zeros(cell_k_domain, dtype=ta.wpfloat, allocator=allocator),
i=gtx.zeros(cell_k_domain, dtype=ta.wpfloat, allocator=allocator),
g=gtx.zeros(cell_k_domain, dtype=ta.wpfloat, allocator=allocator),
)
self._pflx = gtx.zeros(cell_k_domain, dtype=ta.wpfloat, allocator=allocator)
self._pr = gtx.zeros(cell_k_domain, dtype=ta.wpfloat, allocator=allocator)
self._ps = gtx.zeros(cell_k_domain, dtype=ta.wpfloat, allocator=allocator)
self._pi = gtx.zeros(cell_k_domain, dtype=ta.wpfloat, allocator=allocator)
self._pg = gtx.zeros(cell_k_domain, dtype=ta.wpfloat, allocator=allocator)
self._pre = gtx.zeros(cell_k_domain, dtype=ta.wpfloat, allocator=allocator)

self._tendencies: dict[str, fa.CellKField[ta.wpfloat]] = {
"tend_temperature": gtx.zeros(cell_k_domain, dtype=ta.wpfloat, allocator=allocator),
"tend_qv": gtx.zeros(cell_k_domain, dtype=ta.wpfloat, allocator=allocator),
"tend_qc": gtx.zeros(cell_k_domain, dtype=ta.wpfloat, allocator=allocator),
"tend_qr": gtx.zeros(cell_k_domain, dtype=ta.wpfloat, allocator=allocator),
"tend_qs": gtx.zeros(cell_k_domain, dtype=ta.wpfloat, allocator=allocator),
"tend_qi": gtx.zeros(cell_k_domain, dtype=ta.wpfloat, allocator=allocator),
"tend_qg": gtx.zeros(cell_k_domain, dtype=ta.wpfloat, allocator=allocator),
}

self._te_in = gtx.zeros(cell_k_domain, dtype=ta.wpfloat, allocator=allocator)
self._q_in = Q(
v=gtx.zeros(cell_k_domain, dtype=ta.wpfloat, allocator=allocator),
c=gtx.zeros(cell_k_domain, dtype=ta.wpfloat, allocator=allocator),
r=gtx.zeros(cell_k_domain, dtype=ta.wpfloat, allocator=allocator),
s=gtx.zeros(cell_k_domain, dtype=ta.wpfloat, allocator=allocator),
i=gtx.zeros(cell_k_domain, dtype=ta.wpfloat, allocator=allocator),
g=gtx.zeros(cell_k_domain, dtype=ta.wpfloat, allocator=allocator),
)

def _to_tendency(
self,
old: fa.CellKField[ta.wpfloat],
new: fa.CellKField[ta.wpfloat],
out: fa.CellKField[ta.wpfloat],
) -> None:
"""``out = (new - old) / dt`` over the whole column."""
self._calculate_tendency(
dtime=self._dt_seconds,
old_field=old,
new_field=new,
tendency=out,
)
Comment thread
yiluchen1066 marked this conversation as resolved.
Outdated

def _copy_into(self, src: fa.CellKField[ta.wpfloat], dst: fa.CellKField[ta.wpfloat]) -> None:
"""Copy ``src`` into the granule-owned buffer ``dst``."""
self._copy_field(
field=src,
output_field=dst,
)
Comment thread
yiluchen1066 marked this conversation as resolved.
Outdated

def __call__(
self, state: dict[str, model.DataField], time_step: datetime.datetime
) -> dict[str, model.DataField]:
Comment on lines +153 to +155

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question for myself:I suppose this is one of the critical interfaces, though the signature could likely be tightened to something resembling state: (Something?)State, time_step: datetime.datetime? Or the whole class should inherit from a protocol.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor note as well that may need broader discussion: If this is indeed the protocol, my preference is to give this a name rather than using the generic __call__ operator, but I also know we already use __call__ extensively, so may be me still not being used to __call__ being used everywhere.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And further thinking out loud: Could this be based on a more generic TendencyAdaptor? The muphys-specific parts seem quite minimal and the interesting logic seems to be the _to_tendency parts.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and time_step should be datetime.timedelta if we decide to use timedeltas for parametrizing (as opposed to the integer time steps)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

correction: it should be RelativeTime either from standalone_driver or time.py depending on whether #1353 is merged before this one or not

"""Run muphys, then convert its updated state into tendencies.

muphys returns updated state (t_out, q_out); this boundary converts it
to tendencies ``(new - old) / dt`` s. Precip outputs are diagnostics, passed straight through.
"""
# cast from generic ``DataField`` to bare gt4py fields
fields = cast("dict[str, fa.CellKField[ta.wpfloat]]", state)

self._copy_into(fields["te"], self._te_in)
for s in SPECIES:
self._copy_into(fields[f"q{s}"], getattr(self._q_in, s))

self._step(
dz=fields["dz"],
te=self._te_in,
p=fields["p"],
rho=fields["rho"],
q_in=self._q_in,
q_out=self._q_out,
t_out=self._t_out,
pflx=self._pflx,
pr=self._pr,
ps=self._ps,
pi=self._pi,
pg=self._pg,
pre=self._pre,
)

self._to_tendency(fields["te"], self._t_out, self._tendencies["tend_temperature"])
for s in SPECIES:
self._to_tendency(
fields[f"q{s}"], getattr(self._q_out, s), self._tendencies[f"tend_q{s}"]
)

return cast(
"dict[str, model.DataField]",
Comment thread
yiluchen1066 marked this conversation as resolved.
{
**self._tendencies,
"pflx": self._pflx,
"pr": self._pr,
"ps": self._ps,
"pi": self._pi,
"pg": self._pg,
"pre": self._pre,
},
)
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# ICON4Py - ICON inspired code in Python and GT4Py
#
# Copyright (c) 2022-2024, ETH Zurich and MeteoSwiss
# All rights reserved.
#
# Please, refer to the LICENSE file in the root directory.
# SPDX-License-Identifier: BSD-3-Clause

from __future__ import annotations

import dataclasses
import enum


class MuphysScheme(enum.Enum):
"""Selects between the two graupel microphysics formulations.

KOKKOS_MUPHYS follows the muphys C++/Kokkos reference implementation (the
original source of this port, validated by the netCDF-based muphys tests).
AES_GRAUPEL follows the newer MPIM rain-microphysics revisions carried by
icon-nwp (mo_aes_graupel.f90), validated against the aes-graupel serialbox
savepoints.
"""

KOKKOS_MUPHYS = "kokkos_muphys"
AES_GRAUPEL = "aes_graupel"


@dataclasses.dataclass(frozen=True)
class MuphysConfig:
"""Configuration for the muphys microphysics component."""

qnc: float = 50.0e6 # cloud droplet number concentration [m^-3], matches Fortran cloud_num

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ignore my comment here. This is just for myself.
I am surprised that AES uses a fixed qnc. Check why.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it is replicated in Cell-K array form in fortran, but then aes_graupel always uses the first element of the array. so they had thought about this or even started working on cell-dependent qnc but did not implement it in the end

scheme: MuphysScheme = MuphysScheme.AES_GRAUPEL # the driver validates against icon-nwp
Comment thread
yiluchen1066 marked this conversation as resolved.
Outdated
Original file line number Diff line number Diff line change
Expand Up @@ -83,3 +83,40 @@ class IndexConsts(ta.wpfloat, enum.Enum):
prefactor_g = 12.24
exponent_g = 0.217
offset_g = 1.0e-08


class AesGraupelConsts(ta.wpfloat, enum.Enum):
# Constants of the newer MPIM rain-microphysics revisions carried by icon-nwp
# (mo_aes_graupel.f90), used by the AES_GRAUPEL scheme variants only.
# Hydrometeor-density clamp bounds, shared by all polynomial fits and fall speeds
rhox_mn = 3.26216e-08
rhox_mx = 6.97604e-03
# cloud_to_rain: accretion kernel, degree-4 polynomial in log(clamped rho*qr)
a_ac_1 = -2.155543e00
a_ac_2 = -1.148491e00
a_ac_3 = -1.882563e-02
a_ac_4 = 2.941391e-03
a_ac_5 = 5.575598e-05
# rain_to_vapor: evaporation, exp of degree-4 polynomial in log(clamped rho*qr)
a_ev_1 = -5.532194e00
a_ev_2 = 2.432848e-01
a_ev_3 = -4.145391e-02
a_ev_4 = -1.798439e-03
a_ev_5 = -1.405764e-05
# rain fall speed: degree-4 polynomial in log(clamped rho*qr); the constant term
# vm_a_r_1 is NOT multiplied by sqrt(rho_00/rho) in the Fortran (operator
# precedence in mo_aes_graupel.f90 vm()), reproduced faithfully
vm_a_r_1 = -5.91051e-01
vm_a_r_2 = -5.37440e00
vm_a_r_3 = -1.00459e00
vm_a_r_4 = -6.44895e-02
vm_a_r_5 = -1.40361e-03
# ice / snow / graupel fall speeds: prefactor * clamped_rhox**exponent * density correction
vm_prefactor_i = 0.80
vm_exponent_i = 0.160
vm_prefactor_s = 115.60 # 2 * 57.80
vm_exponent_s = 0.16666666666666666
Comment thread
yiluchen1066 marked this conversation as resolved.
Outdated
vm_prefactor_g = 12.24
vm_exponent_g = 0.217
# snow_number: lower clamp on the snow mass density rho*qs
rho_s_mn = 2.0e-7
Original file line number Diff line number Diff line change
Expand Up @@ -27,3 +27,6 @@ class Q_scalar(NamedTuple):
s: ta.wpfloat # Specific snow water
i: ta.wpfloat # Specific ice water content
g: ta.wpfloat # Specific graupel water content


SPECIES: tuple[str, ...] = Q._fields
Loading
Loading