-
Notifications
You must be signed in to change notification settings - Fork 11
Physics driver with Muphys #1301
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
yiluchen1066
wants to merge
92
commits into
main
Choose a base branch
from
physics_driver_l2
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 17 commits
Commits
Show all changes
92 commits
Select commit
Hold shift + click to select a range
717b3e2
first commit: implement physics interface (WIP)
yiluchen1066 e0bfe9e
pre commit: adding licence
yiluchen1066 c38a902
Merge branch 'main' into physics_driver_l2
yiluchen1066 1466c21
add physics state (MuphysState)
yiluchen1066 a3bb29a
add the wrapper for muphys
yiluchen1066 66617a4
rename the muphys granule
yiluchen1066 60c9c6a
up + add the datatest for muphys granule wrapper
yiluchen1066 9f193b9
fix the test
yiluchen1066 47f0aa7
some small fixes and renaming
yiluchen1066 b6d166a
move the muphysState to muphys
yiluchen1066 c325c5f
move the physics_driver to a physics_interface package
yiluchen1066 b035382
move the physics_driver to a physics_interface package (more)
yiluchen1066 730893f
small fix
yiluchen1066 f74672b
some small clean ups
yiluchen1066 2c61eae
Merge branch 'main' into physics_driver_l2
yiluchen1066 e720c40
solve the inplace update in muphys, input_properties and some mypy stuff
yiluchen1066 4722175
Merge branch 'main' into physics_driver_l2
yiluchen1066 fea8cd8
WIP (resolving comments)
yiluchen1066 42ddc80
apply tracers pr
yiluchen1066 bb30857
merge from main and resolve conflict
yiluchen1066 044b2c6
merge from main and resolve conflict
yiluchen1066 596286b
fix the Any type in the protocol
yiluchen1066 98be0a6
add the per-process state to the physics driver
yiluchen1066 5ef96df
small fix
yiluchen1066 5f714e2
Merge branch 'main' into physics_driver_l2
yiluchen1066 eac8adb
more fix on muphys component
yiluchen1066 47d8dab
plug in muphys to standalone driver
yiluchen1066 acac3fd
Merge branch 'main' into physics_driver_l2
yiluchen1066 dd3bc13
bump version
jcanton c7e780f
add experiment description
jcanton 6aa8056
more clean up and renaming
yiluchen1066 acf8a20
rename PhysicsStateProtocol to PhysicsState
yiluchen1066 701686a
solve the KeyError
yiluchen1066 7569596
Merge branch 'main' into add_exp_ape_aes
yiluchen1066 41cc4ae
Merge branch 'main' into physics_driver_l2
yiluchen1066 8aa91e8
small fix on comment
yiluchen1066 f20e891
removing the duplication on the definition of spiecies
yiluchen1066 dd3dd17
porting wip
jcanton 302b573
move constants to constants.py
jcanton 516bc7e
add thermo helpers
jcanton ffdb42b
move this
jcanton 46c5d68
pass global reductions
jcanton cb028d0
add tests
jcanton ddae38d
this should work
jcanton 7520535
mostly refactor
yiluchen1066 f011e9b
refactoring
jcanton 1266ac2
somebody thought this was a great idea.... and then asked a friend to…
jcanton 568d35e
bugfix, now passing
jcanton c001e14
cleanup
jcanton c40475b
pre-commit
jcanton 09b8924
Merge branch 'main' into add_exp_ape_aes
jcanton 4b5b5fb
if it's always present we should not use .get
jcanton 895ed66
add comments on confusing return
jcanton 79f1ec0
tiny lookup save
jcanton ba77f68
kwargs
jcanton 2437069
renaming for clarity is always nice
jcanton 2482ce0
explaining is always nice
jcanton 3f4d9e6
fix regex to be more robust
jcanton 27ac411
it should never be missing
jcanton 731fa38
Merge branch 'main' into add_exp_ape_aes
jcanton c571a20
Merge branch 'main' into physics_driver_l2
yiluchen1066 dee4edd
merge PR #1347
yiluchen1066 59cae8d
adding the datatest for standalone_driver+muphys run
yiluchen1066 55f036c
Move ProcessTimeControl to its own file; split forcing_mode to Physic…
yiluchen1066 89b4b17
make the precip_diagnostic property for IO/plotting
yiluchen1066 59ed7a6
Add time-step-exit and aes-graupel savepoint validation
jcanton b347a62
tiny fix
jcanton 25c2854
Rename aes_graupel to muphys on the icon4py side
jcanton fee4c76
Port the icon-nwp graupel formulation as a selectable MuphysScheme
jcanton 21b9bfb
update on the datatest and rebase with #1363
yiluchen1066 8119c66
small up
yiluchen1066 ad485d1
adding new stencils for updating the exner and theta_v
yiluchen1066 2890d83
Merge branch 'main' into physics_driver_l2
yiluchen1066 9524a28
some small updates and refactors
yiluchen1066 42b46fd
Rename muphys schemes to KOKKOS_MUPHYS / AES_GRAUPEL
yiluchen1066 85dd678
fix the NaN in the test
yiluchen1066 bb80c24
resolve one small commit
yiluchen1066 246f033
merge main
yiluchen1066 f02be4a
chaneg the relativeTime for time_step in component
yiluchen1066 dd3e382
rename everywhere consistently to physicsdriver including the package…
yiluchen1066 8f3352c
small ups
yiluchen1066 336cdda
small clean up on ZERO/ONE...
yiluchen1066 874ac28
cleanups
yiluchen1066 d36d592
pass in grid instead of ncells and nlevs to muphys component
yiluchen1066 6108160
more cleanups
yiluchen1066 08399a3
small clean up on the docs mostly
yiluchen1066 a67a489
merge with main
yiluchen1066 d763a33
small clean up and added physics_driver to the workflow matrix
yiluchen1066 5a95578
Merge branch 'main' into physics_driver_l2
yiluchen1066 241d6a0
fix the pre-commit
yiluchen1066 8d2eb57
Merge branch 'main' into physics_driver_l2
yiluchen1066 e749faf
small cleans and left some small todos
yiluchen1066 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
63 changes: 63 additions & 0 deletions
63
...id_scale_physics/muphys/src/icon4py/model/atmosphere/subgrid_scale_physics/muphys/data.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,63 @@ | ||
| # ICON4Py - ICON inspired code in Python and GT4Py | ||
| # | ||
| # Copyright (c) 2022-2024, ETH Zurich and MeteoSwiss | ||
| # All rights reserved. | ||
| # | ||
| # Please, refer to the LICENSE file in the root directory. | ||
| # SPDX-License-Identifier: BSD-3-Clause | ||
|
|
||
| """Field metadata for the MuphysGranule (Component) input/output contract. | ||
| """ | ||
|
|
||
| from __future__ import annotations | ||
|
|
||
| from icon4py.model.common.states import data, model | ||
|
|
||
|
|
||
| # muphys species keys, in the order of the muphys ``Q`` tuple. | ||
| _SPECIES = ("v", "c", "r", "s", "i", "g") | ||
|
|
||
| # per-species tracer identity, pulled from the common registry. | ||
| _TRACER: dict[str, model.FieldMetaData] = { | ||
| "v": data.COMMON_TRACER_CF_ATTRIBUTES["specific_humidity"], | ||
| "c": data.COMMON_TRACER_CF_ATTRIBUTES["specific_cloud"], | ||
| "r": data.COMMON_TRACER_CF_ATTRIBUTES["specific_rain"], | ||
| "s": data.COMMON_TRACER_CF_ATTRIBUTES["specific_snow"], | ||
| "i": data.COMMON_TRACER_CF_ATTRIBUTES["specific_ice"], | ||
| "g": data.COMMON_TRACER_CF_ATTRIBUTES["specific_graupel"], | ||
| } | ||
|
|
||
|
|
||
| def _tendency_of(base: model.FieldMetaData) -> model.FieldMetaData: | ||
| """Derive a tendency's metadata from the field it is the tendency of.""" | ||
| return dict( | ||
| standard_name=f"tendency_of_{base['standard_name']}", | ||
| units=f"{base['units']} s-1", | ||
| kind="tendency", | ||
| ) | ||
|
|
||
| INPUTS_PROPERTIES: dict[str, model.FieldMetaData] = { | ||
| "dz": dict(standard_name="layer_thickness", units="m"), | ||
| "te": data.DIAGNOSTIC_CF_ATTRIBUTES["temperature"], | ||
| "p": data.DIAGNOSTIC_CF_ATTRIBUTES["pressure"], | ||
| "rho": data.PROGNOSTIC_CF_ATTRIBUTES["air_density"], | ||
| **{f"q{s}": _TRACER[s] for s in _SPECIES}, | ||
| } | ||
|
|
||
| # muphys precip diagnostics -- process-specific (not common fields): pflx is the 3D | ||
| # flux, pr/ps/pi/pg surface rates, pre the surface precip energy flux. | ||
| _PRECIP_PROPERTIES: dict[str, model.FieldMetaData] = { | ||
| "pflx": dict(standard_name="precipitation_flux", units="kg m-2 s-1", kind="diagnostic"), | ||
| "pr": dict(standard_name="rainfall_flux", units="kg m-2 s-1", kind="diagnostic"), | ||
| "ps": dict(standard_name="snowfall_flux", units="kg m-2 s-1", kind="diagnostic"), | ||
| "pi": dict(standard_name="ice_fall_flux", units="kg m-2 s-1", kind="diagnostic"), | ||
| "pg": dict(standard_name="graupel_fall_flux", units="kg m-2 s-1", kind="diagnostic"), | ||
| "pre": dict(standard_name="precipitation_energy_flux", units="W m-2", kind="diagnostic"), | ||
| } | ||
|
yiluchen1066 marked this conversation as resolved.
Outdated
|
||
|
|
||
| # muphys outputs: 7 tendencies (derived from the input identities) + 6 precip diagnostics | ||
| OUTPUTS_PROPERTIES: dict[str, model.FieldMetaData] = { | ||
| "tend_temperature": _tendency_of(data.DIAGNOSTIC_CF_ATTRIBUTES["temperature"]), | ||
| **{f"tend_q{s}": _tendency_of(_TRACER[s]) for s in _SPECIES}, | ||
| **_PRECIP_PROPERTIES, | ||
| } | ||
224 changes: 224 additions & 0 deletions
224
...scale_physics/muphys/src/icon4py/model/atmosphere/subgrid_scale_physics/muphys/granule.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,224 @@ | ||
| # ICON4Py - ICON inspired code in Python and GT4Py | ||
| # | ||
| # Copyright (c) 2022-2024, ETH Zurich and MeteoSwiss | ||
| # All rights reserved. | ||
| # | ||
| # Please, refer to the LICENSE file in the root directory. | ||
| # SPDX-License-Identifier: BSD-3-Clause | ||
|
|
||
| from __future__ import annotations | ||
|
|
||
| import datetime | ||
| import types | ||
| from collections.abc import Callable | ||
| from typing import TYPE_CHECKING, Any, cast | ||
|
|
||
| import gt4py.next as gtx | ||
|
|
||
| from icon4py.model.atmosphere.subgrid_scale_physics.muphys import data as muphys_data | ||
| from icon4py.model.atmosphere.subgrid_scale_physics.muphys.core.definitions import Q | ||
| from icon4py.model.atmosphere.subgrid_scale_physics.muphys.driver import run_full_muphys | ||
| from icon4py.model.common import ( | ||
| dimension as dims, | ||
| field_type_aliases as fa, | ||
| model_backends, | ||
| model_options, | ||
| type_alias as ta, | ||
| ) | ||
| from icon4py.model.common.diagnostic_calculations.stencils import calculate_tendency | ||
|
|
||
|
|
||
| if TYPE_CHECKING: | ||
| import gt4py.next.typing as gtx_typing | ||
|
|
||
| from icon4py.model.common.states import model | ||
|
|
||
| _SPECIES = ("v", "c", "r", "s", "i", "g") | ||
|
|
||
|
|
||
| @gtx.field_operator | ||
| def _copy(field: fa.CellKField[ta.wpfloat]) -> fa.CellKField[ta.wpfloat]: | ||
| return field | ||
|
|
||
|
|
||
| @gtx.program(grid_type=gtx.GridType.UNSTRUCTURED) | ||
| def copy_field( # noqa: PLR0917 # stencil params referenced in domain specs stay positional | ||
| field: fa.CellKField[ta.wpfloat], | ||
| result: fa.CellKField[ta.wpfloat], | ||
| horizontal_start: gtx.int32, | ||
| horizontal_end: gtx.int32, | ||
| vertical_start: gtx.int32, | ||
| vertical_end: gtx.int32, | ||
| ) -> None: | ||
| _copy( | ||
| field, | ||
| out=result, | ||
| domain={ | ||
| dims.CellDim: (horizontal_start, horizontal_end), | ||
| dims.KDim: (vertical_start, vertical_end), | ||
| }, | ||
| ) | ||
|
yiluchen1066 marked this conversation as resolved.
Outdated
|
||
|
|
||
|
|
||
| class MuphysGranule: | ||
|
yiluchen1066 marked this conversation as resolved.
Outdated
|
||
| """L4 per-process adapter wrapping the muphys microphysics program.""" | ||
|
yiluchen1066 marked this conversation as resolved.
Outdated
|
||
|
|
||
| inputs_properties = muphys_data.INPUTS_PROPERTIES | ||
| outputs_properties = muphys_data.OUTPUTS_PROPERTIES | ||
|
|
||
| def __init__( | ||
| self, | ||
| ncells: int, | ||
| nlev: int, | ||
| dt: float, | ||
| qnc: float, | ||
| backend: gtx_typing.Backend | None = None, | ||
| *, | ||
| muphys_step: Callable[..., Any] | None = None, | ||
| ) -> None: | ||
| self._ncells = ncells | ||
| self._nlev = nlev | ||
| self._dt = dt | ||
| self._qnc = qnc | ||
| self._backend = model_options.customize_backend(None, backend) | ||
|
yiluchen1066 marked this conversation as resolved.
Outdated
|
||
|
|
||
| allocator = model_backends.get_allocator(backend) | ||
|
|
||
| if muphys_step is None: | ||
| # TODO (Yilu): what does the comment mean? | ||
| # Build the verified separate-path step (graupel + saturation adjustment). | ||
| # setup_muphys only reads `.ncells`/`.nlev` off `inp`, so a sizes shim suffices. | ||
| sizes = types.SimpleNamespace(ncells=ncells, nlev=nlev) | ||
| muphys_step = run_full_muphys.setup_muphys( | ||
| inp=sizes, # type: ignore[arg-type] # only .ncells/.nlev are read | ||
| dt=dt, | ||
| qnc=qnc, | ||
| backend=backend, | ||
| single_program=False, | ||
| ) | ||
| self._muphys_step = muphys_step | ||
|
yiluchen1066 marked this conversation as resolved.
Outdated
|
||
|
|
||
| domain = gtx.domain({dims.CellDim: ncells, dims.KDim: nlev}) | ||
|
yiluchen1066 marked this conversation as resolved.
Outdated
|
||
| self._t_out = gtx.zeros(domain, dtype=ta.wpfloat, allocator=allocator) | ||
| self._q_out = Q( | ||
| v=gtx.zeros(domain, dtype=ta.wpfloat, allocator=allocator), | ||
| c=gtx.zeros(domain, dtype=ta.wpfloat, allocator=allocator), | ||
| r=gtx.zeros(domain, dtype=ta.wpfloat, allocator=allocator), | ||
| s=gtx.zeros(domain, dtype=ta.wpfloat, allocator=allocator), | ||
| i=gtx.zeros(domain, dtype=ta.wpfloat, allocator=allocator), | ||
| g=gtx.zeros(domain, dtype=ta.wpfloat, allocator=allocator), | ||
| ) | ||
| self._pflx = gtx.zeros(domain, dtype=ta.wpfloat, allocator=allocator) | ||
| self._pr = gtx.zeros(domain, dtype=ta.wpfloat, allocator=allocator) | ||
| self._ps = gtx.zeros(domain, dtype=ta.wpfloat, allocator=allocator) | ||
| self._pi = gtx.zeros(domain, dtype=ta.wpfloat, allocator=allocator) | ||
| self._pg = gtx.zeros(domain, dtype=ta.wpfloat, allocator=allocator) | ||
| self._pre = gtx.zeros(domain, dtype=ta.wpfloat, allocator=allocator) | ||
|
|
||
| self._tendencies: dict[str, fa.CellKField[ta.wpfloat]] = { | ||
| "tend_temperature": gtx.zeros(domain, dtype=ta.wpfloat, allocator=allocator), | ||
| "tend_qv": gtx.zeros(domain, dtype=ta.wpfloat, allocator=allocator), | ||
| "tend_qc": gtx.zeros(domain, dtype=ta.wpfloat, allocator=allocator), | ||
| "tend_qr": gtx.zeros(domain, dtype=ta.wpfloat, allocator=allocator), | ||
| "tend_qs": gtx.zeros(domain, dtype=ta.wpfloat, allocator=allocator), | ||
| "tend_qi": gtx.zeros(domain, dtype=ta.wpfloat, allocator=allocator), | ||
| "tend_qg": gtx.zeros(domain, dtype=ta.wpfloat, allocator=allocator), | ||
| } | ||
|
|
||
| self._te_in = gtx.zeros(domain, dtype=ta.wpfloat, allocator=allocator) | ||
| self._q_in = Q( | ||
| v=gtx.zeros(domain, dtype=ta.wpfloat, allocator=allocator), | ||
| c=gtx.zeros(domain, dtype=ta.wpfloat, allocator=allocator), | ||
| r=gtx.zeros(domain, dtype=ta.wpfloat, allocator=allocator), | ||
| s=gtx.zeros(domain, dtype=ta.wpfloat, allocator=allocator), | ||
| i=gtx.zeros(domain, dtype=ta.wpfloat, allocator=allocator), | ||
| g=gtx.zeros(domain, dtype=ta.wpfloat, allocator=allocator), | ||
| ) | ||
|
|
||
| def _to_tendency( | ||
| self, | ||
| old: fa.CellKField[ta.wpfloat], | ||
| new: fa.CellKField[ta.wpfloat], | ||
| out: fa.CellKField[ta.wpfloat], | ||
| ) -> None: | ||
| """``out = (new - old) / dt`` over the whole column.""" | ||
| program = calculate_tendency.calculate_cell_kdim_field_tendency | ||
| if self._backend is not None: | ||
|
yiluchen1066 marked this conversation as resolved.
Outdated
|
||
| program = program.with_backend(self._backend) | ||
| program( | ||
| dtime=self._dt, | ||
| old_field=old, | ||
| new_field=new, | ||
| tendency=out, | ||
| horizontal_start=0, | ||
| horizontal_end=self._ncells, | ||
| vertical_start=0, | ||
| vertical_end=self._nlev, | ||
| offset_provider={}, | ||
| ) | ||
|
|
||
| def _copy_into(self, src: fa.CellKField[ta.wpfloat], dst: fa.CellKField[ta.wpfloat]) -> None: | ||
| """Copy ``src`` into the granule-owned buffer ``dst``.""" | ||
| program = copy_field | ||
| if self._backend is not None: | ||
|
yiluchen1066 marked this conversation as resolved.
Outdated
|
||
| program = program.with_backend(self._backend) | ||
| program( | ||
| field=src, | ||
|
yiluchen1066 marked this conversation as resolved.
Outdated
|
||
| result=dst, | ||
| horizontal_start=0, | ||
| horizontal_end=self._ncells, | ||
| vertical_start=0, | ||
| vertical_end=self._nlev, | ||
| offset_provider={}, | ||
| ) | ||
|
|
||
| def __call__( | ||
| self, state: dict[str, model.DataField], time_step: datetime.datetime | ||
| ) -> dict[str, model.DataField]: | ||
| """Run muphys, then convert its updated state into tendencies. | ||
|
|
||
| muphys returns updated state (t_out, q_out); this boundary converts it | ||
| to tendencies ``(new - old) / dt`` s. Precip outputs are diagnostics, passed straight through. | ||
| """ | ||
| # cast from generic ``DataFeild`` to bare gt4py fields | ||
|
yiluchen1066 marked this conversation as resolved.
Outdated
|
||
| fields = cast("dict[str, fa.CellKField[ta.wpfloat]]", state) | ||
|
|
||
|
|
||
| self._copy_into(fields["te"], self._te_in) | ||
| for s in _SPECIES: | ||
| self._copy_into(fields[f"q{s}"], getattr(self._q_in, s)) | ||
|
|
||
| self._muphys_step( | ||
| dz=fields["dz"], | ||
| te=self._te_in, | ||
| p=fields["p"], | ||
| rho=fields["rho"], | ||
| q_in=self._q_in, | ||
| q_out=self._q_out, | ||
| t_out=self._t_out, | ||
| pflx=self._pflx, | ||
| pr=self._pr, | ||
| ps=self._ps, | ||
| pi=self._pi, | ||
| pg=self._pg, | ||
| pre=self._pre, | ||
| ) | ||
|
|
||
| self._to_tendency(fields["te"], self._t_out, self._tendencies["tend_temperature"]) | ||
| for s in _SPECIES: | ||
| self._to_tendency( | ||
| fields[f"q{s}"], getattr(self._q_out, s), self._tendencies[f"tend_q{s}"] | ||
| ) | ||
|
|
||
| return cast( | ||
| "dict[str, model.DataField]", | ||
| { | ||
| **self._tendencies, | ||
| "pflx": self._pflx, | ||
| "pr": self._pr, | ||
| "ps": self._ps, | ||
| "pi": self._pi, | ||
| "pg": self._pg, | ||
| "pre": self._pre, | ||
| }, | ||
| ) | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
also from a cheap-friend: