The memory-leak check in CIME/SystemTests/system_tests_common.py reports the elapsed time in "days" using arithmetic that is not actually days. For any run longer than a few days, the reported number is off by orders of magnitude and confusing to users.
Example
A 5-year SMS_Ly5 ELM test produced this TestStatus.log entry:
memleak detected, memory went from 1499.740000 to 1578.320000 in 41129 days
The test only ran for 5 model years (~1825 days), not 41,129.
Root cause
perf_get_memory_list (in CIME/baselines/performance.py) parses cpl.log lines of the form:
... model date = 00051231 ... memory = 1578.320 MB ... highwater ...
using the regex
meminfo = re.compile(r".*model date =\s+(\w+).*memory =\s+(\d+\.?\d+).*highwater")
and stores the first capture group as float(m.group(1)). That value is the model date encoded as YYYYMMDD (e.g. 51231.0), not a day count.
Then in perf_check_for_memory_leak (CIME/SystemTests/system_tests_common.py, around line 1109):
# last day - second day, skip first day, can be too low while initializing
elapsed_days = int(memlist[-1][0]) - int(memlist[1][0])
This subtracts the two encoded stamps directly. For the run above, the second sample was model date = 00010102 and the last sample was model date = 00051231, so:
51231 - 10102 = 41129
which is exactly the "41129 days" printed in the message. The result mixes years (×10000), months (×100), and day-of-month, so it grows by ~10000 per model year plus ~100 per model month and is not a meaningful count of days.
Impact
- Cosmetic / user-facing only. The pass/fail decision itself is unaffected — that uses finalmem/originalmem and the tolerance ratio, both of which are correct.
- The MEMLEAK TestStatus.log message is misleading, and can send developers looking for a bug in the wrong place ("why does it think my 5-year run went for 100+ years?").
- Any downstream tooling that parses this comment and interprets the number as days will be wrong for any run longer than a few days.
Suggested fix
Decode the YYYYMMDD stamp back into (year, month, day) before differencing, then report the elapsed time in years/months/days (dropping zero components), e.g.:
memleak detected, memory went from 1499.740000 to 1578.320000 in 4 years, 11 months, 29 days
The same message should be emitted by any component-specific config.perf_check_for_memory_leak override so the wording stays consistent.
The memory-leak check in CIME/SystemTests/system_tests_common.py reports the elapsed time in "days" using arithmetic that is not actually days. For any run longer than a few days, the reported number is off by orders of magnitude and confusing to users.
Example
A 5-year SMS_Ly5 ELM test produced this TestStatus.log entry:
memleak detected, memory went from 1499.740000 to 1578.320000 in 41129 days
The test only ran for 5 model years (~1825 days), not 41,129.
Root cause
perf_get_memory_list (in CIME/baselines/performance.py) parses cpl.log lines of the form:
... model date = 00051231 ... memory = 1578.320 MB ... highwater ...using the regex
meminfo = re.compile(r".*model date =\s+(\w+).*memory =\s+(\d+\.?\d+).*highwater")and stores the first capture group as float(m.group(1)). That value is the model date encoded as YYYYMMDD (e.g. 51231.0), not a day count.
Then in perf_check_for_memory_leak (CIME/SystemTests/system_tests_common.py, around line 1109):
This subtracts the two encoded stamps directly. For the run above, the second sample was model date = 00010102 and the last sample was model date = 00051231, so:
51231 - 10102 = 41129
which is exactly the "41129 days" printed in the message. The result mixes years (×10000), months (×100), and day-of-month, so it grows by ~10000 per model year plus ~100 per model month and is not a meaningful count of days.
Impact
Suggested fix
Decode the YYYYMMDD stamp back into (year, month, day) before differencing, then report the elapsed time in years/months/days (dropping zero components), e.g.:
memleak detected, memory went from 1499.740000 to 1578.320000 in 4 years, 11 months, 29 days
The same message should be emitted by any component-specific config.perf_check_for_memory_leak override so the wording stays consistent.