Nightly tests fail randomly because of MAGICC download failing

[phackstock]

See example here: https://github.com/iiasa/climate-assessment/actions/runs/18611100199/job/53069336906.

Job output:

--2025-10-18 05:02:47--  http://***
Resolving www.magicc.org (www.magicc.org)... 13.52.188.95, 52.52.192.191, 2600:1f1c:446:4900::259, ...
Connecting to www.magicc.org (www.magicc.org)|13.52.188.95|:80... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://*** [following]
--2025-10-18 05:02:48--  https://***
Connecting to www.magicc.org (www.magicc.org)|13.52.188.95|:443... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://magicc.org/api/download?type=MAGICC7&*** [following]
--2025-10-18 05:02:48--  https://magicc.org/api/download?type=MAGICC7&***
Resolving magicc.org (magicc.org)... 75.2.60.5
Connecting to magicc.org (magicc.org)|75.2.60.5|:443... connected.
Unable to establish SSL connection.
Error: Process completed with exit code 4.

What's weird is that sometimes it works, sometimes it does not.

Since we're running the tests for python 3.11, 3.12 and 3.13 in parallel, one quick hypothesis would be that maybe hitting the MAGICC download endpoint three times in rapid succession causes some kind of issue. If that is in fact the issue, I would see two potential remedies:

1. Limit the number of concurrent jobs to 1 (as described here: https://docs.github.com/en/actions/how-tos/write-workflows/choose-what-workflows-do/run-job-variations#defining-the-maximum-number-of-concurrent-jobs)
2. Download MAGICC only once and then provide it to the three parallel jobs.

@znicholls and @lewisjared do you guys have an idea?

[lewisjared]

I see 3 successful requests in the logs so I'm not sure what happened. I've done some maintenance and made a new deployment with more logging in case it happens again. I don't see why it would hang on 3 concurrent downloads. It's a serverless app so maybe something related to cold-starts.

The MAGICC download can be cached so it doesn't need to be downloaded on every run.

[phackstock]

Thanks a lot @lewisjared, let's see how the next couple of runs will go.

[phackstock]

Seems that the last run worked, looking good so far. I'll keep an eye on this for a few more days and then close the issue if it's still looking good by then.