gdal-r-ci

CI infrastructure for R (and Python) geospatial packages against bleeding-edge GDAL.

GDAL, PROJ, and GEOS are built from source on plain ubuntu:26.04 with standard symbols — no internal PROJ, no dual-library setup. This means full R CMD check including --as-cran and PDF manual rendering works for all packages including sf and terra, with no --no-test-load workarounds. The same chain produces a single working environment for R-and-Python interop where reticulate, rasterio, sf, terra, gdalraster, vapour, and the osgeo Python bindings all link the same GDAL, the same PROJ, the same GEOS, and the same numpy ABI.

Images

Seven images are published to GHCR. The first four publish :latest (release) and :dev (GDAL HEAD) variants; the three leaf images publish :latest only.

Image	Contents	Use for
`ghcr.io/hypertidy/gdal-system`	GDAL + PROJ + GEOS + drivers + uv venv with numpy 2.x	Base for custom images
`ghcr.io/hypertidy/gdal-r`	+ R + dev tooling + tinytex	R package development
`ghcr.io/hypertidy/gdal-r-full`	+ gdalraster, sf, terra, vapour, gdalcubes	Package CI
`ghcr.io/hypertidy/gdal-r-python`	+ rasterio, fiona, xarray, zarr, kerchunk, virtualizarr…	R/Python interop
`ghcr.io/hypertidy/gdal-r-python-extras`	+ R kitchen sink (hypertidy stack, arrow, targets, dev tooling)	Workbench / interactive
`ghcr.io/hypertidy/gdal-r-python-extras-rust`	+ Rust 1.86.0 (clippy, rustfmt, rust-analyzer, lld, cargo-nextest)	Rust + geospatial dev
`ghcr.io/hypertidy/gdal-r-esmf`	gdal-r-python-extras-rust + ESMF 8.9.1 (ESMF_RegridWeightGen), ESMPy, xESMF, TempestRemap	Regridding oracles
`ghcr.io/hypertidy/gdal-r-mesh`	gdal-r-full + wk (surface types), wkpool, cdtr, laridae, decido, RTriangle, SFCGAL + sfcgal, rgl, silicate, quadmesh, polymer	Meshes and triangulation

gdal-system  →  gdal-r  →  gdal-r-full  →  gdal-r-python  →  gdal-r-python-extras  →  gdal-r-python-extras-rust

gdal-r-mesh is a side branch off gdal-r-full:latest; gdal-r-esmf is a leaf off gdal-r-python-extras-rust:latest.

The :dev variants track GDAL HEAD + latest released PROJ/GEOS, rebuilt daily — these are the canary. The :latest variants track the latest releases of all three, rebuilt weekly. gdal-r-python-extras and gdal-r-python-extras-rust have no :dev variants by design; for bleeding-edge work, base on gdal-r-python:dev directly and overlay packages via a writable R_LIBS_USER mount at runtime.

gdal-system

The base layer. Builds GEOS, PROJ, and GDAL from source against a single /usr/local. Also creates the uv-managed Python venv at /opt/gdal-py with numpy 2.x, before GDAL builds — so GDAL's Python bindings link against the venv's numpy ABI from the start. This means every downstream layer inherits a single, internally-consistent numpy 2.x environment with the osgeo bindings correctly bound. The venv lives at every subsequent layer; downstream layers just install packages into it.

gdal-r-python

R-and-Python, not Python-only — the new name reflects that (renamed from gdal-python). The venv inherited from gdal-system already contains numpy and the osgeo bindings. This layer adds Python geospatial packages on top: source-built and linking the system GDAL/PROJ/GEOS: rasterio, fiona, pyogrio, shapely, pyproj, geopandas, odc-geo, rioxarray. Everything else takes wheels. reticulate is installed and pinned to /opt/gdal-py/bin/python via RETICULATE_PYTHON; py_require() calls augment the existing venv rather than spawning ephemeral environments.

gdal-r-python-extras

Adds the everyday-R kitchen sink on top of gdal-r-python via scripts/install-extras.R. Spatial and hypertidy CRAN packages are source-built; everything else takes Posit Public Package Manager binaries. The set is curated rather than exhaustive — anything WIP or rarely reached for stays out and is installed on demand into a writable overlay.

gdal-r-python-extras-rust

Extends gdal-r-python-extras with a pinned Rust dev toolchain (docker/Dockerfile.extras-rust):

Rust 1.86.0 via rustup, with clippy, rustfmt, and rust-analyzer components
lld as the default linker (wired into CARGO_HOME/config.toml) — cuts incremental link times noticeably on large crates
cargo-nextest for a faster, more informative test runner

The critical reason to extend -extras rather than a lighter base: GDAL, PROJ, and GEOS are already present at /usr/local as the single source of truth. Rust crates like proj-sys and gdal-sys will link against those via pkg-config instead of doing their own bundled cmake builds — which means cargo build skips a 10-minute compile step and, more importantly, rwarp's PROJ is byte-for-byte the same PROJ that gdalwarp uses as an oracle.

Commit a rust-toolchain.toml pinning 1.86.0 in any project that consumes this image. The image pre-caches the toolchain; the toml is the repo-side pin. Drift becomes a git diff.

gdal-r-esmf

A leaf off gdal-r-python-extras-rust:latest with the reference regridders (docker/Dockerfile.gdal-r-esmf), so one container can build a Rust regridder, run gdalwarp, and compute the weights both are compared against:

ESMF 8.9.1, built from the release tarball (serial mpiuni, gfortran, NetCDF from apt) into /opt/esmf, with ESMF_RegridWeightGen and the other ESMF apps on PATH and ESMFMKFILE=/opt/esmf/lib/esmf.mk
ESMPy from the same tarball, and xESMF 0.9.2, in the /opt/gdal-py venv
TempestRemap 2.2.0 (GenerateRLLMesh, GenerateOverlapMesh, GenerateOfflineMap, ...) in /usr/local/bin

Every version is a build arg with a checksum. The build runs scripts/smoke-esmf.py: a small global conservative map from xESMF, ESMF_RegridWeightGen and TempestRemap, each of which must preserve a constant field.

docker run --rm -ti ghcr.io/hypertidy/gdal-r-esmf:latest

gdal-r-mesh

A leaf off gdal-r-full:latest for mesh and triangulation work (docker/Dockerfile.gdal-r-mesh), with the packages in config/r-packages-mesh.txt:

wk from the read-surface-types branch, which reads the simple features TIN, TRIANGLE and POLYHEDRALSURFACE types (paleolimbot/wk#236)
wkpool from the sf-surfaces branch, with the TIN round-trip article
cdtr (CDT), laridae (CGAL, via RcppCGAL headers), decido (earcut) and RTriangle (Triangle)
the SFCGAL 2.3 C library, built from source into /usr/local because Ubuntu does not package 2.3, and its R binding sfcgal from GitLab; it needs sf >= 1.1.4, so this image replaces gdal-r-full's CRAN sf with sf from GitHub (built against the same GDAL, PROJ and GEOS) until 1.1-4 reaches CRAN
rgl, with RGL_USE_NULL=TRUE set since containers have no display; save scenes with htmlwidgets::saveWidget(rgl::rglwidget(), ...)
the mesh models silicate, quadmesh and polymer, and, if they install, anglr and meshcore (entries marked ? in the package list are optional: a failure is logged in the build summary and the build goes on)

The R geos package is left out on purpose: it bundles its own GEOS (libgeos), which would put a second GEOS beside the one in /usr/local.

Several entries are development branches, so the image tracks work in progress; pin a digest when you need it fixed. The build runs scripts/smoke-mesh.R: a TIN read by wk in sfc, WKB and WKT form, merged on a vertex pool, the four engines on a square with a hole, an SFCGAL tessellation read back through wk into a pool, an rgl scene on the null device, and the mesh models loading.

docker run --rm -ti ghcr.io/hypertidy/gdal-r-mesh:latest

gdal-conda

The conda-forge counterpart to the chain, defined as a pixi workspace in conda/ and published as ghcr.io/hypertidy/gdal-conda:

Tag	Environment	GDAL	R GDAL packages
`:latest`	`release`	conda-forge `libgdal` + plugins	conda-forge binaries (`r-sf`, `r-terra`, `r-gdalraster`); vapour from source
`:dev`	`dev`	`gdal-master` channel (GDAL's CI rebuilds the conda-forge recipe from master)	gdalraster, sf, terra, vapour from GitHub source

Use it without Docker (Linux x86_64 and macOS arm64):

cd conda
pixi install -e release            # exact builds from pixi.lock
pixi run -e release install-r      # source-built R packages for that env
pixi run -e release smoke          # versions + one libgdal/libproj/libgeos_c
pixi shell -e release              # interactive

or the image:

docker run --rm -ti ghcr.io/hypertidy/gdal-conda:latest

Notes:

R is 4.5 in the conda environments: conda-forge has r-base 4.6, but its R 4.6 package migration has not run yet. The Docker chain uses R 4.6 from CRAN.
The dev environment holds no conda-forge binaries that link GDAL. Those are pinned to the GDAL release they were built against; they load against master today only because master has not bumped the libgdal soname yet.
Activation (GDAL_DATA, PROJ_DATA, UDUNITS2_XML_PATH, compiler variables, ...) is baked into the image: sourced via BASH_ENV for non-interactive bash such as GitHub Actions container steps, /etc/bash.bashrc for interactive shells, and the entrypoint for docker run.
The canary (scheduled-canary.yml) runs the same package/source/variant cells in both gdal-r-full and gdal-conda, and the conda cells again after every gdal-conda build. The image report includes both conda tags.

Image report

A published report at https://hypertidy.github.io/gdal-r-ci/ shows, for every image and tag: versions, compressed and on-disk size, where the bytes are, a strip check, a shared-library census (one file per library family in a process that loads the R and Python stacks together), the GDAL driver set, a compressed-size history recovered from GHCR, and the latest canary results. It is built by .github/workflows/report.yml from the scripts in report/ after each cascade, weekly, and on demand; history accumulates on the report-data branch.

Quick start

# Interactive R session with latest stable GDAL
docker run --rm -ti ghcr.io/hypertidy/gdal-r-full:latest

# Interactive R + Python (reticulate works out of the box)
docker run --rm -ti ghcr.io/hypertidy/gdal-r-python:latest

# Full workbench (R kitchen sink + Python)
docker run --rm -ti ghcr.io/hypertidy/gdal-r-python-extras:latest

# Check version alignment
docker run --rm ghcr.io/hypertidy/gdal-r-full:latest \
    Rscript /opt/scripts/check-r-versions.R

# Run R CMD check on a local package
docker run --rm -v $(pwd):/pkg ghcr.io/hypertidy/gdal-r-full:latest \
    Rscript -e 'rcmdcheck::rcmdcheck("/pkg", args = "--as-cran")'

All images use CMD ["bash"] so docker run -ti lands you in a shell with the full toolchain on PATH. R, Python, and CLI tools are all available interactively without picking a host language up front. A short banner prints on entry showing the GDAL/PROJ/GEOS/R/Python versions baked into that specific image; the banner links back to the relevant section of this README.

See docs/rcmd-check-notes.md for check args and per-package notes (extra deps needed for sf, terra, etc).

Reusable workflow for package CI

Add to your package's .github/workflows/:

name: Check against GDAL latest

on:
  schedule:
    - cron: '0 3 * * 0'
  workflow_dispatch:

jobs:
  gdal-check:
    uses: hypertidy/gdal-r-ci/.github/workflows/check-gdal-release.yml@main

This runs R CMD check against both :latest and :dev images. A failure in :dev only means an upstream GDAL API change that hasn't reached a release yet — file it upstream, not against your package.

Scheduled canary checks

Core packages are tested fortnightly against both release and dev images:

gdalraster — primary test target
sf
terra
vapour
gdalcubes

Failures open issues automatically in this repo.

Package lists

Package contents are explicit — no kitchen sink in the CI-contractual layers.

config/r-packages-base.txt — base R dev tooling in gdal-r
config/r-packages-required.txt — required geo packages in gdal-r-full (build fails if these fail, for :latest only)
config/r-packages-optional.txt — optional geo packages in gdal-r-full (failures logged, build continues)
scripts/install-extras.R — full curated R set installed in gdal-r-python-extras (CRAN spatial, hypertidy stack, AAD pipelines, arrow/duckdb/targets, dev tooling)

For :dev, failures in required packages are logged but don't fail the build — gdalraster failing against GDAL master is canary information worth publishing as an image, not a reason to withhold the optional packages too.

Why build from source?

The osgeo/gdal images build PROJ with renamed symbols (-DPROJ_RENAME_SYMBOLS) so GDAL can use a bleeding-edge internal PROJ without disturbing the system PROJ. This creates two PROJs in one container. R packages that link PROJ directly (sf, terra) use the system PROJ; packages that go through GDAL's headers (gdalraster, vapour) report GDAL's internal PROJ version. When objects cross the boundary the process crashes — this is GDAL issue #13777.

We build GDAL ourselves against a single system PROJ at /usr/local, so there is one libproj.so and everything links it. Full R CMD check works for all packages.

Spatialite is also built from source — the apt package is compiled against the system GEOS (3.14.1 on 26.04), and its headers at /usr/include/geos_c.h conflict with our /usr/local/include/geos_c.h. Building spatialite against our GEOS keeps the entire stack at one consistent version.

See docs/library-alignment.md for full details.

Base image and build args

The chain is based on Ubuntu 26.04 (resolute), which brings current format libraries (HDF5 1.14, netCDF 4.9.3, poppler 26), Python 3.14, gcc 15 and CMake 4. Two build args on gdal-system exist for rollback and fallback:

UBUNTU_VERSION (default 26.04): 24.04 rebuilds the chain on noble.
ARROW_SOURCE (default apache): the Apache Arrow apt repo. ubuntu uses the distro's own Arrow packages instead (23.x on 26.04). gdal-r-python's pyarrow pin follows pkg-config, so either works.

scripts/install-extras.R picks the Posit Package Manager binary repo from the image's codename and falls back to PPM's source mirror if there is no binary index for it (override with PPM_URL).

One library, one ABI

Three classes of duplicate-library bug have surfaced and been closed in this chain. They have different specifics but share a structure:

Bug	Cause	Fix
Dual PROJ	osgeo/gdal images embed PROJ with renamed symbols	Single `/usr/local` PROJ; build GDAL against it
Dual GDAL	pak's `pkg_install()` resolves R packages' GDAL sysreq via apt	`options(pkg.sysreqs = FALSE)` — apt is forbidden, /usr/local satisfies
Dual numpy	apt's python3-numpy is 1.x; venv installs 2.x; osgeo bindings linked against 1.x	Venv created in gdal-system; numpy 2.x installed before GDAL bindings build

Each had the same structural shape: two libraries claiming to be the same library, in one process, with code linked against either one randomly. The solution is always the same in shape — pick one source of truth, rebuild everything against it, eliminate the alternatives. Whether it's /usr/local for system libs, pkg.sysreqs=FALSE for pak, or the up-front venv for numpy, the principle is identity-not-shadowing.

GEOS version capping

The release build caps GEOS to a version known to work with the resolved GDAL release. GDAL 3.12.x was tested against GEOS ≤ 3.13.x; using GEOS 3.14.x with GDAL 3.12.x introduces undefined symbols at runtime. build-scripts/get-versions.sh handles this automatically — see the comments in that file for the cap table.

For :dev, GDAL is always master but PROJ and GEOS use the latest releases (not their main branches). We're testing GDAL API changes, not PROJ/GEOS dev, and keeping PROJ/GEOS at releases ensures R packages can actually build.

Build hygiene

A few patterns make rebuilds fast and image sizes manageable:

Single-numpy-ABI from layer 1. The Python venv at /opt/gdal-py is created in gdal-system with numpy 2.x installed before GDAL builds. GDAL's Python bindings link against this numpy from the start. Every downstream layer inherits the same venv; no --system-site-packages, no post-hoc rebuilds, no ABI ambiguity.
uv with BuildKit cache mounts. --mount=type=cache,target=/root/.cache/uv keeps wheels out of image layers entirely. Cold builds populate the cache; warm rebuilds finish in minutes. uv is the only pip-shaped tool in the chain — no apt python3-pip, no --break-system-packages shenanigans.
PYTHONDONTWRITEBYTECODE=1 plus a __pycache__ sweep at the end of every install RUN. Avoids hundreds of MB of .pyc accumulation across layers.
Source vs binary policy is explicit. Spatial Python packages that link GDAL/PROJ/GEOS are --no-binary; everything else takes wheels. Spatial R packages and hypertidy CRAN are installed via pak::pkg_install("...?source") with pkg.sysreqs = FALSE; everything else takes PPM binaries.
CMake unity build for GDAL halves compile time on the cmake step.
Strip in the same layer that installs. C/C++ libraries are installed with cmake --install --strip / make install-strip, and R package shared objects are stripped by scripts/strip-r-libs.sh at the end of the same RUN that installed them. A strip in a later RUN would copy every touched file up into a new layer and make the image bigger, not smaller.
Minimal Rust profile in gdal-system (rustc, cargo, rust-std only).

Local development

For iterating on Dockerfile or build-script changes without burning CI minutes, test/local-test.sh builds gdal-system locally and runs sanity checks on the result. Cache-friendly: subsequent runs reuse layers up to the first changed line. On a 32-core machine the first build is around 15 minutes; warm rebuilds for changes near the bottom of the Dockerfile are seconds.

bash test/local-test.sh                  # release variant
bash test/local-test.sh dev              # dev variant
NCPUS=16 bash test/local-test.sh         # cap parallelism

To debug a failing build interactively, comment out the failing RUN step temporarily, build to that point, then drop in:

docker run -ti --rm -v $(pwd)/build-scripts:/build-scripts \
    gdal-system:local bash

The -v mount lets you edit the build scripts on the host while iterating inside the container.

Ongoing maintenance

The infrastructure is designed to run itself:

build-gdal-system.yml runs weekly for release, daily for dev
Each image only rebuilds if its upstream digest changed, using repository_dispatch to cascade through gdal-system → gdal-r → gdal-r-full → gdal-r-python → gdal-r-python-extras
The cascade to gdal-r-python-extras only fires for the release variant, since extras has no :dev tag
scheduled-canary.yml runs fortnightly, opens issues on failure

When something needs human attention:

:dev fails, :latest passes — upstream GDAL API change. File issue against the package (e.g. terra's gdal_algs.cpp on CSLConstList type changes; gdalcubes's parallel signature changes).
:latest fails — regression in a released GDAL. Rare and urgent. Check the canary logs to identify whether it's a package, GDAL, or our build infrastructure.
Both fail — probably our infrastructure. Check the build logs for the system layer, run no_cache: true rebuild if needed.
Version alignment warning — should never happen on :latest (build-time check is strict). On :dev it's logged and expected during upstream transitions. Note: GDAL on :dev may report a version label like 3.13.0dev via gdal-config and 3.13.0beta2 via GDALVersionInfo — the alignment script normalises both to 3.13.0 so cosmetic suffix differences don't trigger false alarms.
Suggests-related test or example failures in canary results that don't match a GDAL story — these are real bugs in the canaried packages, surfaced by the deliberately-Suggests-minimal canary environment. File upstream. The canary's narrowness is itself a feature; see docs/r-cmd-check-suggests-and-ordering.md.

Forcing a full rebuild

If something looks stale, trigger build-gdal-system.yml manually with no_cache: true, variant: both. The CACHE_DATE ARG guarantees cache busting for the system layer; downstream GHA caches fall through automatically since their base image digest will have changed.

Overlaying packages at runtime

gdal-r-python-extras is curated, not exhaustive. For packages that aren't baked — anything WIP, or just one-off experiments — install into a writable library at runtime instead of rebuilding the image:

docker run --rm -ti \
  -v $HOME/R-overlay:/opt/r-overlay \
  -e R_LIBS_USER=/opt/r-overlay \
  ghcr.io/hypertidy/gdal-r-python-extras:latest

Inside, pak::pkg_install("hypertidy/zaro", lib = "/opt/r-overlay") installs to the bind-mounted directory, which persists across container runs. The same pattern works on Singularity without bind-mounts because $HOME is writable by default. Use the writable-overlay path for hypertidy WIP packages (zaro, shearwater, cloudcache, ndr, gdalcheck) where rebuild cadence matters more than discoverability.

For Python packages, uv pip install --python /opt/gdal-py/bin/python <pkg> adds to the inherited venv. Inside R, reticulate::py_require("<pkg>") does the same via reticulate's bridge — RETICULATE_PYTHON is pinned so additions land in /opt/gdal-py, not in an ephemeral venv.

Host VM + container workflow for Rust projects

The recommended pattern for interactive Rust development against this image is a clean split: the host VM is the pantry (credentials, raad conventions, data acquisition, anything that touches the outside world); the container is the clean room (build, warp, validate, produce artefacts). The host filesystem enters the container as a bind mount; nothing else crosses the boundary unintentionally.

Basic invocation

From your project directory on the host VM:

cd ~/rwarp-reentry
docker run --rm -ti \
  -v "$(pwd)":/work -w /work \
  ghcr.io/hypertidy/gdal-r-python-extras-rust:latest

The project source is live-edited on the host; cargo build, cargo nextest run, and gdalwarp cross-checks run inside the container against the image's GDAL/PROJ/GEOS stack.

Persisting the build cache

With --rm, anything not bind-mounted evaporates on exit — including target/ and the crates.io registry. Named volumes fix this without touching the host:

docker run --rm -ti \
  -v "$(pwd)":/work -w /work \
  -v rwarp-target:/work/target \
  -v cargo-cache:/opt/cargo/registry \
  ghcr.io/hypertidy/gdal-r-python-extras-rust:latest

rwarp-target overlays on top of the bind mount at /work/target, so incremental builds survive --rm without ever appearing on the host. Host builds (if any) and container builds can never contaminate each other's fingerprints — CARGO_TARGET_DIR on the host is irrelevant inside the container, and the volumes are container-local by definition.

File ownership

Docker runs as root by default. Anything written to /work lands on your VM owned by root. Two options — pick one early, retrofitting is tedious:

Add -u $(id -u):$(id -g) to run as your UID. You may need to docker run --rm -u 0 -v rwarp-target:/work/target <image> chown -R <uid>:<gid> /work/target once to fix the named volume ownership.
Accept root-in-container and keep outputs in a dedicated out/ directory that you chown -R on the host as needed.

Pinning the image by digest

For work that produces figures or artefacts you want to be reproducible months later, pin the image digest rather than :latest:

# get the current digest
docker inspect --format='{{index .RepoDigests 0}}' \
  ghcr.io/hypertidy/gdal-r-python-extras-rust:latest

# use in Makefile / justfile
IMAGE := ghcr.io/hypertidy/gdal-r-python-extras-rust@sha256:<digest>

A discipline worth adopting from day one: have artefact-producing make/just recipes echo the image digest plus the full command line into a sidecar .txt next to each output. "Which GDAL and which rwarp made this figure" then has a mechanical answer.

License

MIT

Navigation Menu

Repository navigation

About

Repository files navigation

gdal-r-ci

Images

gdal-system

gdal-r-python

gdal-r-python-extras

gdal-r-python-extras-rust

gdal-r-esmf

gdal-r-mesh

gdal-conda

Image report

Quick start

Reusable workflow for package CI

Scheduled canary checks

Package lists

Why build from source?

Base image and build args

One library, one ABI

GEOS version capping

Build hygiene

Local development

Ongoing maintenance

Forcing a full rebuild

Overlaying packages at runtime

Host VM + container workflow for Rust projects

Basic invocation

Persisting the build cache

File ownership

Pinning the image by digest

Related

License

About

Releases

Packages

Contributors

Languages

Name		Name	Last commit message	Last commit date
.github/workflows		.github/workflows
build-scripts		build-scripts
conda		conda
config		config
docker		docker
docs		docs
report		report
scripts		scripts
test		test
.gitattributes		.gitattributes
.gitignore		.gitignore
LICENSE.md		LICENSE.md
README.md		README.md

Latest commit

History

Folders and files

Repository files navigation

gdal-r-ci

Images

gdal-system

gdal-r-python

gdal-r-python-extras

gdal-r-python-extras-rust

gdal-r-esmf

gdal-r-mesh

gdal-conda

Image report

Quick start

Reusable workflow for package CI

Scheduled canary checks

Package lists

Why build from source?

Base image and build args

One library, one ABI

GEOS version capping

Build hygiene

Local development

Ongoing maintenance

Forcing a full rebuild

Overlaying packages at runtime

Host VM + container workflow for Rust projects

Basic invocation

Persisting the build cache

File ownership

Pinning the image by digest

Related

License

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages