diff --git a/adr/20260727-plugin-lockfile-integrity.md b/adr/20260727-plugin-lockfile-integrity.md new file mode 100644 index 0000000000..f06423ddd4 --- /dev/null +++ b/adr/20260727-plugin-lockfile-integrity.md @@ -0,0 +1,173 @@ +# Plugin Lockfile Integrity + +- Authors: Claude +- Status: draft +- Date: 2026-07-27 +- Tags: plugins, security, registry, integrity + +## Summary + +Introduce a committed lockfile (`plugins.lock`) that pins a canonical SHA-512 hash of each plugin's **extracted directory** — the unpacked `$id-$version/` tree that Nextflow actually loads and executes. The lockfile is populated automatically the first time a plugin is seen (trust-on-first-use), exactly like `go.sum` / `package-lock.json`, with **no dedicated command**. The trusted hash lives in the pipeline repository under version control and is not re-fetched from the registry on every run. Before loading a plugin, Nextflow re-hashes its extracted directory and verifies it against the committed hash. + +Because it hashes the code that runs rather than the archive it came from, a single mechanism covers **both** integrity surfaces: a tampered/compromised download or registry, **and** a poisoned cache directory (a lower-trust user editing already-extracted files on a shared filesystem). Verification is fully offline, and it uses no ownership or permission heuristics, so it does not produce false-positive warnings on legitimate shared caches. + +## Problem Statement + +Nextflow resolves plugins from a registry (or mirror), downloads a zip archive, extracts it into the local plugin cache under `$id-$version/`, and loads the extracted code. Two integrity gaps exist: + +1. **Cache poisoning (the executed code).** When a `$id-$version/` directory already exists, `PluginUpdater.load0()` short-circuits the download entirely and loads the extracted tree as-is. On a shared, writable plugin cache (`NXF_PLUGINS_DIR` pointed at a group-writable or world-writable location), a lower-trust local user can pre-populate or edit an official pinned coordinate (e.g. `nf-amazon-3.10.0/`) with attacker-controlled class/jar files that then execute inside the victim's Nextflow process. +2. **Runtime trust in the registry.** The registry returns a `sha512sum` during resolution, but that value is fetched fresh every run and only ever compared against the download in transit (pf4j's `CompoundVerifier`, `HttpPluginRepository.groovy:134-135` / `:222`). A compromised registry/mirror that serves a tampered archive with a matching tampered hash passes that check, and silent version drift produces no signal. There is no persistent, user-owned record of what a plugin is *supposed* to be. + +**What is already covered today.** In-transit corruption of a download is caught by `CompoundVerifier` against the registry-supplied hash. The lockfile does not duplicate that. Its job is the *post-download* surface: the extracted code, plus removing runtime trust in the registry's own hash. + +The core issue is that the *trusted* baseline must be owned by the pipeline and committed to version control, and it must describe the artifact that actually executes — the extracted tree — so verification depends on the repository (which the team controls and reviews) rather than on the registry being honest at run time or on the cache being un-tampered. + +## Goals or Decision Drivers + +- **Cover the executed code.** The check must protect the extracted `$id-$version/` tree that Nextflow loads, not merely the archive it was unpacked from. +- **One mechanism, both surfaces.** Supply-chain/registry integrity and cache-poisoning integrity should be answered by the same content hash, not two features. +- **No false positives, no noise.** Verification must not rely on directory ownership or permission heuristics; a legitimate shared or admin-managed cache must be silent. Output happens only on a genuine mismatch. +- **Local, network-free verification.** Once the lockfile exists, checking a plugin requires no network access; the registry is never a runtime dependency of verification. +- **Content-addressed, user-owned trust anchor.** The pinned hash is computed over the actual files and committed to the pipeline repository, reviewed through normal VCS workflows. +- **Zero breakage by default.** Pipelines without a lockfile behave exactly as today; the feature is opt-in by the mere presence of the file, defaulting to `warn` on mismatch (mirroring PR #7308's warn-first rollout). + +## Non-goals + +- **A dedicated `lock` command.** The lockfile is populated automatically on first load; there is no `nextflow plugins lock` subcommand to run or maintain. +- **Transitive / dependency lock resolution.** Only coordinates that are actually loaded are pinned; there is no dependency-graph resolution. +- **Registry-server changes.** Entirely client-side; the registry contract is unchanged. +- **Signature / provenance verification.** The lockfile pins a content hash, not a cryptographic signature or attestation. +- **Catching in-transit download corruption.** Already handled by pf4j's `CompoundVerifier`. + +## Considered Options + +### Option A: Load-time checksum comparison against the registry + +Fetch the plugin, then re-query the registry for the expected `sha512sum` at load time and compare. No committed lockfile. + +- Bad, because it makes the **registry a runtime dependency** of every plugin load — breaking air-gapped and offline execution, the environments where integrity matters most. +- Bad, because the registry becomes both the source of the artifact *and* the source of the trusted hash: a compromised registry defeats the check entirely. +- Bad, because it provides no reproducibility anchor and does nothing for cache poisoning. + +### Option B: Committed archive-hash lockfile + +Commit a `plugins.lock` pinning the SHA-512 of each plugin **archive** (the zip), verified on download and (by retaining the zip) on warm-cache runs. + +- Good, because it is a committed, offline, content-addressed anchor for supply-chain integrity and drift. +- Bad, because on a warm cache **the archive is not what executes** — Nextflow runs the extracted tree. Re-hashing a retained zip cannot detect edits to already-unpacked files, so it does **not** cover cache poisoning (the executed code) at all. It would still need PR #7308's directory guard as a complement. +- Bad, because it requires **retaining the archive** next to the extracted tree — roughly doubling cache footprint — and introduces an "archive absent" case (every pre-feature cache, and admin caches that ship only extracted dirs) that produces notices without adding protection. + +### Option C: Committed extracted-tree-hash lockfile (adopted) + +Commit a `plugins.lock` pinning a canonical SHA-512 over the **extracted `$id-$version/` directory contents**. Before loading a plugin, re-hash its directory and compare to the committed entry. + +- Good, because it hashes exactly the bytes that get executed, so a **single** hash covers supply-chain/drift **and** cache poisoning. +- Good, because it uses **no ownership/permission heuristic** — it is silent on any legitimate cache (private, shared read-only, admin service-account) and speaks only on a real content mismatch. This eliminates the false-positive/warning-noise problem of a directory-ownership guard. +- Good, because it needs **no retained archive** (no doubled footprint) and has **no "archive absent" case** — the extracted tree is always present when a plugin loads. +- Good, because verification of a locally-present directory is network-free. +- Bad (accepted), because it must re-hash the extracted tree; for large cloud plugins this cost is non-trivial and motivates the caching optimisation described below. +- Bad (accepted), because, like any lockfile, the first pin is trust-on-first-use — it trusts the artifact present when the coordinate is first seen. + +## Solution or decision outcome + +**Option C — committed extracted-tree-hash lockfile** — is adopted. It places a content-addressed, user-owned trust anchor over the code that actually runs, keeps verification network-free, produces no ownership-based false positives, and with one mechanism covers both the supply-chain surface (Option B's goal) and the cache-poisoning surface (PR #7308's goal). It therefore **supersedes** both the archive-hash approach and the directory-ownership guard. + +## Rationale & discussion + +### Threat model + +- **Cache poisoning of the executed code.** A lower-trust user edits the extracted `$id-$version/` tree on a shared/writable cache. Re-hashing the tree at load detects any change to the files, **independently of who owns the directory or its permissions** — so it catches cases an ownership heuristic misses (e.g. a group-writable cache owned by a trusted service account) and never mis-fires on a legitimate read-only shared cache. +- **Registry/mirror compromise and drift.** A tampered archive extracts to a different tree, so its hash no longer matches the committed entry. This holds on every run after the lock was honestly generated. +- **In-transit corruption.** Already caught by `CompoundVerifier`; not a lockfile novelty. + +The first pin of each coordinate is **trust-on-first-use**: it trusts the tree present when first seen (which, on a cold download, was just fetched and checked in transit against the registry hash). Once committed to VCS, every later run is anchored to that reviewed baseline and no longer trusts the registry or the cache. + +### Lockfile format and location + +- **File name**: `plugins.lock`, in the pipeline project root next to `nextflow.config`, committed to VCS. +- **Format**: JSON, chosen for diff-friendliness under code review, with a stable (sorted) key order. +- **Keying**: by resolved `id@version`. + +```json +{ + "version": 1, + "plugins": { + "nf-amazon@2.0.0": { + "sha512": "cbc4..." + } + } +} +``` + +- `sha512` is a canonical hash of the **extracted directory tree**: for every regular file, in sorted relative-path order, the digest absorbs the relative path and the file bytes (not timestamps or permissions), so it is stable across extractions and platforms while detecting any change to executable content. + +### Generation — automatic on first load (trust-on-first-use) + +There is **no dedicated command**. The lockfile is populated as a side effect of loading a plugin, as `go` writes `go.sum` and `npm` writes `package-lock.json`: + +- The feature is dormant unless a `plugins.lock` file is present. To start, create an empty one (`touch plugins.lock`) and run the pipeline once. +- The first time a coordinate is loaded and it is **missing** from the lock, its extracted-tree hash is **appended**. Reviewing the resulting diff and committing it establishes the baseline. +- The pinned value is computed from the files on disk, so it is a genuine content hash. + +Auto-pinning only ever **adds a missing coordinate**. An entry already present is **never silently rewritten** — a plugin whose tree differs from a committed entry is a verification *failure* (mode-gated below), not a silent update, exactly as `go.sum` refuses to quietly change a recorded hash. Re-pinning a legitimately changed plugin is explicit: delete the stale entry and re-run. + +### Verification flow (in `PluginUpdater`) + +Verification happens once per plugin in `load0()`, immediately before `loadPluginFromPath()`, covering both a fresh download and a reused (warm) cache with the same code path. **No branch ever re-fetches from the registry as a remedy** — network access happens only to obtain a plugin genuinely absent from the cache, exactly as today. + +- Compute the canonical hash of the extracted `$id-$version/` directory. +- If the coordinate is **not in the lock** → append it (trust-on-first-use). +- If it **matches** the committed entry → proceed. +- If it **differs** → mode-gated (below). + +Verification of a locally-present directory requires no network. There is no "artifact absent" case: if a plugin is being loaded, its extracted directory exists by definition. + +### Performance and the re-hash cost + +Verification re-hashes the extracted directory on every load when a lockfile is present. For small plugins this is negligible next to JVM and Nextflow start-up; for large cloud plugins (bundled SDKs of tens to hundreds of MB) it is material. + +The initial implementation performs the full hash each time and does **not** pre-optimise. A follow-up optimisation, added only once the cost is shown to matter, avoids the re-hash when nothing changed: + +- Keep a **local, non-committed** sidecar (e.g. under `$NXF_PLUGINS_DIR`) recording, per plugin, the last verified tree hash and a cheap fingerprint of the tree — the set of `(relative-path, size, mtime)`. +- On load, stat the files (no reads) and rebuild the fingerprint; if it is unchanged and the lock entry is unchanged, reuse the last verified result and skip the full hash. +- **Safety of the shortcut:** `mtime`/`size` are forgeable by anyone who can write the files, so the fingerprint shortcut is trusted **only when the cache cannot change under you** — i.e. a directory you own that is not writable by group or others. On a shared/writable cache the fingerprint is not trusted and the full hash always runs. Ownership is thus used to select the *strategy* (fast vs full), never to emit a warning — so the noise problem of an ownership guard does not reappear. + +This keeps steady-state launches on a private cache nearly free, runs the full hash exactly where tampering is possible, and stays silent in all cases except a genuine mismatch. + +### Modes and rollout + +- **Opt-in by presence**: no `plugins.lock` → dormant, behaviour unchanged, zero breakage. +- When the file exists, mismatch behaviour is controlled by `NXF_PLUGINS_LOCK_MODE`, reusing the `warn`/`strict`/`off` tri-state plumbing pattern from PR #7308's `NXF_PLUGINS_STRICT_MODE`, but independent from it. + +| Mode | Tree hash mismatch (locked entry) | Coordinate missing from lock | +|------|-----------------------------------|------------------------------| +| `strict` | **Abort** with a re-pin hint | Auto-pin (trust-on-first-use) | +| `warn` (default when the file is present) | Log a warning once and proceed | Auto-pin | +| `off` | Skip verification | Auto-pin | + +Mode gates only the **mismatch** outcome. A coordinate missing from the lock is auto-pinned, not gated. Default is `warn` for a friendly rollout; `strict` is opt-in for teams that want a fail-closed guarantee. Unlike a directory-ownership guard, **either default is silent in normal operation** — a mismatch is a real, rare, actionable event, not per-run noise on healthy caches. + +### Relationship to PR #7308 + +PR #7308 proposed a `PluginSecurity` directory guard that flags a plugin directory as untrusted when it is foreign-owned or world-writable, to defend the extracted tree on shared caches. This ADR's extracted-tree hash defends the **same** surface by content instead of by ownership, which is both stronger (it catches any modification regardless of ownership, including the group-writable-trusted-owner case the heuristic misses) and quieter (it never warns on a legitimate shared/read-only cache). It also covers the supply-chain surface the guard did not. This mechanism therefore **supersedes PR #7308**, which can be closed in its favour. The zero-config baseline for the extracted-tree surface remains the operational guidance to keep the plugin cache private (per-user); the lockfile is the committed, enforceable layer on top for teams that want it. + +### Reused and new components + +| Component | Module | Change | +|-----------|--------|--------| +| `PluginLockFile` (new) | nf-commons | Read/write/round-trip of `plugins.lock`; blank/malformed-file handling | +| `PluginLockVerifier` (new) | nf-commons | Canonical extracted-tree hash; auto-pin on first load (TOFU); mismatch mode gating | +| `PluginUpdater` | nf-commons | Verify/pin the extracted directory in `load0()` before loading. No retained archive, no dedicated command | +| `HttpPluginRepository` | nf-commons | No change — `CompoundVerifier` still checks downloads against the registry `sha512sum` at fetch time | +| `NXF_PLUGINS_LOCK_MODE` | nf-commons | New env var, tri-state (`strict`/`warn`/`off`), default `warn`, independent of `NXF_PLUGINS_STRICT_MODE` | + +## Testing + +- **`PluginLockFile`**: read, write, round-trip; blank/`touch`ed file parses as empty; malformed file throws. +- **`sha512Tree`**: identical trees in different locations hash equal; any file-content change changes the hash; output is a 128-char lowercase hex digest. +- **Auto-pin (TOFU)**: an enabled but empty lock, given an extracted directory for an unlocked coordinate, appends the tree hash to the file on disk; a dormant (no file) verifier never pins. +- **Verification**: + - matching tree passes; + - mismatch **aborts** in `strict`, **warns once** in `warn`, is **ignored** in `off`; + - **cache poisoning**: pinning a good tree then editing an extracted file in place is detected as a mismatch — regardless of directory ownership. +- **No-network guarantee**: verification and pinning operate purely on local files; no branch re-fetches from the registry. diff --git a/docs/plugins/using-plugins.mdx b/docs/plugins/using-plugins.mdx index 82a370749c..fe8d04c888 100644 --- a/docs/plugins/using-plugins.mdx +++ b/docs/plugins/using-plugins.mdx @@ -56,6 +56,30 @@ Plugin declarations in Nextflow configuration files are ignored when specifying When Nextflow downloads plugins, it caches them in the directory specified by `NXF_PLUGINS_DIR` (`$HOME/.nextflow/plugins` by default). +:::note +The plugin cache is shared across pipelines and is not access-controlled. On multi-tenant or shared systems, use a private cache directory per user (set `NXF_PLUGINS_DIR` to a location only you can write) to avoid loading plugin artifacts populated by another user. +::: + +### Lockfile + + + +A `plugins.lock` file pins the exact plugin code a pipeline expects. For each plugin it records a `sha512` hash of the **extracted plugin directory** — the code Nextflow actually loads and runs — keyed by `id@version`. The file is meant to be committed to the pipeline repository so that everyone running the pipeline executes the same plugin code. + +The lockfile is populated automatically, like `go.sum` or `package-lock.json` — there is no separate command. To enable it, create an empty file in the pipeline directory and run the pipeline once: + +```bash +touch plugins.lock +``` + +The first time each plugin is loaded, its hash is added to `plugins.lock`. Review the resulting file and commit it. On subsequent runs Nextflow re-hashes each plugin's extracted directory and verifies it against the committed hash. When no `plugins.lock` file is present, the feature is dormant and has no effect. + +Verification is fully offline — it re-hashes the files already in the local cache and never contacts the plugin registry. Because it hashes the extracted code rather than the download, it detects both a tampered or compromised download and a plugin directory that was modified after extraction (for example by another user on a shared cache), independently of file ownership or permissions. An existing entry is never rewritten automatically — if a plugin legitimately changes, delete its entry and run again to re-pin it. + +Use [`NXF_PLUGINS_LOCK_MODE`][using-plugins-env-vars] to control what happens on a mismatch: `warn` (default) logs a warning and continues, `strict` aborts the run, and `off` skips verification. + +The lockfile complements the private-cache guidance above: keeping the cache private prevents untrusted code from being written in the first place, while the lockfile detects any change to the plugin code that is actually loaded. + ## Offline usage When running Nextflow in an offline environment, any required plugins must be downloaded and moved into the offline environment prior to any runs. @@ -75,6 +99,7 @@ To use Nextflow plugins in an offline environment: ::: [install-standalone]: ../install#standalone-distribution +[using-plugins-env-vars]: ../reference/env-vars#nxf_plugins_lock_mode [using-plugins-config]: ./using-plugins#configuration [using-plugins-identifiers]: ./using-plugins#identifiers diff --git a/docs/reference/env-vars.mdx b/docs/reference/env-vars.mdx index a8cdc2d883..d924206973 100644 --- a/docs/reference/env-vars.mdx +++ b/docs/reference/env-vars.mdx @@ -210,6 +210,12 @@ Whether to use the default plugins when no plugins are specified in the Nextflow The path where the plugin archives are loaded and stored (default: `$NXF_HOME/plugins`). +##### `NXF_PLUGINS_LOCK_MODE` + + + +Controls how Nextflow reacts when a plugin's extracted directory does not match the hash recorded in the `plugins.lock` file: `warn` logs a warning once per plugin and continues, `strict` aborts the run, and `off` skips verification silently (default: `warn`). Verification is dormant when no `plugins.lock` file is present. + ##### `NXF_PLUGINS_REGISTRY_URL` diff --git a/modules/nf-commons/src/main/nextflow/plugin/PluginLockFile.groovy b/modules/nf-commons/src/main/nextflow/plugin/PluginLockFile.groovy new file mode 100644 index 0000000000..5041047755 --- /dev/null +++ b/modules/nf-commons/src/main/nextflow/plugin/PluginLockFile.groovy @@ -0,0 +1,177 @@ +/* + * Copyright 2013-2026, Seqera Labs + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package nextflow.plugin + +import java.nio.file.Files +import java.nio.file.Path + +import com.google.gson.Gson +import com.google.gson.GsonBuilder +import com.google.gson.JsonSyntaxException +import groovy.transform.CompileStatic +import groovy.transform.EqualsAndHashCode +import groovy.transform.ToString +import groovy.util.logging.Slf4j + +/** + * Model the {@code plugins.lock} file. It holds a format version number and a map + * of plugin fully-qualified ids (ie. {@code id@version}) to the corresponding + * {@link Entry} carrying the {@code sha512} checksum of the plugin archive. + * + * The file is serialised as pretty-printed JSON with a stable (sorted) key order to + * keep diffs deterministic. + * + * @author Paolo Di Tommaso + */ +@Slf4j +@CompileStatic +@EqualsAndHashCode(includeFields = true) +@ToString(includeNames = true, includeFields = true) +class PluginLockFile { + + /** + * The current lock file format version + */ + static final int CURRENT_VERSION = 1 + + /** + * Represent a single locked plugin entry + */ + @EqualsAndHashCode + @ToString(includeNames = true) + static class Entry { + String sha512 + + Entry() {} + + Entry(String sha512) { + this.sha512 = sha512 + } + } + + private int version = CURRENT_VERSION + + private Map plugins = new TreeMap() + + int getVersion() { version } + + void setVersion(int value) { this.version = value } + + /** + * @return An immutable view of the locked plugin entries, keyed by fully-qualified id + */ + Map getEntries() { + return Collections.unmodifiableMap(plugins) + } + + /** + * Lookup a locked entry by its fully-qualified id ie. {@code id@version}. + * + * @param fqid The plugin fully-qualified id + * @return The corresponding {@link Entry} or {@code null} if not present + */ + Entry getEntry(String fqid) { + return plugins.get(fqid) + } + + /** + * Add or update a locked entry. + * + * @param fqid The plugin fully-qualified id ie. {@code id@version} + * @param entry The {@link Entry} to associate with the given id + * @return The object itself to enable method chaining + */ + PluginLockFile addEntry(String fqid, Entry entry) { + if( !fqid ) + throw new IllegalArgumentException("Plugin lock entry id cannot be empty") + if( entry == null ) + throw new IllegalArgumentException("Plugin lock entry cannot be null") + plugins.put(fqid, entry) + return this + } + + /** + * @return {@code true} when no plugin entries are held + */ + boolean isEmpty() { + return plugins.isEmpty() + } + + /** + * Serialise this lock file as pretty-printed JSON with a stable key order. + * + * @param path The target file path + */ + void write(Path path) { + final json = gson0().toJson(toModel()) + Files.write(path, json.getBytes('UTF-8')) + } + + private Map toModel() { + // use a plain map so that only 'version' and 'plugins' are emitted, with + // plugins held in a TreeMap to guarantee a stable, sorted key order + final result = new LinkedHashMap() + result.put('version', version) + result.put('plugins', new TreeMap(plugins)) + return result + } + + private static Gson gson0() { + return new GsonBuilder().setPrettyPrinting().create() + } + + /** + * Read and parse a {@code plugins.lock} file. + * + * @param path The lock file path + * @return A {@link PluginLockFile}; an empty (dormant) instance when the file does not exist + * @throws IllegalStateException when the file content cannot be parsed + */ + static PluginLockFile read(Path path) { + if( path == null || !Files.exists(path) ) { + log.debug "Plugins lock file does not exist: $path - returning an empty lock" + return new PluginLockFile() + } + + final text = new String(Files.readAllBytes(path), 'UTF-8') + // a blank or freshly `touch`ed file is a valid, empty lock (bootstrap case) + if( !text.trim() ) + return new PluginLockFile() + try { + final model = gson0().fromJson(text, ModelBean) + if( model == null ) + throw new IllegalStateException("Invalid plugins lock file - empty content: $path") + final result = new PluginLockFile() + result.version = model.version + if( model.plugins ) + result.plugins.putAll(model.plugins) + return result + } + catch( JsonSyntaxException e ) { + throw new IllegalStateException("Invalid plugins lock file - malformed JSON: $path", e) + } + } + + /** + * Deserialization bean matching the on-disk JSON structure + */ + static class ModelBean { + int version + Map plugins + } + +} diff --git a/modules/nf-commons/src/main/nextflow/plugin/PluginLockVerifier.groovy b/modules/nf-commons/src/main/nextflow/plugin/PluginLockVerifier.groovy new file mode 100644 index 0000000000..28fa532801 --- /dev/null +++ b/modules/nf-commons/src/main/nextflow/plugin/PluginLockVerifier.groovy @@ -0,0 +1,191 @@ +/* + * Copyright 2013-2026, Seqera Labs + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package nextflow.plugin + +import java.nio.file.FileVisitResult +import java.nio.file.Files +import java.nio.file.Path +import java.nio.file.SimpleFileVisitor +import java.nio.file.attribute.BasicFileAttributes +import java.security.MessageDigest + +import groovy.transform.CompileStatic +import groovy.util.logging.Slf4j +import nextflow.SysEnv +import nextflow.exception.AbortOperationException + +/** + * Verifies an extracted plugin directory against the {@code plugins.lock} file. + * + * The feature is opt-in by the presence of the lock file: it is dormant (a no-op) when no + * {@code plugins.lock} exists. When the file is present, a canonical hash of the extracted + * {@code $id-$version/} directory - the code that Nextflow actually loads and executes - is + * re-computed locally (no network) and compared to the committed lock entry. Because it hashes the + * unpacked tree rather than the archive, it detects both a tampered/compromised download and a + * poisoned cache directory (a lower-trust user editing already-extracted files on a shared cache). + * + * Following the {@code go.sum} / {@code package-lock.json} model the lock is populated + * automatically: the first time a coordinate is seen and it is missing from the lock, its tree + * hash is appended (trust-on-first-use). An existing entry is never rewritten silently — a + * mismatch against a committed entry is a verification failure, gated by + * {@code NXF_PLUGINS_LOCK_MODE}: + *
    + *
  • {@code strict} - abort with an {@link AbortOperationException}
    • + *
  • {@code warn} (default) - log a warning once per coordinate and proceed
    • + *
  • {@code off} - skip verification silently
    • + *
+ * + * @author Paolo Di Tommaso + */ +@Slf4j +@CompileStatic +class PluginLockVerifier { + + static enum Mode { STRICT, WARN, OFF } + + private final Path lockPath + + private final PluginLockFile lock + + private final boolean enabled + + private final Set notified = Collections.synchronizedSet(new HashSet()) + + /** + * @param lockPath The {@code plugins.lock} path; the feature is enabled only when this file + * exists. A {@code null} or missing path leaves the verifier dormant. + */ + PluginLockVerifier(Path lockPath) { + this.lockPath = lockPath + this.enabled = lockPath != null && Files.exists(lockPath) + this.lock = PluginLockFile.read(lockPath) + } + + /** + * @return {@code true} when a lock file is present ie. verification/pinning is enabled + */ + boolean isEnabled() { + return enabled + } + + /** + * Resolve the verification mode from the {@code NXF_PLUGINS_LOCK_MODE} environment variable. + * + * @return The resolved {@link Mode}; {@link Mode#WARN} when unset or unrecognised + */ + static Mode getMode() { + final value = SysEnv.get('NXF_PLUGINS_LOCK_MODE') + if( !value ) + return Mode.WARN + switch( value.toLowerCase() ) { + case 'strict': return Mode.STRICT + case 'warn': return Mode.WARN + case 'off': return Mode.OFF + default: + log.warn "Invalid NXF_PLUGINS_LOCK_MODE value: '$value' - using default 'warn'" + return Mode.WARN + } + } + + /** + * Verify - and, on first sight, pin - an extracted plugin directory against the lock. + * + * @param fqid The plugin fully-qualified id ie. {@code id@version} + * @param pluginDir The extracted plugin directory ({@code $id-$version/}) + */ + void verify(String fqid, Path pluginDir) { + if( !enabled ) + return + if( pluginDir == null || !Files.isDirectory(pluginDir) ) { + log.debug "Cannot verify plugin '$fqid' against the plugins lock file - directory not available: $pluginDir" + return + } + + final actual = sha512Tree(pluginDir) + final entry = lock.getEntry(fqid) + + // coordinate not yet locked: pin it on first sight (trust-on-first-use), never fail + if( entry == null ) { + pin(fqid, actual) + return + } + // matches the committed hash + if( actual == entry.sha512 ) + return + + // mismatch: the extracted plugin differs from what the lock pinned + final reason = "Plugin '$fqid' does not match the plugins lock file\n- expected: ${entry.sha512}\n- actual : ${actual}" + switch( getMode() ) { + case Mode.OFF: + log.debug "Plugins lock verification failed (ignored, mode=off) - $reason" + break + case Mode.STRICT: + throw new AbortOperationException("$reason\n- delete the entry from the plugins lock file and re-run to re-pin, or restore the expected plugin") + case Mode.WARN: + // warn only once per coordinate to avoid log spam + if( notified.add(fqid) ) + log.warn "$reason\n- delete the entry from the plugins lock file and re-run to re-pin" + break + } + } + + /** + * Append a new entry to the lock and persist it (trust-on-first-use). Existing entries are + * never overwritten by this path — {@link #verify} routes an already-locked coordinate through + * the hash comparison instead. + */ + private synchronized void pin(String fqid, String sha512) { + lock.addEntry(fqid, new PluginLockFile.Entry(sha512)) + if( lockPath != null ) + lock.write(lockPath) + log.info "Added plugin '$fqid' to the plugins lock file" + } + + /** + * Compute a canonical sha512 digest over the content of a directory tree. The digest covers, + * for every regular file (visited in sorted relative-path order for determinism), its relative + * path and its bytes - not timestamps or permissions - so it is stable across extractions and + * platforms while still detecting any change to the files that will be executed. + * + * @param dir The directory to hash + * @return The lowercase hex-encoded sha512 digest (128 chars) + */ + static String sha512Tree(Path dir) { + final md = MessageDigest.getInstance('SHA-512') + // TreeMap keyed by relative path -> deterministic, sorted iteration order + final files = new TreeMap() + Files.walkFileTree(dir, new SimpleFileVisitor() { + @Override + FileVisitResult visitFile(Path file, BasicFileAttributes attrs) { + files.put(dir.relativize(file).toString().replace('\\', '/'), file) + return FileVisitResult.CONTINUE + } + }) + final buffer = new byte[8192] + for( Map.Entry it : files.entrySet() ) { + md.update(it.key.getBytes('UTF-8')) + md.update((byte) 0) + try (InputStream is = Files.newInputStream(it.value)) { + int read + while( (read = is.read(buffer)) != -1 ) + md.update(buffer, 0, read) + } + md.update((byte) 0) + } + return HexFormat.of().formatHex(md.digest()) + } +} diff --git a/modules/nf-commons/src/main/nextflow/plugin/PluginUpdater.groovy b/modules/nf-commons/src/main/nextflow/plugin/PluginUpdater.groovy index 0ae4412901..d9247f7383 100644 --- a/modules/nf-commons/src/main/nextflow/plugin/PluginUpdater.groovy +++ b/modules/nf-commons/src/main/nextflow/plugin/PluginUpdater.groovy @@ -20,6 +20,7 @@ import static java.nio.file.StandardCopyOption.* import java.nio.file.Files import java.nio.file.Path +import java.nio.file.Paths import java.util.regex.Pattern import com.github.zafarkhaja.semver.Version @@ -73,6 +74,8 @@ class PluginUpdater extends UpdateManager { private DefaultPlugins defaultPlugins = DefaultPlugins.INSTANCE + private PluginLockVerifier lockVerifier + protected PluginUpdater(CustomPluginManager pluginManager) { super(pluginManager) this.pluginManager = pluginManager @@ -357,6 +360,23 @@ class PluginUpdater extends UpdateManager { return load0(id, version) } + /** + * The {@code plugins.lock} file location used to verify plugins against their pinned hash. + * Defaults to a {@code plugins.lock} file in the current working directory. + */ + protected Path lockFilePath() { + return Paths.get('plugins.lock') + } + + /** + * Lazily create the {@link PluginLockVerifier}. The lock file is read once and cached. + */ + protected synchronized PluginLockVerifier getLockVerifier() { + if( lockVerifier == null ) + lockVerifier = new PluginLockVerifier(lockFilePath()) + return lockVerifier + } + private Path download0(String id, String version) { // 0. check if version is specified if( !version ) @@ -509,6 +529,10 @@ class PluginUpdater extends UpdateManager { log.warn("Plugin '${pluginPath.getFileName()}' installation looks corrupted - Delete the following directory and run nextflow again: $pluginPath") } + // verify (or pin) the extracted plugin directory against the plugins lock, before loading + // and executing its code. This covers both a fresh download and a reused (warm) cache. + getLockVerifier().verify("${id}@${version}", pluginPath) + // load the plugin from the file system PluginWrapper wrapper = pluginManager.loadPluginFromPath(pluginPath) diff --git a/modules/nf-commons/src/test/nextflow/plugin/PluginLockFileTest.groovy b/modules/nf-commons/src/test/nextflow/plugin/PluginLockFileTest.groovy new file mode 100644 index 0000000000..84740ff179 --- /dev/null +++ b/modules/nf-commons/src/test/nextflow/plugin/PluginLockFileTest.groovy @@ -0,0 +1,151 @@ +/* + * Copyright 2013-2026, Seqera Labs + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package nextflow.plugin + +import java.nio.file.Files +import java.nio.file.Path + +import spock.lang.Specification + +/** + * + * @author Paolo Di Tommaso + */ +class PluginLockFileTest extends Specification { + + def 'should round-trip write and read' () { + given: + def path = Files.createTempFile('plugins', '.lock') + and: + final lock = new PluginLockFile() + lock.version = 1 + lock.addEntry('nf-amazon@2.0.0', new PluginLockFile.Entry('sha512-aaa')) + lock.addEntry('nf-google@1.5.0', new PluginLockFile.Entry('sha512-ggg')) + + when: + lock.write(path) + and: + final copy = PluginLockFile.read(path) + + then: + copy.version == 1 + copy.getEntry('nf-amazon@2.0.0') == new PluginLockFile.Entry('sha512-aaa') + copy.getEntry('nf-google@1.5.0') == new PluginLockFile.Entry('sha512-ggg') + copy.entries.size() == 2 + and: + copy == lock + + cleanup: + Files.deleteIfExists(path) + } + + def 'should write pretty json with stable key order' () { + given: + def path = Files.createTempFile('plugins', '.lock') + and: + final lock = new PluginLockFile() + lock.version = 1 + // add out of order + lock.addEntry('nf-zeta@1.0.0', new PluginLockFile.Entry('sha512-z')) + lock.addEntry('nf-alpha@1.0.0', new PluginLockFile.Entry('sha512-a')) + + when: + lock.write(path) + final text = path.text + + then: + // pretty printed (contains newlines and indentation) + text.contains('\n') + // keys are sorted: alpha appears before zeta + text.indexOf('nf-alpha@1.0.0') < text.indexOf('nf-zeta@1.0.0') + + cleanup: + Files.deleteIfExists(path) + } + + def 'should return empty for missing file' () { + given: + final path = Path.of('/no/such/plugins.lock') + + when: + final lock = PluginLockFile.read(path) + + then: + lock != null + lock.isEmpty() + lock.entries.isEmpty() + } + + def 'should return empty for a blank (touched) file' () { + given: + def path = Files.createTempFile('plugins', '.lock') + path.text = ' \n' + + when: + final lock = PluginLockFile.read(path) + + then: + lock != null + lock.isEmpty() + + cleanup: + Files.deleteIfExists(path) + } + + def 'should throw for malformed file' () { + given: + def path = Files.createTempFile('plugins', '.lock') + path.text = '{ this is not valid json ]' + + when: + PluginLockFile.read(path) + + then: + thrown(IllegalStateException) + + cleanup: + Files.deleteIfExists(path) + } + + def 'should lookup an entry by id and version' () { + given: + final lock = new PluginLockFile() + lock.addEntry('nf-amazon@2.0.0', new PluginLockFile.Entry('sha512-aaa')) + + expect: + lock.getEntry('nf-amazon@2.0.0') == new PluginLockFile.Entry('sha512-aaa') + lock.getEntry('nf-unknown@9.9.9') == null + } + + def 'should add and update an entry' () { + given: + final lock = new PluginLockFile() + + when: + lock.addEntry('nf-amazon@2.0.0', new PluginLockFile.Entry('sha512-old')) + then: + lock.getEntry('nf-amazon@2.0.0').sha512 == 'sha512-old' + lock.entries.size() == 1 + + when: + lock.addEntry('nf-amazon@2.0.0', new PluginLockFile.Entry('sha512-new')) + then: + lock.getEntry('nf-amazon@2.0.0').sha512 == 'sha512-new' + lock.entries.size() == 1 + } + +} diff --git a/modules/nf-commons/src/test/nextflow/plugin/PluginLockVerifierTest.groovy b/modules/nf-commons/src/test/nextflow/plugin/PluginLockVerifierTest.groovy new file mode 100644 index 0000000000..3312210e70 --- /dev/null +++ b/modules/nf-commons/src/test/nextflow/plugin/PluginLockVerifierTest.groovy @@ -0,0 +1,274 @@ +/* + * Copyright 2013-2026, Seqera Labs + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package nextflow.plugin + +import java.nio.file.Files +import java.nio.file.Path + +import nextflow.SysEnv +import nextflow.exception.AbortOperationException +import spock.lang.Specification +import spock.lang.Unroll + +/** + * + * @author Paolo Di Tommaso + */ +class PluginLockVerifierTest extends Specification { + + /** Create an extracted-plugin-like directory tree with the given relative-path->content files */ + private Path pluginDir(Map files) { + final dir = Files.createTempDirectory('plugin') + files.each { rel, content -> + final f = dir.resolve(rel) + Files.createDirectories(f.parent) + Files.write(f, content.getBytes('UTF-8')) + } + return dir + } + + /** Write a lock file holding the given fqid->sha512 entries and return its path */ + private Path lockFileWith(Map entries) { + final path = Files.createTempFile('plugins', '.lock') + final lock = new PluginLockFile() + entries.each { k, v -> lock.addEntry(k, new PluginLockFile.Entry(v)) } + lock.write(path) + return path + } + + @Unroll + def 'should resolve lock mode from env [#VALUE]' () { + given: + SysEnv.push(VALUE != null ? [NXF_PLUGINS_LOCK_MODE: VALUE] : [:]) + + expect: + PluginLockVerifier.getMode() == EXPECTED + + cleanup: + SysEnv.pop() + + where: + VALUE | EXPECTED + null | PluginLockVerifier.Mode.WARN + 'warn' | PluginLockVerifier.Mode.WARN + 'WARN' | PluginLockVerifier.Mode.WARN + 'strict' | PluginLockVerifier.Mode.STRICT + 'STRICT' | PluginLockVerifier.Mode.STRICT + 'off' | PluginLockVerifier.Mode.OFF + 'OFF' | PluginLockVerifier.Mode.OFF + 'nonsense' | PluginLockVerifier.Mode.WARN + } + + // --------------------------------------------------------------------------- + // tree hashing + + def 'sha512Tree should be deterministic and content-sensitive' () { + given: + def a = pluginDir(['classes/A.class': 'aaa', 'META-INF/MANIFEST.MF': 'mmm']) + def b = pluginDir(['classes/A.class': 'aaa', 'META-INF/MANIFEST.MF': 'mmm']) + def c = pluginDir(['classes/A.class': 'aaa', 'META-INF/MANIFEST.MF': 'CHANGED']) + + expect: + // identical content in different dirs -> identical hash + PluginLockVerifier.sha512Tree(a) == PluginLockVerifier.sha512Tree(b) + // any content change -> different hash + PluginLockVerifier.sha512Tree(a) != PluginLockVerifier.sha512Tree(c) + and: + PluginLockVerifier.sha512Tree(a).length() == 128 + PluginLockVerifier.sha512Tree(a) ==~ /[0-9a-f]{128}/ + + cleanup: + [a, b, c].each { it.deleteDir() } + } + + // --------------------------------------------------------------------------- + // dormant / enabled + + def 'should be dormant when no lock file exists' () { + given: + SysEnv.push([NXF_PLUGINS_LOCK_MODE: 'strict']) + and: + def dir = pluginDir(['classes/A.class': 'aaa']) + def verifier = new PluginLockVerifier(Path.of('/no/such/plugins.lock')) + + expect: + !verifier.isEnabled() + + when: + verifier.verify('nf-foo@1.0.0', dir) + then: + noExceptionThrown() + + cleanup: + SysEnv.pop() + dir.deleteDir() + } + + def 'should be enabled when an (even empty) lock file exists' () { + given: + def path = Files.createTempFile('plugins', '.lock') + path.text = '' + + expect: + new PluginLockVerifier(path).isEnabled() + + cleanup: + Files.deleteIfExists(path) + } + + // --------------------------------------------------------------------------- + // trust-on-first-use pinning + + def 'should pin a coordinate missing from the lock on first sight (TOFU)' () { + given: + SysEnv.push([NXF_PLUGINS_LOCK_MODE: 'strict']) + and: + def dir = pluginDir(['classes/A.class': 'hello']) + final sha = PluginLockVerifier.sha512Tree(dir) + and: + // an empty but present lock file -> feature enabled, nothing pinned yet + def lockPath = lockFileWith([:]) + def verifier = new PluginLockVerifier(lockPath) + + when: + verifier.verify('nf-foo@1.0.0', dir) + + then: + noExceptionThrown() + and: + // the entry was appended to the lock file on disk + def written = PluginLockFile.read(lockPath) + written.getEntry('nf-foo@1.0.0').sha512 == sha + + cleanup: + SysEnv.pop() + dir.deleteDir() + Files.deleteIfExists(lockPath) + } + + // --------------------------------------------------------------------------- + // verification of an already-locked coordinate + + def 'should pass verification when the plugin matches the lock' () { + given: + SysEnv.push([NXF_PLUGINS_LOCK_MODE: 'strict']) + and: + def dir = pluginDir(['classes/A.class': 'hello']) + final sha = PluginLockVerifier.sha512Tree(dir) + def lockPath = lockFileWith(['nf-foo@1.0.0': sha]) + def verifier = new PluginLockVerifier(lockPath) + + when: + verifier.verify('nf-foo@1.0.0', dir) + + then: + noExceptionThrown() + + cleanup: + SysEnv.pop() + dir.deleteDir() + Files.deleteIfExists(lockPath) + } + + def 'strict mode should abort when the extracted plugin does not match' () { + given: + SysEnv.push([NXF_PLUGINS_LOCK_MODE: 'strict']) + and: + def dir = pluginDir(['classes/A.class': 'poisoned']) + def lockPath = lockFileWith(['nf-foo@1.0.0': 'deadbeef']) + def verifier = new PluginLockVerifier(lockPath) + + when: + verifier.verify('nf-foo@1.0.0', dir) + + then: + def e = thrown(AbortOperationException) + e.message.contains('does not match') + + cleanup: + SysEnv.pop() + dir.deleteDir() + Files.deleteIfExists(lockPath) + } + + def 'warn mode should not abort on a mismatch' () { + given: + SysEnv.push([NXF_PLUGINS_LOCK_MODE: 'warn']) + and: + def dir = pluginDir(['classes/A.class': 'poisoned']) + def lockPath = lockFileWith(['nf-foo@1.0.0': 'deadbeef']) + def verifier = new PluginLockVerifier(lockPath) + + when: + verifier.verify('nf-foo@1.0.0', dir) + // second call to exercise the "log once" path + verifier.verify('nf-foo@1.0.0', dir) + + then: + noExceptionThrown() + + cleanup: + SysEnv.pop() + dir.deleteDir() + Files.deleteIfExists(lockPath) + } + + def 'off mode should ignore a mismatch' () { + given: + SysEnv.push([NXF_PLUGINS_LOCK_MODE: 'off']) + and: + def dir = pluginDir(['classes/A.class': 'poisoned']) + def lockPath = lockFileWith(['nf-foo@1.0.0': 'deadbeef']) + def verifier = new PluginLockVerifier(lockPath) + + when: + verifier.verify('nf-foo@1.0.0', dir) + + then: + noExceptionThrown() + + cleanup: + SysEnv.pop() + dir.deleteDir() + Files.deleteIfExists(lockPath) + } + + def 'should catch cache poisoning regardless of directory ownership' () { + given: + // pin the good tree, then tamper an extracted file in place (the shared-cache attack) + SysEnv.push([NXF_PLUGINS_LOCK_MODE: 'strict']) + and: + def dir = pluginDir(['classes/A.class': 'good', 'lib/x.jar': 'jar']) + final good = PluginLockVerifier.sha512Tree(dir) + def lockPath = lockFileWith(['nf-foo@1.0.0': good]) + def verifier = new PluginLockVerifier(lockPath) + and: + // attacker edits an already-extracted file + Files.write(dir.resolve('classes/A.class'), 'evil'.getBytes('UTF-8')) + + when: + verifier.verify('nf-foo@1.0.0', dir) + + then: + thrown(AbortOperationException) + + cleanup: + SysEnv.pop() + dir.deleteDir() + Files.deleteIfExists(lockPath) + } +}