Conditional GET (ETag / Last-Modified) for Maven metadata downloads

[knutwannheden]

Motivation

Caches that store maven-metadata.xml version lists with a freshness window (for example a persistent cross-JVM cache with a 1-hour TTL) pay a recurring "cold metadata" tax: once the window elapses, the next resolution re-downloads and re-parses the metadata for every GAV across every configured repository, even when nothing has changed. Released POMs are immutable and cache cleanly, so the cost is specifically the version-list metadata that drives latest.release, version ranges, RELEASE / LATEST, BOM / plugin version discovery, and snapshot timestamps. The TTL can't simply be lengthened because metadata is genuinely mutable — new versions get published, and latest.release-style recipes would silently miss them. The correct fix is an HTTP conditional request: send If-None-Match (ETag) / If-Modified-Since (Last-Modified) and handle 304 Not Modified, whose empty body lets us skip both the download and the parse while refreshing the entry. Maven Central, Artifactory, and Nexus all support these on maven-metadata.xml.

Examples

MavenPomCache exchanges a single MavenMetadataCacheEntry for maven-metadata.xml — it carries the value, its ETag / Last-Modified validators, and an expired verdict. A cache that wants conditional GETs persists the validators and, once an entry's freshness window has elapsed, hands it back marked expired so the downloader revalidates it instead of re-downloading:

class MyCache implements MavenPomCache {
    // Persist the value together with the ETag / Last-Modified the origin returned.
    @Override
    public void putMavenMetadata(URI repo, GroupArtifactVersion gav, MavenMetadataCacheEntry metadata) {
        store(repo, gav, metadata, /* freshAt */ now());
    }

    // Serve a fresh entry directly; once it has expired, hand it back marked expired so the
    // downloader revalidates it with a conditional GET rather than re-downloading and re-parsing.
    @Override
    public @Nullable MavenMetadataCacheEntry getMavenMetadata(URI repo, GroupArtifactVersion gav) {
        Entry e = lookup(repo, gav);
        return e == null ? null : e.value.withExpired(isStale(e.freshAt));
    }
}

On the next resolution after the entry expires, the downloader sends If-None-Match / If-Modified-Since; a 304 reuses the cached value (no parse) and re-stores it (refreshing freshness), while a 200 replaces the value and captures the new validators. A cache that never marks an entry expired (the in-memory default) keeps serving it directly with no network call.

Summary

• Added MavenMetadataCacheEntry — a value type carrying a cached MavenMetadata, its ETag / Last-Modified validators, and an expired read-time verdict. Write-side entries are built with MavenMetadataCacheEntry.fresh(...) (never expired); caches that expire entries recompute the verdict on read (e.g. via withExpired(...)).
• Reworked MavenPomCache's metadata methods into a single get/put pair carrying that entry: getMavenMetadata returns a @Nullable MavenMetadataCacheEntry (a null return is a cache miss; a non-null entry carries the value — or a negative result when its metadata is null — together with any validators, and is marked expired when the caller should revalidate), and putMavenMetadata accepts the same entry. This replaces the previous Optional<MavenMetadata>-returning get and the bare put.
• Breaking change: the two methods are no longer default and their signatures changed, so every MavenPomCache implementation must adopt the new entry type. Updated in this PR: InMemoryMavenPomCache, RocksdbMavenPomCache (metadata remains a no-op there), and CompositeMavenPomCache.
• MavenPomDownloader.downloadMetadata consults the single get, serves a fresh hit (value or negative) directly with no network call, and on a miss or expired entry issues a conditional GET (If-None-Match / If-Modified-Since) using the entry's validators; a 304 reuses the cached value (skipping the parse), a 200 captures the new ETag / Last-Modified. The 304 status is surfaced through the HttpSender response rather than being treated as an error.
• The downloader writes the metadata cache only when it actually produced the value this iteration (download, file read, derivation, or 304 revalidation) rather than re-writing a value served straight from the cache, so a stored validator is never overwritten with nulls on a cache hit.
• CompositeMavenPomCache needs no conditional-GET-specific code: forwarding the unified get/put through both layers carries the validators automatically. It promotes only a still-fresh l2 hit into l1, leaving an expired l2 entry (validators only) for the downloader to revalidate rather than seeding l1 with a stale value.

Test plan

• MavenPomDownloaderTest.ConditionalMetadataRequests.reusesCachedValueOn304 — the second request sends If-None-Match, the server replies 304, and the cached value is reused without re-parsing a body.
• downloadsNewMetadataWhenChanged — a conditional GET whose validator no longer matches receives a fresh 200 (new version surfaces), the new ETag is stored, and the following revalidation is a 304 against the new ETag.
• sendsNoConditionalHeadersWhenCacheRetainsNoValidator — a cache that always misses performs only plain, unconditional downloads.
• CompositeMavenPomCacheTest — the unified put forwards a value and its validators to the persistent layer, and an l2 entry (with validators) surfaces through the composite on an l1 miss.
• Full :rewrite-maven:test suite plus :rewrite-gradle consumers, and license checks, pass.

[jkschneider]

Like to see if we can do this without adding more overloads.

[knutwannheden]

I've simplified the APIs now. Please check if that works for you.