Antalya 26.6: Bump to 26.6.4

[zvonand]

Changelog category (leave one):

• Not for changelog (changelog entry is not required)

CI/CD Options

Exclude tests:

• Fast test
• Integration Tests
• Stateless tests
• Stateful tests
• Unit tests
• Performance tests
• Aarch64 tests
• All with ASAN
• All with TSAN
• All with MSAN
• All with UBSAN
• All with Coverage
• All Regression
• Disable CI Cache

Regression jobs to run:

• Fast suites (mostly <1h)
• Aggregate Functions (2h)
• Alter (1.5h)
• Benchmark (30m)
• CAS (content-addressed storage; Antalya only)
• ClickHouse Keeper (1h)
• Iceberg (2h)
• LDAP (1h)
• OAuth (5m)
• Parquet (1.5h)
• RBAC (1.5h)
• SSL Server (1h)
• S3 (2h)
• S3 Export (2h)
• Swarms (30m)
• Tiered Storage (2h)

[github-actions]

Workflow [PR], commit [dfa2e9f]

[zvonand]

@blau-ai

[blau-ai]

CI triage for #2309 (bump to 26.6.4)

Verdict: the red is dominated by infra/flaky noise plus a mechanical DCO failure. Out of everything, only one failure looks like a genuine, deterministic regression that this bump introduces — 03363_hive_style_partition — and one adjacent test worth watching. Nothing here looks like a blocker you can't land, but 03363 deserves a look before merge.

Head SHA analysed: ee0a9959b00. Base: antalya-26.6.

Check	Class	PR-caused?
DCO	mechanical (missing Signed-off-by)	expected for a bump — see below
Stateless 03363_hive_style_partition (amd_debug seq, amd_debug s3 seq, asan db-disk seq)	real regression, 3/3 reruns	YES — from backport ClickHouse#111842, new in this PR
Stateless 04627_object_storage_lazy_hive_partitioning (several suites)	timeout in DB cleanup / "queries hung"	maybe — new test from same backport; watch
Integration test_schema_inference iceberg (targeted 12/12, db-disk 4/8)	node went down (EOF → connection refused)	no — infra-suspected
Integration test_auth_token_profile_events (7/8)	1 flaky, retry_failed (assert 0 >= 1)	no — flaky
Stateless CAS suites (cas s3 storage, cas storage) + Regression cas_*, tiered_storage_cas, settings, cas_s3_cache_*	S3 Connection refused (e.code()=111) to cas_s3, "Server died", some 5h-cancelled	no — CAS test-harness/infra
Stateless tests (amd_binary, cas storage)	ERROR, no results ("Job failed to produce Result … CI runner issue")	no — runner/infra

---

1. DCO — mechanical, expected for a release bump

cncf/dco2 requires every commit to carry a Signed-off-by trailer. This PR is a merge of the upstream 26.6.4 release + backport robot commits; 0 of the 100+ new commits are signed off (they're upstream/robot-clickhouse commits, so this is unavoidable). This is not a code problem and can't reasonably be fixed by rewriting hundreds of upstream commits — resolve it the way bump PRs normally are (maintainer/admin override of the DCO gate at merge time).

2. 03363_hive_style_partition — the one real, PR-introduced failure ⚠️

Fails reproducibly (3/3 reruns) on multiple independent runners — this is not flaky and not infra (the errors are all clean server-side logic errors, not connection failures). Both this test and its C++ come from backport ClickHouse#111842 "Resolve hive partitioning sample path lazily" (20b656d7c2c), which is new in this PR (not on antalya-26.6), so the regression is introduced here.

The failing statement:

-- Contains only partition columns in schema and use_hive_partitioning=1. The sample path for
-- hive partitioning detection is resolved lazily, so CREATE succeeds and the first use fails.
CREATE TABLE s3_table_half_schema_with_format (year UInt64)
  engine=S3(s3_conn, filename='half_baked/**.parquet', format=Parquet)
  SETTINGS use_hive_partitioning=1;

Code: 117. DB::Exception: A hive partitioned file can't contain only partition columns.
Try reading it with `use_hive_partitioning=0` and/or `partition_strategy != hive`. (INCORRECT_DATA)

The whole point of ClickHouse#111842 is that this CREATE should defer the sample-path resolution and succeed, and only the next statement (SELECT … -- {serverError INCORRECT_DATA}) should throw. Here the CREATE itself throws INCORRECT_DATA (from HivePartitioningUtils.cpp:181), so the deferral isn't taking effect for this explicit-schema + globbed-path (half_baked/**.parquet) case, and the rest of the .sql is aborted.

Suggested fix (for the backport author to confirm): the deferral is gated by

// src/Storages/ObjectStorage/StorageObjectStorage.cpp (added by 20b656d7c2c)
hive_partitioning_sample_path_deferred =
    !is_table_function && need_resolve_sample_path && !need_resolve_columns_or_format;

For a CREATE TABLE with explicit columns+format this should be true, yet resolution still happens at CREATE. Please diff StorageObjectStorage.cpp (the constructor's need_resolve_sample_path / need_resolve_columns_or_format computation and the eager-resolution block) against upstream ClickHouse#111842 — the backport likely dropped a companion hunk so a glob path still gets listed eagerly. If the eager throw is actually intended in 26.6, then the test's expectation (added in the same commit) needs updating instead. Upstream ClickHouse#111842 is green, so comparing the two constructors is the fastest way to pinpoint it. Happy to prepare a patch (either the C++ fix or the test-expectation update) on a blau/* branch and open it as a separate PR if you'd like — just say which.

3. 04627_object_storage_lazy_hive_partitioning — watch

Also new from ClickHouse#111842. It doesn't fail on a result mismatch — it times out in the test harness' DB-cleanup (_cleanup_database → clickhouse_execute_http blocked), and some suites also report "Some queries hung". That can be plain slowness under debug+asan, but given ClickHouse#111842 adds a hive_partitioning_resolution_mutex on the read path, a hang is at least plausible. Re-run first; if it keeps hanging, check it together with the 03363 investigation.

4. Everything else — infra / flaky, not PR-caused

• Iceberg test_schema_inference (targeted 12/12, db-disk 4/8): the first test hits ATTEMPT_TO_READ_AFTER_EOF, every subsequent one is Connection refused (…:9000) NETWORK_ERROR — one CH node dropped and never came back, failing all selected tests. No Received signal/segfault/coredump text in the report. Reads as node death/infra; safe to re-run. (If it reproduces, pull that node's clickhouse-server.log.zst to rule out a crash.)
• CAS stateless + all RegressionTestsRelease / CAS* / TieredStorageCAS / settings: dominated by S3Exception … Connection refused, bucket test/cas_s3/… and "Server died"; several regression jobs were cancelled at the 5h wall-clock. This is the Antalya CAS storage harness (minio-backed cache/shared-storage) being unreachable — environment, not the diff. Re-run.
• test_auth_token_profile_events (7/8): single retry_failed with assert refreshed >= 1 → assert 0 >= 1; classic flaky profile-event race. Safe to re-run.
• Stateless tests (amd_binary, cas storage): ERROR with no test results — "Job failed to produce Result due to a script error or CI runner issue". Pure runner failure.

---

Recommendation

Re-run the CAS/iceberg/04627 jobs (infra + flaky). Before merge, please look at 03363_hive_style_partition — it's a deterministic, PR-introduced failure from the ClickHouse#111842 backport. I can open a fix PR for it if you want; let me know whether you'd prefer the C++ deferral fix or a test-expectation update, and I'll put it on a blau/* branch.

I can't build or run ClickHouse in this environment, so the 03363 root-cause above is from static analysis of the backport diff + the CI report; the exact fix should be confirmed by CI on whatever patch we land.

[zvonand]

@blau-ai fix the above-mentioned 03363_hive_style_partition into this branch

[blau-ai]

Fixed 03363_hive_style_partition — committed to this branch

Done — pushed directly to bump/antalya-26.6/26.6.4 as you asked: commit 3440514 ("Defer hive partitioning sample path in StorageObjectStorageCluster").

Root cause

This is a backport gap, not a flaky/infra failure. It reproduced 3/3 reruns in all three sequential stateless jobs with the same clean server-side error on the CREATE (not the SELECT):

Code: 117. DB::Exception: A hive partitioned file can't contain only partition columns.
Try reading it with `use_hive_partitioning=0` and/or `partition_strategy != hive`. (INCORRECT_DATA)
(query: CREATE TABLE s3_table_half_schema_with_format (year UInt64)
        engine=S3(s3_conn, filename='half_baked/**.parquet', format=Parquet)
        SETTINGS use_hive_partitioning=1;)

Backport ClickHouse#111842 ("Resolve hive partitioning sample path lazily") taught StorageObjectStorage to defer listing the object storage — when the schema and format are already known, the hive sample-path is resolved on first use, so CREATE succeeds and only the later SELECT throws INCORRECT_DATA. The updated test expects exactly that:

CREATE TABLE s3_table_half_schema_with_format (year UInt64) engine=S3(...) SETTINGS use_hive_partitioning=1;
SELECT DISTINCT * FROM s3_table_half_schema_with_format; -- {serverError INCORRECT_DATA}

But in this fork a plain CREATE TABLE ... engine=S3(...) is served by the Altinity-specific StorageObjectStorageCluster wrapper (registerStorageObjectStorage.cpp:99), not by bare StorageObjectStorage. The backport only patched the inner storage; the cluster wrapper kept its own eager block (StorageObjectStorageCluster.cpp:190-205) that still called getPathSample() + setupHivePartitioningForObjectStorage() at CREATE. With a schema of only partition columns, that throws INCORRECT_DATA from HivePartitioningUtils.cpp:181 right at CREATE — before deferral ever kicks in.

The fix

Mirror the inner storage's deferral in the cluster wrapper: compute the same hive_partitioning_sample_path_deferred condition and skip the eager getPathSample() when it holds. The inner pure_storage already carries the identical deferral and resolves it lazily on read (readFallBackToPure → pure_storage->read → resolveHivePartitioningSamplePathIfDeferred), so the first SELECT now throws INCORRECT_DATA as the test expects.

const bool need_resolve_sample_path = context_->getSettingsRef()[Setting::use_hive_partitioning]
    && !configuration->isDataLakeConfiguration()
    && !configuration->getPartitionStrategy();
const bool hive_partitioning_sample_path_deferred =
    !is_table_function && need_resolve_sample_path && !need_resolve_columns_or_format;

if (updated_configuration && sample_path.empty()
        && need_resolve_sample_path
        && !hive_partitioning_sample_path_deferred)
{
    sample_path = getPathSample(context_);
}

Why it's safe / scoped

The deferral only triggers for a non-table-function table with an explicit schema + explicit format, use_hive_partitioning=1, no partition_strategy, and not a datalake — i.e. only the s3_table_half_schema_with_format case (line 81). Every other statement in 03363 uses partition_strategy='hive', use_hive_partitioning=0, or an s3(...) table function, so none are affected. When deferred, setupHivePartitioningForObjectStorage() runs with an empty sample path (empty hive columns, no throw) — the same shape the inner StorageObjectStorage constructor already uses when deferring.

Likely bonus

04627_object_storage_lazy_hive_partitioning (also new from ClickHouse#111842) was timing out in DB cleanup — consistent with CREATE TABLE 04627_unreachable ENGINE = S3('http://localhost:1/...') hanging on the same eager getPathSample() for the full S3 retry budget. This change should let that CREATE return immediately too. Worth re-checking on the next run.

Caveat

I can't build or run ClickHouse in this environment, so this is validated by static analysis of the backport diff + the CI report, not compiled locally. Please let CI on this branch confirm it. The other red on this PR (DCO on the release bump, CAS/iceberg S3 Connection refused, test_auth_token_profile_events flake) is unrelated infra/mechanical noise as covered in my earlier triage.

[zvonand]

@blau-ai

[zvonand]

@blau-ai

[blau-ai]

CI triage for #2309 (26.6.4 release-sync)

Verdict — 6 red checks: 0 caused by anything unique to this PR, 1 real (deterministic) bug inherited from antalya-26.6, 1 policy/metadata check, 4 experimental-CAS-config instability.

Every failing test suite is either the one deterministic Iceberg bug or an experimental CAS s3 storage config (cas s3 storage, cas_selects, settings, tiered_storage_cas). All the non-CAS suites that actually ran are green (Fast test, Integration … 1/8, 2/8, builds, install, compatibility, CI Tests, Grype). Nothing here traces to the version-bump/backport payload itself.

---

1. DCO — fail (policy, not code)

There are 100 commits incorrectly signed off … Sign-off not found

These are the upstream 26.6 backport commits that make up the release sync; they don't carry Signed-off-by. Expected for a bump/backport PR. Not fixable by me — it would require rewriting 100 commit messages (history rewrite / force-push), which I'm not permitted to do. Resolve the usual way for release PRs: admin-merge / DCO override, or a maintainer re-applies with --signoff. Not a code problem.

2. Integration tests (amd_asan_ubsan, db disk, old analyzer, 4/8) — fail — real bug, but pre-existing on antalya-26.6

Two deterministic failures:

• test_storage_iceberg_with_spark/test_writes.py::test_writes_decimal_wide_minmax_pruning[s3]
• …[local]

Code: 36. DB::Exception: Iceberg decimal type supports precision up to 38, got 76. (BAD_ARGUMENTS)
(query: CREATE TABLE … (d128 Decimal(38, 10), d256 Decimal(76, 20), control Int64) ENGINE=IcebergS3(…))

This is a self-contradiction already sitting on the base branch, not introduced by this PR:

• Upstream Support decimal types for writing in data lakes ClickHouse/ClickHouse#115832 (merged as 9091be4a672, ancestor of antalya-26.6) removed the precision guard in getIcebergType and added test_writes_decimal_wide_minmax_pruning, which creates Decimal(76, 20) and requires the CREATE to succeed.
• e3edda0f5ee "Reject Iceberg decimal with precision above 38 again" (@zvonand, also an ancestor of antalya-26.6) re-added the precision > 38 throw at src/Storages/ObjectStorage/DataLakes/Iceberg/Utils.cpp:585.

Both the guard and the test are present on origin/antalya-26.6, so this shard is red on the base branch too — inherited here, not caused by the bump.

It's still a genuine bug worth fixing, and it's a design call for whoever owns the guard (@zvonand):

• If wide Iceberg decimals should stay rejected (the guard's stated cross-engine-safety rationale): the upstream test must be adapted or skipped — it exercises Decimal(76, 20), which is exactly what the guard forbids, so it can never pass while the guard stands. Simplest is to drop/pytest.mark.skip test_writes_decimal_wide_minmax_pruning (or narrow d256 to a supported precision, though that defeats the test's point). There are also the matching unit tests (IcebergTypeMapping.DecimalPrecisionAboveSpecLimitIsRejected etc.) the guard was meant to satisfy — keep those.
• If wide decimals should be supported (upstream's stance): revert e3edda0f5ee and instead adjust the Altinity unit tests that expect rejection.

I'd lean toward the first (keep the guard, adapt the test) since the guard is the more recent, deliberate decision — but it needs your call. Happy to prepare the diff (test-side skip or guard revert) as a separate blau/* PR on your say-so.

3. Stateless tests (amd_asan_ubsan, cas s3 storage, parallel, 1/2 & 2/2) — fail — infra / CAS-config instability

1/2: 14 failed of ~5600; 2/2: 0 failed but flagged "Some queries hung". The 1/2 failures are a cascade of server exhaustion, not independent test bugs — the tail is dominated by:

Code: 241 … MEMORY_LIMIT_EXCEEDED
Internal query (CREATE/DROP DATABASE) failed: HTTPError

i.e. once the server hits memory pressure under the CAS s3 config, subsequent unrelated tests fail on DB setup and queries hang. This is resource/instability on the experimental CAS storage layer, not something a version bump changes. Safe to re-run; if it persists it's a CAS-config capacity issue, not a PR regression.

4. Stateless tests (amd_binary, cas s3 storage, parallel) — fail — CAS-layer-specific

Single failure 02265_column_ttl, reproducible (19/19, and 19/19 without randomized settings — so not a settings-randomization flake). Root cause is CAS-storage part exchange:

Code: 210. NETWORK_ERROR: Source localhost did not prove it still holds the manifest
it offered for part 20100101_0_0_1 by relink; the relink is abandoned …

The "manifest … by relink" path is specific to the CAS storage feature. This is a property of that experimental config's replicated fetch, not of the 26.6.4 bump. Belongs with a CAS-storage owner, not this PR.

5. Regression — cas_selects (8 scenarios), settings (4 scenarios), tiered_storage_cas (errored, ~2h) — CAS experimental suites, likely infra/pre-existing

All three are Altinity CAS/tiered-storage regression suites. tiered_storage_cas "errored" after ~2h — that reads as a timeout, not an assertion failure. Same pattern as #3/#4: failures concentrated entirely in the CAS configuration. I couldn't attribute these to the bump payload; recommend re-running and comparing against a recent antalya-26.6 regression run — if base is red on the same CAS scenarios, they're pre-existing.

---

Bottom line

• Nothing to fix in the bump payload — the non-CAS matrix is green.
• DCO is expected for a 100-commit backport PR; needs a maintainer/admin, not a code change.
• The Iceberg decimal shard (Update README.md #2) is the only real, deterministic defect, and it's inherited from antalya-26.6 (guard e3edda0f5ee vs. upstream test both live on base). Worth fixing on the base branch; @zvonand, tell me which way you want it (keep guard → adapt test, or revert guard) and I'll open a blau/* PR.
• All the CAS suites (Update README.md #3–Update README.md #5) are experimental-config instability/timeouts — re-run and compare to base; not bump-caused.