Antalya-26.6: iceberg v3 multi-arg transforms ,added logic to read source_ids and not throw an error

[subkanthi]

#2316
implementation similar to apache/iceberg-python#3630

Changelog category (leave one):

• New Feature

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Implement iceberg v3 feature when the transformations are created in multiple columns, source-ids data is stored in the manifest.
This PR will read iceberg tables created with source-ids and not throw an error.

CI/CD Options

Exclude tests:

• Fast test
• Integration Tests
• Stateless tests
• Stateful tests
• Unit tests
• Performance tests
• Aarch64 tests
• All with ASAN
• All with TSAN
• All with MSAN
• All with UBSAN
• All with Coverage
• All Regression
• Disable CI Cache

Regression jobs to run:

• Fast suites (mostly <1h)
• Aggregate Functions (2h)
• Alter (1.5h)
• Benchmark (30m)
• CAS (content-addressed storage; Antalya only)
• ClickHouse Keeper (1h)
• Iceberg (2h)
• LDAP (1h)
• OAuth (5m)
• Parquet (1.5h)
• RBAC (1.5h)
• SSL Server (1h)
• S3 (2h)
• S3 Export (2h)
• Swarms (30m)
• Tiered Storage (2h)

[github-actions]

Workflow [PR], commit [c320405]

[subkanthi]

@blau-ai

[blau-ai]

CI triage for #2318

Verdict: none of the red checks are caused by this PR. This PR changes only 8 Iceberg data-lake C++ files (ManifestFile*, ManifestFileIterator, Utils, ChunkPartitioner, MetadataGenerator, Constant.h, + a gtest). Every failure below is in a code path the diff does not touch, and the two Iceberg-looking ones are already tracked as pre-existing branch bugs.

Breakdown: 2 integration failures — pre-existing & tracked · 1 stateless — flaky · 1 stateless — pre-existing (cas storage) · Regression TestFlows — pre-existing branch state.

---

Integration tests (amd_asan_ubsan, db disk, old analyzer, 4/8) — 12/1102 ❌ NOT this PR (pre-existing, tracked)

All 12 are test_storage_iceberg_with_spark/test_schema_inference.py::test_schema_inference[*]. It's 1 real crash + 11 cascades: [s3-1-True] kills node1 (ATTEMPT_TO_READ_AFTER_EOF), the other 11 then get Connection refused (…:9000).

This is issue #2216 — a signed-integer overflow parsing Iceberg Decimal bounds with scale > 18 in IcebergFieldParseHelpers.cpp:147, reachable on antalya-26.6 since #2145 (merged 2026-08-11). It fails 12/12 on the base branch on every run since 2026-08-11. The faulty code (IcebergFieldParseHelpers.cpp, IDataLakeMetadata.cpp) is not in this PR's diff — same suite name, different code path. → no action for this PR; tracked in #2216.

Integration tests (amd_asan_ubsan, db disk, old analyzer, 7/8) — 1/635 ❌ NOT this PR (pre-existing, tracked)

test_database_iceberg_lakekeeper_catalog/test.py::test_auth_token_profile_events — deterministic assert 0 >= 1 (reproduces on retry). This is issue #2323: a test-only defect introduced by #2222 (merged 2026-09-04) that fails on every integration build of the branch. Unrelated to partition transforms. → tracked in #2323.

Stateless tests (amd_binary, cas storage / cas s3 storage) — 02265_column_ttl ❌ NOT this PR (flaky/infra)

Praktika labelled it flaky (1/38 reruns failed on one job). Failure is a NETWORK_ERROR in MergeTree/DataPartsExchange.cpp relink during a replicated fetch on shared (cas) storage — nothing to do with Iceberg. The check also carries do_not_block_pipeline_on_failure: true. → safe to re-run.

Stateless tests (amd_asan_ubsan, cas s3 storage, parallel, 2/2) — 02435_rollback_cancelled_queries ❌ NOT this PR (pre-existing, cas storage)

Reproducible (3/3): a cancelled INSERT isn't rolled back, so system.processes shows an extra query. This is transaction/rollback behavior on Altinity cas (shared) storage — a config the Iceberg-only diff cannot affect. Pre-existing to the cas-storage config.

Regression (TestFlows): cas_alter_attach_1, cas_s3_cache_*, clickhouse_keeper_no_ssl_1, selects, settings ❌ NOT this PR

These suites exercise ALTER/ATTACH, S3 cache, Keeper, settings and selects — none read Iceberg data-lake code. They reflect the branch's existing regression state, not this diff.

---

Bottom line

The PR itself looks healthy from CI's perspective: the changed code compiled in every build and the Iceberg-specific failures are the branch's known pre-existing crashes (#2216, #2323), not regressions from this diff. No fix is needed on this PR to make these checks pass — they need the underlying branch issues resolved (or a re-run for the flaky 02265). If you want, I can re-run the flaky stateless job.

_{Triaged from the praktika S3 report for 880b51f. I can't build/run ClickHouse here, so this is evidence-based from CI artifacts; correctness of the change itself is ultimately validated by green checks once the branch bugs are addressed.}

[subkanthi]

pyiceberg/trino or spark versions must be updated to create integration tests.