-
Notifications
You must be signed in to change notification settings - Fork 3.4k
Site: Add blog post for iceberg-rust 0.10.0 #17032
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
dannycjones
wants to merge
6
commits into
apache:main
Choose a base branch
from
dannycjones:blog-iceberg-rust-0.10
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+159
−0
Draft
Changes from 1 commit
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
1735336
Site: Add blog post for iceberg-rust 0.10.0
dannycjones b5b1ebe
Update blog post based on feedback, other improvements
dannycjones 363b27f
Merge branch 'main' into blog-iceberg-rust-0.10
dannycjones 6348c20
Update for RC3
dannycjones 5cd02b3
Update based on PR feedback
dannycjones 5e4589b
Address weird whitespacing
dannycjones File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
147 changes: 147 additions & 0 deletions
147
site/docs/blog/posts/2026-07-01-iceberg-rust-0.10.0-release.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,147 @@ | ||
| --- | ||
| date: 2026-07-06 | ||
| title: Apache Iceberg Rust 0.10.0 Release | ||
| slug: apache-iceberg-rust-0.10.0-release | ||
| authors: | ||
| - iceberg-pmc | ||
| categories: | ||
| - release | ||
| --- | ||
|
|
||
| <!-- | ||
| - Licensed to the Apache Software Foundation (ASF) under one or more | ||
| - contributor license agreements. See the NOTICE file distributed with | ||
| - this work for additional information regarding copyright ownership. | ||
| - The ASF licenses this file to You under the Apache License, Version 2.0 | ||
| - (the "License"); you may not use this file except in compliance with | ||
| - the License. You may obtain a copy of the License at | ||
| - | ||
| - http://www.apache.org/licenses/LICENSE-2.0 | ||
| - | ||
| - Unless required by applicable law or agreed to in writing, software | ||
| - distributed under the License is distributed on an "AS IS" BASIS, | ||
| - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| - See the License for the specific language governing permissions and | ||
| - limitations under the License. | ||
| --> | ||
|
|
||
| The Apache Iceberg community is pleased to announce version 0.10.0 of [iceberg-rust](https://github.com/apache/iceberg-rust). | ||
|
|
||
| This release covers development work from early March through late June 2026 and is the result of merging **238 PRs** from **40 contributors**. See the [changelog] for the complete list of changes. | ||
|
|
||
| [changelog]: https://github.com/apache/iceberg-rust/blob/main/CHANGELOG.md#v0100---2026-06-29 | ||
|
|
||
| `iceberg-rust` is a native Rust implementation of the Apache Iceberg specification, providing high-performance libraries for reading, writing, and managing Iceberg tables in Rust applications and through Python bindings (`pyiceberg-core`). | ||
|
|
||
| <!-- more --> | ||
|
|
||
| ## Release Highlights | ||
|
|
||
| There has been plenty of contributions across the `iceberg-rust` library, here's a highlight of a few. | ||
|
|
||
| ### Schema Evolution | ||
|
|
||
| This release adds the [`update_schema` action](https://github.com/apache/iceberg-rust/pull/2120), enabling programmatic table schema updates. This allows `iceberg-rust` users to add, drop, rename, and update columns through a builder interface. | ||
|
dannycjones marked this conversation as resolved.
Outdated
|
||
|
|
||
| ### Snapshot Expiration | ||
|
|
||
| A new [`ExpireSnapshotsAction`](https://github.com/apache/iceberg-rust/pull/2591) provides a high-level API for expiring old snapshots, respecting `history.expire.*` table properties and cleaning up associated statistics file metadata for the expired snapshots. | ||
|
dannycjones marked this conversation as resolved.
Outdated
|
||
|
|
||
| ### Custom Runtime Support | ||
|
|
||
| The [`Runtime` trait](https://github.com/apache/iceberg-rust/pull/2308) can now be injected into catalogs, decoupling `iceberg-rust` from a hard tokio dependency at the catalog layer. This enables embedding in applications with custom async runtimes. | ||
|
|
||
| ### Writer Improvements | ||
|
|
||
| - **[Table property-driven Parquet settings](https://github.com/apache/iceberg-rust/pull/2561)**: `ParquetWriterBuilder::from_table_properties` now honors `write.parquet.*` table properties, eliminating the need for manual configuration | ||
| - **[Content-defined chunking](https://github.com/apache/iceberg-rust/pull/2375)**: CDC table properties for optimizing Parquet file boundaries | ||
| - **[Purge table support](https://github.com/apache/iceberg-rust/pull/2232)**: Catalogs can now purge tables, deleting both metadata and data files | ||
|
|
||
| ### Enhanced DataFusion Integration | ||
|
|
||
| - **[EXPLAIN shows pushed-down limits](https://github.com/apache/iceberg-rust/pull/2360)**: `IcebergTableScan` now displays limit pushdown in EXPLAIN output for query debugging | ||
| - **[IsNaN predicate pushdown](https://github.com/apache/iceberg-rust/pull/2592)**: NaN-preserving numeric expressions are now pushed into scans | ||
| - **[Empty insert handling](https://github.com/apache/iceberg-rust/pull/2712)**: INSERT operations that produce no rows now correctly return a single row with count 0 | ||
|
|
||
| ### Reader and Scan Improvements | ||
|
|
||
| - **[Scan I/O metrics](https://github.com/apache/iceberg-rust/pull/2349)**: New `read_with_metrics()` API exposes bytes read and other I/O statistics during scans | ||
| - **[Name-mapped field IDs](https://github.com/apache/iceberg-rust/pull/2612)**: Projection and predicate pushdown now correctly use name-mapped field IDs for tables without embedded field IDs | ||
| - **[FixedBinary(N) support](https://github.com/apache/iceberg-rust/pull/2348)**: Added support for fixed-length binary types | ||
| - **[Snappy codec for Avro](https://github.com/apache/iceberg-rust/pull/2573)**: Avro files compressed with Snappy can now be read | ||
|
|
||
| ### Table Encryption (Foundational) | ||
|
|
||
| Significant groundwork has been laid toward Iceberg's table encryption specification, including [AES-GCM cryptographic primitives](https://github.com/apache/iceberg-rust/pull/2026), [streaming encryption/decryption](https://github.com/apache/iceberg-rust/pull/2286), a [KMS trait](https://github.com/apache/iceberg-rust/pull/2339), [StandardKeyMetadata](https://github.com/apache/iceberg-rust/pull/2340), an [Encryption Manager](https://github.com/apache/iceberg-rust/pull/2383), and [manifest-list-level decryption plumbing](https://github.com/apache/iceberg-rust/pull/2453). Writes to encrypted tables are [explicitly blocked](https://github.com/apache/iceberg-rust/pull/2626) until the full write path is complete. End-to-end read and write support for encrypted tables is expected in a future release. See the [table encryption RFC](https://github.com/apache/iceberg-rust/pull/2183) for the full design. | ||
|
|
||
| ### Storage Improvements | ||
|
|
||
| #### New Storage Backend - HuggingFace Hub | ||
|
|
||
| A new [HuggingFace Hub storage backend](https://github.com/apache/iceberg-rust/pull/2375) was contributed, enabling direct access to Iceberg tables hosted on HuggingFace datasets. | ||
|
|
||
| #### Other improvements | ||
|
|
||
| - **[`delete_stream` on Storage trait](https://github.com/apache/iceberg-rust/pull/2216)**: Batch delete operations for efficient cleanup | ||
| - **[OpenDAL resolving storage](https://github.com/apache/iceberg-rust/pull/2231)**: Automatically resolves the appropriate storage backend based on file path scheme | ||
| - **[Timeout layer for OpenDAL](https://github.com/apache/iceberg-rust/pull/2455)**: Added `TimeoutLayer` inside `RetryLayer` to prevent indefinite hangs | ||
| - **[S3 virtual-host-style addressing](https://github.com/apache/iceberg-rust/pull/2330)**: Default to virtual-host-style S3 addressing for broader compatibility | ||
|
|
||
| ## Bug Fixes | ||
|
|
||
| Notable correctness fixes in this release: | ||
|
|
||
| - **[INT96 timestamp values](https://github.com/apache/iceberg-rust/pull/2301)**: Fixed incorrect Parquet INT96 timestamp interpretation | ||
| - **[Nested type column indices](https://github.com/apache/iceberg-rust/pull/2307)**: Fixed `build_fallback_field_id_map` producing incorrect indices for schemas with nested types | ||
| - **[Delete-only manifests in FastAppend](https://github.com/apache/iceberg-rust/pull/2545)**: Preserved delete-only manifests that were incorrectly dropped | ||
| - **[DeleteFileIndex lost wakeup](https://github.com/apache/iceberg-rust/pull/2696)**: Fixed a hang caused by a lost-wakeup race in `get_deletes_for_data_file` | ||
| - **[Row group byte range filtering](https://github.com/apache/iceberg-rust/pull/2615)**: Fixed row duplication for sub-row-group file splits | ||
| - **[Cross-engine decimal compatibility](https://github.com/apache/iceberg-rust/pull/2538)**: Added space in decimal type serialization for compatibility with other engines | ||
| - **[Puffin magic number validation](https://github.com/apache/iceberg-rust/pull/2416)**: Added missing validation for magic number at the start of a Puffin file | ||
| - **[Glue catalog filtering](https://github.com/apache/iceberg-rust/pull/2570)**: Only list Iceberg tables, not all Glue tables | ||
|
|
||
| ## Breaking Changes | ||
|
|
||
| - **[Purge table](https://github.com/apache/iceberg-rust/pull/2232)**: Added `purge_table` to the `Catalog` trait; some catalog implementations moved their data-deletion logic from `drop_table` to `purge_table` | ||
| - **[Compression codec enum](https://github.com/apache/iceberg-rust/pull/2288)**: Enhanced with additional variants | ||
| - **[Custom Runtime in Catalog](https://github.com/apache/iceberg-rust/pull/2308)**: Added `with_runtime` to the `CatalogBuilder` trait | ||
| - **[DefaultLocationGenerator](https://github.com/apache/iceberg-rust/pull/2604)**: `new()` now borrows `TableMetadata` instead of taking ownership | ||
| - **[MSRV](https://github.com/apache/iceberg-rust/pull/2652)**: Bumped to Rust 1.94 | ||
|
|
||
| ## Dependency Updates | ||
|
|
||
| - [Upgraded to DataFusion 53.1](https://github.com/apache/iceberg-rust/pull/2206) and [Arrow 58](https://github.com/apache/iceberg-rust/pull/2206) | ||
|
dannycjones marked this conversation as resolved.
Outdated
|
||
| - [Upgraded OpenDAL to 0.57](https://github.com/apache/iceberg-rust/pull/2588) | ||
| - [Upgraded reqwest to 0.13.3](https://github.com/apache/iceberg-rust/pull/2429) | ||
| - [Upgraded tokio to 1.52.1](https://github.com/apache/iceberg-rust/pull/2374) | ||
| - [Upgraded rand to 0.10.1](https://github.com/apache/iceberg-rust/pull/2449) | ||
|
|
||
| ## Infrastructure Improvements | ||
|
|
||
| - **Public API stability**: [cargo-public-api checks](https://github.com/apache/iceberg-rust/pull/2525) on every PR to catch unintentional API breakage | ||
| - **Trusted publishing**: [Migrated to trusted publishing](https://github.com/apache/iceberg-rust/pull/2593) for crates.io releases | ||
| - **Standalone compilation**: [CI verifies each crate compiles independently](https://github.com/apache/iceberg-rust/pull/2389) | ||
| - **Security**: Pinned all third-party actions to Apache-approved SHAs; ASF allowlist check on every PR | ||
| - **Performance**: Disabled debuginfo and incremental compilation in CI; Rust caches saved only from main | ||
|
|
||
| ## What's Next | ||
|
|
||
| Work is already underway on several features for upcoming releases: | ||
|
|
||
| - **Table encryption**: With the foundational encryption stack landed, the community is working toward end-to-end encrypted reads and writes | ||
| - **Additional commit types**: Support for `RewriteDataFiles`, `RowDelta`, and `Overwrite` commit operations | ||
| - **Extensibility**: Continued investment in abstractions for pluggable storage backends and runtime configuration | ||
|
|
||
| ## Getting Involved | ||
|
|
||
| The `iceberg-rust` project welcomes contributions. We use GitHub [issues](https://github.com/apache/iceberg-rust/issues) for tracking work and the [Apache Iceberg Community Slack](https://iceberg.apache.org/community/#slack) for discussions. | ||
|
|
||
| The easiest way to get started is to: | ||
|
|
||
| 1. Try `iceberg-rust` with your workloads and report any issues you encounter | ||
| 2. Review the [contributor guide](https://github.com/apache/iceberg-rust/blob/main/CONTRIBUTING.md) | ||
| 3. Look for [good first issues](https://github.com/apache/iceberg-rust/contribute) | ||
|
|
||
| Code review is also a very welcome contribution - please provide feedback on pull requests where you feel comfortable to do so! | ||
|
|
||
| For more information, visit the [iceberg-rust repository](https://github.com/apache/iceberg-rust). | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.