diff --git a/content/influxdb3/enterprise/reference/cli/influxdb3/manage/retry-upgrade-to-pacha-tree.md b/content/influxdb3/enterprise/reference/cli/influxdb3/manage/retry-upgrade-to-pacha-tree.md index 25046c532a..6ef413c0e7 100644 --- a/content/influxdb3/enterprise/reference/cli/influxdb3/manage/retry-upgrade-to-pacha-tree.md +++ b/content/influxdb3/enterprise/reference/cli/influxdb3/manage/retry-upgrade-to-pacha-tree.md @@ -26,6 +26,11 @@ The command re-checks every previously failed upgrade source against the catalog The reset itself only changes durable state. The migration resumes the next time a compactor node starts with `--upgrade-pacha-tree`. +Starting in v3.11.8 and v3.12.2, the command refuses to run while any ingest node +in the upgrade lists Parquet files it couldn't convert. +Follow the [retry procedure](#retry-after-an-ingest-conversion-failure) to resolve +these failures before you run the command. + ## Usage @@ -55,3 +60,22 @@ influxdb3 manage retry-upgrade-to-pacha-tree \ ``` If any sources are left unresolved, run the command again to re-attempt them. + +## Retry after an ingest conversion failure + +Starting in v3.11.8 and v3.12.2, an ingest node reports exhausted conversion +attempts in `system.upgrade_parquet_node`. +The cluster upgrade remains `failed` until you reset it with this command. + +To reset the failed upgrade and resume the migration, follow these steps: + +1. [Query `system.upgrade_parquet_node`](/influxdb3/enterprise/admin/query-system-data/#query-storage-engine-tables) + and check `last_message` for the file and error. +2. Fix the cause of the conversion failure. +3. Restart each affected ingest node to retry its files. +4. Run `influxdb3 manage retry-upgrade-to-pacha-tree` as shown in the [example](#example). + If the command reports that an ingest node still lists unconverted files, + resolve that node's failure and retry. +5. Restart a compactor node with + [`--upgrade-pacha-tree`](/influxdb3/enterprise/reference/internals/storage-engine/#upgrade-from-parquet) + to resume the migration. diff --git a/content/shared/influxdb3-admin/query-system-data/_index.md b/content/shared/influxdb3-admin/query-system-data/_index.md index d0761100cd..c6f391f81d 100644 --- a/content/shared/influxdb3-admin/query-system-data/_index.md +++ b/content/shared/influxdb3-admin/query-system-data/_index.md @@ -335,6 +335,12 @@ The `conversion_*` and `import_*` columns report conversion and import progress. These columns are null on `ingest` rows. +Starting in v3.11.8 and v3.12.2, if a Parquet file fails every conversion attempt, +the `status` column of the node's `ingest` row is `failed`. +The `last_message` column identifies the file and its error. +To resume the upgrade, follow the +[retry procedure](/influxdb3/enterprise/reference/cli/influxdb3/manage/retry-upgrade-to-pacha-tree/#retry-after-an-ingest-conversion-failure). + **Query telemetry** — the query telemetry endpoint provides detailed execution statistics for analyzing query performance: diff --git a/content/shared/v3-core-enterprise-release-notes/_index.md b/content/shared/v3-core-enterprise-release-notes/_index.md index 00faa459d6..a8d02f7fc5 100644 --- a/content/shared/v3-core-enterprise-release-notes/_index.md +++ b/content/shared/v3-core-enterprise-release-notes/_index.md @@ -6,6 +6,88 @@ > All updates to Core are automatically included in Enterprise. > The Enterprise sections below only list updates exclusive to Enterprise. +## v3.12.2 {date="2026-10-29"} + + + +### Core + +#### Features + +- **`influxdb3 debug catalog repair`**: This offline command, added in v3.11.6, is now also available on 3.12.2. It repairs a catalog that fails to load because of a duplicate table name or a reused table or database ID. Stop every node before you run it with `--execute`. +- **Processing engine Python updated to 3.13.18**: Picks up the security fixes released since 3.13.15. + +#### Bug fixes + +- **Queries that exceed memory limits returned `500`**: The HTTP query endpoints now return `429 Too Many Requests` when a query exhausts the DataFusion memory pool. Previously, they returned `500`. For CSV, JSON, and JSON Lines output, this applies when the query fails before it returns any rows. An error after the first rows still ends the HTTP `200` response early. Some Core queries still return `500`, such as a query on a table that runs out of memory while deduplicating data. Enterprise already mapped resource-exhaustion errors to `429`. +- **Catalog deadlock when checking a deleted table**: Checking whether a table is deleted now takes the catalog lock once. Previously, a catalog change that arrived between its two reads could deadlock the catalog. +- **WAL plugins received values under the wrong column names**: Rows passed to a WAL trigger's plugin now use the column names of their own record batch. Previously, when a flush held batches with the same columns in a different order, later batches used the first batch's column names. +- Other bug fixes and performance improvements + +### Enterprise + +All Core updates are included in Enterprise. +Additional Enterprise-specific updates: + +#### Features + +- **Faster [file index](/influxdb3/enterprise/admin/file-index/) loading (Parquet engine)**: Loading a table's file index, including during a storage engine upgrade, now sorts each posting list once. Previously, every generation re-sorted each list it touched, so a table with many generations loaded slowly and used more memory. +- **Faster reads of compaction metadata (Parquet engine)**: Compaction and snapshot metadata files are now parsed in one pass, including during a storage engine upgrade. Previously, each was parsed twice. +- **Table index cache starts once the server is ready (Parquet engine)**: On ingest nodes, the table index cache now starts its background work when the server reports ready, instead of at the beginning of startup. +- **Fewer file downloads per query (upgraded storage engine)**: A query now downloads only the run set files whose key range covers the queried field family. Previously, it also downloaded files that held only the table's other field families. +- **Faster merging of overlapping files (upgraded storage engine)**: When a query merges rows from several files, each output column is now built in one vectorized step instead of row by row. + +#### Bug fixes + +- **Storage engine upgrade completed with unconverted Parquet files**: If a Parquet file fails every conversion attempt, the ingest node now reports `failed` in `system.upgrade_parquet_node`. The `last_message` column identifies the file and its error. The cluster upgrade also fails. To resume the upgrade, follow the [retry procedure](/influxdb3/enterprise/reference/cli/influxdb3/manage/retry-upgrade-to-pacha-tree/#retry-after-an-ingest-conversion-failure). Previously, the node dropped the file and reported its upgrade as complete. +- **Storage engine upgrade stalled after an ingest node was removed**: The upgrade now drops a removed ingest node from its roster once its Parquet files are converted or absorbed. Previously, a removed ingest node held the cluster at `upgrading` forever. +- **Overwrites lost after a query node restart (upgraded storage engine)**: After a restart, a query node now reads gen0 files in the order they were written, so the latest write to a series and timestamp wins. Previously, a query could return the value from before an overwrite until compaction. +- **Query node could miss writes from a stopping ingest node (upgraded storage engine)**: A query node now keeps following an ingest node that is [`stopping`](/influxdb3/enterprise/admin/node-lifecycle/) and fetches its remaining WAL files after it acknowledges the stop. Previously, the query node could stop fetching before the node flushed its last writes. +- **Trailing WAL fetch could hang after an ingest node stopped (upgraded storage engine)**: A query node's fetch of a stopped ingest node's remaining WAL files now completes. Previously, it could wait forever on files its workers had already fetched. +- **Nodes stopped applying compaction updates after falling behind (Parquet engine)**: A node that refreshes its view of compacted data no longer reads the detail of each generation the compaction removed. Previously, a node more than [`--compaction-cleanup-wait`](/influxdb3/enterprise/reference/config-options/#compaction-cleanup-wait) behind the compactor failed every refresh with `generation detail not found` until it restarted. +- **Retention stopped after Gen1 file cleanup deleted a snapshot (Parquet engine)**: The table index cache now skips a snapshot that [Gen1 file cleanup](/influxdb3/enterprise/admin/gen1-file-cleanup/) already deleted. Previously, the missing snapshot failed the cache initialization, and retention didn't run until the next restart. +- **Restarted node listed as stopped by its previous process**: A process that is shutting down no longer marks its node stopped once another process has registered under the same `--node-id`. Previously, a slow shutdown could mark the running replacement as stopped, and on the upgraded storage engine query nodes stopped following its WAL. +- **Schema changes lost when upgrading a Core node to Enterprise**: [Upgrading from Core](/influxdb3/enterprise/admin/upgrade-from-core/) now promotes the node's live catalog. Previously, if the Core node had first run a release before 3.10, the upgrade promoted a stale catalog checkpoint, so schema changes made since then were missing. +- Other bug fixes and performance improvements + +## v3.11.8 {date="2026-10-29"} + + + +### Core + +#### Features + +- **Processing engine Python updated to 3.13.18**: Picks up the security fixes released since 3.13.15. + +#### Bug fixes + +- **Queries that exceed memory limits returned `500`**: The HTTP query endpoints now map DataFusion memory-pool exhaustion to `429 Too Many Requests` instead of `500`. If the error occurs during query execution, `pretty` and `parquet` output return `429`. For CSV, JSON, and JSON Lines output, execution errors still end the HTTP `200` response early, even before the first rows. Some Core queries still return `500`, such as a query on a table that runs out of memory while deduplicating data. Enterprise already mapped resource-exhaustion errors to `429`. +- **Catalog deadlock when checking a deleted table**: Checking whether a table is deleted now takes the catalog lock once. Previously, a catalog change that arrived between its two reads could deadlock the catalog. +- **WAL plugins received values under the wrong column names**: Rows passed to a WAL trigger's plugin now use the column names of their own record batch. Previously, when a flush held batches with the same columns in a different order, later batches used the first batch's column names. +- Other bug fixes and performance improvements + +### Enterprise + +All Core updates are included in Enterprise. +Additional Enterprise-specific updates: + +#### Features + +- **Faster [file index](/influxdb3/enterprise/admin/file-index/) loading (Parquet engine)**: Loading a table's file index, including during a storage engine upgrade, now sorts each posting list once. Previously, every generation re-sorted each list it touched, so a table with many generations loaded slowly and used more memory. +- **Faster reads of compaction metadata (Parquet engine)**: Compaction and snapshot metadata files are now parsed in one pass, including during a storage engine upgrade. Previously, each was parsed twice. +- **Table index cache starts once the server is ready (Parquet engine)**: On ingest nodes, the table index cache now starts its background work when the server reports ready, instead of at the beginning of startup. + +#### Bug fixes + +- **Storage engine upgrade completed with unconverted Parquet files**: If a Parquet file fails every conversion attempt, the ingest node now reports `failed` in `system.upgrade_parquet_node`. The `last_message` column identifies the file and its error. The cluster upgrade also fails. To resume the upgrade, follow the [retry procedure](/influxdb3/enterprise/reference/cli/influxdb3/manage/retry-upgrade-to-pacha-tree/#retry-after-an-ingest-conversion-failure). Previously, the node dropped the file and reported its upgrade as complete. +- **Storage engine upgrade stalled after an ingest node was removed**: The upgrade now drops a removed ingest node from its roster once its Parquet files are converted or absorbed. Previously, a removed ingest node held the cluster at `upgrading` forever. +- **Overwrites lost after a query node restart (upgraded storage engine)**: After a restart, a query node now reads gen0 files in the order they were written, so the latest write to a series and timestamp wins. Previously, a query could return the value from before an overwrite until compaction. +- **Nodes stopped applying compaction updates after falling behind (Parquet engine)**: A node that refreshes its view of compacted data no longer reads the detail of each generation the compaction removed. Previously, a node more than [`--compaction-cleanup-wait`](/influxdb3/enterprise/reference/config-options/#compaction-cleanup-wait) behind the compactor failed every refresh with `generation detail not found` until it restarted. +- **Retention stopped after Gen1 file cleanup deleted a snapshot (Parquet engine)**: The table index cache now skips a snapshot that [Gen1 file cleanup](/influxdb3/enterprise/admin/gen1-file-cleanup/) already deleted. Previously, the missing snapshot failed the cache initialization, and retention didn't run until the next restart. +- **Restarted node listed as stopped by its previous process**: A process that is shutting down no longer marks its node stopped once another process has registered under the same `--node-id`. Previously, a slow shutdown could mark the running replacement as stopped, and on the upgraded storage engine query nodes stopped following its WAL. +- Other bug fixes and performance improvements + ## v3.12.1 {date="2026-10-14"} ### Core diff --git a/data/products.yml b/data/products.yml index 1998f9f885..0b34e7cef7 100644 --- a/data/products.yml +++ b/data/products.yml @@ -13,7 +13,7 @@ influxdb3_core: list_order: 2 search_includes_resources: false latest: core - latest_patch: 3.12.1 + latest_patch: 3.12.2 placeholder_host: localhost:8181 scheme: http limits: @@ -60,7 +60,7 @@ influxdb3_enterprise: list_order: 2 search_includes_resources: false latest: enterprise - latest_patch: 3.12.1 + latest_patch: 3.12.2 placeholder_host: localhost:8181 scheme: http limits: