Skip to content

ES|QL: fix transport crash for conflicted sub-fields - #156326

Draft
alighahramani-alig wants to merge 2 commits into
elastic:mainfrom
alighahramani-alig:esql-conflicted-subfield-transport-152322
Draft

ES|QL: fix transport crash for conflicted sub-fields#156326
alighahramani-alig wants to merge 2 commits into
elastic:mainfrom
alighahramani-alig:esql-conflicted-subfield-transport-152322

Conversation

@alighahramani-alig

@alighahramani-alig alighahramani-alig commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

ES|QL can crash when a query plan is shipped to a data node: a cross-index type
conflict on a multi-field sub-field is coordinator-only metadata, but it survives
nested inside its healthy parent field's properties map and throws on
serialization (InvalidMappedField must never leave the coordinator /
CompactInvalidMappedField shouldn't be transported). This makes
UnionTypesCleanup recurse into a field's properties and replace such nested
conflict fields with a transportable UnsupportedEsField, so the parent
FieldAttribute serializes cleanly. Surfaced by SDH #10177 (a customer on 9.4.3
hit this via a TS query); relates to #152322.

Why only TS, why it's safe, why no version guard

Why only TS? The conflict is on a sub-field, so it only bites once its
parent is shipped to a data node. FROM ships only the fields you name; TS
additionally force-resolves every time-series dimension, so it pulls in the parent
— and its conflicted sub-field — even when the query never names it. (A top-level
conflict never gets here: it is turned into an unsupported field before planning.)

Why is it safe? The conflict child was already unusable, so swapping it for an
UnsupportedEsField keeps the same meaning; explicit x.sub::type casts are
resolved earlier and untouched; and any query that shipped the parent used to
crash, so there is no working behavior to lose. EsField#withProperties keeps the
parent's concrete type (keyword/text/date) so exact-match and sort still work.

Why no version guard? Cleanup runs on the coordinator and emits
UnsupportedEsField, a wire type every node already understands — nothing new
crosses the wire. A new coordinator → old data node is fine; an old coordinator
still throws on its own write (the pre-existing bug, unchanged).

Tested by a 2-node internalClusterTest reproducer plus analyzer tests. Relates to
#152350 — this also handles InvalidMappedTsField sub-fields and preserves the
parent subtype.

@alighahramani-alig alighahramani-alig self-assigned this Aug 10, 2026
@alighahramani-alig alighahramani-alig added >bug :Analytics/ES|QL AKA ESQL backport-needed Indicate whether a gh issue needs to backport to any active release. labels Aug 10, 2026
@elasticsearchmachine

Copy link
Copy Markdown
Collaborator

Hi @alighahramani-alig, I've created a changelog YAML for you.

@github-actions

github-actions Bot commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

🔍 Preview links for changed docs

⏳ Building and deploying preview... View progress

This comment will be updated with preview links when the build is complete.

@github-actions

Copy link
Copy Markdown
Contributor

ℹ️ Important: Docs version tagging

👋 Thanks for updating the docs! Just a friendly reminder that our docs are now cumulative. This means all 9.x versions are documented on the same page and published off of the main branch, instead of creating separate pages for each minor version.

We use applies_to tags to mark version-specific features and changes.

Expand for a quick overview

When to use applies_to tags:

✅ At the page level to indicate which products/deployments the content applies to (mandatory)
✅ When features change state (e.g. preview, ga) in a specific version
✅ When availability differs across deployments and environments

What NOT to do:

❌ Don't remove or replace information that applies to an older version
❌ Don't add new information that applies to a specific version without an applies_to tag
❌ Don't forget that applies_to tags can be used at the page, section, and inline level

🤔 Need help?

@alighahramani-alig
alighahramani-alig force-pushed the esql-conflicted-subfield-transport-152322 branch from 9687688 to 3865e19 Compare August 10, 2026 19:30
A cross-index type conflict on a multi-field sub-field is stored as
coordinator-only metadata inside its parent field's properties map.
When the healthy parent is serialized to a data node, the child's
writeContent throws ("InvalidMappedField must never leave the
coordinator" / "CompactInvalidMappedField shouldn't be transported").

UnionTypesCleanup only sanitized top-level conflicted attributes. Make
it recurse into properties, replacing nested TypeConflictedField and
InvalidMappedTsField children with a transportable UnsupportedEsField.
Add a subtype-preserving EsField#withProperties so the rebuilt parent
keeps its concrete type.

Relates to elastic#152322.
@alighahramani-alig
alighahramani-alig force-pushed the esql-conflicted-subfield-transport-152322 branch from 3865e19 to d9ccd38 Compare August 10, 2026 20:02
@elasticsearchmachine

Copy link
Copy Markdown
Collaborator

Hi @alighahramani-alig, I've created a changelog YAML for you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

:Analytics/ES|QL AKA ESQL backport-needed Indicate whether a gh issue needs to backport to any active release. >bug v9.6.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants