[MSE] Add opt-in adaptive sorting for global ordered windows

[xiangfu0]

Build opt-in adaptive sorting for global ordered windows on #19711 using the logical sorted-input contract. AUTO chooses on the finalized logical tree before stage allocation: warm evidence produces an unbounded sender Sort and an explicit k-way merge exchange; cold, negative or incomplete evidence produces a plain exchange followed by an unbounded receiver Sort. The cold path now retains complete window input beyond the broker response limit, including following frames and RANGE peers.

Persistent decisions use query identity plus deterministic logical exchange paths and input/collation fingerprints; per-query bindings map those identities to allocated receiver stages only for observations. The existing four-sender/400k-row heuristic, bounded cache, TTL, probe ownership and cooldown remain. Explicit true/false never train AUTO, and missing, unknown, mixed, unreadable or SNAPSHOT versions and multi-cluster queries fail closed. Cold profiling retains compatible protobuf field 8 and bounded per-stream sampling; warm plans use the distinct merge-receive node with plain mailbox transport.

Try SET useMultistageEngine=true; SET windowSortOnSender='auto'; SELECT SUM(v) OVER (ORDER BY ts) FROM myTable;.

Validation: 718 unique planner, runtime and broker cases pass on the canonical source represented by restacked head b43d2c8b43 with warnings/deprecation enabled on JDK 25; scoped formatting/license/checkstyle checks pass, with zero compiler warnings on added lines. The shared test cleanup freshly reran 123 cases; remaining unchanged-source evidence is reused after exact tree verification, and all seven module checks ran on the final restacked head. A reproducible local 4×100k-row operator comparison checks complete row parity across random, sorted, reverse and tie-heavy inputs, including sender-sort cost; concurrent host load and broad confidence intervals limit timing inference. The controlled local whole-query benchmark below is complete; full hosted CI for the final remote head and reviewer approval remain merge gates. This source port does not close #19395's unbounded read-ahead issue.

Final dependency head is 755721f7fd1548f3e94f81c86f4f64cfae4c66c8; existing stack base names remain unchanged. The release-version capability gate is a cached point-in-time check and does not atomically prevent a concurrent downgrade.

The inherited fixture followup independently passes all 593 ResourceBasedQueryPlansTest cases on this exact source tree with compiler warnings/deprecations enabled, zero failures/errors/skips and all four planner formatting/license/style goals passing. Its native feature patch is byte-identical to the preceding published stack; only core owns the 124 corrected window-input fetch expectations. These local results do not establish hosted CI status.

The existing stack base is retained. Full unit/integration PR workflows are filtered to master; no complete hosted CI result exists for this child at its current stack base. After the prerequisite PRs land, the agreed base update and exact-head CI are still required. No merge, automatic merge, CI suppression or master integration was performed.

Controlled whole-query AUTO validation

Three fresh JVMs executed 576 queries and checked all 230,400,000 returned rows against exact LONG-schema/payload oracles on this head. Four real query servers each read one immutable 100k-row segment. Real QueryEnvironment planning, QueryDispatcher/QueryRunner execution and mailbox transport exercised AUTO cold profiling, warm sender selection, periodic receiver reprobes, following frames beyond the response limit and RANGE peers spanning all four senders. Each of 16 tie keys had 25,000 global peers. All three runs exited normally; an independent review checked every plan/statistics tree, recomputed distributions and verified cleanup.

Random warm sender-selected AUTO medians were 144.91 ms versus OFF 269.56 ms for following, and 175.21 ms versus 321.47 ms for RANGE. These summarize only the 27 warm sender-selected samples per frame, excluding AUTO cold/probe requests. Reverse/tie receiver AUTO medians were about 12–16% higher than OFF in this experiment. Fixed workload order, shared CPU/heap, JIT/cache/GC and host load limit inference; these data establish neither production speedups nor absence of regressions.

The fixture uses public supported-capability test configuration and in-memory routing registration on one host, with three loopback gRPC mailbox senders and one in-memory sender. It excludes broker HTTP/auth, Helix/ZK/controller, live version discovery, mixed-version/concurrent-upgrade qualification, failure coverage and arbitrary BIG_DECIMAL/null workloads. The production SNAPSHOT fail-closed gate and all source files remain unchanged. All owned JVMs, 27 endpoints and generated temporary segment directories were cleaned up. Production qualification and exact-head hosted stack CI remain separate gates.

[codecov-commenter]

Codecov Report

❌ Patch coverage is 77.62431% with 81 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (xiangfu0/codex1/sorted-exchange-ordering@755721f). Learn more about missing BASE report.

Files with missing lines	Patch %	Lines
...not/broker/requesthandler/WindowSortAutoTuner.java	75.00%	16 Missing and 26 partials ⚠️
...requesthandler/MultiStageBrokerRequestHandler.java	27.77%	13 Missing ⚠️
...query/runtime/operator/MailboxReceiveOperator.java	82.97%	3 Missing and 5 partials ⚠️
...e/rel/rules/PinotWindowExchangeNodeInsertRule.java	75.00%	1 Missing and 3 partials ⚠️
.../java/org/apache/pinot/query/QueryEnvironment.java	88.00%	2 Missing and 1 partial ⚠️
...he/pinot/query/planner/explain/PlanNodeMerger.java	0.00%	2 Missing ⚠️
...he/pinot/query/planner/logical/PlanFragmenter.java	81.81%	0 Missing and 2 partials ⚠️
...t/query/planner/logical/WindowSortAutoPlanner.java	94.11%	0 Missing and 2 partials ⚠️
.../apache/pinot/core/util/DataBlockExtractUtils.java	66.66%	0 Missing and 1 partial ⚠️
.../calcite/rel/logical/PinotLogicalSortExchange.java	85.71%	1 Missing ⚠️
... and 3 more

Additional details and impacted files

@@                             Coverage Diff                             @@
##             xiangfu0/codex1/sorted-exchange-ordering   #19714   +/-   ##
===========================================================================
  Coverage                                            ?   68.26%           
  Complexity                                          ?     1450           
===========================================================================
  Files                                               ?     3529           
  Lines                                               ?   229576           
  Branches                                            ?    36479           
===========================================================================
  Hits                                                ?   156723           
  Misses                                              ?    60504           
  Partials                                            ?    12349           

Flag	Coverage Δ
integration	100.00% <ø> (?)
integration1	100.00% <ø> (?)
integration2	0.00% <ø> (?)
java-25	68.26% <77.62%> (?)
lane-a	100.00% <ø> (?)
lane-b	0.00% <ø> (?)
temurin	68.26% <77.62%> (?)
unittests	68.26% <77.62%> (?)
unittests1	58.22% <85.22%> (?)
unittests2	39.88% <47.79%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[xiangfu0]

CI follow-up at f8a15779 adds six exact stacked-base pull_request branch filters (+6/-0). Feature code/tests and existing master/push, path, permission and job rules are preserved.

All 14 current-head checks passed: unit/integration, compatibility, quickstart, Java 11 client, linter and Trivy (checks). Maintainer review remains pending.