Skip to content

Document multi-stage IdSet support #1088

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
xiangfu0 merged 1 commit into from
Oct 7, 2026
Merged

Document multi-stage IdSet support #1088

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion build-with-pinot/querying-and-sql/filtering-with-idset.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ description: Learn how to write fast queries for looking up IDs in a list of val


{% hint style="info" %}
Filtering with IdSet is **only supported with the single-stage query engine (v1)**.
`IDSET` and `IN_ID_SET` work in both the single-stage and multi-stage query engines. In the multi-stage engine, `IN_ID_SET` can be used in leaf filters, post-join filters, join conditions, and `HAVING`. The string-subquery forms `IN_SUBQUERY` and `IN_PARTITIONED_SUBQUERY` remain single-stage features; multi-stage single-query subquery forms are not yet supported.
{% endhint %}

A common use case is filtering on an id field with a list of values. This can be done with the IN clause, but using IN doesn't perform well with large lists of IDs. For large lists of IDs, we recommend using an IdSet.
Expand Down Expand Up @@ -37,6 +37,8 @@ Note that when a Bloom Filter is used, the filter results are approximate - you

This function returns 1 if a column contains a value specified in the IdSet and 0 if it does not.

The lookup value must have the same stored type as the values used to build the IdSet. With null handling enabled, a NULL lookup value yields NULL for a non-empty set, but FALSE for an empty or NULL set. Thus `IN_ID_SET(nullableCol, <empty set>) = 0` also keeps NULL rows.

### IN\_SUBQUERY

> IN\_SUBQUERY(columnName, subQuery)
Expand Down
Loading