Skip to content

fix order by optimization#2251

Merged
jycor merged 5 commits intomainfrom
james/order
Jan 11, 2024
Merged

fix order by optimization#2251
jycor merged 5 commits intomainfrom
james/order

Conversation

@jycor
Copy link
Copy Markdown
Contributor

@jycor jycor commented Jan 10, 2024

This hopefully fixes 24 sqllogictests.

We introduced an optimization where we would drop Sort nodes for queries that included ORDER BYs if there were IndexLookups that were a matching prefix over the sortfields. The idea was that since indexes are already in order there was no need to sort.

However, there is a case when that isn't necessarily true. If the index is created from a filter over multiple columns, specifically when that filter contains an OR expression, it is possible to iterate over columns in a non-sorted order. During analysis, the filters are converted into several non-overlapping ranges; it is possible for a range expression in the range to overlap with another range's range expression, but for the two ranges to not overlap. It's clearer to look at the test.

We didn't catch this bug earlier, as it only affects queries in dolt. We iterate over rows in the in-memory tables in such a way that the rows will still appear in order.

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants