Skip to content

Implement ORDER BY spilling #14836

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
highker merged 10 commits into from
Jul 24, 2020
Merged

Implement ORDER BY spilling #14836

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
f2e6a83
Revoke memory after initial output page has been produced in tests
Jul 11, 2020
80890da
Allow memory revoke only during operator finish phase
Jul 13, 2020
df4ae85
Produce more than single page in testHashAggregation
Jul 13, 2020
9aa39fb
Extract DummySpillerFactory from TestHashAggregationOperator
Oct 19, 2018
3af951b
Use WorkProcessor in OrderByOperator
Jul 9, 2020
e24359d
Add Spill To Disk for ORDER BY
Jul 10, 2020
52adeba
Use OrderingCompiler in OrderBy spilling
Jul 10, 2020
28962d5
Convert revocable memory to user memory on OrderBy finish
Jul 13, 2020
1dc65c6
Extract order by queries tests to separate class
Jul 13, 2020
13db5e5
Close spiller on Operator#close for ORDER BY spilling
Jul 22, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion presto-benchmark/src/main/java/com/facebook/presto/benchmark/OrderByBenchmark.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,10 +19,12 @@
import com.facebook.presto.operator.OrderByOperator.OrderByOperatorFactory;
import com.facebook.presto.operator.PagesIndex;
import com.facebook.presto.spi.plan.PlanNodeId;
import com.facebook.presto.sql.gen.OrderingCompiler;
import com.facebook.presto.testing.LocalQueryRunner;
import com.google.common.collect.ImmutableList;

import java.util.List;
import java.util.Optional;

import static com.facebook.presto.benchmark.BenchmarkQueryRunner.createLocalQueryRunner;
import static com.facebook.presto.common.block.SortOrder.ASC_NULLS_LAST;
Expand Down Expand Up @@ -53,7 +55,10 @@ protected List<? extends OperatorFactory> createOperatorFactories()
ROWS,
ImmutableList.of(0),
ImmutableList.of(ASC_NULLS_LAST),
new PagesIndex.TestingFactory(false));
new PagesIndex.TestingFactory(false),
false,
Optional.empty(),
new OrderingCompiler());

return ImmutableList.of(tableScanOperator, limitOperator, orderByOperator);
}
Expand Down
28 changes: 28 additions & 0 deletions presto-hive/src/test/java/com/facebook/presto/hive/TestHiveDistributedOrderByQueries.java
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
/*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.facebook.presto.hive;

import com.facebook.presto.tests.AbstractTestOrderByQueries;

import static com.facebook.presto.hive.HiveQueryRunner.createQueryRunner;
import static io.airlift.tpch.TpchTable.getTables;

public class TestHiveDistributedOrderByQueries
extends AbstractTestOrderByQueries
{
public TestHiveDistributedOrderByQueries()
{
super(() -> createQueryRunner(getTables()));
}
}
Loading