[SPARK-24020][SQL] Sort-merge join inner range optimization#21109

Closed

zecevicp wants to merge 52 commits intoapache:masterfrom

astronomy-commons:branch-pz-smj

Commits on Sep 7, 2018

Secondary sort Sort Merge Join optimization. Not finished yet.

Petar Zecevic
authored and
Petar Zecevic
committed
SortMergeJoin secondary sort optimization

Petar Zecevic
authored and
Petar Zecevic
committed
Sort-Merge "inner range join" (secondary sort) - code generation

Petar Zecevic
authored and
Petar Zecevic
committed
Sort-Merge "inner range join" (secondary sort) - two bug fixes - works now

Petar Zecevic
authored and
Petar Zecevic
committed
Code simplification

Petar Zecevic
authored and
Petar Zecevic
committed
Bug fix

Petar Zecevic
authored and
Petar Zecevic
committed
Scalastyle fixes

Petar Zecevic
authored and
Petar Zecevic
committed
Scalastyle fixes

Petar Zecevic
authored and
Petar Zecevic
committed
SMJ range join unit tests

Petar Zecevic
authored and
Petar Zecevic
committed
Scalastyle

Petar Zecevic
authored and
Petar Zecevic
committed
Scalastyle

Petar Zecevic
authored and
Petar Zecevic
committed
Scalastyle

Petar Zecevic
authored and
Petar Zecevic
committed
Fix generated code - dequeue method missing

Petar Zecevic
authored and
Petar Zecevic
committed
Bug fix: include other binary comparisons in range conditions match

Petar Zecevic
authored and
Petar Zecevic
committed
Test fix: sortWithinPartitions; Bug Fix: check references in rangeConditions, not columns

Petar Zecevic
authored and
Petar Zecevic
committed
Test fix

Petar Zecevic
authored and
Petar Zecevic
committed
Test fix

Petar Zecevic
authored and
Petar Zecevic
committed
Fix required child ordering for inner range queries

Petar Zecevic
authored and
Petar Zecevic
committed
Parameter for turning off inner range optimization

Petar Zecevic
authored and
Petar Zecevic
committed
Scala style

Petar Zecevic
authored and
Petar Zecevic
committed
Bug fix - NPE when inner range optimization turned off

Petar Zecevic
authored and
Petar Zecevic
committed
Adding test case when inner range optimization is turned off

Petar Zecevic
authored and
Petar Zecevic
committed
Stala style

Petar Zecevic
authored and
Petar Zecevic
committed
Stala style

Petar Zecevic
authored and
Petar Zecevic
committed
Remove range condition extraction when inner range join optimization is disabled

Petar Zecevic
authored and
Petar Zecevic
committed
Scala style

Petar Zecevic
authored and
Petar Zecevic
committed
Unit test fix

Petar Zecevic
authored and
Petar Zecevic
committed
Unit test fix

Petar Zecevic
authored and
Petar Zecevic
committed
- Turning off inner range optimization when whole stage code generation is turned off.

Petar Zecevic
authored and
Petar Zecevic
committed
Switch off inner range optimization when whole stage codegen is off.

Petar Zecevic
authored and
Petar Zecevic
committed
SMJ inner range optimization benchmarks

Petar Zecevic
authored and
Petar Zecevic
committed
Removing "expensive function" from the SMJ inner range optimization benchmark.

Petar Zecevic
authored and
Petar Zecevic
committed
SMJ inner range optimization with wholeStage codegen turned off - code + tests

Petar Zecevic
authored and
Petar Zecevic
committed
Unit test fix. Benchmark results update.

Petar Zecevic
authored and
Petar Zecevic
committed
Scalastyle for comments

Petar Zecevic
authored and
Petar Zecevic
committed
Code changes based on review comments.

Petar Zecevic
authored and
Petar Zecevic
committed
Code review changes

Petar Zecevic
authored and
Petar Zecevic
committed
Removing exception when numRowsInMemoryBufferThreshold is reached in InMemoryUnsafeRowQueue class.

Petar Zecevic
authored and
Petar Zecevic
committed
Scala style

Petar Zecevic
authored and
Petar Zecevic
committed
Unneeded import

Petar Zecevic
authored and
Petar Zecevic
committed
A dot

Petar Zecevic
authored and
Petar Zecevic
committed
A dot

Petar Zecevic
authored and
Petar Zecevic
committed
A dot

Petar Zecevic
authored and
Petar Zecevic
committed
A dot

Petar Zecevic
authored and
Petar Zecevic
committed
A dot

Petar Zecevic
authored and
Petar Zecevic
committed
Fixes for some rebase issues.

ctslater
authored and
Petar Zecevic
committed
Merge with upstream

zecevicp
authored and
Petar Zecevic
committed
Merge with upstream

zecevicp
authored and
Petar Zecevic
committed
SMJ inner range spill over implementation and tests

zecevicp
authored and
Petar Zecevic
committed
External unsafe row dequeue test extension

zecevicp
authored and
Petar Zecevic
committed
A dot

zecevicp
authored and
Petar Zecevic
committed

Commits on Dec 10, 2018

Merge branch 'master' into branch-pz-smj
zecevicp
authored

[SPARK-24020][SQL] Sort-merge join inner range optimization#21109zecevicp wants to merge 52 commits intoapache:masterapache/spark:masterfrom astronomy-commons:branch-pz-smjastronomy-commons/axs-spark:branch-pz-smjCopy head branch name to clipboard

Commits

Commits on Sep 7, 2018

Commits on Dec 10, 2018

[SPARK-24020][SQL] Sort-merge join inner range optimization#21109
zecevicp wants to merge 52 commits intoapache:masterfrom
astronomy-commons:branch-pz-smj