Dont use row block field block offsets if not needed by radek-kondziolka · Pull Request #10825 · trinodb/trino

radek-kondziolka · 2022-01-27T11:35:55Z

Add support for RowType to BenchmarkDataGenerator and benchmarks
Do not use/serialize/deserialize fieldBlockOffset when not needed
- When there are no null rows within RowBlock then it's not needed to store fieldBlockOffsets.
  This improves performance and reduces network traffic as fieldBlockOffsets array is not serialized or deserialized.

Benchmark  (before/after)                      (nullChance)  Mode  Cnt  Score   Error  Units
(before)   BenchmarkBlockSerde.deserializeRow  0             avgt  60   17.348  ±      0.109  ns/op
(after)    BenchmarkBlockSerde.deserializeRow  0             avgt  60   15.205  ±      0.084  ns/op
(before)   BenchmarkBlockSerde.deserializeRow  .01           avgt  60   18.020  ±      0.104  ns/op
(after)    BenchmarkBlockSerde.deserializeRow  .01           avgt  60   18.279  ±      0.113  ns/op
(before)   BenchmarkBlockSerde.deserializeRow  .10           avgt  60   16.859  ±      0.140  ns/op
(after)    BenchmarkBlockSerde.deserializeRow  .10           avgt  60   16.494  ±      0.157  ns/op
(before)   BenchmarkBlockSerde.deserializeRow  .50           avgt  60   10.273  ±      0.044  ns/op
(after)    BenchmarkBlockSerde.deserializeRow  .50           avgt  60   10.491  ±      0.063  ns/op
(before)   BenchmarkBlockSerde.deserializeRow  .90           avgt  60   4.351   ±      0.020  ns/op
(after)    BenchmarkBlockSerde.deserializeRow  .90           avgt  60   4.311   ±      0.013  ns/op
(before)   BenchmarkBlockSerde.deserializeRow  .99           avgt  60   2.919   ±      0.027  ns/op
(after)    BenchmarkBlockSerde.deserializeRow  .99           avgt  60   2.851   ±      0.008  ns/op
(before)   BenchmarkBlockSerde.serializeRow    0             avgt  60   20.862  ±      0.117  ns/op
(after)    BenchmarkBlockSerde.serializeRow    0             avgt  60   16.941  ±      0.089  ns/op
(before)   BenchmarkBlockSerde.serializeRow    .01           avgt  60   21.588  ±      0.097  ns/op
(after)    BenchmarkBlockSerde.serializeRow    .01           avgt  60   20.045  ±      0.102  ns/op
(before)   BenchmarkBlockSerde.serializeRow    .10           avgt  60   20.667  ±      0.068  ns/op
(after)    BenchmarkBlockSerde.serializeRow    .10           avgt  60   19.448  ±      0.126  ns/op
(before)   BenchmarkBlockSerde.serializeRow    .50           avgt  60   13.217  ±      0.090  ns/op
(after)    BenchmarkBlockSerde.serializeRow    .50           avgt  60   11.839  ±      0.042  ns/op
(before)   BenchmarkBlockSerde.serializeRow    .90           avgt  60   7.404   ±      0.017  ns/op
(after)    BenchmarkBlockSerde.serializeRow    .90           avgt  60   5.615   ±      0.009  ns/op
(before)   BenchmarkBlockSerde.serializeRow    .99           avgt  60   5.517   ±      0.007  ns/op
(after)    BenchmarkBlockSerde.serializeRow    .99           avgt  60   3.534   ±      0.009  ns/op

sopel39 · 2022-01-27T12:10:38Z

Is it really a draft? If you think it's ready make it non-draft?

core/trino-spi/src/main/java/io/trino/spi/block/AbstractRowBlock.java

core/trino-spi/src/main/java/io/trino/spi/block/RowBlock.java

core/trino-spi/src/main/java/io/trino/spi/block/RowBlockEncoding.java

core/trino-spi/src/test/java/io/trino/spi/block/TestRowBlock.java

core/trino-main/src/test/java/io/trino/block/BenchmarkRowBlock.java

sopel39 · 2022-01-31T11:05:57Z

mind automation (maven checks)

core/trino-spi/src/main/java/io/trino/spi/block/AbstractRowBlock.java

core/trino-spi/src/main/java/io/trino/spi/block/RowBlockEncoding.java

sopel39 · 2022-02-09T17:26:42Z

core/trino-spi/src/main/java/io/trino/spi/block/RowBlockEncoding.java

streams are not good in tight loops. You can use io.airlift.slice.Slices#wrappedIntArray(int[], int, int) to point at specific offset without copying data

core/trino-spi/src/test/java/io/trino/spi/block/TestRowBlock.java

core/trino-main/src/test/java/io/trino/execution/buffer/BenchmarkBlockSerde.java

testing/trino-tests/src/test/java/io/trino/execution/TestEventListenerWithSplits.java

core/trino-spi/src/main/java/io/trino/spi/block/RowBlockEncoding.java

sopel39 · 2022-02-11T11:16:48Z

core/trino-spi/src/test/java/io/trino/spi/block/TestFromFieldBlocksCreatorForRowBlock.java

rename it back to testRowBlock

skrzypo987

Added some comments, will look again tomoroow

core/trino-spi/src/main/java/io/trino/spi/block/AbstractRowBlock.java

skrzypo987

LGTM % comments

core/trino-main/src/test/java/io/trino/execution/buffer/BenchmarkBlockSerde.java

core/trino-main/src/test/java/io/trino/execution/buffer/BenchmarkDataGenerator.java

sopel39

lgtm % comments

core/trino-main/src/test/java/io/trino/execution/buffer/BenchmarkBlockSerde.java

core/trino-spi/src/main/java/io/trino/spi/block/RowBlockEncoding.java

core/trino-spi/src/test/java/io/trino/spi/block/TestRowBlock.java

When there are no null rows within RowBlock then it's not needed to store fieldBlockOffsets. This improves performance and reduces network traffic as fieldBlockOffsets array is not serialized or deserialized. Tests TestRowBlock and TestRowBlockEncoding were added. Benchmark result: Benchmark (before/after) (nullChance) Mode Cnt Score Error Units (before) BenchmarkBlockSerde.deserializeRow 0 avgt 60 17.348 ± 0.109 ns/op (after) BenchmarkBlockSerde.deserializeRow 0 avgt 60 15.205 ± 0.084 ns/op (before) BenchmarkBlockSerde.deserializeRow .01 avgt 60 18.020 ± 0.104 ns/op (after) BenchmarkBlockSerde.deserializeRow .01 avgt 60 18.279 ± 0.113 ns/op (before) BenchmarkBlockSerde.deserializeRow .10 avgt 60 16.859 ± 0.140 ns/op (after) BenchmarkBlockSerde.deserializeRow .10 avgt 60 16.494 ± 0.157 ns/op (before) BenchmarkBlockSerde.deserializeRow .50 avgt 60 10.273 ± 0.044 ns/op (after) BenchmarkBlockSerde.deserializeRow .50 avgt 60 10.491 ± 0.063 ns/op (before) BenchmarkBlockSerde.deserializeRow .90 avgt 60 4.351 ± 0.020 ns/op (after) BenchmarkBlockSerde.deserializeRow .90 avgt 60 4.311 ± 0.013 ns/op (before) BenchmarkBlockSerde.deserializeRow .99 avgt 60 2.919 ± 0.027 ns/op (after) BenchmarkBlockSerde.deserializeRow .99 avgt 60 2.851 ± 0.008 ns/op (before) BenchmarkBlockSerde.serializeRow 0 avgt 60 20.862 ± 0.117 ns/op (after) BenchmarkBlockSerde.serializeRow 0 avgt 60 16.941 ± 0.089 ns/op (before) BenchmarkBlockSerde.serializeRow .01 avgt 60 21.588 ± 0.097 ns/op (after) BenchmarkBlockSerde.serializeRow .01 avgt 60 20.045 ± 0.102 ns/op (before) BenchmarkBlockSerde.serializeRow .10 avgt 60 20.667 ± 0.068 ns/op (after) BenchmarkBlockSerde.serializeRow .10 avgt 60 19.448 ± 0.126 ns/op (before) BenchmarkBlockSerde.serializeRow .50 avgt 60 13.217 ± 0.090 ns/op (after) BenchmarkBlockSerde.serializeRow .50 avgt 60 11.839 ± 0.042 ns/op (before) BenchmarkBlockSerde.serializeRow .90 avgt 60 7.404 ± 0.017 ns/op (after) BenchmarkBlockSerde.serializeRow .90 avgt 60 5.615 ± 0.009 ns/op (before) BenchmarkBlockSerde.serializeRow .99 avgt 60 5.517 ± 0.007 ns/op (after) BenchmarkBlockSerde.serializeRow .99 avgt 60 3.534 ± 0.009 ns/op

core/trino-main/src/test/java/io/trino/execution/buffer/BenchmarkDataGenerator.java

sopel39 · 2022-02-23T12:34:43Z

small comment

sopel39 · 2022-02-23T13:40:00Z

Too small, specific change for RN.

cla-bot bot added the cla-signed label Jan 27, 2022

radek-kondziolka force-pushed the rk/dont_use_RowBlock_fieldBlockOffsets branch from c3fff75 to 4b07829 Compare January 27, 2022 11:53

radek-kondziolka requested review from lukasz-stec and sopel39 January 27, 2022 12:07

radek-kondziolka marked this pull request as ready for review January 27, 2022 12:54

sopel39 reviewed Jan 31, 2022

View reviewed changes

radek-kondziolka force-pushed the rk/dont_use_RowBlock_fieldBlockOffsets branch 3 times, most recently from c3a8f3c to 45904d9 Compare February 8, 2022 09:59

radek-kondziolka requested a review from sopel39 February 8, 2022 10:29

radek-kondziolka force-pushed the rk/dont_use_RowBlock_fieldBlockOffsets branch 5 times, most recently from 130c1bd to 27e0015 Compare February 9, 2022 15:03

sopel39 changed the title ~~Rk/dont use row block field block offsets~~ Dont use row block field block offsets if not needed Feb 9, 2022

sopel39 reviewed Feb 9, 2022

View reviewed changes

radek-kondziolka force-pushed the rk/dont_use_RowBlock_fieldBlockOffsets branch 2 times, most recently from ab279c6 to f48db07 Compare February 10, 2022 19:56

radek-kondziolka requested a review from sopel39 February 10, 2022 20:03

radek-kondziolka force-pushed the rk/dont_use_RowBlock_fieldBlockOffsets branch 3 times, most recently from 8d269bb to a163612 Compare February 11, 2022 10:26

sopel39 requested a review from skrzypo987 February 11, 2022 10:51

sopel39 reviewed Feb 11, 2022

View reviewed changes

radek-kondziolka force-pushed the rk/dont_use_RowBlock_fieldBlockOffsets branch 3 times, most recently from c4cd596 to 3fdc374 Compare February 11, 2022 12:22

radek-kondziolka force-pushed the rk/dont_use_RowBlock_fieldBlockOffsets branch 2 times, most recently from 4175e77 to 245c8d5 Compare February 11, 2022 12:24

skrzypo987 reviewed Feb 15, 2022

View reviewed changes

core/trino-spi/src/main/java/io/trino/spi/block/AbstractRowBlock.java Outdated Show resolved Hide resolved

radek-kondziolka force-pushed the rk/dont_use_RowBlock_fieldBlockOffsets branch from 245c8d5 to afd150a Compare February 16, 2022 09:45

radek-kondziolka requested review from skrzypo987 and sopel39 February 16, 2022 10:06

skrzypo987 approved these changes Feb 17, 2022

View reviewed changes

core/trino-main/src/test/java/io/trino/execution/buffer/BenchmarkBlockSerde.java Outdated Show resolved Hide resolved

core/trino-main/src/test/java/io/trino/execution/buffer/BenchmarkDataGenerator.java Outdated Show resolved Hide resolved

radek-kondziolka force-pushed the rk/dont_use_RowBlock_fieldBlockOffsets branch from afd150a to d0ed9a0 Compare February 17, 2022 11:32

sopel39 approved these changes Feb 22, 2022

View reviewed changes

Add support for RowType to BenchmarkDataGenerator and benchmarks

660b887

radek-kondziolka force-pushed the rk/dont_use_RowBlock_fieldBlockOffsets branch from d0ed9a0 to 2a2a3d5 Compare February 22, 2022 16:11

radek-kondziolka requested review from skrzypo987 and sopel39 February 22, 2022 16:11

radek-kondziolka force-pushed the rk/dont_use_RowBlock_fieldBlockOffsets branch from 2a2a3d5 to f1459b2 Compare February 22, 2022 20:38

skrzypo987 approved these changes Feb 23, 2022

View reviewed changes

sopel39 approved these changes Feb 23, 2022

View reviewed changes

core/trino-main/src/test/java/io/trino/execution/buffer/BenchmarkDataGenerator.java Show resolved Hide resolved

sopel39 merged commit bb60e2d into trinodb:master Feb 23, 2022

mosabua mentioned this pull request Feb 23, 2022

Add Trino 372 release notes #11094

Merged

github-actions bot added this to the 372 milestone Feb 23, 2022

Conversation

radek-kondziolka commented Jan 27, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sopel39 commented Jan 27, 2022

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sopel39 commented Jan 31, 2022

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sopel39 Feb 9, 2022

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sopel39 Feb 11, 2022

Choose a reason for hiding this comment

Uh oh!

skrzypo987 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

skrzypo987 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

sopel39 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sopel39 commented Feb 23, 2022

Uh oh!

sopel39 commented Feb 23, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

3 participants

radek-kondziolka commented Jan 27, 2022 •

edited

Loading

sopel39 commented Feb 23, 2022 •

edited

Loading