Conversation
core/trino-spi/src/main/java/io/trino/spi/block/ArrayBlock.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Would it make sense for this to be
public interface ValueBlock<T extends ValueBlock>Then the methods would be
@Override
T copyPositions(...)This would require implementations to override all of the methods with the correct return type.
There was a problem hiding this comment.
Generally, I don't like making these interfaces generic as they aren't used in ways that the generic type would help.... basically all uses would be Value<?> or even worse Value<? extends Value>. I think we can satisfy this by adding a test that verifies no methods on an implementation return ValueBlock (so they are forced to override the return)
core/trino-spi/src/test/java/io/trino/spi/function/TestScalarFunctionAdapter.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
We could add a byte array equals method to Slice
There was a problem hiding this comment.
yep... we could also go the other way and unwarp here. For now, this follows the style of the other operators.
There was a problem hiding this comment.
Do we need this version? Or can the engine invert the above?
There was a problem hiding this comment.
It could, but the conversion code isn't that bright right now.
lib/trino-hive-formats/src/main/java/io/trino/hive/formats/encodings/text/MapEncoding.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Why don't all of these method additions need revapi changes? Maybe it doesn't flag on new methods?
There was a problem hiding this comment.
We only check for backwards incompatible changes
...c/main/java/io/trino/plugin/deltalake/transactionlog/checkpoint/CheckpointEntryIterator.java
Outdated
Show resolved
Hide resolved
The code was only checking of the block could have a null and not if it actually contained a null.
Avro will convert int to long and float to double during read
The aggregation framework already unwinds RLE and dictionary wrappers, so this is mostly type changes.
The generated full loops have specialized paths for dictionary or run length encoded data, and for masked or unmasked data.
The equals method on Field was complex and incomplete. Removing these simplifies the code and removes some bugs.
Merge RleAwarePositionsAppender into UnnestingPositionsAppender and simplify combined code
These implementations delegate to Block, but the block implementation for these types do not support getSlice
|
/test-with-secrets sha=e872b5c29db025e8673099ff64f4f25e6d2b8aee |
|
The CI workflow run with tests that require additional secrets finished as failure: https://github.com/trinodb/trino/actions/runs/6583554942 |
Description
This PR adds the
ValueBlockinterface to the SPI. A value block is a type ofBlockthat contains actual values as opposed to the synthetic wrapper blocksDictionaryBlock,RunLengthEncodedBlock, andLaxyBlock. EachTypenow declares theValueBlockclass used to represent the type. When interacting with data for a type, and implementation can safely cast aValueBlockinstance to theTypesdeclaredValueBlockclass.Additionally, there is a new VALUE_BLOCK_POSITION (and related VALUE_BLOCK_POSITION_NOT_NULL) calling convention that will ensure that synthetic blocks are unwrapped on the caller's side. This call convention is preferred as it empowers the caller to unwrap blocks outside of the core computation loop.
All aggregation functions have been updated to use ValueBlocks with the synthetic blocks unwrapped outside of the core loop. This change does require that each aggregation parameter have a separate position instead of a shared position as the unwrapped blocks can be unaligned.
This is part of #14237
Fixes #13267
Release notes
(X) Release notes are required, with the following suggested text: