[native] Add session property for partial aggregation and spill#23527
[native] Add session property for partial aggregation and spill#23527kewang1024 merged 1 commit intoprestodb:masterfrom
Conversation
|
Should this be documented on https://prestodb.io/docs/current/presto_cpp/properties.html ? |
xiaoxmeng
left a comment
There was a problem hiding this comment.
@kewang1024 LGTM. Thanks!
xiaoxmeng
left a comment
There was a problem hiding this comment.
@kewang1024 do we still have the sanity check for the native session conversion in Prestissimo? Thanks!
| public static final String NATIVE_EXECUTION_PROCESS_REUSE_ENABLED = "native_execution_process_reuse_enabled"; | ||
| public static final String NATIVE_DEBUG_VALIDATE_OUTPUT_FROM_OPERATORS = "native_debug_validate_output_from_operators"; | ||
|
|
||
| public static final String NATIVE_MAX_PARTIAL_AGGREGATION_MEMORY = "native_max_partial_aggregation_memory"; |
There was a problem hiding this comment.
Do you want to expose max_extended_partial_aggregation_memory as well? Velox supports to dynamically adjust aggregation memory limit in a range defined by the above two configs? Thanks!
aec9036 to
6151448
Compare
Actually this is session property which is different than the configs, so should not be added here. |
9bf6160 to
5839bcb
Compare
There was a problem hiding this comment.
Thanks @kewang1024 for this code.
Also as per @steveburnett's comment we should document atleast native_max_spill_bytes at https://prestodb.io/docs/current/presto_cpp/properties.html
| /// The maximum allowed spill file size. | ||
| static constexpr const char* kMaxSpillFileSize = "native_max_spill_file_size"; | ||
|
|
||
| static constexpr const char* kMaxSpillBytes = "native_max_spill_bytes"; |
There was a problem hiding this comment.
It might be good to specify all the operators (Hashjoin, HashAgg, OrderBy, RowNumber, Window etc) to which this value applies
There was a problem hiding this comment.
I don't have much context on spilling so might not have the full list, @xiaoxmeng do you know the full list?
I can add in a separate PR. We need this change to make an internal release cut.
There was a problem hiding this comment.
@kewang1024 @aditi-pandit: this applies at task level and is not specific to a particular operator.
yingsu00
left a comment
There was a problem hiding this comment.
@kewang1024 This PR needs the release note. Will you be able to add one? Thanks.
5839bcb to
9fdc14d
Compare
9fdc14d to
05981a2
Compare
Like the Presto Properties Reference page, the Presto C++ Properties Reference page begins with the statement "The following pages are not a complete list of all configuration and session properties ". I know the sentence before that reads "This section describes Presto C++ configuration properties." which is exclusionary, but that could be changed to "This section describes Presto C++ configuration and session properties." and we don't have a separate session properties page in the Presto doc. (Maybe we should.) I don't think we need a separate page for Presto C++ session properties, so I think all three of these session properties could be added to Presto C++ Properties Reference. We could highlight the distinction for session properties in the description. For example, in your current draft adding If you or others feel strongly we need a separate page for Presto C++ Session Properties, I am open to the idea. Help me understand why a separate page is important. |
Uh oh!
There was an error while loading. Please reload this page.