Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove unused AggregateOptions struct and scalar_update_factor config setting #12241

Merged
merged 4 commits into from
Sep 2, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 0 additions & 11 deletions datafusion/common/src/config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -500,17 +500,6 @@ config_namespace! {
///
/// [`SessionConfig`]: https://docs.rs/datafusion/latest/datafusion/prelude/struct.SessionConfig.html
pub struct AggregateOptions {
/// Specifies the threshold for using `ScalarValue`s to update
/// accumulators during high-cardinality aggregations for each input batch.
///
/// The aggregation is considered high-cardinality if the number of affected groups
/// is greater than or equal to `batch_size / scalar_update_factor`. In such cases,
/// `ScalarValue`s are utilized for updating accumulators, rather than the default
/// batch-slice approach. This can lead to performance improvements.
///
/// By adjusting the `scalar_update_factor`, you can balance the trade-off between
/// more efficient accumulator updates and the number of groups affected.
pub scalar_update_factor: usize, default = 10
}
}

Expand Down
13 changes: 0 additions & 13 deletions datafusion/execution/src/config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -383,19 +383,6 @@ impl SessionConfig {
self.options.execution.batch_size
}

/// Get the currently configured scalar_update_factor for aggregate
pub fn agg_scalar_update_factor(&self) -> usize {
self.options.execution.aggregate.scalar_update_factor
}

/// Customize scalar_update_factor for aggregate
pub fn with_agg_scalar_update_factor(mut self, n: usize) -> Self {
// scalar update factor must be greater than zero
assert!(n > 0);
self.options.execution.aggregate.scalar_update_factor = n;
self
}

/// Enables or disables the coalescence of small batches into larger batches
pub fn with_coalesce_batches(mut self, enabled: bool) -> Self {
self.options.execution.coalesce_batches = enabled;
Expand Down
Loading