-
Notifications
You must be signed in to change notification settings - Fork 761
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
refactor(query): Multiple states Aggregate function #17148
Conversation
Docker Image for PR
|
Docker Image for PR
|
Benchmark table CREATE TABLE t1 (
category_id INT NOT NULL,
region_id INT NOT NULL,
user_id BIGINT NOT NULL,
product_id INT NOT NULL,
transaction_date DATETIME NOT NULL,
transaction_amount DECIMAL(10,2) NOT NULL,
discount_amount DECIMAL(10,2) NOT NULL,
is_returned BOOLEAN NOT NULL DEFAULT 0,
inventory_count INT NOT NULL,
last_updated TIMESTAMP NOT NULL
); query SELECT category_id, region_id, user_id, product_id,
COUNT(*) AS transaction_count,
SUM(transaction_amount) AS total_transaction_amount,
SUM(discount_amount) AS total_discount_amount,
MAX(transaction_date) AS latest_transaction_date,
MAX(inventory_count) AS max_inventory,
MAX(transaction_amount) AS max_returned_transaction_amount
FROM t1
GROUP BY category_id, region_id, user_id, product_id
ignore_result; states layout this pr: StatesLayout {
layout: Layout { size: 93, align: 8 },
// (index,offset)
states_loc: [[Custom(0, 0)], [Custom(1, 8), Bool(2, 88)], [Custom(3, 24), Bool(4, 89)], [Custom(5, 40), Bool(6, 90)], [Custom(7, 80), Bool(8, 91)], [Custom(9, 56), Bool(10, 92)]]
} before: layout: Layout { size: 128, align: 8 })
offsets: [0, 8, 32, 56, 80, 96] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Almost LGTM. Please fix the comments and conflicts.
I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/
Summary
AggrStateRegistry
andget_states_layout
to reduce the memory consumption of aggregated states.Payload::append_rows
from one-by-one to array.fixes: #16984
Tests
Type of change
This change is