Fix retained size after Page deserialization#16248
Fix retained size after Page deserialization#16248viczhang861 wants to merge 1 commit intoprestodb:masterfrom
Conversation
Slice in SerializedPage includes all blocks. When VariableWidthBlock is deserilized using VariableWidthBlockEncoding, VariableWidthBlock should be backed by a new slice with copied data rather than a view of the slice with all blocks.
5cbc33a to
cef0753
Compare
|
I would love to avoid an extra copy for |
|
This pull request has been automatically marked as stale because it has not had recent activity. If you'd still like this PR merged, please comment on the task, make sure you've addressed reviewer comments, and rebase on the latest master. Thank you for your contributions! |
|
Hi @viczhang861 , Do you think you can get this fixed? We would like to get it reviewed and merged. Please let me know if you need any additional help. Thanks! |
we fixed this issue with #17641 |
Slice in SerializedPage includes all blocks. When VariableWidthBlock
is deserialized using VariableWidthBlockEncoding, VariableWidthBlock
should be backed by a new slice with copied data rather than the
slice for all blocks.
Otherwise, every block holds a slice of the entire page, which leads to
wrong calculation of Page retained size. Depends on number of affected
blocks in a page, memory could be off by N times (N >= count of VariableWidthBlock).
As a result, spilled query will have incorrect retained size and fail.
Test plan
Tested in production, previously OOMed spilled query succeeds after this change.
Performance regression test
612 production queries (CPU time >= 1 hour) are run with and without this patch. Total cpu hours is 24680 for test job and 23956 for control run.