Fix raw input stats for ScanFilterAndProject#13873
Fix raw input stats for ScanFilterAndProject#13873mbasmanova merged 2 commits intoprestodb:masterfrom
Conversation
4c71e20 to
3c96f6a
Compare
presto-main/src/main/java/com/facebook/presto/operator/TableScanOperator.java
Outdated
Show resolved
Hide resolved
3c96f6a to
f1d73d5
Compare
mbasmanova
left a comment
There was a problem hiding this comment.
@pettyjamesm James, thanks for fixing this issue.
nit: comment title shouldn't capitalize all words: Fix raw input stats for ScanFilterAndProject
There was a problem hiding this comment.
I don't quite understand why this call is needed. If LazyBlocks are not loaded, the data is not read, hence no stats to update.
There was a problem hiding this comment.
This is a last chance update that shouldn't be necessary if everything else is working as expected, but attempts to harden the bytes read accounting against future refactors. I can remove it if you'd prefer, but I would note that the previous omission of accounting wouldn't have yielded incorrect results if this fallback logic in finish were there.
There was a problem hiding this comment.
@pettyjamesm I would prefer to remove this call (as long as it is not necessary).
mbasmanova
left a comment
There was a problem hiding this comment.
nit: commit title shouldn't capitalize all words: Refactor stats recording in TableScanOperator
|
Looks good % one question about calling |
0783e90 to
d8312b0
Compare
mbasmanova
left a comment
There was a problem hiding this comment.
@pettyjamesm Looks good % unnecessary recordPageSourceRawInputStats call in ScanFilterAndProject.
Previously, LazyBlocks loaded from a PageSource would update the processed input stat, but would not inherently trigger the raw input bytes to be recomputed. This wasn't necessarily a problem, but could result in raw input data being left uncounted until the next call to getOutput (which might never occur).
d8312b0 to
a49865b
Compare
|
Removed |
Reorganizes TableScanOperator stats collection to match the changes made to ScanFilterAndProjectOperator.
a49865b to
1fa774b
Compare
|
Different test failures across two source identical builds are in different modules and seem spurious/unrelated. |
Previously,
LazyBlocks loaded from aPageSourcewould update theprocessed input bytes upon being loaded, but would not recompute the raw input
bytes. The next call to getOutput would update the raw input stats, but if that never occurs then the reads were left uncounted.