Add metrics for page filter and projection execution time#14135
Add metrics for page filter and projection execution time#14135raunaqmorarka merged 4 commits intotrinodb:masterfrom
Conversation
core/trino-main/src/main/java/io/trino/operator/project/PageProcessor.java
Outdated
Show resolved
Hide resolved
core/trino-main/src/main/java/io/trino/operator/project/PageProcessor.java
Outdated
Show resolved
Hide resolved
core/trino-main/src/main/java/io/trino/operator/project/PageProcessorMetrics.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Also, why are the two metrics being tracked differently? (recordXXX vs setXXX). That tells me there's something funny going on with the abstraction.
There was a problem hiding this comment.
I'm trying re-use the recording of projection execution time by ExpressionProfiler here so that we don't need to call System.nanoTime twice for recording start and stop of projection.
I've tweaked the code to avoid using double and make the tracking more similar for both fields but it doesn't look exactly the same.
There was a problem hiding this comment.
Why are we not using the same mechanism for filters? Maybe we should do that first so that both use the same approach.
There was a problem hiding this comment.
Currently PageProcessor does not perform profiling and tuning of batch sizes based on filter execution time, it does that only for projections. I'm not sure why that is the case or if adding if would be useful. If we need to add that, that seems more appropriate for a separate PR.
core/trino-main/src/main/java/io/trino/operator/project/PageProcessorMetrics.java
Outdated
Show resolved
Hide resolved
core/trino-main/src/main/java/io/trino/operator/project/PageProcessorMetrics.java
Outdated
Show resolved
Hide resolved
core/trino-main/src/main/java/io/trino/sql/gen/ExpressionProfiler.java
Outdated
Show resolved
Hide resolved
fa9496c to
698dce0
Compare
There was a problem hiding this comment.
The new shape is better, but I have a slight concern about the fact that both methods now look the same singnature-wise but behave very differently. I think this could be improved by renaming this method to recordFilterTimeSince to indicate that it's taking care of the "until" measurement.
Once we fix how we're measuring timing for filters (per my other comment), we can change it to look like the recordProjectionTime method below.
698dce0 to
7b7c3da
Compare
Description
Adds metrics for page filter and projection execution time to operator metrics which
are available in EXPLAIN ANALYZE VERBOSE.
Non-technical explanation
Add metrics for filter and projection execution time
Release notes
( ) This is not user-visible and no release notes are required.
( ) Release notes are required, please propose a release note for me.
(x) Release notes are required, with the following suggested text: