Add blocked stats for input and output#11625
Conversation
arhimondr
force-pushed
the
blocked-stats
branch
from
e1deea1 to
b9b5285
Compare
|
|
||
| builder.append(indentString(1)) | ||
| .append(format("CPU: %s, Scheduled: %s, Input: %s (%s); per task: avg.: %s std.dev.: %s, Output: %s (%s)\n", | ||
| .append(format("CPU: %s, Scheduled: %s, Blocked %s (Input: %s, Output: %s), Input: %s (%s); per task: avg.: %s std.dev.: %s, Output: %s (%s)\n", |
There was a problem hiding this comment.
I am a bit concerned if it is readable. We have Input: and Output: in the string twice. But maybe it is fine.
There was a problem hiding this comment.
Yeah, I also thought about that :-) Though ran out of ideas how to make it more readable. Basically i was considering a couple of options:
- Moving the
Blockedsection to the end of the line. But then it feels likeCPU / Scheduled / Blockedshould be next to each other. - Instead of spelling
Input: ... Output: ...I was also thinking of simplifying it to(I: ..., O: ...)(since it's rather a "niche" information and one looking is assumed to already understand what they are looking at). But then it felt like it is still better to be more explicit so it is slightly less confusing for somebody who doesn't know the context.
Happy to hear your thoughts
There was a problem hiding this comment.
Maybe Blocked total/input/output? But it is even less explicit than I:.., O:....
I think we can keep it. Having this info in single line implies that it will be not super trivial to parse. And I do not think we want to split it to multiple lines (surely not in this PR).
| { | ||
| StringBuilder output = new StringBuilder(); | ||
| if (node.getStats().isEmpty() || !(plan.getTotalCpuTime().isPresent() && plan.getTotalScheduledTime().isPresent())) { | ||
| if (node.getStats().isEmpty() || !(plan.getTotalCpuTime().isPresent() && plan.getTotalScheduledTime().isPresent() && plan.getTotalBlockedTime().isPresent())) { |
There was a problem hiding this comment.
is that intentional to not print anything if any of the stats is missing?
There was a problem hiding this comment.
Stats are available when
- Generating a distributed plan for
QueryCompletedEvent - When running
EXPLAIN ANALYZE
Stats will be missing when running a simple EXPLAIN (TYPE DISTRIBUTED) ...
|
What does "blocked input" and "blocked output" mean exactly? What are the possible reasons for each of them? |
|
This is the time a task is blocked reading input data or writing input data (for example data produced by upstream tasks) |
Thanks! I have a follow-up question: Thank you! |
|
The high |
4 participants
Description
Having the blocked time as a top level statistics for input and output of a driver / pipeline / task / query will help with debugging issues related to exchange throughput.
Improvement
Core engine
-Related issues, pull requests, and links
-Documentation
(x) No documentation is needed.
( ) Sufficient documentation is included in this PR.
( ) Documentation PR is available with #prnumber.
( ) Documentation issue #issuenumber is filed, and can be handled later.
Release notes
(x) No release notes entries required.
( ) Release notes entries required with the following suggested text: