feat: add DistributedQueryExec::explain_executed_plan and show logical plan in display - #2042
Merged
Merged
Conversation
Member
Author
|
@timsaucer I'm still working on this, but fyi |
andygrove
marked this pull request as ready for review
July 22, 2026 13:26
3 tasks
avantgardnerio
added a commit
that referenced
this pull request
Aug 8, 2026
…als (#2254) PR #2038 added `optional uint32 partition = 16` to `OperatorMetric` in the proto. PR #2042 was based on pre-#2038 main and added a test with three `OperatorMetric { … }` struct literals that don't set `partition`. The merge of #2042 skipped a rebase, so the combined tree stopped compiling on main (E0063 at 338, 341, 356). The other three literals in this file (394, 398, 414) already set `partition: None`. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
No tracking issue was filed for this change. Happy to open one if preferred.
Rationale for this change
After running a distributed query, there is no convenient way for a client to see the physical plan that actually executed on the cluster.
DistributedQueryExeconly holds the logical plan, and until now its display showed a singlescheduler_url=...line. The distributed physical plan (the per-stage DAG with shuffle boundaries) is built and retained on the scheduler and was only surfaced throughEXPLAIN ANALYZE, always bundled with runtime metrics. This PR lets a client fetch that executed plan on demand (with or without metrics) and makes the operator's default display more informative.What changes are included in this PR?
DistributedQueryExec::explain_executed_plan(&self, session_config, with_metrics), which fetches the executed per-stage plan for an already-run query via the existingget_job_metricsgRPC and returns it as a printable string.with_metrics = truematches theEXPLAIN ANALYZEoutput;falseomits themetrics=[...]suffixes. It returns an error if called before the query has executed.RecordBatchbuilder into a reusableformat_job_plan(job_metrics, with_metrics), and widenfetch_job_metricstopub(crate). ExistingEXPLAIN ANALYZEoutput is unchanged (the record-batch builder now delegates toformat_job_plan(.., true)).DistributedQueryExec'sDisplayAs(Default/Verbose) to also render the submitted logical plan viaLogicalPlan::display_indent(). This is synchronous and makes no scheduler round-trip; theTreeRenderarm is unchanged.No proto, planner, or scheduler changes. Covered by unit tests (formatter without metrics, display includes logical plan, error-before-execution) and an end-to-end integration test in
context_checks.rsthat runs a query and asserts the executed plan renders with and without metrics.Are there any user-facing changes?
Yes, additive and non-breaking:
DistributedQueryExec::explain_executed_plan.DistributedQueryExec's default display now includes the submitted logical plan in addition to the scheduler URL.No breaking changes to public APIs.