Skip to content

feat: add DistributedQueryExec::explain_executed_plan and show logical plan in display - #2042

Merged
milenkovicm merged 5 commits into
apache:mainfrom
andygrove:explain-executed-plan
Aug 8, 2026
Merged

feat: add DistributedQueryExec::explain_executed_plan and show logical plan in display#2042
milenkovicm merged 5 commits into
apache:mainfrom
andygrove:explain-executed-plan

Conversation

@andygrove

Copy link
Copy Markdown
Member

Which issue does this PR close?

No tracking issue was filed for this change. Happy to open one if preferred.

Rationale for this change

After running a distributed query, there is no convenient way for a client to see the physical plan that actually executed on the cluster. DistributedQueryExec only holds the logical plan, and until now its display showed a single scheduler_url=... line. The distributed physical plan (the per-stage DAG with shuffle boundaries) is built and retained on the scheduler and was only surfaced through EXPLAIN ANALYZE, always bundled with runtime metrics. This PR lets a client fetch that executed plan on demand (with or without metrics) and makes the operator's default display more informative.

What changes are included in this PR?

  • Add DistributedQueryExec::explain_executed_plan(&self, session_config, with_metrics), which fetches the executed per-stage plan for an already-run query via the existing get_job_metrics gRPC and returns it as a printable string. with_metrics = true matches the EXPLAIN ANALYZE output; false omits the metrics=[...] suffixes. It returns an error if called before the query has executed.
  • Extract the per-stage plan formatting out of the explain-analyze RecordBatch builder into a reusable format_job_plan(job_metrics, with_metrics), and widen fetch_job_metrics to pub(crate). Existing EXPLAIN ANALYZE output is unchanged (the record-batch builder now delegates to format_job_plan(.., true)).
  • Extend DistributedQueryExec's DisplayAs (Default/Verbose) to also render the submitted logical plan via LogicalPlan::display_indent(). This is synchronous and makes no scheduler round-trip; the TreeRender arm is unchanged.

No proto, planner, or scheduler changes. Covered by unit tests (formatter without metrics, display includes logical plan, error-before-execution) and an end-to-end integration test in context_checks.rs that runs a query and asserts the executed plan renders with and without metrics.

Are there any user-facing changes?

Yes, additive and non-breaking:

  • New public method DistributedQueryExec::explain_executed_plan.
  • DistributedQueryExec's default display now includes the submitted logical plan in addition to the scheduler URL.

No breaking changes to public APIs.

@andygrove

Copy link
Copy Markdown
Member Author

@timsaucer I'm still working on this, but fyi

@andygrove
andygrove marked this pull request as ready for review July 22, 2026 13:26

@milenkovicm milenkovicm left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @andygrove

@milenkovicm
milenkovicm merged commit e1e39da into apache:main Aug 8, 2026
17 checks passed
avantgardnerio added a commit that referenced this pull request Aug 8, 2026
…als (#2254)

PR #2038 added `optional uint32 partition = 16` to `OperatorMetric` in
the proto. PR #2042 was based on pre-#2038 main and added a test with
three `OperatorMetric { … }` struct literals that don't set `partition`.
The merge of #2042 skipped a rebase, so the combined tree stopped
compiling on main (E0063 at 338, 341, 356).

The other three literals in this file (394, 398, 414) already set
`partition: None`.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants