feat: dominance-based hybrid query plan scheduler (opt-in)

[jensneuse]

Summary

• Adds an opt-in query plan scheduler (WithBuildScheduleTree()) that emits Parallel(Sequence(...), Sequence(...)) shapes when independent dependency chains can run fully in parallel — cutting critical-path latency on the dominant federation pattern.
• Picks between two complementary algorithms (scheduleSP eager-inline, scheduleLevel WCC + ASAP-roots) using a provably correct symbolic path-set dominance check. Guarantees no regression vs the level-based output under any duration distribution.
• Refactors the executor to safely run nested Sequence / Parallel fetch trees with a three-phase leaf protocol (prepare / load / merge); flat plans keep the existing fast path with zero mutex overhead.

Why

Today's level-based grouping (orderSequenceByDependencies + createParallelNodes) synchronises every dependency level. When two independent entity chains co-exist (e.g. me → currentPractice plus an unrelated organisations fetch), the second chain has to wait for the first chain's join even though no dependency requires it.
The optimal plan is Parallel(Sequence(user, practice), organisations), which cuts critical path from max(user, organisations) + practice to max(user + practice, organisations).

Algorithm — TL;DR

1. Build the fetch DAG.
2. Run scheduleSP (component-aware eager-inline with multi-parent merge intersection) and scheduleLevel (component-aware level-based) on the same DAG.
3. Validate both trees.
4. Return SP iff dominates(SP, Level) is true, where dominance is set-containment of root-to-leaf paths: every path in A is contained in some path in B. This implies makespan_A(d) ≤ makespan_B(d) for every non-negative duration vector — a finite, polynomial-time, provably correct check.
5. Otherwise return Level. Conservative no-regression fallback.

Full theorem and proof in docs/adr/0001-query-plan-scheduler.md.

Wiring

The new processor is opt-in via WithBuildScheduleTree(). The default postprocess pipeline is unchanged — existing fixtures and behaviour are stable. A follow-up PR will flip the default and audit fixture diffs.

Scope

• v2/pkg/engine/resolve/loader.go — three-phase split (prepare/load/merge) inside resolveSingle for SingleFetch / EntityFetch / BatchEntityFetch. mergeMu sync.Mutex and useMergeMu bool on Loader. fetchTreeHasNestedParallel walk at fetch-tree entry. resolveParallel branches: existing fast path for all-Single children (no lock), resolveParallelNested for the new shapes (lock-aware via maybeLock).
• v2/pkg/engine/postprocess/build_schedule_tree.go — scheduleSP, scheduleLevel, dominates, buildScheduleTree, validateSchedule with leaf-flattening and fetch-kind-aware providedPath for SingleFetch/EntityFetch/BatchEntityFetch.
• v2/pkg/engine/postprocess/postprocess.go — WithBuildScheduleTree() opt-in option.
• v2/pkg/engine/postprocess/build_schedule_tree_scenarios_test.go — 23 hand-built scenarios + option-wiring + scenario-17 validator stress test.
• v2/pkg/engine/postprocess/build_schedule_tree_property_test.go — exhaustive enumeration n≤6, random n=3..15 (fixed seed), smoke n=50/200, fixture skew-stress.
• v2/pkg/engine/datasource/graphql_datasource/query_order_baseline_test.go — planner-direct baseline test (pins today's level-based shape; the new processor is opt-in so this still passes).
• docs/adr/0001-query-plan-scheduler.md — ADR with algorithms, dominance theorem + proof, executor split, residuals, worked examples.
• docs/superpowers/specs/2026-05-08-*.md — design specs that drove the implementation.

Acknowledged residuals

Documented in the ADR and in scenario fixtures:

• Gap A — non-SP DAGs where no SP tree achieves the critical path under all durations (representation limit; closing requires a richer runtime model).
• Gap B — incomparable trees under skew (the dominance fallback picks Level conservatively; some duration distributions favour SP at runtime, but we cannot tell without runtime statistics).

Test plan

• CI green
• go test ./pkg/engine/postprocess/ — all 23 scenarios + 4 property tests pass
• go test ./pkg/engine/resolve/ — clean
• go test -race ./pkg/engine/postprocess/ ./pkg/engine/resolve/ — clean
• go test -run TestQueryOrderBaseline ./pkg/engine/datasource/graphql_datasource/ — passes (default behaviour unchanged)
• Reviewer sanity-check the dominance theorem proof in the ADR
• Reviewer skim of the 23 scenario fixtures and at least one of the property tests
• Decide whether to keep the cycle-detection panic or change FetchTreeProcessor interface to return error (open question)
• Plan the follow-up PR that flips the default and audits fixture diffs

Notes

This is a draft. No fixture flips, no default behaviour changes, no executor regressions — only new opt-in code paths plus a behaviour-preserving executor refactor.

🤖 Generated with Claude Code

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 5b266776-8452-4438-a22b-ddc3667ebda6

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch improve-query-order

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[ysmolski]

Resolved by #1594