Skip to content

feat: Cache LINQ query plans per filter shape (fixes #5013) - #5

Closed
erdtsieck wants to merge 2 commits into
masterfrom
erdtsieck-query-plan-cache-5013
Closed

feat: Cache LINQ query plans per filter shape (fixes #5013)#5
erdtsieck wants to merge 2 commits into
masterfrom
erdtsieck-query-plan-cache-5013

Conversation

@erdtsieck

Copy link
Copy Markdown
Owner

Summary

Implements JasperFx#5013: an opt-in query plan cache that reuses compiled LINQ plans across ad-hoc queries sharing the same structural "shape" (e.g. conditional Where() chains behind optional filters) but differing only in the captured filter values. This lets endpoints with dynamic filter combinations get compiled-query-like performance without giving up IQueryable ergonomics.

Correctness is unaffected for every existing code path — the cache is entirely additive and disabled by default.

Design

  1. ExpressionShapeVisitor (src/Marten/Linq/Caching/) walks the expression tree and computes a stable SHA-256 shape key. It only understands a narrow, allow-listed subset — Where, OrderBy/OrderByDescending, ThenBy/ThenByDescending, Skip, Take, Select, plus simple comparisons/member access. Anything else (Include(), Stats(), custom SQL, StartsWith(), GroupBy, SelectMany, etc.) flags the shape as unsupported, and it's never cached — falls straight back to today's behavior. While walking, each closure-captured/constant sub-expression is recorded as a positional "slot" whose value may vary between calls.
  2. LinqPlanRecorder builds the cached plan on a miss: it replaces every slot with a distinct sentinel value (reusing the same internal UniqueValueSource/QueryCompiler.Finders machinery that already backs ICompiledQuery), compiles that template exactly once through the normal LinqQueryParser pipeline, and matches the resulting NpgsqlBatch's parameters back to slots purely by sentinel-value identity. If any parameter can't be attributed to a slot (e.g. a multi-tenant tenant_id), the shape is rejected outright and never cached — this is the key correctness guard against replaying a stale/foreign value.
  3. On a cache hit, the template NpgsqlBatch is cloned with fresh parameter values pulled from the new expression's slots (NpgsqlParameter.Value is mutable, unlike the underlying Weasel CommandParameter that produced the original SQL). This skips LINQ parsing and SQL generation entirely and jumps straight to execution with the cached IQueryHandler.
  4. QueryPlanCache is a bounded, thread-safe, FIFO-approximate-LRU cache (ConcurrentDictionary + insertion-order ConcurrentQueue).

Configuration

var opts = new StoreOptions();
opts.Linq.QueryPlanCache = QueryPlanCache.PerShape(maxEntries: 1024);
// or:
opts.Linq.EnableQueryPlanCaching(maxEntries: 1024);

Per-query opt-in:

var lines = await query.ToListAsync(cancellation, QueryPlanCaching.Cached);

Naming deviation from the issue

The issue's example used QueryPlan.Cached, but Marten.Linq.QueryPlan already exists as the Postgres EXPLAIN plan model — overloading it with a .Cached static member would be confusing. This PR introduces a distinct QueryPlanCaching enum (Default / Cached) instead.

Scope (MVP)

Only the ToListAsync<TResult>(CancellationToken, QueryPlanCaching) terminal operator is wired up to the cache in this pass, matching the issue's own example. CountAsync, AnyAsync, streaming (ToAsyncEnumerable), etc. are left uncached for now — the underlying ExpressionShapeVisitor / QueryPlanCache / LinqPlanRecorder infrastructure is generic enough to extend to those later.

Sentinel-value generation currently supports string, Guid, int, long, float, decimal, DateTime, DateTimeOffset, enums, and arrays of those (reusing QueryCompiler.Finders). Types without a registered finder (e.g. bool) simply never get cached — a safe MVP limitation, not a correctness risk.

Tests

New src/LinqTests/query_plan_cache_Tests.cs covers:

  • Same shape / different values → same cache key
  • Different shapes → different keys
  • Different filter combinations (e.g. Where alone vs. Where + OrderBy) → different keys
  • Unsupported shapes (e.g. StartsWith) are correctly flagged as not cacheable
  • Cached plan produces identical results to the uncached path
  • Cache respects maxEntries
  • Default (disabled) behavior requires explicit opt-in; cache stays empty until enabled

Verification

  • dotnet build on src/Marten, src/LinqTests, and src/CoreTests — no errors.
  • Full LinqTests suite (single TFM, net10.0) run against Postgres: 1405/1405 passed, including the 7 new tests, with zero regressions vs. baseline.

Adds an opt-in query plan cache that reuses compiled LINQ plans across
ad-hoc queries sharing the same structural shape (e.g. conditional
Where chains behind optional filters) but differing only in captured
filter values.

- ExpressionShapeVisitor computes a stable SHA-256 shape key by
  walking a narrow, allow-listed subset of the expression tree
  (Where/OrderBy/ThenBy/Skip/Take/Select, simple comparisons/member
  access), while recording each closure-captured/constant sub-
  expression as a positional value slot. Anything outside that
  allow-list (Include, Stats, custom SQL, StartsWith, GroupBy, etc.)
  is flagged unsupported and never cached.
- LinqPlanRecorder builds a template pass on a cache miss: it replaces
  every slot with a unique sentinel value (reusing the existing
  internal UniqueValueSource/QueryCompiler.Finders machinery from
  ICompiledQuery support), compiles that template through the normal
  LinqQueryParser pipeline exactly once, and matches the resulting
  NpgsqlBatch's parameters back to slots purely by sentinel value
  identity. If any parameter can't be attributed to a slot (e.g. a
  multi-tenant tenant_id), the shape is never cached.
- On a cache hit, the cached NpgsqlBatch template is cloned with fresh
  parameter values (NpgsqlParameter.Value is mutable, unlike the
  Weasel CommandParameter feeding the original SQL fragment), so a hit
  skips LINQ parsing and SQL generation entirely.
- QueryPlanCache is a bounded, FIFO-approximate-LRU, thread-safe cache
  configured via StoreOptions.Linq.QueryPlanCache =
  QueryPlanCache.PerShape(maxEntries) or
  StoreOptions.Linq.EnableQueryPlanCaching(maxEntries). Disabled by
  default.
- Per-query opt-in via
  query.ToListAsync(token, QueryPlanCaching.Cached) (new overloads on
  MartenLinqQueryable<T> and QueryableExtensions).

Naming note: the issue's example API used QueryPlan.Cached, but
Marten.Linq.QueryPlan already exists as the Postgres EXPLAIN plan
model, so this uses a distinct QueryPlanCaching enum instead to avoid
confusion between the two unrelated concepts.

MVP scope: only the ToListAsync<TResult>(token, QueryPlanCaching)
terminal operator is wired up to the cache. Other terminals
(CountAsync, AnyAsync, streaming, etc.) are left uncached for now; the
underlying ExpressionShapeVisitor/QueryPlanCache/LinqPlanRecorder
infrastructure is generic enough to extend to them later.

Adds src/LinqTests/query_plan_cache_Tests.cs covering: stable shape
keys across different values, differing keys across differing shapes
and filter combinations, unsupported-shape detection, cached vs.
uncached result parity, max-entries bounding, and default opt-out
behavior.
Adds a new section to docs/documents/querying/compiled-queries.md
covering the query plan cache: when it helps (conditional Where
chains that can't use ICompiledQuery), how to enable it via
StoreOptions.Linq.QueryPlanCache / EnableQueryPlanCaching(), the
per-query ToListAsync(token, QueryPlanCaching.Cached) opt-in, how
shape keys are computed, the bounded/LRU cache sizing, and current
scope limitations. Also clarifies the distinction from the unrelated
IQueryPlan<T> specification pattern and the Postgres EXPLAIN
QueryPlan/ExplainAsync() feature already documented on this page.

Verified locally with markdownlint-cli and cspell against docs/**/*.md
per the repo's documentation linting conventions -- both clean.
@erdtsieck

Copy link
Copy Markdown
Owner Author

Superseded by the correctly-targeted upstream PR: JasperFx#5018

@erdtsieck erdtsieck closed this Jul 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant