Cache LINQ query plans per filter shape — compiled-query performance for conditional Where chains

[erdtsieck]

Motivation

Compiled queries (ICompiledQuery / ICompiledListQuery) remove the LINQ expression parsing cost per execution, and with UseMartenCompiledQueryResultPolicy they compose beautifully with raw JSON streaming in Wolverine.HTTP. But they freeze the query shape at plan time, which makes them unusable for the most common real-world endpoint: a list with optional filters.

Every overview endpoint in our claims platform looks like this:

IQueryable<PendingClientClaimLine> query = session.Query<PendingClientClaimLine>();
if (vanaf.HasValue)                       query = query.Where(x => x.Begindatum >= vanafAsDateOnly);
if (totEnMet.HasValue)                    query = query.Where(x => x.Begindatum <= totEnMetAsDateOnly);
if (bsn.HasValue)                         query = query.Where(x => x.ClientBsn == bsn);
else if (!string.IsNullOrWhiteSpace(client)) query = query.Where(x => x.ClientNaam.VolledigeNaam.NgramSearch(client));
if (prestatieId.HasValue)                 query = query.Where(x => x.PrestatieId == prestatieId);

Five optional filters → up to 2^5 distinct shapes, but each individual shape is completely static: same expression trees, same operators, only parameter values differ. Today every request re-parses and re-plans the whole expression. Encoding this as a compiled query is impossible (myParam == null || ... guards would be baked in at plan compilation), so these endpoints are locked out of the compiled-query tier entirely.

Proposal

An opt-in caching layer that recognizes recurring query shapes and reuses their compiled plans:

var opts = new StoreOptions();
opts.Linq.QueryPlanCache = QueryPlanCache.PerShape(maxEntries: 1024);

Sketch:

1. Before full LINQ compilation, compute a shape key: a structural hash of the expression tree that treats captured closure values/constants as parameter slots (type + position) rather than values. The conditional-Where pattern above yields one key per filter combination.
2. Cache hit → reuse the compiled SQL + parameter-extraction delegates; only re-bind the slot values from the new expression's closures. Miss → compile as today, store.
3. Bounded LRU; anything that defeats safe parameterization (dynamically built member access, non-parameterizable constructs like the explicit OR-chain we build for id IN (...) workarounds) simply never caches — correctness identical to today, caching is purely a fast path.

Explicit opt-in per query is a fine first cut if global shape-hashing is too aggressive:

var lines = await query.ToListAsync(cancellation, QueryPlan.Cached);

Why in Marten and not user land

The shape key and the value-slot extraction need the internals of Marten's LINQ-to-SQL compilation (which subtrees become parameters, which become SQL structure). Outside Marten you can only cache whole result sets, not plans.

Expected impact

For hot list endpoints the LINQ parse/plan step is the dominant CPU cost after raw streaming removes serialization (the SQL itself is index-backed and fast). Shape caching gives conditional-filter endpoints the same per-request profile as hand-written compiled queries — without contorting them into 2^N compiled-query classes or forfeiting streaming.

Prior art

EF Core does exactly this transparently (query cache keyed on expression shape with parameter extraction); the concept is proven, the ask is the Marten/Weasel equivalent.