Expose SliceGroup.TryFindUpstreamCache for downstream projection enrichment#205
Merged
jeremydmiller merged 1 commit intomainfrom May 6, 2026
Merged
Expose SliceGroup.TryFindUpstreamCache for downstream projection enrichment#205jeremydmiller merged 1 commit intomainfrom
jeremydmiller merged 1 commit intomainfrom
Conversation
…types Composite-projection downstream stages can already consult the upstream stages' in-memory aggregate cache when they go through the EnrichWith<T>().AddReferences() chain (which internally calls the private findCache<TEntityId, TEntity>). Custom enrichment callbacks inside EnrichUsingEntityQuery had no way to do the same lookup for a type other than the EnrichWith<T> type — which is the exact scenario that lured users into SQL-querying upstream stage output and getting empty results because the writes are still queued in the in-memory ProjectionUpdateBatch and not yet committed (see JasperFx/marten#4329). Add a public TryFindUpstreamCache<TEntityId, TEntity> on SliceGroup that returns the per-tenant upstream cache, or false when no upstream stage of the composite is producing entities of that type. This is a strict superset of the existing internal lookup; nothing else changes. Refs JasperFx/marten#4329. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
jeremydmiller
added a commit
that referenced
this pull request
May 6, 2026
Public API addition: SliceGroup<TDoc, TId>.TryFindUpstreamCache<TEntityId, TEntity> lets composite-projection downstream stages look up the in-memory aggregate cache of an arbitrary upstream entity type from inside a custom enrichment callback (notably EnrichUsingEntityQuery). Refs JasperFx/marten#4329, #205. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
jeremydmiller
added a commit
to JasperFx/marten
that referenced
this pull request
May 6, 2026
…egration test, docs JasperFx.Events 1.34.0 ships a public SliceGroup<TDoc, TId>.TryFindUpstreamCache that lets a custom enrichment callback look up an upstream stage's in-memory aggregate cache for arbitrary entity types — the previously-only-internal hook that EnrichWith<T>().AddReferences() relies on. This commit: - Bumps JasperFx 1.29.0 → 1.29.1 and JasperFx.Events 1.33.1 → 1.34.0 in Directory.Packages.props. - Adds Bug_4329_try_find_upstream_cache integration test that runs the exact shape from #4329 (single composite batch where stage 1 produces an Order and stage 2 needs to read it). Without this API the only way for stage 2 to read upstream Order data inside a custom EnrichEventsAsync would be to query SQL (which returns empty) or to listen for Updated<Order>; with the new API the downstream stage can pull the in-flight Order from the upstream cache by id. - Wires the new option into the "Cross-stage document visibility" section of composite.md and the EnrichUsingEntityQuery warning in enrichment.md, marked with a JasperFx.Events 1.34 badge. The composite.md sample is sourced from #region sample_try_find_upstream_cache in the new test file. Refs #4329, JasperFx/jasperfx#205. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
jeremydmiller
added a commit
to JasperFx/marten
that referenced
this pull request
May 7, 2026
…egration test, docs JasperFx.Events 1.34.0 ships a public SliceGroup<TDoc, TId>.TryFindUpstreamCache that lets a custom enrichment callback look up an upstream stage's in-memory aggregate cache for arbitrary entity types — the previously-only-internal hook that EnrichWith<T>().AddReferences() relies on. This commit: - Bumps JasperFx 1.29.0 → 1.29.1 and JasperFx.Events 1.33.1 → 1.34.0 in Directory.Packages.props. - Adds Bug_4329_try_find_upstream_cache integration test that runs the exact shape from #4329 (single composite batch where stage 1 produces an Order and stage 2 needs to read it). Without this API the only way for stage 2 to read upstream Order data inside a custom EnrichEventsAsync would be to query SQL (which returns empty) or to listen for Updated<Order>; with the new API the downstream stage can pull the in-flight Order from the upstream cache by id. - Wires the new option into the "Cross-stage document visibility" section of composite.md and the EnrichUsingEntityQuery warning in enrichment.md, marked with a JasperFx.Events 1.34 badge. The composite.md sample is sourced from #region sample_try_find_upstream_cache in the new test file. Refs #4329, JasperFx/jasperfx#205. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Companion change for JasperFx/marten#4329.
A composite projection runs all of its stages against a single in-memory `IProjectionBatch` that flushes to the database once, after every stage completes. Document writes produced by stage 1 are still queued in memory while stage 2 enriches, so a SQL query in a downstream stage cannot see them — a trap that's easy to fall into when writing custom `EnrichUsingEntityQuery` callbacks.
`SliceGroup<TDoc, TId>` already has a private `findCache<TEntityId, TEntity>()` helper that walks `Upstream` and pulls the upstream stage's in-memory aggregate cache for entities of type `TEntity`. This is the mechanism the `EnrichWith().AddReferences()` chain uses to read in-flight upstream output without going to SQL.
This PR exposes that lookup as a public API so custom enrichment callbacks (especially inside `EnrichUsingEntityQuery`) can do the same lookup for arbitrary upstream entity types — not only the `TEntity` of the enclosing `EnrichWith`.
```csharp
public bool TryFindUpstreamCache<TEntityId, TEntity>(
[NotNullWhen(true)] out IAggregateCache<TEntityId, TEntity>? cache);
```
Returns `false` when no upstream stage of the composite is producing entities of that type. Otherwise, returns a per-tenant cache backed by the upstream stage's in-memory aggregate cache.
Test plan
🤖 Generated with Claude Code