Cross-stage document visibility docs + TryFindUpstreamCache support (#4329)#4334
Merged
jeremydmiller merged 2 commits intomasterfrom May 7, 2026
Merged
Conversation
Adds a "Cross-stage document visibility" section to composite.md explaining that downstream stages cannot SQL-query the in-flight writes of upstream stages in the same composite batch, with pointers to the supported alternatives (Updated<T>, EnrichWith<T>, ReferencePeerView<T>). Cross-links a warning into the EnrichUsingEntityQuery section in enrichment.md so users hit the guidance at the API surface where the trap is easiest to fall into. Refs #4329. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…egration test, docs JasperFx.Events 1.34.0 ships a public SliceGroup<TDoc, TId>.TryFindUpstreamCache that lets a custom enrichment callback look up an upstream stage's in-memory aggregate cache for arbitrary entity types — the previously-only-internal hook that EnrichWith<T>().AddReferences() relies on. This commit: - Bumps JasperFx 1.29.0 → 1.29.1 and JasperFx.Events 1.33.1 → 1.34.0 in Directory.Packages.props. - Adds Bug_4329_try_find_upstream_cache integration test that runs the exact shape from #4329 (single composite batch where stage 1 produces an Order and stage 2 needs to read it). Without this API the only way for stage 2 to read upstream Order data inside a custom EnrichEventsAsync would be to query SQL (which returns empty) or to listen for Updated<Order>; with the new API the downstream stage can pull the in-flight Order from the upstream cache by id. - Wires the new option into the "Cross-stage document visibility" section of composite.md and the EnrichUsingEntityQuery warning in enrichment.md, marked with a JasperFx.Events 1.34 badge. The composite.md sample is sourced from #region sample_try_find_upstream_cache in the new test file. Refs #4329, JasperFx/jasperfx#205. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This was referenced May 6, 2026
This was referenced May 7, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two-part response to #4329 — a documentation gap and a missing public API surface.
A composite projection runs all of its stages against a single in-memory
IProjectionBatchthat flushes to the database once, after every stage completes. Document writes produced by an upstream stage are not yet visible to a SQL query issued from a downstream stage of the same batch — the query goes to PostgreSQL, which has not received them. DuringRebuildProjectionAsyncthe trap is even easier to fall into because no documents have been committed yet. The user in #4329 hit this when they wrotequerySession.Query<Appointment>().ToList()insideEnrichUsingEntityQueryand got an empty result for documents an upstream stage was producing.Docs
:::warningcallout on theEnrichUsingEntityQuerysection in enrichment.md at the API surface where the trap is easiest to fall into.New supported pattern
The doc above lists three pre-existing options (
Updated<T>events,EnrichWith<T>().AddReferences(),ReferencePeerView<T>()) plus a fourth, new option:group.TryFindUpstreamCache<TId, T>(out var cache)— exposed in JasperFx/jasperfx#205, shipped as JasperFx.Events 1.34.0. Lets a custom enrichment callback (notably insideEnrichUsingEntityQuery) reach into the upstream stage's in-memory aggregate cache for arbitrary entity types — not only theTEntityof the enclosingEnrichWith<T>.Bumps + new integration test
JasperFx1.29.0 → 1.29.1,JasperFx.Events1.33.1 → 1.34.0 inDirectory.Packages.props.Bug_4329_try_find_upstream_cacheintegration test running the exact Composite projections: Query result of documents of upstream projection always empty #4329 shape (one composite batch, stage 1 produces anOrder, stage 2 reads it insideEnrichEventsAsync). The test fails the visibility-empty path the user reported and now passes viaTryFindUpstreamCache. The doc sample is sourced from this test's#region sample_try_find_upstream_cacheblock.Test plan
Bug_4329_try_find_upstream_cache.downstream_stage_can_read_upstream_in_flight_order_via_upstream_cachepasses on net10.0DaemonTests/Composites/*(6 tests) pass on net10.0 with the bumped package versionsnpm run docsrenders the new section, anchor link, andsample_try_find_upstream_cachesnippet🤖 Generated with Claude Code