Skip to content

Cross-stage document visibility docs + TryFindUpstreamCache support (#4329)#4334

Merged
jeremydmiller merged 2 commits intomasterfrom
docs-issue-4329-composite-stage-visibility
May 7, 2026
Merged

Cross-stage document visibility docs + TryFindUpstreamCache support (#4329)#4334
jeremydmiller merged 2 commits intomasterfrom
docs-issue-4329-composite-stage-visibility

Conversation

@jeremydmiller
Copy link
Copy Markdown
Member

@jeremydmiller jeremydmiller commented May 6, 2026

Summary

Two-part response to #4329 — a documentation gap and a missing public API surface.

A composite projection runs all of its stages against a single in-memory IProjectionBatch that flushes to the database once, after every stage completes. Document writes produced by an upstream stage are not yet visible to a SQL query issued from a downstream stage of the same batch — the query goes to PostgreSQL, which has not received them. During RebuildProjectionAsync the trap is even easier to fall into because no documents have been committed yet. The user in #4329 hit this when they wrote querySession.Query<Appointment>().ToList() inside EnrichUsingEntityQuery and got an empty result for documents an upstream stage was producing.

Docs

  • New Cross-stage document visibility section in composite.md explaining the rule and pointing at the four supported ways to read upstream stage output.
  • :::warning callout on the EnrichUsingEntityQuery section in enrichment.md at the API surface where the trap is easiest to fall into.

New supported pattern

The doc above lists three pre-existing options (Updated<T> events, EnrichWith<T>().AddReferences(), ReferencePeerView<T>()) plus a fourth, new option:

  • group.TryFindUpstreamCache<TId, T>(out var cache) — exposed in JasperFx/jasperfx#205, shipped as JasperFx.Events 1.34.0. Lets a custom enrichment callback (notably inside EnrichUsingEntityQuery) reach into the upstream stage's in-memory aggregate cache for arbitrary entity types — not only the TEntity of the enclosing EnrichWith<T>.

Bumps + new integration test

Test plan

  • Bug_4329_try_find_upstream_cache.downstream_stage_can_read_upstream_in_flight_order_via_upstream_cache passes on net10.0
  • All DaemonTests/Composites/* (6 tests) pass on net10.0 with the bumped package versions
  • npm run docs renders the new section, anchor link, and sample_try_find_upstream_cache snippet

🤖 Generated with Claude Code

Adds a "Cross-stage document visibility" section to composite.md
explaining that downstream stages cannot SQL-query the in-flight
writes of upstream stages in the same composite batch, with pointers
to the supported alternatives (Updated<T>, EnrichWith<T>, ReferencePeerView<T>).
Cross-links a warning into the EnrichUsingEntityQuery section in
enrichment.md so users hit the guidance at the API surface where the
trap is easiest to fall into.

Refs #4329.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…egration test, docs

JasperFx.Events 1.34.0 ships a public SliceGroup<TDoc, TId>.TryFindUpstreamCache
that lets a custom enrichment callback look up an upstream stage's in-memory
aggregate cache for arbitrary entity types — the previously-only-internal hook
that EnrichWith<T>().AddReferences() relies on.

This commit:

- Bumps JasperFx 1.29.0 → 1.29.1 and JasperFx.Events 1.33.1 → 1.34.0 in
  Directory.Packages.props.
- Adds Bug_4329_try_find_upstream_cache integration test that runs the exact
  shape from #4329 (single composite batch where stage 1 produces an Order and
  stage 2 needs to read it). Without this API the only way for stage 2 to read
  upstream Order data inside a custom EnrichEventsAsync would be to query SQL
  (which returns empty) or to listen for Updated<Order>; with the new API the
  downstream stage can pull the in-flight Order from the upstream cache by id.
- Wires the new option into the "Cross-stage document visibility" section of
  composite.md and the EnrichUsingEntityQuery warning in enrichment.md, marked
  with a JasperFx.Events 1.34 badge. The composite.md sample is sourced from
  #region sample_try_find_upstream_cache in the new test file.

Refs #4329, JasperFx/jasperfx#205.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@jeremydmiller jeremydmiller changed the title Docs: clarify cross-stage document visibility in composite projections (#4329) Cross-stage document visibility docs + TryFindUpstreamCache support (#4329) May 6, 2026
@jeremydmiller jeremydmiller merged commit 356ea48 into master May 7, 2026
6 checks passed
@jeremydmiller jeremydmiller deleted the docs-issue-4329-composite-stage-visibility branch May 7, 2026 00:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant