diff --git a/.claude/agents/architecture-reviewer.md b/.claude/agents/architecture-reviewer.md index d8714d39..03859a04 100644 --- a/.claude/agents/architecture-reviewer.md +++ b/.claude/agents/architecture-reviewer.md @@ -69,7 +69,14 @@ Specific bug-classes that have bitten this repo before. When the target file mat - **`.RequireAuthorization()` at group level** unless explicitly public. - **List endpoints clamp pagination** server-side (`ClampPaging` or equivalent, cap ≤ 100). - **Rate-limiter shape (Should-consider on single-instance, Must-fix when scaled to 2+ instances).** ASP.NET Core's `RequireRateLimiting` + `AddFixedWindowLimiter` / `AddSlidingWindowLimiter` uses an in-memory counter store. Single-instance: correct. 2+ instances: the limit silently multiplies by N — each instance enforces its own counter, a client hitting any instance gets a fresh allowance. NextAurora is single-instance everywhere today (Catalog deployed on one Fly Machine; rest local), so in-memory is right *for now*. Flag: (a) a new `RequireRateLimiting` on an endpoint without a comment at the call site or registration site (`Program.cs` `AddFixedWindowLimiter`) justifying why in-memory is correct + naming the swap-to-Redis trigger, OR (b) a deployment-config PR (`Dockerfile.*`, `fly.toml`, GitHub Actions deploy workflow) that scales a rate-limited service past 1 Machine without a paired swap to a Redis-backed limiter. The fix when scale-out lands: Redis-backed limiter using the existing HybridCache Redis, with the increment + TTL pair wrapped in a Lua `EVAL` so it's atomic under concurrency. See CLAUDE.md "Security Requirements → Rate Limiting". -- **Long-running work shape (Must-fix on minutes-scale handlers, Should-consider on >1s).** If a write endpoint synchronously awaits something that can take more than ~1s — multi-step external API chain (e.g. Stripe + tax calc + fraud check sequentially), aggregation over thousands of rows, bulk import, report generation — it's the wrong shape. The HTTP request holds a thread, a DB connection, and a concurrency-budget slot for the full duration, so a small spike on that one endpoint can take the rest of the API down with it. Reshape as 202 Accepted: validate + persist a tracking row + publish a Wolverine message + return `202` with the job/correlation ID in the body and a `Location` header pointing to a status endpoint. A background handler does the work; the client polls or receives a push (SignalR/SSE/email). The synchronous parts commit in one EF transaction via `AutoApplyTransactions`. NextAurora already has the full machinery (Wolverine + Service Bus + outbox + saga handlers) — the rule is "use it when a handler would otherwise block on minutes-scale work." Reference shape: `POST /api/v1/orders` (place → publish `OrderPlaced` → return OrderId immediately; PaymentService + ShippingService handle the downstream work async via the saga). Same rule applies to Wolverine handlers themselves: if the handler body runs for minutes, the work belongs in a follow-up message handler, not in-line. See CLAUDE.md "Performance Rules → Long-running work belongs on the message bus." +- **Long-running work shape (Must-fix on minutes-scale handlers, Should-consider on >1s).** If a write endpoint synchronously awaits something that can take more than ~1s — multi-step external API chain (e.g. Stripe + tax calc + fraud check sequentially), aggregation over thousands of rows, bulk import, report generation — it's the wrong shape. The HTTP request holds a thread, a DB connection, and a concurrency-budget slot for the full duration, so a small spike on that one endpoint can take the rest of the API down with it. Reshape as 202 Accepted: validate + persist a tracking row + publish a Wolverine message + return `202` with the job/correlation ID in the body and a `Location` header pointing to a status endpoint. A background handler does the work; the client polls or receives a push (SignalR/SSE/email). The synchronous parts commit in one EF transaction via `AutoApplyTransactions`. NextAurora already has the full machinery (Wolverine + RabbitMQ + outbox + saga handlers) — the rule is "use it when a handler would otherwise block on minutes-scale work." Reference shape: `POST /api/v1/orders` (place → publish `OrderPlaced` → return OrderId immediately; PaymentService + ShippingService handle the downstream work async via the saga). Same rule applies to Wolverine handlers themselves: if the handler body runs for minutes, the work belongs in a follow-up message handler, not in-line. See CLAUDE.md "Performance Rules → Long-running work belongs on the message bus." + +### When reviewing `**/Program.cs` messaging blocks (Wolverine/RabbitMQ wiring) + +- **Topology completeness vs the first-boot race (CRITICAL until #168 lands).** Fanout exchanges silently discard unroutable messages, and AutoProvision declares topology lazily per-service — a consumer's queue+binding exists only after that consumer's first boot. Flag any new publisher whose consumers' queues are not also declared publisher-side (or otherwise guaranteed to exist before first publish). See CLAUDE.md. +- **Durability is per-direction (Must-fix on new listeners until #169 lands).** `UseDurableOutboxOnAllSendingEndpoints()` covers only sends; default listeners are buffered (acked before handlers run). New `ListenToRabbitQueue(...)` endpoints need `UseDurableInboxOnAllListeningEndpoints()` (store-backed services) or `ProcessInline()` (stateless services). See CLAUDE.md. +- **Exchange/queue names as shared constants, not inline literals.** A typo'd `BindExchange("payment_events")` is silently auto-provisioned and the consumer never receives real events — no error anywhere. Names duplicated between `BindExchange(...).ToQueue("x")` and `ListenToRabbitQueue("x")` must be a single const; cross-service names belong in a shared topology constants home (see #168 discussion). +- **Config-key shape consistency.** `Wolverine:AutoProvision` is read colon-form via `UseSetting`/config; environment overrides use `Wolverine__AutoProvision`. Flag a third variant. ### When reviewing `**/*RecoveryJob*.cs` or any `BackgroundService` / cron-style sweeper diff --git a/.claude/architecture-map.md b/.claude/architecture-map.md index 0cb305cd..288436d7 100644 --- a/.claude/architecture-map.md +++ b/.claude/architecture-map.md @@ -41,7 +41,7 @@ PlaceOrder (HTTP POST → OrderService) │ │ 2. Order aggregate saved + OrderPlacedEvent staged in outbox (same tx) │ - └─→ OrderPlacedEvent (Service Bus topic) + └─→ OrderPlacedEvent (RabbitMQ exchange) ├─→ PaymentService consumes (OrderPlacedHandler → ProcessPayment) │ │ │ │ Payment aggregate saved + PaymentCompletedEvent staged in outbox @@ -79,9 +79,9 @@ All events live in `NextAurora.Contracts/Events/`: - `PaymentFailedEvent` — published by PaymentService - `ShipmentDispatchedEvent` — published by ShippingService -Service Bus topology: one topic per event type, one subscription per consumer-source pair. -Subscription names are globally unique within the namespace (Aspire 13+ rule). -Convention: `{consumer}-{source-events}-sub` (e.g. `notify-orders-sub`). +RabbitMQ topology: one fanout exchange per event type, one queue per consumer bound to it. +Convention: `{consumer}-{source-events}` (e.g. `notify-orders` = NotificationService consuming +`order-events`). Wolverine declares + AutoProvisions the exchanges/queues/bindings. --- @@ -150,7 +150,7 @@ the failure mode. - **REST (HTTP)** — frontend ↔ services only. URL-segment versioned `/api/v1/...`. - **gRPC (sync)** — OrderService → CatalogService for real-time product validation + stock reservation. Versioned via `.proto` `package` declarations. -- **Service Bus (async)** — all workflow events. Wolverine transports + transactional outbox. +- **RabbitMQ (async)** — all workflow events. Wolverine transport + transactional outbox. --- diff --git a/.claude/audits/INDEX.md b/.claude/audits/INDEX.md index 34136cd8..4c0895fd 100644 --- a/.claude/audits/INDEX.md +++ b/.claude/audits/INDEX.md @@ -38,3 +38,4 @@ Persistent log of every `/article-audit` run. One row per audit; full reasoning | 2026-06-12 | [The Async/Await Habit That Quietly Slows ASP.NET Core Under Load](2026-06-12-async-await-habits-under-load.md) (Kerim Kara, Activated Thinker) | ✅ All five teaser categories encoded, mostly more rigorously (build-time bans + hooks vs advice); paywall-limited — intro only, re-audit on paste | No action | | 2026-06-12 | [.NET Microservices Guide — What To Do and What To Avoid](2026-06-12-dotnet-microservices-do-avoid.md) (Kerim Kara, Real World .NET) | ⚙️ Re-audited after user pasted full body (initial pass paywall-limited). All 10 do/avoid items covered; article's own examples violate the canon 3× (publish-after-save dual-write vs outbox; ValidateAudience=false vs JWT rule; IOrderService layer vs VSA). Recurring implicit stance: internal-gRPC trust + no-gateway undocumented (2nd hit today) | Offered small encoding issue (internal-trust + no-gateway triggers); pending user decision | | 2026-06-12 | [Building Dapr Workflows in .NET With Aspire](2026-06-12-dapr-workflows-aspire.md) (Milan Jovanović, The .NET Weekly) | ✅ Considered + rejected with depth — project-decisions.md §22 ("Dapr — considered, not adopted") + architecture.md:283 (choreography saga, no central orchestrator) + perf doc already names Temporal/Durable Functions/Step Functions as the durable-execution alternatives. 202-Accepted + idempotency + outbox all encoded | No new issue — §22's table maps the classic 5 building blocks but not **Dapr Workflow**; fold a Workflow row + orchestration-vs-choreography trigger into open #86 | +| 2026-06-15 | [Wolverine + Azure Service Bus Emulator](2026-06-15-wolverine-asb-emulator.md) (WolverineFx official docs) | ⚙️ Official recipe (`UseAzureServiceBus` + `transport.ManagementConnectionString` on :5300 + AutoProvision) matches in-flight feat-branch fix and **contradicts two `main` rules** — CLAUDE.md:230 + architecture.md:399 claim "emulator has no management API" (false; it does, only subscription admin returns 500). Also confirms the `ListenToAzureServiceBusSubscription(sub, c => c.TopicName=topic)` API vs our wrong `"{topic}/{sub}"` form | No new issue — tracked on #148; doc corrections land with that branch. Confirms feat-branch "Approach B" is officially correct; subscription-admin-500 limitation is ours to document (upstream omits it) | diff --git a/.claude/scripts/check-claude-md-refs.sh b/.claude/scripts/check-claude-md-refs.sh index 50d56132..7021b646 100755 --- a/.claude/scripts/check-claude-md-refs.sh +++ b/.claude/scripts/check-claude-md-refs.sh @@ -30,11 +30,21 @@ matches=$(grep -rln "See CLAUDE.md" \ | grep -v "^${CANONICAL}\$" \ || true) -# No matches: silent no-op (nothing to remind about). -if [ -z "$matches" ]; then +# AI-instruction surfaces are ALWAYS on the review list, marker or not — so this list is +# built BEFORE any no-match early-exit. They paraphrase canon for other AI tools (Copilot, +# CodeRabbit), and drift there silently regenerates retired patterns — +# .github/copilot-instructions.md still taught the removed transport months after the swap +# because it carried no marker. See CLAUDE.md. +ai_surfaces="" +for f in "$REPO_ROOT/.github/copilot-instructions.md" "$REPO_ROOT/.coderabbit.yaml"; do + [ -f "$f" ] && ai_surfaces="${ai_surfaces}${f}"$'\n' +done + +# Nothing at all to remind about: silent no-op. +if [ -z "$matches" ] && [ -z "$ai_surfaces" ]; then exit 0 fi # Emit additionalContext via hookSpecificOutput so the model sees the reminder. -msg=$(printf 'CLAUDE.md was edited. Files containing the "See CLAUDE.md" marker (review each for staleness against the new rule):\n%s' "$matches") +msg=$(printf 'CLAUDE.md was edited. Files containing the "See CLAUDE.md" marker (review each for staleness against the new rule):\n%s\n\nAI-instruction surfaces (ALWAYS check these — they paraphrase canon for other AI tools and drift silently):\n%s' "${matches:-(none)}" "$ai_surfaces") jq -n --arg m "$msg" '{hookSpecificOutput: {hookEventName: "PostToolUse", additionalContext: $m}}' diff --git a/.claude/scripts/check-tombstones.sh b/.claude/scripts/check-tombstones.sh new file mode 100755 index 00000000..93184c8b --- /dev/null +++ b/.claude/scripts/check-tombstones.sh @@ -0,0 +1,62 @@ +#!/usr/bin/env bash +# Tombstone audit — removed-identifier drift control. See CLAUDE.md. +# +# When a subsystem is removed (a transport, a metric, an API), its identifiers get +# tombstoned in .claude/tombstones.txt. This script fails if any tombstoned pattern +# appears in tracked files outside the allowlist (.claude/tombstones-allowlist.txt). +# +# Why: the compiler catches stale identifiers in code; NOTHING catches them in docs, +# comments, and config. The RabbitMQ swap (#159) left 15+ docs teaching Azure Service +# Bus as current — found by an ultracode review, not by the loop. This closes that gap +# mechanically: the sweep's completion criterion is "this script passes", not "the docs +# someone remembered are updated". +# +# Usage: .claude/scripts/check-tombstones.sh (run from anywhere; CI runs it per PR) + +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" +cd "$ROOT" + +TOMBSTONES=".claude/tombstones.txt" +ALLOWLIST=".claude/tombstones-allowlist.txt" + +excludes=() +while IFS= read -r line; do + [ -z "$line" ] && continue + case "$line" in \#*) continue ;; esac + excludes+=(":(exclude)$line") +done < "$ALLOWLIST" + +fail=0 +while IFS= read -r pattern; do + [ -z "$pattern" ] && continue + case "$pattern" in \#*) continue ;; esac + # git grep over tracked files only; -i case-insensitive, -E extended regex. + # Exit codes: 0 = matches (violation), 1 = clean, >=2 = error (e.g. invalid + # regex) — an invalid tombstone must FAIL the audit, not silently disable it. + set +e + hits=$(git grep -inE "$pattern" -- '.' "${excludes[@]}" 2>&1) + status=$? + set -e + if [ "$status" -ge 2 ]; then + echo "TOMBSTONE AUDIT ERROR — pattern '$pattern' failed to evaluate (git grep exit $status):" + echo "$hits" | sed -n '1,5p' + fail=1 + elif [ "$status" -eq 0 ] && [ -n "$hits" ]; then + echo "TOMBSTONE VIOLATION — pattern '$pattern':" + # sed -n '1,30p' rather than piping through head: under pipefail, head + # closing the pipe early would SIGPIPE the producer and abort the script. + echo "$hits" | sed -n '1,30p' | sed 's/^/ /' + echo "" + fail=1 + fi +done < "$TOMBSTONES" + +if [ "$fail" -eq 1 ]; then + echo "Removed identifiers are resurfacing (or were never fully swept)." + echo "Fix the references — or, for genuinely historical/comparative mentions," + echo "add the file to $ALLOWLIST with a justification comment." + exit 1 +fi +echo "Tombstone audit clean." diff --git a/.claude/tombstones-allowlist.txt b/.claude/tombstones-allowlist.txt new file mode 100644 index 00000000..7d1a936f --- /dev/null +++ b/.claude/tombstones-allowlist.txt @@ -0,0 +1,21 @@ +# Files exempt from the tombstone audit — each entry earns its place by containing +# LEGITIMATE historical or comparative mentions of removed identifiers. Keep this list +# short; an allowlisted file can re-drift invisibly, so prefer fixing over allowlisting. +# +# Historical decision records (past-tense by nature): +docs/project-decisions.md +docs/full-saga-deployment-plan.md +docs/war-story-wolverine6-outbox-atomicity.md +# Comparative/portable guides where the removed transport is a subject of comparison: +docs/messaging-transport-selection.md +docs/performance-and-data-correctness.md +# Canon: carries the "evaluated and removed" rationale + durable-pub/sub pattern comparison: +CLAUDE.md +# Past-tense removal note (line ~57) + AWS-alternative comparison section: +docs/architecture.md +# The control's own definition files: +.claude/tombstones.txt +.claude/tombstones-allowlist.txt +.claude/scripts/check-tombstones.sh +# Audit log of external-article reviews (historical verdicts, intentionally tracked): +.claude/audits/INDEX.md diff --git a/.claude/tombstones.txt b/.claude/tombstones.txt new file mode 100644 index 00000000..2037d480 --- /dev/null +++ b/.claude/tombstones.txt @@ -0,0 +1,16 @@ +# Removed-identifier tombstones — one extended-regex pattern per line (case-insensitive). +# When a subsystem/identifier is REMOVED from the codebase, its names go here so CI fails +# any doc/comment/config that still presents it as current. This is the identifier-level +# counterpart of the file-move discipline. See CLAUDE.md. +# +# Files where historical/comparative mentions are legitimate are listed in +# tombstones-allowlist.txt. Everything else must be clean. +# +# --- Azure Service Bus transport (removed in #159; RabbitMQ is the transport) --- +azureservicebus +azure service bus +azure\.messaging\.servicebus +servicebus\.windows\.net +service bus +# Old ASB subscription names (renamed to queue names without the -sub suffix) +(payment-orders|notify-orders|order-payments|shipping-payments|notify-payments|order-shipping|notify-shipping)-sub diff --git a/.coderabbit.yaml b/.coderabbit.yaml index 3ff2fefe..d3783464 100644 --- a/.coderabbit.yaml +++ b/.coderabbit.yaml @@ -261,7 +261,7 @@ reviews: one EF transaction via AutoApplyTransactions. NextAurora already has the full infrastructure for this — Wolverine + - Service Bus + outbox + saga handlers — so the rule is "use it when a + RabbitMQ + outbox + saga handlers — so the rule is "use it when a handler would otherwise block on minutes-scale work." Reference shape: POST /api/v1/orders (place → publish OrderPlaced → return OrderId immediately; PaymentService + ShippingService handle the downstream @@ -523,6 +523,40 @@ reviews: Same reasoning as the AppHost.cs rule above — reviewers reason against the .svg picture of the request lifecycle. Flag missing pairs. + TRANSPORT-REMOVAL SWEEP. When a messaging/transport package is removed + (e.g. a Wolverine transport swap), removing the `PackageReference` is not + enough — flag two orphans: (1) a dead OpenTelemetry `AddSource("...")` for + that transport's ActivitySource in the `.WithTracing(...)` block here + (silently drops messaging spans from the dashboard), and (2) an orphaned + `PackageVersion` left in `Directory.Packages.props` with no remaining + `PackageReference`. Both bit the ASB→RabbitMQ swap (PR #159). + + - path: "**/Program.cs" + instructions: | + MESSAGING WIRING (Wolverine/RabbitMQ — see CLAUDE.md Package Management + + Communication Patterns traps). (1) Fanout exchanges DISCARD unroutable + messages and AutoProvision declares topology lazily per-service: flag a new + publisher whose consumers' queues aren't guaranteed to exist before first + publish (first-boot loss window — CLAUDE.md trap, tracked #168). (2) + Durability is per-direction: UseDurableOutboxOnAllSendingEndpoints covers + only sends; new ListenToRabbitQueue endpoints without + UseDurableInboxOnAllListeningEndpoints (or ProcessInline for stateless + services) are buffered and lose in-flight messages on crash (tracked #169). + (3) Exchange/queue name literals duplicated between BindExchange().ToQueue() + and ListenToRabbitQueue() must be a single const — a typo'd name is silently + auto-provisioned and the consumer starves with no error. + + - path: "docs/**/*.md" + instructions: | + TOMBSTONED IDENTIFIERS (see CLAUDE.md "Identifier-move discipline"). Flag + any PRESENT-TENSE claim that a removed subsystem is current — the tombstone + list lives in .claude/tombstones.txt (currently: the removed cloud-broker + transport's API names and its old *-sub queue names). Historical/comparative mentions are fine in + the allowlisted decision docs (.claude/tombstones-allowlist.txt). Also flag + docs that assert a metric, trace source, or config key exists when the diff + (or repo) shows nothing emits/reads it — a documented-but-dead signal is + worse than none. + - path: "**" instructions: | FILE-MOVE DISCIPLINE (CLAUDE.md "File-move discipline"). When this PR's diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index f3aed5b2..84c6257e 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -277,7 +277,7 @@ When generating code for NextAurora, follow CLAUDE.md for: - **Coding standards** — `.NET 10` / C# 13, `_` field prefix, `Async` suffix, `I` interface prefix, `var` when type is apparent, `TreatWarningsAsErrors` - **Central Package Management** — `Directory.Packages.props` declares versions; `` in csproj never includes `Version=` - **Static analyzer compliance** — Meziantou, SonarAnalyzer, Roslynator at error severity (notably MA0002: `Dictionary` requires `StringComparer.Ordinal`) -- **Performance Rules** — `AsNoTracking()` + projection on reads, no N+1, async + `CancellationToken` everywhere, pagination caps, bulk ops via `ExecuteUpdateAsync`, optimistic concurrency tokens, **Wolverine transactional outbox** semantics, `DbContext` thread-safety, structured logging templates, no logging in tight loops, brief DB connection holds, cache invalidation in the write path, immutable migrations, measure before optimizing +- **Performance Rules** — `AsNoTracking()` + projection on reads, no N+1, async + `CancellationToken` everywhere, pagination caps, bulk ops via `ExecuteUpdateAsync`, optimistic concurrency tokens, **Wolverine transactional outbox** semantics, `DbContext` thread-safety, structured logging templates, no logging in tight loops, brief DB connection holds, cache invalidation in the write path, immutable migrations, measure before optimizing. See CLAUDE.md. - **Observability & context propagation** — `CorrelationId`/`UserId`/`SessionId` flow via `CorrelationIdMiddleware` (HTTP) and `ContextPropagationMiddleware` (Wolverine). Never extract context manually inside handlers — the middleware handles it - **Authentication** — JWT Bearer + Keycloak realm wired in `NextAurora.ServiceDefaults`; selective `.RequireAuthorization()`; buyer-scope checks on order endpoints - **Error handling** — `GlobalExceptionHandler` returns RFC 7807 ProblemDetails; never expose internal state, IDs, or stack traces to clients @@ -292,14 +292,14 @@ When generating code for NextAurora, follow CLAUDE.md for: ### What changed (so prior guidance doesn't mislead) -The platform migrated to Wolverine for command/query dispatch and event publishing. Anything you may have read about MediatR `IPipelineBehavior`, `LoggingBehavior`, `ValidationBehavior`, hand-rolled `ServiceBusEventPublisher`/`LoggingEventPublisher`, or `EventLogs`/`/admin/events` replay endpoints is **historical** — those are gone. The current pipeline is: +The platform migrated to Wolverine for command/query dispatch and event publishing, with RabbitMQ as the messaging transport in every environment. Anything you may have read about MediatR `IPipelineBehavior`, `LoggingBehavior`, `ValidationBehavior`, hand-rolled `ServiceBusEventPublisher`/`LoggingEventPublisher`, or `EventLogs`/`/admin/events` replay endpoints is **historical** — those are gone. See CLAUDE.md. The current pipeline is: -``` +```text FluentValidation policy (opts.UseFluentValidation) → ContextPropagationMiddleware (logger scope, baggage restore) → handler ``` -Outgoing events go through `WolverineEventPublisher` → `IMessageBus.PublishAsync` → Wolverine transactional outbox (`wolverine` schema in each service's database) → background dispatcher to Azure Service Bus. +Outgoing events publish via the Wolverine-enlisted APIs — the `IMessageContext` parameter injected into `HandleAsync` (handlers) or `IDbContextOutbox` (non-handler code); a constructor-injected `IMessageBus`/`IEventPublisher` publishes inline, OUTSIDE the transaction, and is only for fire-and-forget — → Wolverine transactional outbox (`wolverine` schema in each service's database) → background dispatcher to RabbitMQ (fanout exchanges). See CLAUDE.md. If your suggestion would reintroduce any of the removed concepts above, stop and check CLAUDE.md and the perf guide first. diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index e54f9b45..27104fea 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -237,6 +237,17 @@ jobs: done < <(find docs -maxdepth 2 -type f -name '*.svg' 2>/dev/null || true) exit "$fail" + # Tombstone audit: removed identifiers (a deleted transport's API names, queue + # names, metric names, trace sources) must not survive in docs/comments/config + # as if they were current. The compiler catches stale identifiers in code; + # nothing catches them in prose — the RabbitMQ swap left 15+ docs teaching the + # removed transport as current, found by review instead of mechanically. + # Patterns: .claude/tombstones.txt; historical/comparative exemptions: + # .claude/tombstones-allowlist.txt. See CLAUDE.md "File-and-identifier-move + # discipline". + - name: Tombstone audit — removed identifiers must not resurface + run: .claude/scripts/check-tombstones.sh + # Testcontainers-based integration tests, in their own job: they need Docker (the # ubuntu-latest runner ships it at the standard /var/run/docker.sock, so Testcontainers # auto-detects — no DOCKER_HOST override, unlike macOS Docker Desktop locally). Kept diff --git a/CLAUDE.md b/CLAUDE.md index d27871db..b33a343e 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -4,7 +4,7 @@ ## Project Overview -NextAurora is a .NET 10 microservices e-commerce platform using Aspire, Azure Service Bus, gRPC, EF Core, and Blazor. It follows DDD, CQRS, and event-driven architecture. +NextAurora is a .NET 10 microservices e-commerce platform using Aspire, RabbitMQ (via Wolverine), gRPC, EF Core, and a React SPA storefront. It follows DDD, CQRS, and event-driven architecture. ## Architecture Principles @@ -170,6 +170,8 @@ Two tiers: The original "default to no comments" guidance still applies when *adding new code that doesn't fit tier 1* — don't sprinkle WHAT-comments across plumbing, never leave PR-relative comments ("added for X", "fixes #123"), never comment around well-named identifiers. The tier-1 carve-out is about *teaching the architecture*, not narrating every method. +**Comments link durable docs, not STATUS.md.** STATUS.md is rolling state — its sections churn every few weeks, so a code comment pointing at it dangles silently (the test-factory comments pointed readers at a STATUS.md real-wire-test entry that had moved to dev-loop.md). Point comments at the stable home (a `docs/*.md` section, an issue number); STATUS.md may point at code, never the reverse. + ## Debugging Discipline When a debugging session surfaces a non-obvious failure mode — framework version traps, configuration silently overriding another, an ordering gotcha, an API behavior that contradicts the docs, a backwards-incompatible change between major versions — capture the lesson **before moving on**: @@ -184,6 +186,8 @@ The bar isn't "document every bug fix." It's: **if the failure mode would surpri **File-move discipline — when deleting or renaming a file**, grep the repo for refs to the *old* path and update them in the same PR. Look in: `*.md` (docs, READMEs, INDEX), inline comments in `*.cs`, `Dockerfile*` COPY/ADD lines, `*.csproj` ProjectReference Include, `.github/workflows/*.yml` `run:` blocks, `.coderabbit.yaml` path_instructions. **Same compounding-loop principle as the CLAUDE.md rule above**, just for file structure instead of rule text. Failing to do this is the same drift class as the simplicity refactor fallout — `Dockerfile.catalog` referenced `CatalogService/CatalogService.Api/CatalogService.Api.csproj` for months after PR #31 collapsed CatalogService to a single project, broken silently until someone tried to redeploy. Enforcement is layered: (a) the `check-file-moves.sh` PostToolUse hook prints stale-ref candidates after `git mv` / `git rm`, (b) the CI broken-link audit in `.github/workflows/ci.yml` catches stale markdown links at PR-merge gate, (c) the `.coderabbit.yaml` "**" path_instruction flags missed refs at review time, (d) this rule documents the discipline. +**Identifier-move discipline — the same rule for removed *identifiers*, with a grep-to-zero completion criterion.** The compiler catches stale identifiers in code; **nothing catches them in prose** — so when removing a public identifier (a transport's API names, queue/exchange names, metric names, ActivitySource names, config keys), grep `*.md`, inline comments, `.github/`, `scripts/` for it and sweep every hit in the same PR. **The sweep is done when `git grep` returns zero non-allowlisted hits — not when the docs you remembered are updated** (memory-based sweeps left 15+ docs teaching Azure Service Bus as current after the RabbitMQ swap, plus a metric documented as the DLQ alarm that nothing increments — found by review, not by the loop). Add the removed identifiers to `.claude/tombstones.txt`; the CI tombstone audit (`.claude/scripts/check-tombstones.sh`) then fails any resurfacing; genuinely historical/comparative mentions go in `.claude/tombstones-allowlist.txt` with a justification. + **Doc-and-diagram discipline — docs and diagrams are the review surface, not byproducts.** When reviewers (human or CodeRabbit) want to *observe* what the system does today, they read `docs/architecture.md` and look at `docs/nextaurora-architecture.svg`. If those are stale, every review reasons against a fiction — and drift becomes invisible until someone hits the discrepancy in production. So when code or config changes affect what a doc or diagram *depicts*, the doc/diagram updates in the same PR — or the PR description names the deferred follow-up issue. Concrete pairings (extend as new categories appear): - **Topology changes** (new service / new transport / new DB / new external dep): `NextAurora.AppHost/AppHost.cs` ↔ `docs/architecture.md` ↔ `docs/nextaurora-architecture.{svg,excalidraw}` @@ -226,15 +230,14 @@ This rule is for humans, AI assistants, and future-you. Don't wait to be asked. - Individual `.csproj` files reference packages WITHOUT version attributes - Shared build settings in `Directory.Build.props` - **Aspire SDK and runtime packages must match — including minor versions.** The `Aspire.AppHost.Sdk/X.Y.Z` declared in `NextAurora.AppHost.csproj` and the `Aspire.Hosting.*` package versions in `Directory.Packages.props` need to match exactly (or the SDK ≥ packages). Major mismatches surface at *build/startup* as `TypeLoadException` (internal types like `PublishingContext`). Minor mismatches surface at *runtime* as DCP rejecting startup with `Newer version of the Aspire.Hosting.AppHost package is required`. Bump SDK and packages together as one change. -- **Service Bus subscription names are globally unique within the namespace** (Aspire 13+). Don't reuse the same subscription name on different topics — `DistributedApplicationException` at AppHost startup. Convention: `{consumer}-{source-events}-sub` (e.g. `notify-orders-sub`, `notify-payments-sub`). When adding a new subscription in `AppHost.cs`, also update the matching `ListenToAzureServiceBusSubscription("{topic}/{sub}")` string in the consuming service's `Program.cs`. -- **Aspire 13+ Azure resources need explicit local-dev fallbacks.** `AddAzureServiceBus` requires a chained `.RunAsEmulator()` for local runs (the implicit emulator behavior from Aspire 9 is gone). `AddAzureApplicationInsights` has no local emulator at all — gate it on `builder.ExecutionContext.IsPublishMode` and skip in dev. Without these, AppHost's resource pane shows "Missing subscription configuration" and every service that `WithReference`s the resource fails to start. +- **Aspire 13+ Azure resources need explicit local-dev fallbacks.** `AddAzureApplicationInsights` has no local emulator — gate it on `builder.ExecutionContext.IsPublishMode` and skip in dev. Without that, AppHost's resource pane shows "Missing subscription configuration" and every service that `WithReference`s it fails to start. - **`WithReference(x)` ≠ wait-for-healthy in Aspire 13.** `WithReference` only injects connection strings / endpoints; the service starts as soon as its env vars are resolvable, even if the target is still warming up. Containers like the Service Bus emulator take 30-60s to be healthy on first run; services that race past that crash with "connection refused" and exit. **Hard rule: every `WithReference` on a non-trivial dependency (DB, messaging, identity, peer service) gets a matching `.WaitFor(x)`.** Without it, the Aspire dashboard shows services as "Finished" instead of "Running" because they exited before infra was ready. -- **Wolverine `AutoProvision()` is incompatible with the Service Bus emulator — disable it for local dev.** `AutoProvision()` (default on; `Wolverine:AutoProvision` true) provisions topics/subscriptions at startup via the Service Bus **management API** (`ServiceBusAdministrationClient`). The **emulator implements only the AMQP data plane, not the management API**, so every check fails, retries 4×, and the host dies with `BrokerInitializationException: Unable to initialize the Broker asb in time` — a sub-second, deterministic startup crash hitting *every* Wolverine service (catalog has no ASB, so it survives, which makes it look selective). Locally the topology is already declared in `AppHost.cs` (`AddServiceBusTopic`/`AddServiceBusSubscription` write the emulator's config), so provisioning is impossible *and* redundant. **Fix: AppHost injects `Wolverine__AutoProvision=false` into each Wolverine service** (mirrors the integration-test harnesses); in Publish mode against real Azure it stays on to create entities. See [docs/architecture.md](docs/architecture.md) Infrastructure section. - -## Communication Patterns +- **Messaging transport is RabbitMQ** (every environment — local dev, CI, and the Hetzner deployment, so dev matches prod). Wolverine maps the saga's pub/sub onto **fanout exchanges** (one per event family: `order-events`, `payment-events`, `shipping-events`) with a **queue per consumer** bound to each; `AutoProvision()` declares them against the live broker, gated by `Wolverine:AutoProvision` (default on; off in integration tests, which stub the transport via `DisableAllExternalWolverineTransports()` + a fake `amqp://` connection string). The full saga flows locally — verified end-to-end (place an order → reaches `Shipped` in seconds). The transport is swappable (Wolverine abstracts it — a ~5-line block per service); **Azure Service Bus was evaluated and removed** — its local emulator can't run the saga (subscription admin returns HTTP 500 *and* Wolverine's system queues can't auto-provision against it) and there's no Azure deployment today, so carrying a second wiring earned nothing. Re-add ASB only if Azure becomes a real target. Full story: [docs/full-saga-deployment-plan.md](docs/full-saga-deployment-plan.md) (D3) + #148. +- **Fanout exchanges silently DISCARD unroutable messages — AutoProvision declares topology lazily, per service.** A consumer's queue+binding exists only after that consumer's first boot; an event published before then is dropped by the broker while the outbox marks it delivered (no retry, no DLQ). Every fresh broker (each local AppHost run — no persistent volume — and the first deploy) has this first-boot loss window, because services get only `WaitFor(messaging)`, not inter-service ordering. The old ASB setup pre-declared full topology in AppHost, so this is a swap regression. Known gap, fix tracked: #168 (publisher-side declaration of consumer queues is the preferred shape). +- **Wolverine durability is per-direction: `UseDurableOutboxOnAllSendingEndpoints()` covers ONLY the send side.** Default listeners are *buffered* — the broker is acked as messages enter the in-memory buffer, *before* handlers run, so a consumer crash loses the buffer (a *stopped* service resumes; a *crashing* one loses in-flight messages). Consume-side at-least-once needs `UseDurableInboxOnAllListeningEndpoints()` (message stores already wired for Order/Payment/Shipping) or `ProcessInline()` (Notification — no store). Known gap, fix tracked: #169. Until both #168 and #169 land, treat "can't lose a message on either side" as publish-side-only. -- **Async events** (Azure Service Bus): For workflow orchestration (order -> payment -> shipping -> notification) -- **Durability ≠ replay — don't reach for a stream just to avoid losing messages.** A common misconception is "pub/sub loses the message if a subscriber is down, so use a stream (Kafka/Event Hubs) when you can't afford loss." Not so: message loss depends on whether the subscription is **durable**, not on queue-vs-stream. The misconception comes from **Redis Pub/Sub specifically**, which *is* fire-and-forget — no persistence, a subscriber that's down when a message publishes misses it forever; there, the user's intuition is right and Redis **Streams** (persistent, consumer-group offsets, replayable) is the durable answer *within the Redis ecosystem*. But that's a property of Redis Pub/Sub being non-durable, not of pub/sub as a pattern: **durable** pub/sub (Azure Service Bus topics+subscriptions, RabbitMQ durable queues, AWS SNS→SQS) does NOT lose messages — the broker persists per-subscriber until ack. NextAurora's stack already can't lose a message on either side — the **transactional outbox** guards the publish side (event persisted in the same DB transaction as the entity write, dispatched with retry, so "entity saved but event lost" can't happen), and **durable Service Bus subscriptions / RabbitMQ durable queues** guard the consume side (the broker holds the message per-subscriber until ack, so a down service resumes from where it left off). At-least-once delivery means the real risk is *duplication, not loss* — which is why every handler is idempotent (see "Key Conventions: Event handlers must be idempotent"). **Reach for a stream (Kafka, Azure Event Hubs, Redis Streams) only when you need what a durable queue can't give: replay from an offset, multi-day retention, an ordered append-only event log, or N independent consumers each re-reading history at their own pace** — *not* merely "don't lose messages." NextAurora has no such need today (it deliberately deleted the hand-rolled `EventLogs` replay table; any future replay rides Wolverine's own message store). Adding a stream to prevent loss the outbox + durable-queue + idempotency stack already prevents is the same speculative over-engineering the factory-pattern rule warns against. Full transport-selection decision guide (Redis Pub/Sub vs Streams vs RabbitMQ vs ASB vs SNS+SQS vs Kafka/Event Hubs — portable across systems): [docs/messaging-transport-selection.md](docs/messaging-transport-selection.md). +- **Async events** (RabbitMQ via Wolverine): For workflow orchestration (order -> payment -> shipping -> notification) +- **Durability ≠ replay — don't reach for a stream just to avoid losing messages.** A common misconception is "pub/sub loses the message if a subscriber is down, so use a stream (Kafka/Event Hubs) when you can't afford loss." Not so: message loss depends on whether the subscription is **durable**, not on queue-vs-stream. The misconception comes from **Redis Pub/Sub specifically**, which *is* fire-and-forget — no persistence, a subscriber that's down when a message publishes misses it forever; there, the user's intuition is right and Redis **Streams** (persistent, consumer-group offsets, replayable) is the durable answer *within the Redis ecosystem*. But that's a property of Redis Pub/Sub being non-durable, not of pub/sub as a pattern: **durable** pub/sub (Azure Service Bus topics+subscriptions, RabbitMQ durable queues, AWS SNS→SQS) does NOT lose messages — the broker persists per-subscriber until ack. NextAurora's stack is built so a message can't be lost on either side (publish side fully guarded today; consume side once #168/#169 land — see the trap bullets above) — the **transactional outbox** guards the publish side (event persisted in the same DB transaction as the entity write, dispatched with retry, so "entity saved but event lost" can't happen), and **RabbitMQ durable queues** guard the consume side (the broker holds the message per-consumer-queue until ack, so a down service resumes from where it left off — with the two caveats in the trap bullets above: lazily-declared topology (#168) and buffered listeners (#169) currently weaken this to publish-side-only). At-least-once delivery means the real risk is *duplication, not loss* — which is why every handler is idempotent (see "Key Conventions: Event handlers must be idempotent"). **Reach for a stream (Kafka, Azure Event Hubs, Redis Streams) only when you need what a durable queue can't give: replay from an offset, multi-day retention, an ordered append-only event log, or N independent consumers each re-reading history at their own pace** — *not* merely "don't lose messages." NextAurora has no such need today (it deliberately deleted the hand-rolled `EventLogs` replay table; any future replay rides Wolverine's own message store). Adding a stream to prevent loss the outbox + durable-queue + idempotency stack already prevents is the same speculative over-engineering the factory-pattern rule warns against. Full transport-selection decision guide (Redis Pub/Sub vs Streams vs RabbitMQ vs ASB vs SNS+SQS vs Kafka/Event Hubs — portable across systems): [docs/messaging-transport-selection.md](docs/messaging-transport-selection.md). - **gRPC** (sync): For real-time queries between services (OrderService -> CatalogService product validation). gRPC is versioned separately via `.proto` `package` declarations. - **REST** (HTTP): For frontend-to-service communication only. URL-segment versioned via `Asp.Versioning.Http` — every endpoint lives under `/api/v{version}/...`. Default version is `1.0`; the version segment is required (`AssumeDefaultVersionWhenUnspecified = false`). **Always use `app.MapV1ApiGroup("Tag", "resource")`** (helper in `NextAurora.ServiceDefaults`) to register a versioned route group — it returns a `RouteGroupBuilder` rooted at `/api/v1/resource` and applies the version + tag in one call. Don't hand-roll `NewVersionedApi(...).MapGroup(...).HasApiVersion(...)` chains — drift across services is the failure mode. To add v2 later, register a side-by-side group with `.HasApiVersion(new ApiVersion(2, 0))`; v1 keeps working untouched. - **No `IRequestHandler` / `IFooHandler` interface per handler.** Handlers are plain classes (e.g. `PlaceOrderHandler` in `OrderService/Features/PlaceOrder.cs`). Wolverine assembly-scans `Features/` and binds message-type → handler-type directly via `opts.Discovery.IncludeAssembly(...)` — no `IRequestHandler`-style shim is required. The MediatR-style *"one interface per handler for testability / discoverability"* pattern doesn't apply: Wolverine's bus *is* the abstraction, and tests resolve the handler concretely (see the *"Handlers resolved directly in tests must be DI-registered"* rule below). Avoid introducing handler interfaces speculatively — they fail the *"consumer substitution"* test in the same way `IFooRepository` did. @@ -284,7 +287,7 @@ These are always-on headlines. **Full rationale, edge cases, and worked examples - **Test structure — AAA with narrative comments.** Every test is structured as **Arrange → Act → Assert** with `// ARRANGE`, `// ACT`, `// ASSERT` markers (all caps, em-dash explanation on the same line). Each phase carries a *story comment*: explain what's being set up and *why it matters*, what's being called, and what each assertion is verifying. A junior dev should be able to read a single test top-to-bottom and understand the contract being checked + the failure mode being guarded against — without having to read the SUT first. When the ASSERT phase verifies multiple invariants, number them and explain why each matters (especially for security boundaries, idempotency guards, and ordering-sensitive operations like cache-after-save). Trivial happy-path tests can be shorter; security/concurrency/idempotency tests get the full story. Reference templates: [ProductAuthorizationTests.cs](tests/CatalogService.Tests.Integration/ProductAuthorizationTests.cs) (IDOR-prevention + rejected-write invariants, integration-tier), [OrderSagaTests.cs](tests/OrderService.Tests.Integration/OrderSagaTests.cs) (saga consume-side idempotency under at-least-once delivery, integration-tier), [OrderTests.cs](tests/OrderService.Tests.Unit/Domain/OrderTests.cs) (aggregate state-machine invariants, unit-tier — domain-only, no `DbContext`). - Integration tests with Testcontainers for infrastructure — `tests/{Service}.Tests.Integration`, booting the real API via `WebApplicationFactory`. Four service slices ship and each runs as its own step in `.github/workflows/ci.yml` so a single-slice failure doesn't mask the rest: **CatalogService** (Postgres + Redis — caching + concurrency token + IDOR), **OrderService** (SQL Server + Wolverine stubbed transport — outbox, saga handlers, `RowVersion` token), **PaymentService** (SQL Server — outbox-in-non-handler wrap, retry/recovery job), **ShippingService** (Postgres — IDOR-by-order saga handlers). Pattern documented in each project's README. - **Integration tests need Docker.** On macOS, Docker Desktop's socket is at `~/.docker/run/docker.sock`, not `/var/run/docker.sock` — Testcontainers fails fast with `DockerUnavailableException` unless `DOCKER_HOST` points there (or Docker Desktop's "default Docker socket" toggle is on). CI runners have it at the standard path. -- **Fake credentials in test fixtures are suppressed with inline `// gitleaks:allow` markers — there is no project-level gitleaks config.** Some factory fakes have to be protocol-syntax-valid: the fake Azure Service Bus connection string in `OrderApiFactory.cs` / `PaymentApiFactory.cs` / `ShippingApiFactory.cs` is required because `Program.cs` parses `UseAzureServiceBus(GetConnectionString("messaging"))` eagerly at registration time, *before* `DisableAllExternalWolverineTransports()` stubs the transport. The convention: keep the high-entropy base64-encoded **self-labeling** literal (the project's fake `SharedAccessKey` base64-decodes to `fake-shared-key-for-testing-only`) and add `// gitleaks:allow` to the end of the line containing the literal. Gitleaks' default `generic-api-key` rule fires on the entropy of `key=value` strings regardless of value content, so realistic-looking fakes get flagged at PR-scan time. The fix is the **inline marker**, not lowering the fake's entropy below scanner thresholds. **Why no `.gitleaks.toml` lives at repo root:** a path+regex `[[allowlists]]` block was tried and removed — global `[[allowlists]]` requires gitleaks ≥ 8.25.0 (the action ships 8.24.x) and `MatchCondition` defaults to OR not AND, so the block both failed to fire on the pinned scanner version AND would have over-matched if it did. The inline marker is reliable across all 8.x versions, scoped to the exact line, and self-documenting at the call site. Pasting a real key into the same file still trips because the marker only suppresses THAT specific finding line, and pasting the fake literal anywhere without the marker still trips. Do NOT also reproduce the high-entropy literal in CLAUDE.md or any other prose — the scanner walks every diff, including documentation. **The diff-range residue trap:** gitleaks scans every commit in the PR range, not just HEAD, so a fix-in-a-later-commit doesn't suppress findings introduced un-marked in an earlier commit — squash the branch if you hit this. Reference: any of the three factory files for the marker pattern; [.github/workflows/gitleaks.yml](.github/workflows/gitleaks.yml) for the CI wiring. +- **Fake connection strings in test fixtures must be protocol-syntax-valid; prefer LOW-entropy fakes; high-entropy fakes need inline `// gitleaks:allow` markers.** The factory fakes exist because each `Program.cs` parses `UseRabbitMq(GetConnectionString("messaging"))` eagerly at registration time, *before* `DisableAllExternalWolverineTransports()` stubs the transport — today's fake is the plain low-entropy `amqp://guest:guest@localhost:5672`, which gitleaks never flags, so **no markers are currently needed anywhere**. If a future fixture genuinely requires a high-entropy fake (e.g. a base64 key a parser validates): make the literal **self-labeling** (decodes to something like `fake-...-for-testing-only`), suppress with `// gitleaks:allow` on that exact line — NOT a repo-level `.gitleaks.toml` (global `[[allowlists]]` needs gitleaks ≥ 8.25.0, the action ships 8.24.x, and `MatchCondition` ORs instead of ANDs — tried and removed; the inline marker is version-proof, line-scoped, self-documenting). Never reproduce a high-entropy literal in CLAUDE.md or docs prose (the scanner walks every diff). **Diff-range residue trap:** gitleaks scans every commit in the PR range, not just HEAD — a fix-in-a-later-commit doesn't suppress an un-marked earlier commit; squash the branch if you hit this. CI wiring: [.github/workflows/gitleaks.yml](.github/workflows/gitleaks.yml). - **IDOR test is required for every new endpoint that returns or mutates a scoped entity.** Add an integration test that authenticates as buyer X, requests a resource owned by buyer Y, and asserts the response is 404 (NOT 200, NOT 403 — see Security Requirements). The absence of such a test is exactly how the original `GET /api/v1/orders/{id}` IDOR survived undetected for the lifetime of the codebase. A `dotnet build` clean and unit tests passing aren't sufficient — *authorization behavior is only proven by an authorization-failure test*. - **Outbox-in-non-handler test.** Code paths that publish events from outside a Wolverine handler (BackgroundService sweepers, recovery jobs) need an integration test that asserts a row appears in `wolverine.outgoing_envelopes` in the same transaction as the entity write. See the outbox-outside-handler trap in Observability → Transactional Outbox. The PaymentRecoveryJob outbox bug survived because no test asserted that the staged envelope actually persisted. - **Handlers resolved directly in tests must be DI-registered.** If an integration test does `scope.ServiceProvider.GetRequiredService()`, the handler must be `AddScoped()`'d in `AddXInfrastructure`. Wolverine's handler-discovery does not populate `IServiceCollection` — it builds its own internal map. See "Communication Patterns → Wolverine handler discovery is NOT DI registration" for the full mechanism. Failure mode: `InvalidOperationException: No service for type 'X.Handler' has been registered` at first test run. Catch this at PR review time, not in CI — `/check-rules` audit pattern for the test diff: every `GetRequiredService<*Handler>()` line needs a matching `AddScoped<*Handler>()` in DI. diff --git a/CatalogService/Infrastructure/Data/CatalogDbContext.cs b/CatalogService/Infrastructure/Data/CatalogDbContext.cs index 64cdf0dc..ed7f5007 100644 --- a/CatalogService/Infrastructure/Data/CatalogDbContext.cs +++ b/CatalogService/Infrastructure/Data/CatalogDbContext.cs @@ -15,7 +15,7 @@ namespace CatalogService.Infrastructure.Data; /// EF then includes WHERE xmin = @originalXmin on every UPDATE; if another transaction /// touched the row first, the WHERE matches zero rows and EF throws /// . The handler layer catches that and either retries -/// (Service Bus events) or returns 409 Conflict (HTTP) — see GlobalExceptionHandler and +/// (RabbitMQ events) or returns 409 Conflict (HTTP) — see GlobalExceptionHandler and /// AddConcurrencyRetry. Net result: last-write-wins is impossible. /// /// diff --git a/Directory.Packages.props b/Directory.Packages.props index b7ec97a7..683cd4ed 100644 --- a/Directory.Packages.props +++ b/Directory.Packages.props @@ -14,8 +14,8 @@ StreamJsonRpc ships with MessagePack >= 2.5.301. --> - + @@ -36,7 +36,6 @@ - - + diff --git a/NextAurora.AppHost/AppHost.cs b/NextAurora.AppHost/AppHost.cs index 7474af89..cedef936 100644 --- a/NextAurora.AppHost/AppHost.cs +++ b/NextAurora.AppHost/AppHost.cs @@ -2,7 +2,7 @@ using Aspire.Hosting.Azure; // .NET Aspire orchestration root for NextAurora. Defines the entire local-development topology: -// every container (Postgres, SQL Server, Redis, Service Bus emulator, Keycloak, App Insights), +// every container (Postgres, SQL Server, Redis, RabbitMQ, Keycloak, App Insights), // every service project, and every dependency edge between them. When you run this project, // Aspire spins up the containers, sets per-service connection strings/auth via environment // variables (the magic behind `WithReference`), boots the services in dependency order, and @@ -13,7 +13,6 @@ // it's stated here. That makes it easy to reason about what runs in any environment. const string keycloakHostname = "http://localhost:8080/"; -const string disableAutoProvisionValue = "false"; var builder = DistributedApplication.CreateBuilder(args); @@ -37,36 +36,14 @@ // once we wire it (architecture.md "Future Considerations"). var redis = builder.AddRedis("cache"); -// Azure Service Bus — Aspire 13 requires explicit `.RunAsEmulator()` for local dev (in 9.x -// the emulator was implicit). Without this, AppHost reports "Missing subscription configuration" -// at startup because Aspire treats the resource as needing a real Azure subscription. -// See CLAUDE.md. -var serviceBus = builder.AddAzureServiceBus("messaging") - .RunAsEmulator(); - -// Topic / subscription topology. Each service that publishes events owns a topic; subscribers -// get their own subscription per topic so they can be scaled and dead-lettered independently. -// -// Subscription naming: `{consumer}-{source-events}-sub`. Aspire 13 requires subscription names -// to be globally unique within the bus namespace (not scoped per topic), hence the source -// suffix. The strings here must match the `ListenToAzureServiceBusSubscription("{topic}/{sub}")` -// calls in each service's Program.cs. -var orderEventsTopic = serviceBus.AddServiceBusTopic("order-events"); -orderEventsTopic.AddServiceBusSubscription("payment-orders-sub"); // PaymentService consumes -orderEventsTopic.AddServiceBusSubscription("notify-orders-sub"); // NotificationService consumes - -var paymentEventsTopic = serviceBus.AddServiceBusTopic("payment-events"); -paymentEventsTopic.AddServiceBusSubscription("order-payments-sub"); // OrderService consumes -paymentEventsTopic.AddServiceBusSubscription("shipping-payments-sub"); // ShippingService consumes -paymentEventsTopic.AddServiceBusSubscription("notify-payments-sub"); // NotificationService consumes (failure notifications) - -var shippingEventsTopic = serviceBus.AddServiceBusTopic("shipping-events"); -shippingEventsTopic.AddServiceBusSubscription("order-shipping-sub"); // OrderService consumes -shippingEventsTopic.AddServiceBusSubscription("notify-shipping-sub"); // NotificationService consumes - -// Direct queue (not topic) for "send a notification right now" requests — single consumer, -// fan-in only. NotificationService listens here in addition to the topic subscriptions. -serviceBus.AddServiceBusQueue("send-notification"); +// --- Messaging broker: RabbitMQ --- +// One container + the management UI (a demo artifact at :15672). Wolverine declares the +// exchanges/queues/bindings itself via AutoProvision against the live broker, so no topology is +// hand-declared here. RabbitMQ is also the deployed broker (Hetzner) — dev/prod parity. The +// connection string is exposed under "messaging". The transport stays swappable (~5-line Wolverine +// block per service); the previous cloud-broker wiring was evaluated and removed — its emulator +// couldn't run the saga locally and there's no Azure deployment today. See CLAUDE.md + docs/full-saga-deployment-plan.md (D3) + #148. +var messaging = builder.AddRabbitMQ("messaging").WithManagementPlugin(); // Application Insights only when running in Publish mode (i.e. real deploys to Azure). // Aspire 13 has no local emulator for App Insights — keeping this in for local dev causes @@ -118,48 +95,37 @@ static IResourceBuilder WithOptionalAppInsights( .WithReference(realm, configurationPrefix: keycloakConfigPrefix).WaitFor(realm) .WithEnvironment("Frontend__AllowedOrigins", SpaDevOrigin); -// Wolverine's AutoProvision() uses the Service Bus *management* API (ServiceBusAdministrationClient) -// to create/verify topics + subscriptions at host startup. The Service Bus emulator does NOT -// implement the management API — only the AMQP data plane — so AutoProvision retries, times out, -// and the host dies with `BrokerInitializationException: Unable to initialize the Broker asb in -// time`. Locally the topology is already declared above (AddServiceBusTopic/AddServiceBusSubscription -// write the emulator's config), so provisioning is both impossible AND redundant. Disable it for -// the four Wolverine services in dev; in Publish mode against real Azure, AutoProvision stays on -// (Wolverine:AutoProvision defaults true) to create the entities. Mirrors the test harnesses' -// `Wolverine:AutoProvision=false`. See CLAUDE.md. -const string disableAutoProvision = "Wolverine__AutoProvision"; +// Wires a Wolverine service to RabbitMQ: connection-string reference + WaitFor (Aspire 13 needs an +// explicit WaitFor for health). Wolverine AutoProvisions its exchanges/queues against the live +// broker at startup. See CLAUDE.md. +IResourceBuilder WithMessaging(IResourceBuilder project) +{ + return project.WithReference(messaging).WaitFor(messaging); +} // OrderService also references catalogService — that gives it the gRPC client config to call // into Catalog for product validation during order placement. -var orderService = WithOptionalAppInsights( +var orderService = WithMessaging(WithOptionalAppInsights( builder.AddProject("order-service") .WithReference(ordersDb).WaitFor(ordersDb) - .WithReference(serviceBus).WaitFor(serviceBus) .WithReference(catalogService).WaitFor(catalogService), appInsights) .WithReference(realm, configurationPrefix: keycloakConfigPrefix).WaitFor(realm) - .WithEnvironment("Frontend__AllowedOrigins", SpaDevOrigin) - .WithEnvironment(disableAutoProvision, disableAutoProvisionValue); + .WithEnvironment("Frontend__AllowedOrigins", SpaDevOrigin)); -WithOptionalAppInsights( +WithMessaging(WithOptionalAppInsights( builder.AddProject("payment-service") - .WithReference(paymentsDb).WaitFor(paymentsDb) - .WithReference(serviceBus).WaitFor(serviceBus), appInsights) - .WithReference(realm, configurationPrefix: keycloakConfigPrefix).WaitFor(realm) - .WithEnvironment(disableAutoProvision, disableAutoProvisionValue); + .WithReference(paymentsDb).WaitFor(paymentsDb), appInsights) + .WithReference(realm, configurationPrefix: keycloakConfigPrefix).WaitFor(realm)); -WithOptionalAppInsights( +WithMessaging(WithOptionalAppInsights( builder.AddProject("shipping-service") - .WithReference(shippingDb).WaitFor(shippingDb) - .WithReference(serviceBus).WaitFor(serviceBus), appInsights) + .WithReference(shippingDb).WaitFor(shippingDb), appInsights) .WithReference(realm, configurationPrefix: keycloakConfigPrefix).WaitFor(realm) - .WithEnvironment("Frontend__AllowedOrigins", SpaDevOrigin) - .WithEnvironment(disableAutoProvision, disableAutoProvisionValue); + .WithEnvironment("Frontend__AllowedOrigins", SpaDevOrigin)); // NotificationService is stateless — no DB reference, just messaging + telemetry. -WithOptionalAppInsights( - builder.AddProject("notification-service") - .WithReference(serviceBus).WaitFor(serviceBus), appInsights) - .WithEnvironment(disableAutoProvision, disableAutoProvisionValue); +WithMessaging(WithOptionalAppInsights( + builder.AddProject("notification-service"), appInsights)); // --- Frontend --- // Storefront and SellerPortal reference the API services so service-discovery resolves diff --git a/NextAurora.AppHost/NextAurora.AppHost.csproj b/NextAurora.AppHost/NextAurora.AppHost.csproj index 6cb7b16a..4acc83fb 100644 --- a/NextAurora.AppHost/NextAurora.AppHost.csproj +++ b/NextAurora.AppHost/NextAurora.AppHost.csproj @@ -8,8 +8,8 @@ + - diff --git a/NextAurora.ServiceDefaults/Extensions.cs b/NextAurora.ServiceDefaults/Extensions.cs index 9d46aa47..6887772e 100644 --- a/NextAurora.ServiceDefaults/Extensions.cs +++ b/NextAurora.ServiceDefaults/Extensions.cs @@ -1,453 +1,456 @@ -using Asp.Versioning; -using JasperFx.CodeGeneration.Model; -using JasperFx.Core; -using Microsoft.AspNetCore.Authentication.JwtBearer; -using Microsoft.AspNetCore.Builder; -using Microsoft.AspNetCore.Diagnostics.HealthChecks; -using Microsoft.AspNetCore.Http; -using Microsoft.AspNetCore.Routing; -using Microsoft.EntityFrameworkCore; -using Microsoft.Extensions.Configuration; -using Microsoft.Extensions.DependencyInjection; -using Microsoft.Extensions.Diagnostics.HealthChecks; -using Microsoft.Extensions.Logging; -using Microsoft.Extensions.ServiceDiscovery; -using NextAurora.ServiceDefaults.Messaging; -using NextAurora.ServiceDefaults.Metrics; -using NextAurora.ServiceDefaults.Middleware; -using OpenTelemetry; -using OpenTelemetry.Metrics; -using OpenTelemetry.Trace; -using Wolverine; -using Wolverine.ErrorHandling; - -namespace Microsoft.Extensions.Hosting; - -// Adds common Aspire services: service discovery, resilience, health checks, and OpenTelemetry. -// This project should be referenced by each service project in your solution. -// To learn more about using this project, see https://aka.ms/dotnet/aspire/service-defaults -public static class Extensions -{ - private const string HealthEndpointPath = "/health"; - private const string AlivenessEndpointPath = "/alive"; - - public static TBuilder AddServiceDefaults(this TBuilder builder) where TBuilder : IHostApplicationBuilder - { - builder.ConfigureOpenTelemetry(); - - builder.AddDefaultHealthChecks(); - - builder.Services.AddExceptionHandler(); - builder.Services.AddProblemDetails(); - - builder.Services.AddSingleton(); - - builder.Services.AddServiceDiscovery(); - - builder.AddDefaultAuthentication(); - - builder.AddFrontendCors(); - - builder.AddNextAuroraApiVersioning(); - - builder.Services.ConfigureHttpClientDefaults(http => - { - // Turn on resilience by default - http.AddStandardResilienceHandler(); - - // Turn on service discovery by default - http.AddServiceDiscovery(); - }); - - return builder; - } - - public static TBuilder ConfigureOpenTelemetry(this TBuilder builder) where TBuilder : IHostApplicationBuilder - { - builder.Logging.AddOpenTelemetry(logging => - { - logging.IncludeFormattedMessage = true; - logging.IncludeScopes = true; - }); - - builder.Services.AddOpenTelemetry() - .WithMetrics(metrics => - { - metrics.AddAspNetCoreInstrumentation() - .AddHttpClientInstrumentation() - .AddRuntimeInstrumentation() - .AddMeter("NextAurora"); - }) - .WithTracing(tracing => - { - tracing.AddSource(builder.Environment.ApplicationName) - .AddSource("Azure.Messaging.ServiceBus") - // "NextAurora.Messaging" is the ActivitySource used by all Service Bus - // processors. Registering it here causes consumer spans to appear in the - // Aspire dashboard and any connected distributed tracing backend. - .AddSource("NextAurora.Messaging") - .AddAspNetCoreInstrumentation(tracing => - // Exclude health check requests from tracing - tracing.Filter = context => - !context.Request.Path.StartsWithSegments(HealthEndpointPath, StringComparison.OrdinalIgnoreCase) - && !context.Request.Path.StartsWithSegments(AlivenessEndpointPath, StringComparison.OrdinalIgnoreCase) - ) - .AddGrpcClientInstrumentation() - .AddHttpClientInstrumentation(); - }); - - builder.AddOpenTelemetryExporters(); - - return builder; - } - - private static void AddOpenTelemetryExporters(this TBuilder builder) where TBuilder : IHostApplicationBuilder - { - var useOtlpExporter = !string.IsNullOrWhiteSpace(builder.Configuration["OTEL_EXPORTER_OTLP_ENDPOINT"]); - - if (useOtlpExporter) - { - builder.Services.AddOpenTelemetry().UseOtlpExporter(); - } - } - - public static TBuilder AddDefaultHealthChecks(this TBuilder builder) where TBuilder : IHostApplicationBuilder - { - builder.Services.AddHealthChecks() - // Add a default liveness check to ensure app is responsive - .AddCheck("self", () => HealthCheckResult.Healthy(), ["live"]); - - return builder; - } - - public static WebApplication MapDefaultEndpoints(this WebApplication app) - { - // ORDER MATTERS. CorrelationIdMiddleware sits BETWEEN UseAuthentication and - // UseAuthorization for two reasons: - // - // 1. It reads ClaimTypes.NameIdentifier from context.User to populate the - // UserId scope key. UseAuthentication populates context.User, so the - // middleware must run AFTER it. Otherwise UserId is silently always null. - // - // 2. Placing it BEFORE UseAuthorization (rather than after) means the - // UserId scope is active during the Authorization step itself. Any - // 401/403 denials get logged with the authenticated user's ID — - // preserving the audit trail for "user X tried to access resource they - // shouldn't." Running after Authorization would drop that signal, - // losing visibility into unauthorized-attempt patterns. - // - // UseExceptionHandler stays first so it wraps every error below. UseCors runs - // before UseAuthentication so CORS preflights (OPTIONS, which never carry a - // bearer token) are answered without touching the auth pipeline. The policy is - // only registered when Frontend:AllowedOrigins is configured — services with no - // browser-facing surface skip the middleware entirely. - app.UseExceptionHandler(); - if (!string.IsNullOrWhiteSpace(app.Configuration["Frontend:AllowedOrigins"])) - { - app.UseCors(FrontendCorsPolicyName); - } - app.UseAuthentication(); - app.UseMiddleware(); - app.UseAuthorization(); - - // All health checks must pass for app to be considered ready to accept traffic after starting - app.MapHealthChecks(HealthEndpointPath); - - // Only health checks tagged with the "live" tag must pass for app to be considered alive - app.MapHealthChecks(AlivenessEndpointPath, new HealthCheckOptions - { - Predicate = r => r.Tags.Contains("live") - }); - - return app; - } - - private const string FrontendCorsPolicyName = "frontend"; - - /// - /// Explicit CORS for the browser SPA — registered ONLY when Frontend:AllowedOrigins - /// is configured (semicolon-separated origin list, injected per service by AppHost). - /// Per CLAUDE.md "Security Requirements → CORS": explicit policy, known origins only — - /// never AllowAnyOrigin. X-Correlation-Id is exposed so the SPA's - /// observability surface can read the correlation ID off every response; without - /// WithExposedHeaders, browsers hide non-safelisted response headers from JS even - /// when the request itself succeeds. - /// - private static void AddFrontendCors(this TBuilder builder) where TBuilder : IHostApplicationBuilder - { - var configured = builder.Configuration["Frontend:AllowedOrigins"]; - if (string.IsNullOrWhiteSpace(configured)) - { - return; - } - - var origins = configured.Split(';', StringSplitOptions.RemoveEmptyEntries | StringSplitOptions.TrimEntries); - - builder.Services.AddCors(options => options.AddPolicy(FrontendCorsPolicyName, policy => policy - .WithOrigins(origins) - .AllowAnyHeader() - .AllowAnyMethod() - .WithExposedHeaders("X-Correlation-Id"))); - } - - private static void AddDefaultAuthentication(this TBuilder builder) where TBuilder : IHostApplicationBuilder - { - // Authority resolution, in priority order: - // 1. Authentication:Authority — explicit override (prod / appsettings). - // 2. Keycloak:AuthServerUrl + Keycloak:Realm — what the Aspire Keycloak.AuthServices - // hosting integration injects from `WithReference(realm, configurationPrefix: "Keycloak")`. - // It does NOT inject a single `Keycloak:Url`; it injects the base server URL and the - // realm name separately, and the OIDC authority is `{AuthServerUrl}/realms/{Realm}`. - // Reading the injected keys (not a hardcoded URL) keeps this correct across Aspire's - // dynamic endpoints. Both the SPA and the API resolve to the same base, so the token's - // issuer matches the validated authority (Keycloak's issuer is host-reflective). See CLAUDE.md. - var authority = builder.Configuration["Authentication:Authority"]; - if (string.IsNullOrEmpty(authority)) - { - var authServerUrl = builder.Configuration["Keycloak:AuthServerUrl"]?.TrimEnd('/'); - var realm = builder.Configuration["Keycloak:Realm"]; - if (!string.IsNullOrEmpty(authServerUrl) && !string.IsNullOrEmpty(realm)) - { - authority = $"{authServerUrl}/realms/{realm}"; - } - } - - if (string.IsNullOrEmpty(authority)) - { - // No identity provider configured — register auth services with no-op defaults - // so UseAuthentication/UseAuthorization don't throw, but no tokens are validated. - builder.Services.AddAuthentication(); - builder.Services.AddAuthorization(); - return; - } - - // RequireHttpsMetadata is fail-closed outside Development. An http authority in - // Production must fail LOUDLY at startup, not silently fetch OIDC discovery + JWKS - // over plaintext (an active MITM could inject signing keys and forge tokens every - // service would accept). The explicit Authentication:RequireHttpsMetadata=false key - // is the auditable opt-out for legitimate internal-http deployments (e.g. Keycloak - // behind a service mesh); the default derives it only for local dev. See CLAUDE.md. - var requireHttpsMetadata = builder.Configuration.GetValue("Authentication:RequireHttpsMetadata", (bool?)null); - if (!requireHttpsMetadata.HasValue) - { - requireHttpsMetadata = authority.StartsWith("https://", StringComparison.OrdinalIgnoreCase) - || !builder.Environment.IsDevelopment(); - } - - if (!requireHttpsMetadata.Value) - { - // An opt-out (derived or explicit) should be visible in the app's logs, never - // silent. PostConfigure runs when the options are first resolved, after the DI - // logging pipeline exists — so this lands in the real log stream, not a side channel. - builder.Services.AddOptions(JwtBearerDefaults.AuthenticationScheme) - .PostConfigure((_, loggerFactory) => - loggerFactory.CreateLogger("NextAurora.ServiceDefaults.Authentication") - .LogWarning("JWT bearer RequireHttpsMetadata is DISABLED (authority: {Authority}) — OIDC metadata/JWKS are fetched without TLS. Acceptable for local dev only.", authority)); - } - - builder.Services.AddAuthentication(JwtBearerDefaults.AuthenticationScheme) - .AddJwtBearer(options => - { - options.Authority = authority; - options.Audience = builder.Configuration["Authentication:Audience"] ?? "nextaurora-api"; - options.RequireHttpsMetadata = requireHttpsMetadata.Value; - options.TokenValidationParameters = new Microsoft.IdentityModel.Tokens.TokenValidationParameters - { - ValidateAudience = true, - ValidateIssuer = true, - ValidateLifetime = true, - // Explicit — JWT Bearer's implicit default already validates the - // signature against JWKS-discovered keys, but making it explicit - // makes the security posture auditable + prevents a future config - // change from accidentally disabling signature validation. - ValidateIssuerSigningKey = true, - // Default ClockSkew is 5 minutes — revoked/expired tokens remain - // accepted for 5 extra minutes. The realm pins 5-MINUTE access tokens - // (nextaurora-realm.json accessTokenLifespan: 300), so the default skew - // would double every token's effective lifetime. 30 seconds covers - // reasonable inter-server clock drift without a long replay window. - ClockSkew = TimeSpan.FromSeconds(30), - NameClaimType = "preferred_username", - RoleClaimType = "realm_access.roles", - }; - }); - - builder.Services.AddAuthorization(); - } - - /// - /// URL-segment API versioning. Default version is 1.0; clients must include the version in - /// the route (`/api/v1/...`). The ApiExplorer integration makes versioned endpoints visible - /// in OpenAPI docs with version-aware group names. Called automatically by - /// . - /// - /// - /// Why URL-segment over header-based versioning: URL versioning is what most public - /// REST APIs (Stripe, GitHub, Twitter, AWS) use. Pros: visible in logs/dashboards, cacheable - /// (HTTP caches key on URL), debuggable from a browser, plays well with Swagger. The - /// "header versioning is more RESTful" argument is academic — the practical wins of URL - /// versioning dominate. - /// - /// - /// Why AssumeDefaultVersionWhenUnspecified = false: the version segment is - /// required. Hitting /api/products returns 400, not silent v1. This avoids the - /// classic mistake of silently treating un-versioned calls as v1, which makes future v2 - /// migrations a behavior-change debugging nightmare. - /// - /// - private static void AddNextAuroraApiVersioning(this TBuilder builder) where TBuilder : IHostApplicationBuilder - { - builder.Services - .AddApiVersioning(options => - { - options.DefaultApiVersion = new ApiVersion(1, 0); - options.AssumeDefaultVersionWhenUnspecified = false; - options.ReportApiVersions = true; - options.ApiVersionReader = new UrlSegmentApiVersionReader(); - }) - .AddApiExplorer(options => - { - // 'v'VVV → groups become "v1", "v2", "v1.1" etc. in OpenAPI. - options.GroupNameFormat = "'v'VVV"; - // Replace `{version:apiVersion}` placeholder with the actual version in - // generated docs/URLs (so Swagger UI shows "/api/v1/products" not the template). - options.SubstituteApiVersionInUrl = true; - }); - } - - /// - /// Canonical helper for registering a versioned route group at - /// /api/v{version}/{template}. Every endpoint group in every service goes through - /// this — that way the policy (default version, tag application, route prefix) can't drift - /// across services. Equivalent to: - /// - /// app.NewVersionedApi(tag) - /// .MapGroup("/api/v{version:apiVersion}/" + template) - /// .HasApiVersion(new ApiVersion(1, 0)) - /// .WithTags(tag); - /// - /// - /// How v2 will work: register a side-by-side group with the same template but - /// HasApiVersion(new ApiVersion(2, 0)). Existing v1 callers keep hitting the v1 - /// handler unchanged. - /// - /// - /// SOLID — DRY without ceremony: this method exists because the alternative — every - /// endpoint extension repeating four lines of boilerplate — is the kind of duplication that - /// rots over time (somebody copies it, somebody else types it slightly differently, the - /// versioning policy quietly diverges). One helper, one rule, one place to change. - /// - /// - public static RouteGroupBuilder MapV1ApiGroup(this WebApplication app, string tag, string template) - { - ArgumentException.ThrowIfNullOrWhiteSpace(tag); - ArgumentException.ThrowIfNullOrWhiteSpace(template); - - var trimmed = template.TrimStart('/'); - return app.NewVersionedApi(tag) - .MapGroup($"/api/v{{version:apiVersion}}/{trimmed}") - .HasApiVersion(new ApiVersion(1, 0)) - .WithTags(tag); - } - - /// - /// Registers Wolverine context propagation middleware for both directions: - /// - /// ContextPropagationMiddleware (incoming) — reads CorrelationId/UserId/SessionId - /// from the envelope headers and opens a logger scope so every log line emitted by - /// the handler carries those fields. Mirrors what CorrelationIdMiddleware - /// does for HTTP requests. - /// OutgoingContextMiddleware (outgoing) — copies those same IDs from - /// Activity baggage onto outbound envelope headers so the next service in the saga - /// picks them up. - /// - /// Call inside UseWolverine() in every service. Without this, observability falls - /// apart at the Service Bus boundary — you can't trace one transaction across services. - /// - public static WolverineOptions AddNextAuroraContextPropagation(this WolverineOptions opts) - { - opts.Policies.AddMiddleware(); - opts.Policies.AddMiddleware(); - return opts; - } - - /// - /// On , retry the message handler up to three - /// times with increasing backoff (50ms, 100ms, 250ms). After exhaustion the message goes - /// to the dead-letter queue. - /// - /// - /// Why retry rather than fail: concurrency conflicts in the saga are *expected* — - /// when PaymentCompletedEvent and ShipmentDispatchedEvent arrive at OrderService - /// near-simultaneously, both handlers fetch the same order, both try to mutate, one wins. - /// The loser's retry refetches the now-updated order; its status guard then either no-ops - /// (if the operation became invalid) or applies cleanly. This is the right behavior — not - /// an error — and shouldn't surface as a 5xx. - /// - /// - /// Pairs with the HTTP path: GlobalExceptionHandler catches the same exception - /// on the HTTP side and returns 409 Conflict with a refetch-and-try-again hint. - /// - /// - /// Call inside UseWolverine() in every service that handles events on tracked - /// aggregates with concurrency tokens (Order, Payment, Shipping). - /// - /// - public static WolverineOptions AddConcurrencyRetry(this WolverineOptions opts) - { - opts.OnException() - .RetryWithCooldown(50.Milliseconds(), 100.Milliseconds(), 250.Milliseconds()); - return opts; - } - - /// - /// Permit Wolverine's generated handler code to resolve dependencies from the DI container - /// (service location) instead of failing the build. - /// - /// - /// Why this is needed (Wolverine 6 breaking change): Wolverine generates code that - /// constructs each handler and resolves its dependencies. When a dependency can be *inlined* - /// (concrete type, or a simple registration the generator can reproduce), it emits - /// new Dependency(...). When it can't — e.g. an interface registered with a factory - /// lambda such as IProductCache over HybridCache, or EF's DbContext via - /// a pooling factory — it falls back to serviceProvider.GetRequiredService<T>(). - /// Wolverine 6 flipped the default to - /// , which turns that fallback into a startup - /// exception. Setting restores the 5.x - /// behavior. - /// - /// - /// This is NOT the service-locator anti-pattern. Our handlers use constructor injection; - /// the container resolution happens once in generated bootstrap code, not via an ambient - /// IServiceProvider threaded through business logic. The alternative — pre-generated - /// static codegen with method-injected handlers — is the production-grade path and is tracked - /// as a follow-up; until then this keeps the dynamic-codegen developer loop working. - /// Call inside UseWolverine() in every service. - /// - /// - public static WolverineOptions AllowHandlerServiceLocation(this WolverineOptions opts) - { - opts.ServiceLocationPolicy = ServiceLocationPolicy.AlwaysAllowed; - return opts; - } - - /// - /// Apply pending EF Core migrations at startup. Resolves the DbContext from a fresh DI - /// scope, calls Database.MigrateAsync, returns. - /// - /// - /// Dev-only by convention: this is called inside if (app.Environment.IsDevelopment()) - /// in every service's Program.cs. Why not production? With multiple replicas behind a load - /// balancer, all replicas would race to apply migrations on startup — the first wins, the - /// rest see history-table conflicts. In production, migrations should run as a separate - /// pre-deploy step, then app pods start without migration at all. - /// - /// - public static async Task MigrateDatabaseAsync(this IServiceProvider services, CancellationToken ct = default) - where TContext : DbContext - { - using var scope = services.CreateScope(); - var context = scope.ServiceProvider.GetRequiredService(); - await context.Database.MigrateAsync(ct); - } -} +using Asp.Versioning; +using JasperFx.CodeGeneration.Model; +using JasperFx.Core; +using Microsoft.AspNetCore.Authentication.JwtBearer; +using Microsoft.AspNetCore.Builder; +using Microsoft.AspNetCore.Diagnostics.HealthChecks; +using Microsoft.AspNetCore.Http; +using Microsoft.AspNetCore.Routing; +using Microsoft.EntityFrameworkCore; +using Microsoft.Extensions.Configuration; +using Microsoft.Extensions.DependencyInjection; +using Microsoft.Extensions.Diagnostics.HealthChecks; +using Microsoft.Extensions.Logging; +using Microsoft.Extensions.ServiceDiscovery; +using NextAurora.ServiceDefaults.Messaging; +using NextAurora.ServiceDefaults.Metrics; +using NextAurora.ServiceDefaults.Middleware; +using OpenTelemetry; +using OpenTelemetry.Metrics; +using OpenTelemetry.Trace; +using Wolverine; +using Wolverine.ErrorHandling; + +namespace Microsoft.Extensions.Hosting; + +// Adds common Aspire services: service discovery, resilience, health checks, and OpenTelemetry. +// This project should be referenced by each service project in your solution. +// To learn more about using this project, see https://aka.ms/dotnet/aspire/service-defaults +public static class Extensions +{ + private const string HealthEndpointPath = "/health"; + private const string AlivenessEndpointPath = "/alive"; + + public static TBuilder AddServiceDefaults(this TBuilder builder) where TBuilder : IHostApplicationBuilder + { + builder.ConfigureOpenTelemetry(); + + builder.AddDefaultHealthChecks(); + + builder.Services.AddExceptionHandler(); + builder.Services.AddProblemDetails(); + + builder.Services.AddSingleton(); + + builder.Services.AddServiceDiscovery(); + + builder.AddDefaultAuthentication(); + + builder.AddFrontendCors(); + + builder.AddNextAuroraApiVersioning(); + + builder.Services.ConfigureHttpClientDefaults(http => + { + // Turn on resilience by default + http.AddStandardResilienceHandler(); + + // Turn on service discovery by default + http.AddServiceDiscovery(); + }); + + return builder; + } + + public static TBuilder ConfigureOpenTelemetry(this TBuilder builder) where TBuilder : IHostApplicationBuilder + { + builder.Logging.AddOpenTelemetry(logging => + { + logging.IncludeFormattedMessage = true; + logging.IncludeScopes = true; + }); + + builder.Services.AddOpenTelemetry() + .WithMetrics(metrics => + { + metrics.AddAspNetCoreInstrumentation() + .AddHttpClientInstrumentation() + .AddRuntimeInstrumentation() + .AddMeter("NextAurora"); + }) + .WithTracing(tracing => + { + tracing.AddSource(builder.Environment.ApplicationName) + // Wolverine's own ActivitySource — emits the message send/receive/handle spans + // for the saga regardless of transport (RabbitMQ today). This is the span source + // you follow in the Aspire dashboard to watch an order walk Order→Payment→Shipping. + .AddSource("Wolverine") + // "NextAurora.Messaging" is the project's own ActivitySource (context-propagation + // middleware). Registering it makes those spans appear in the Aspire dashboard + // and any connected distributed tracing backend. + .AddSource("NextAurora.Messaging") + .AddAspNetCoreInstrumentation(tracing => + // Exclude health check requests from tracing + tracing.Filter = context => + !context.Request.Path.StartsWithSegments(HealthEndpointPath, StringComparison.OrdinalIgnoreCase) + && !context.Request.Path.StartsWithSegments(AlivenessEndpointPath, StringComparison.OrdinalIgnoreCase) + ) + .AddGrpcClientInstrumentation() + .AddHttpClientInstrumentation(); + }); + + builder.AddOpenTelemetryExporters(); + + return builder; + } + + private static void AddOpenTelemetryExporters(this TBuilder builder) where TBuilder : IHostApplicationBuilder + { + var useOtlpExporter = !string.IsNullOrWhiteSpace(builder.Configuration["OTEL_EXPORTER_OTLP_ENDPOINT"]); + + if (useOtlpExporter) + { + builder.Services.AddOpenTelemetry().UseOtlpExporter(); + } + } + + public static TBuilder AddDefaultHealthChecks(this TBuilder builder) where TBuilder : IHostApplicationBuilder + { + builder.Services.AddHealthChecks() + // Add a default liveness check to ensure app is responsive + .AddCheck("self", () => HealthCheckResult.Healthy(), ["live"]); + + return builder; + } + + public static WebApplication MapDefaultEndpoints(this WebApplication app) + { + // ORDER MATTERS. CorrelationIdMiddleware sits BETWEEN UseAuthentication and + // UseAuthorization for two reasons: + // + // 1. It reads ClaimTypes.NameIdentifier from context.User to populate the + // UserId scope key. UseAuthentication populates context.User, so the + // middleware must run AFTER it. Otherwise UserId is silently always null. + // + // 2. Placing it BEFORE UseAuthorization (rather than after) means the + // UserId scope is active during the Authorization step itself. Any + // 401/403 denials get logged with the authenticated user's ID — + // preserving the audit trail for "user X tried to access resource they + // shouldn't." Running after Authorization would drop that signal, + // losing visibility into unauthorized-attempt patterns. + // + // UseExceptionHandler stays first so it wraps every error below. UseCors runs + // before UseAuthentication so CORS preflights (OPTIONS, which never carry a + // bearer token) are answered without touching the auth pipeline. The policy is + // only registered when Frontend:AllowedOrigins is configured — services with no + // browser-facing surface skip the middleware entirely. + app.UseExceptionHandler(); + if (!string.IsNullOrWhiteSpace(app.Configuration["Frontend:AllowedOrigins"])) + { + app.UseCors(FrontendCorsPolicyName); + } + app.UseAuthentication(); + app.UseMiddleware(); + app.UseAuthorization(); + + // All health checks must pass for app to be considered ready to accept traffic after starting + app.MapHealthChecks(HealthEndpointPath); + + // Only health checks tagged with the "live" tag must pass for app to be considered alive + app.MapHealthChecks(AlivenessEndpointPath, new HealthCheckOptions + { + Predicate = r => r.Tags.Contains("live") + }); + + return app; + } + + private const string FrontendCorsPolicyName = "frontend"; + + /// + /// Explicit CORS for the browser SPA — registered ONLY when Frontend:AllowedOrigins + /// is configured (semicolon-separated origin list, injected per service by AppHost). + /// Per CLAUDE.md "Security Requirements → CORS": explicit policy, known origins only — + /// never AllowAnyOrigin. X-Correlation-Id is exposed so the SPA's + /// observability surface can read the correlation ID off every response; without + /// WithExposedHeaders, browsers hide non-safelisted response headers from JS even + /// when the request itself succeeds. + /// + private static void AddFrontendCors(this TBuilder builder) where TBuilder : IHostApplicationBuilder + { + var configured = builder.Configuration["Frontend:AllowedOrigins"]; + if (string.IsNullOrWhiteSpace(configured)) + { + return; + } + + var origins = configured.Split(';', StringSplitOptions.RemoveEmptyEntries | StringSplitOptions.TrimEntries); + + builder.Services.AddCors(options => options.AddPolicy(FrontendCorsPolicyName, policy => policy + .WithOrigins(origins) + .AllowAnyHeader() + .AllowAnyMethod() + .WithExposedHeaders("X-Correlation-Id"))); + } + + private static void AddDefaultAuthentication(this TBuilder builder) where TBuilder : IHostApplicationBuilder + { + // Authority resolution, in priority order: + // 1. Authentication:Authority — explicit override (prod / appsettings). + // 2. Keycloak:AuthServerUrl + Keycloak:Realm — what the Aspire Keycloak.AuthServices + // hosting integration injects from `WithReference(realm, configurationPrefix: "Keycloak")`. + // It does NOT inject a single `Keycloak:Url`; it injects the base server URL and the + // realm name separately, and the OIDC authority is `{AuthServerUrl}/realms/{Realm}`. + // Reading the injected keys (not a hardcoded URL) keeps this correct across Aspire's + // dynamic endpoints. Both the SPA and the API resolve to the same base, so the token's + // issuer matches the validated authority (Keycloak's issuer is host-reflective). See CLAUDE.md. + var authority = builder.Configuration["Authentication:Authority"]; + if (string.IsNullOrEmpty(authority)) + { + var authServerUrl = builder.Configuration["Keycloak:AuthServerUrl"]?.TrimEnd('/'); + var realm = builder.Configuration["Keycloak:Realm"]; + if (!string.IsNullOrEmpty(authServerUrl) && !string.IsNullOrEmpty(realm)) + { + authority = $"{authServerUrl}/realms/{realm}"; + } + } + + if (string.IsNullOrEmpty(authority)) + { + // No identity provider configured — register auth services with no-op defaults + // so UseAuthentication/UseAuthorization don't throw, but no tokens are validated. + builder.Services.AddAuthentication(); + builder.Services.AddAuthorization(); + return; + } + + // RequireHttpsMetadata is fail-closed outside Development. An http authority in + // Production must fail LOUDLY at startup, not silently fetch OIDC discovery + JWKS + // over plaintext (an active MITM could inject signing keys and forge tokens every + // service would accept). The explicit Authentication:RequireHttpsMetadata=false key + // is the auditable opt-out for legitimate internal-http deployments (e.g. Keycloak + // behind a service mesh); the default derives it only for local dev. See CLAUDE.md. + var requireHttpsMetadata = builder.Configuration.GetValue("Authentication:RequireHttpsMetadata", (bool?)null); + if (!requireHttpsMetadata.HasValue) + { + requireHttpsMetadata = authority.StartsWith("https://", StringComparison.OrdinalIgnoreCase) + || !builder.Environment.IsDevelopment(); + } + + if (!requireHttpsMetadata.Value) + { + // An opt-out (derived or explicit) should be visible in the app's logs, never + // silent. PostConfigure runs when the options are first resolved, after the DI + // logging pipeline exists — so this lands in the real log stream, not a side channel. + builder.Services.AddOptions(JwtBearerDefaults.AuthenticationScheme) + .PostConfigure((_, loggerFactory) => + loggerFactory.CreateLogger("NextAurora.ServiceDefaults.Authentication") + .LogWarning("JWT bearer RequireHttpsMetadata is DISABLED (authority: {Authority}) — OIDC metadata/JWKS are fetched without TLS. Acceptable for local dev only.", authority)); + } + + builder.Services.AddAuthentication(JwtBearerDefaults.AuthenticationScheme) + .AddJwtBearer(options => + { + options.Authority = authority; + options.Audience = builder.Configuration["Authentication:Audience"] ?? "nextaurora-api"; + options.RequireHttpsMetadata = requireHttpsMetadata.Value; + options.TokenValidationParameters = new Microsoft.IdentityModel.Tokens.TokenValidationParameters + { + ValidateAudience = true, + ValidateIssuer = true, + ValidateLifetime = true, + // Explicit — JWT Bearer's implicit default already validates the + // signature against JWKS-discovered keys, but making it explicit + // makes the security posture auditable + prevents a future config + // change from accidentally disabling signature validation. + ValidateIssuerSigningKey = true, + // Default ClockSkew is 5 minutes — revoked/expired tokens remain + // accepted for 5 extra minutes. The realm pins 5-MINUTE access tokens + // (nextaurora-realm.json accessTokenLifespan: 300), so the default skew + // would double every token's effective lifetime. 30 seconds covers + // reasonable inter-server clock drift without a long replay window. + ClockSkew = TimeSpan.FromSeconds(30), + NameClaimType = "preferred_username", + RoleClaimType = "realm_access.roles", + }; + }); + + builder.Services.AddAuthorization(); + } + + /// + /// URL-segment API versioning. Default version is 1.0; clients must include the version in + /// the route (`/api/v1/...`). The ApiExplorer integration makes versioned endpoints visible + /// in OpenAPI docs with version-aware group names. Called automatically by + /// . + /// + /// + /// Why URL-segment over header-based versioning: URL versioning is what most public + /// REST APIs (Stripe, GitHub, Twitter, AWS) use. Pros: visible in logs/dashboards, cacheable + /// (HTTP caches key on URL), debuggable from a browser, plays well with Swagger. The + /// "header versioning is more RESTful" argument is academic — the practical wins of URL + /// versioning dominate. + /// + /// + /// Why AssumeDefaultVersionWhenUnspecified = false: the version segment is + /// required. Hitting /api/products returns 400, not silent v1. This avoids the + /// classic mistake of silently treating un-versioned calls as v1, which makes future v2 + /// migrations a behavior-change debugging nightmare. + /// + /// + private static void AddNextAuroraApiVersioning(this TBuilder builder) where TBuilder : IHostApplicationBuilder + { + builder.Services + .AddApiVersioning(options => + { + options.DefaultApiVersion = new ApiVersion(1, 0); + options.AssumeDefaultVersionWhenUnspecified = false; + options.ReportApiVersions = true; + options.ApiVersionReader = new UrlSegmentApiVersionReader(); + }) + .AddApiExplorer(options => + { + // 'v'VVV → groups become "v1", "v2", "v1.1" etc. in OpenAPI. + options.GroupNameFormat = "'v'VVV"; + // Replace `{version:apiVersion}` placeholder with the actual version in + // generated docs/URLs (so Swagger UI shows "/api/v1/products" not the template). + options.SubstituteApiVersionInUrl = true; + }); + } + + /// + /// Canonical helper for registering a versioned route group at + /// /api/v{version}/{template}. Every endpoint group in every service goes through + /// this — that way the policy (default version, tag application, route prefix) can't drift + /// across services. Equivalent to: + /// + /// app.NewVersionedApi(tag) + /// .MapGroup("/api/v{version:apiVersion}/" + template) + /// .HasApiVersion(new ApiVersion(1, 0)) + /// .WithTags(tag); + /// + /// + /// How v2 will work: register a side-by-side group with the same template but + /// HasApiVersion(new ApiVersion(2, 0)). Existing v1 callers keep hitting the v1 + /// handler unchanged. + /// + /// + /// SOLID — DRY without ceremony: this method exists because the alternative — every + /// endpoint extension repeating four lines of boilerplate — is the kind of duplication that + /// rots over time (somebody copies it, somebody else types it slightly differently, the + /// versioning policy quietly diverges). One helper, one rule, one place to change. + /// + /// + public static RouteGroupBuilder MapV1ApiGroup(this WebApplication app, string tag, string template) + { + ArgumentException.ThrowIfNullOrWhiteSpace(tag); + ArgumentException.ThrowIfNullOrWhiteSpace(template); + + var trimmed = template.TrimStart('/'); + return app.NewVersionedApi(tag) + .MapGroup($"/api/v{{version:apiVersion}}/{trimmed}") + .HasApiVersion(new ApiVersion(1, 0)) + .WithTags(tag); + } + + /// + /// Registers Wolverine context propagation middleware for both directions: + /// + /// ContextPropagationMiddleware (incoming) — reads CorrelationId/UserId/SessionId + /// from the envelope headers and opens a logger scope so every log line emitted by + /// the handler carries those fields. Mirrors what CorrelationIdMiddleware + /// does for HTTP requests. + /// OutgoingContextMiddleware (outgoing) — copies those same IDs from + /// Activity baggage onto outbound envelope headers so the next service in the saga + /// picks them up. + /// + /// Call inside UseWolverine() in every service. Without this, observability falls + /// apart at the message-broker boundary — you can't trace one transaction across services. + /// + public static WolverineOptions AddNextAuroraContextPropagation(this WolverineOptions opts) + { + opts.Policies.AddMiddleware(); + opts.Policies.AddMiddleware(); + return opts; + } + + /// + /// On , retry the message handler up to three + /// times with increasing backoff (50ms, 100ms, 250ms). After exhaustion the message goes + /// to the dead-letter queue. + /// + /// + /// Why retry rather than fail: concurrency conflicts in the saga are *expected* — + /// when PaymentCompletedEvent and ShipmentDispatchedEvent arrive at OrderService + /// near-simultaneously, both handlers fetch the same order, both try to mutate, one wins. + /// The loser's retry refetches the now-updated order; its status guard then either no-ops + /// (if the operation became invalid) or applies cleanly. This is the right behavior — not + /// an error — and shouldn't surface as a 5xx. + /// + /// + /// Pairs with the HTTP path: GlobalExceptionHandler catches the same exception + /// on the HTTP side and returns 409 Conflict with a refetch-and-try-again hint. + /// + /// + /// Call inside UseWolverine() in every service that handles events on tracked + /// aggregates with concurrency tokens (Order, Payment, Shipping). + /// + /// + public static WolverineOptions AddConcurrencyRetry(this WolverineOptions opts) + { + opts.OnException() + .RetryWithCooldown(50.Milliseconds(), 100.Milliseconds(), 250.Milliseconds()); + return opts; + } + + /// + /// Permit Wolverine's generated handler code to resolve dependencies from the DI container + /// (service location) instead of failing the build. + /// + /// + /// Why this is needed (Wolverine 6 breaking change): Wolverine generates code that + /// constructs each handler and resolves its dependencies. When a dependency can be *inlined* + /// (concrete type, or a simple registration the generator can reproduce), it emits + /// new Dependency(...). When it can't — e.g. an interface registered with a factory + /// lambda such as IProductCache over HybridCache, or EF's DbContext via + /// a pooling factory — it falls back to serviceProvider.GetRequiredService<T>(). + /// Wolverine 6 flipped the default to + /// , which turns that fallback into a startup + /// exception. Setting restores the 5.x + /// behavior. + /// + /// + /// This is NOT the service-locator anti-pattern. Our handlers use constructor injection; + /// the container resolution happens once in generated bootstrap code, not via an ambient + /// IServiceProvider threaded through business logic. The alternative — pre-generated + /// static codegen with method-injected handlers — is the production-grade path and is tracked + /// as a follow-up; until then this keeps the dynamic-codegen developer loop working. + /// Call inside UseWolverine() in every service. + /// + /// + public static WolverineOptions AllowHandlerServiceLocation(this WolverineOptions opts) + { + opts.ServiceLocationPolicy = ServiceLocationPolicy.AlwaysAllowed; + return opts; + } + + /// + /// Apply pending EF Core migrations at startup. Resolves the DbContext from a fresh DI + /// scope, calls Database.MigrateAsync, returns. + /// + /// + /// Dev-only by convention: this is called inside if (app.Environment.IsDevelopment()) + /// in every service's Program.cs. Why not production? With multiple replicas behind a load + /// balancer, all replicas would race to apply migrations on startup — the first wins, the + /// rest see history-table conflicts. In production, migrations should run as a separate + /// pre-deploy step, then app pods start without migration at all. + /// + /// + public static async Task MigrateDatabaseAsync(this IServiceProvider services, CancellationToken ct = default) + where TContext : DbContext + { + using var scope = services.CreateScope(); + var context = scope.ServiceProvider.GetRequiredService(); + await context.Database.MigrateAsync(ct); + } +} diff --git a/NextAurora.ServiceDefaults/Messaging/ContextPropagationMiddleware.cs b/NextAurora.ServiceDefaults/Messaging/ContextPropagationMiddleware.cs index d6865c1f..6247cb48 100644 --- a/NextAurora.ServiceDefaults/Messaging/ContextPropagationMiddleware.cs +++ b/NextAurora.ServiceDefaults/Messaging/ContextPropagationMiddleware.cs @@ -8,7 +8,7 @@ namespace NextAurora.ServiceDefaults.Messaging; /// Wolverine incoming-message middleware: restores the three observability identifiers /// (CorrelationId, UserId, SessionId) from envelope headers into the /// current Activity baggage and opens a structured logger scope. Mirrors what -/// CorrelationIdMiddleware does for incoming HTTP requests, but for incoming Service Bus +/// CorrelationIdMiddleware does for incoming HTTP requests, but for incoming RabbitMQ /// messages — so the same scope keys appear on every log line whether the work was triggered /// over HTTP or async messaging. /// diff --git a/NextAurora.ServiceDefaults/Metrics/NextAuroraMetrics.cs b/NextAurora.ServiceDefaults/Metrics/NextAuroraMetrics.cs index 8ff84887..d813d22a 100644 --- a/NextAurora.ServiceDefaults/Metrics/NextAuroraMetrics.cs +++ b/NextAurora.ServiceDefaults/Metrics/NextAuroraMetrics.cs @@ -15,7 +15,7 @@ public sealed class NextAuroraMetrics : IDisposable public Counter ShipmentsDispatched { get; } public Counter NotificationsSent { get; } /// - /// Incremented whenever a Service Bus message is abandoned (returned for retry or DLQ). + /// Incremented whenever a message is abandoned (returned for retry or dead-lettered). /// Tagged with Subject and service name. Alert on this counter rising to detect DLQ /// pile-ups before they cause user-visible outages. /// diff --git a/NextAurora.ServiceDefaults/Middleware/CorrelationIdMiddleware.cs b/NextAurora.ServiceDefaults/Middleware/CorrelationIdMiddleware.cs index 061b8aff..5b58e373 100644 --- a/NextAurora.ServiceDefaults/Middleware/CorrelationIdMiddleware.cs +++ b/NextAurora.ServiceDefaults/Middleware/CorrelationIdMiddleware.cs @@ -21,7 +21,7 @@ namespace NextAurora.ServiceDefaults.Middleware; /// /// All three values are stored in two places: /// 1. Activity.Current baggage — accessed by OutgoingContextMiddleware when stamping outgoing -/// Service Bus messages, and by ContextPropagationMiddleware when enriching handler logs. +/// RabbitMQ messages, and by ContextPropagationMiddleware when enriching handler logs. /// 2. logger.BeginScope() — automatically prepended to every log line written during /// this request, including deep inside handlers and repositories. /// diff --git a/NotificationService/NotificationService.csproj b/NotificationService/NotificationService.csproj index ebcfcb63..c087d01c 100644 --- a/NotificationService/NotificationService.csproj +++ b/NotificationService/NotificationService.csproj @@ -5,7 +5,7 @@ - + diff --git a/NotificationService/Program.cs b/NotificationService/Program.cs index 148da476..ebec3dd8 100644 --- a/NotificationService/Program.cs +++ b/NotificationService/Program.cs @@ -5,7 +5,7 @@ using NotificationService.Infrastructure; using Scalar.AspNetCore; using Wolverine; -using Wolverine.AzureServiceBus; +using Wolverine.RabbitMQ; var builder = WebApplication.CreateBuilder(args); @@ -14,25 +14,21 @@ builder.Host.UseWolverine(opts => { var connectionString = builder.Configuration.GetConnectionString("messaging")!; - var azureServiceBus = opts.UseAzureServiceBus(connectionString); - - // AutoProvision creates topics/subscriptions via the Service Bus management API at host - // startup. Disabled in two environments: integration tests (fake ASB string hangs) and - // local dev (the emulator has no management API → BrokerInitializationException). The - // AppHost injects Wolverine__AutoProvision=false for the emulator. Gate on a config flag - // (defaults true) so real Azure still provisions. See OrderService/Program.cs + CLAUDE.md. + // RabbitMQ transport. NotificationService is listen-only (the saga sink): bind a per-source + // queue to each event exchange, plus the direct send-notification queue. AutoProvision is gated + // (default on) for consistency with the other services. See OrderService/Program.cs + CLAUDE.md. + var rabbit = opts.UseRabbitMq(factory => factory.Uri = new Uri(connectionString)); if (builder.Configuration.GetValue("Wolverine:AutoProvision", defaultValue: true)) { - azureServiceBus.AutoProvision(); + rabbit.AutoProvision(); } - - // Listen to events from other services - opts.ListenToAzureServiceBusSubscription("order-events/notify-orders-sub"); - opts.ListenToAzureServiceBusSubscription("payment-events/notify-payments-sub"); - opts.ListenToAzureServiceBusSubscription("shipping-events/notify-shipping-sub"); - - // Listen to direct command queue - opts.ListenToAzureServiceBusQueue("send-notification"); + rabbit.BindExchange("order-events", ExchangeType.Fanout).ToQueue("notify-orders"); + rabbit.BindExchange("payment-events", ExchangeType.Fanout).ToQueue("notify-payments"); + rabbit.BindExchange("shipping-events", ExchangeType.Fanout).ToQueue("notify-shipping"); + opts.ListenToRabbitQueue("notify-orders"); + opts.ListenToRabbitQueue("notify-payments"); + opts.ListenToRabbitQueue("notify-shipping"); + opts.ListenToRabbitQueue("send-notification"); // Single-project assembly — Wolverine auto-discovers handlers from the entry assembly, // so no explicit IncludeAssembly call is needed. diff --git a/OrderService/Domain/Order.cs b/OrderService/Domain/Order.cs index 4e89b4f0..9dcee11d 100644 --- a/OrderService/Domain/Order.cs +++ b/OrderService/Domain/Order.cs @@ -74,7 +74,7 @@ public static Order Create(Guid buyerId, string currency, List lines) /// /// Transition: Placed → Paid. Triggered by the saga when PaymentCompletedEvent /// is received. The status guard is the idempotency mechanism: if this event arrives twice - /// (Service Bus is at-least-once, redelivery happens), the second call throws and the + /// (the broker delivers at-least-once, redelivery happens), the second call throws and the /// handler treats it as a no-op rather than corrupting state. /// public void MarkAsPaid() diff --git a/OrderService/Features/PaymentCompletedHandler.cs b/OrderService/Features/PaymentCompletedHandler.cs index dbb66101..f58bb515 100644 --- a/OrderService/Features/PaymentCompletedHandler.cs +++ b/OrderService/Features/PaymentCompletedHandler.cs @@ -11,7 +11,7 @@ namespace OrderService.Features; /// ). /// /// -/// Idempotency layered three ways — important because Service Bus delivers at-least-once +/// Idempotency layered three ways — important because RabbitMQ delivers at-least-once /// and a redelivery here is normal, not exceptional: /// /// diff --git a/OrderService/Features/PaymentFailedHandler.cs b/OrderService/Features/PaymentFailedHandler.cs index 6462c236..66d48369 100644 --- a/OrderService/Features/PaymentFailedHandler.cs +++ b/OrderService/Features/PaymentFailedHandler.cs @@ -11,7 +11,7 @@ namespace OrderService.Features; /// . /// /// -/// Idempotency layered three ways — important because Service Bus delivers at-least-once +/// Idempotency layered three ways — important because RabbitMQ delivers at-least-once /// and a redelivery here is normal, not exceptional: /// /// diff --git a/OrderService/Features/ShipmentDispatchedHandler.cs b/OrderService/Features/ShipmentDispatchedHandler.cs index cfb389ac..cce92b8b 100644 --- a/OrderService/Features/ShipmentDispatchedHandler.cs +++ b/OrderService/Features/ShipmentDispatchedHandler.cs @@ -14,7 +14,7 @@ namespace OrderService.Features; /// Saga ordering: we trust the natural ordering: payment must succeed before shipping /// can be created (by CreateShipmentHandler in ShippingService, which itself reacts to /// PaymentCompletedEvent). If the events somehow arrive out of order at this handler — -/// extremely unlikely with single-subscription Service Bus, but possible after a DLQ replay — +/// extremely unlikely with a single consumer queue per event, but possible after a DLQ replay — /// the status guard catches it: an order still in Placed can't go to Shipped, so /// we silently skip. /// diff --git a/OrderService/OrderService.csproj b/OrderService/OrderService.csproj index d8ec243c..d87479c2 100644 --- a/OrderService/OrderService.csproj +++ b/OrderService/OrderService.csproj @@ -22,7 +22,7 @@ - + diff --git a/OrderService/Program.cs b/OrderService/Program.cs index 55f78897..9670b94c 100644 --- a/OrderService/Program.cs +++ b/OrderService/Program.cs @@ -7,7 +7,7 @@ using OrderService.Infrastructure.Data; using Scalar.AspNetCore; using Wolverine; -using Wolverine.AzureServiceBus; +using Wolverine.RabbitMQ; using Wolverine.EntityFrameworkCore; using Wolverine.FluentValidation; using Wolverine.SqlServer; @@ -19,29 +19,28 @@ builder.Host.UseWolverine(opts => { var connectionString = builder.Configuration.GetConnectionString("messaging")!; - var azureServiceBus = opts.UseAzureServiceBus(connectionString); - // AutoProvision creates topics/subscriptions via the Service Bus *management* API - // (ServiceBusAdministrationClient) at host startup — a real network operation against the - // configured endpoint. Two environments must disable it: - // 1. Integration tests use a fake ASB connection string, so provisioning hangs/times out - // (it fires before DisableAllExternalWolverineTransports() takes effect). - // 2. Local dev (Aspire) uses the Service Bus emulator, which implements only the AMQP data - // plane — NOT the management API — so AutoProvision retries 4× and the host dies with - // BrokerInitializationException. The AppHost declares the topology and injects - // Wolverine__AutoProvision=false for each Wolverine service. - // Gate on a config flag (defaults true) so real Azure / Publish mode still provisions. See CLAUDE.md. + // Channel names (RabbitMQ fanout exchanges). + const string orderEvents = "order-events"; + const string paymentEvents = "payment-events"; + const string shippingEvents = "shipping-events"; + + // RabbitMQ transport (local dev, CI, and the Hetzner deployment). Wolverine declares the + // exchanges/queues/bindings via AutoProvision against the live broker: this service's published + // events go to a fanout exchange, and each consumed subscription is a queue bound to its source + // exchange. AutoProvision is gated (default on) so it's OFF for integration tests, which stub the + // transport and would otherwise block on the fake connection string. See CLAUDE.md. + var rabbit = opts.UseRabbitMq(factory => factory.Uri = new Uri(connectionString)); if (builder.Configuration.GetValue("Wolverine:AutoProvision", defaultValue: true)) { - azureServiceBus.AutoProvision(); + rabbit.AutoProvision(); } - - // Publish outgoing events to their topics - opts.PublishMessage().ToAzureServiceBusTopic("order-events"); - - // Listen to incoming events from other services - opts.ListenToAzureServiceBusSubscription("payment-events/order-payments-sub"); - opts.ListenToAzureServiceBusSubscription("shipping-events/order-shipping-sub"); + rabbit.DeclareExchange(orderEvents, e => e.ExchangeType = ExchangeType.Fanout); + rabbit.BindExchange(paymentEvents, ExchangeType.Fanout).ToQueue("order-payments"); + rabbit.BindExchange(shippingEvents, ExchangeType.Fanout).ToQueue("order-shipping"); + opts.PublishMessage().ToRabbitExchange(orderEvents); + opts.ListenToRabbitQueue("order-payments"); + opts.ListenToRabbitQueue("order-shipping"); // Transactional outbox: persist outgoing messages to SQL Server in the same // transaction as the entity write, then dispatch via background flush. diff --git a/PaymentService/Domain/IPaymentGateway.cs b/PaymentService/Domain/IPaymentGateway.cs index 8abdef13..4e2d9240 100644 --- a/PaymentService/Domain/IPaymentGateway.cs +++ b/PaymentService/Domain/IPaymentGateway.cs @@ -8,7 +8,7 @@ namespace PaymentService.Domain; /// /// Why is required, not optional. The Gateway /// handler (PaymentProcessingRequestedHandler) runs on Wolverine's at-least-once -/// delivery semantics — Service Bus or the local in-memory queue can hand the same +/// delivery semantics — the broker or the local in-memory queue can hand the same /// PaymentProcessingRequested message to the handler more than once (process death /// after a successful Stripe call but before MarkAsCompleted is the canonical /// race). Without an idempotency key, redelivery makes the second Stripe call too — the diff --git a/PaymentService/Domain/Payment.cs b/PaymentService/Domain/Payment.cs index 8c23e06f..d2ba8e98 100644 --- a/PaymentService/Domain/Payment.cs +++ b/PaymentService/Domain/Payment.cs @@ -10,7 +10,7 @@ namespace PaymentService.Domain; /// Idempotency under saga retries: the OrderPlacedHandler in this service first /// checks if a Payment already exists for the order before creating one, and the status guards /// in / prevent double-processing if -/// the Service Bus redelivers a message. Combined with the OrderId unique index, we get +/// the broker redelivers a message. Combined with the OrderId unique index, we get /// "exactly-once outcome" even with at-least-once delivery. /// /// diff --git a/PaymentService/Features/ProcessPayment.cs b/PaymentService/Features/ProcessPayment.cs index 1bc60d20..0a44755e 100644 --- a/PaymentService/Features/ProcessPayment.cs +++ b/PaymentService/Features/ProcessPayment.cs @@ -15,7 +15,7 @@ namespace PaymentService.Features; /// /// Why split. The Stripe call is sub-second in the happy path but seconds-to-30s /// on degraded gateway states. Doing it inline in ProcessPaymentHandler held the HTTP -/// request open (and a Wolverine handler slot + DbContext + Service Bus message lease on the +/// request open (and a Wolverine handler slot + DbContext + broker message lease on the /// saga path) for the entire duration. The 202 Accepted rule says: validate + persist /// intent + publish a Wolverine message + return; let a follow-up handler do the slow work. /// See CLAUDE.md "Performance Rules → Long-running work belongs on the message bus". diff --git a/PaymentService/Infrastructure/Data/PaymentDbContext.cs b/PaymentService/Infrastructure/Data/PaymentDbContext.cs index c9eea464..9e51a0da 100644 --- a/PaymentService/Infrastructure/Data/PaymentDbContext.cs +++ b/PaymentService/Infrastructure/Data/PaymentDbContext.cs @@ -11,7 +11,7 @@ namespace PaymentService.Infrastructure.Data; /// /// /// Why a unique index on OrderId: the saga can deliver OrderPlacedEvent -/// more than once (Service Bus retries, DLQ replays). The handler does an existence check +/// more than once (broker redeliveries, DLQ replays). The handler does an existence check /// first, but races between two simultaneous deliveries could still attempt two inserts. The /// unique index turns the second insert into a database error rather than two payments for one /// order. Defense in depth. diff --git a/PaymentService/PaymentService.csproj b/PaymentService/PaymentService.csproj index c73d0c3e..e6cec2c7 100644 --- a/PaymentService/PaymentService.csproj +++ b/PaymentService/PaymentService.csproj @@ -13,7 +13,7 @@ - + diff --git a/PaymentService/Program.cs b/PaymentService/Program.cs index 4694e39c..d8f64b33 100644 --- a/PaymentService/Program.cs +++ b/PaymentService/Program.cs @@ -9,7 +9,7 @@ using PaymentService.Infrastructure.Data; using Scalar.AspNetCore; using Wolverine; -using Wolverine.AzureServiceBus; +using Wolverine.RabbitMQ; using Wolverine.EntityFrameworkCore; using Wolverine.FluentValidation; using Wolverine.SqlServer; @@ -42,24 +42,23 @@ builder.Host.UseWolverine(opts => { var connectionString = builder.Configuration.GetConnectionString("messaging")!; - var azureServiceBus = opts.UseAzureServiceBus(connectionString); - // AutoProvision creates topics/subscriptions via the Service Bus management API at host - // startup. Disabled in two environments: integration tests (fake ASB string hangs) and - // local dev (the emulator has no management API → BrokerInitializationException). The - // AppHost injects Wolverine__AutoProvision=false for the emulator. Gate on a config flag - // (defaults true) so real Azure still provisions. See OrderService/Program.cs + CLAUDE.md. + // Channel names (RabbitMQ fanout exchanges). + const string paymentEvents = "payment-events"; + const string orderEvents = "order-events"; + + // RabbitMQ transport. Wolverine declares the exchange/queue/binding via AutoProvision (gated so + // it's off for transport-stubbed integration tests). See OrderService/Program.cs + CLAUDE.md. + var rabbit = opts.UseRabbitMq(factory => factory.Uri = new Uri(connectionString)); if (builder.Configuration.GetValue("Wolverine:AutoProvision", defaultValue: true)) { - azureServiceBus.AutoProvision(); + rabbit.AutoProvision(); } - - // Publish outgoing events to their topics - opts.PublishMessage().ToAzureServiceBusTopic("payment-events"); - opts.PublishMessage().ToAzureServiceBusTopic("payment-events"); - - // Listen to incoming events from other services - opts.ListenToAzureServiceBusSubscription("order-events/payment-orders-sub"); + rabbit.DeclareExchange(paymentEvents, e => e.ExchangeType = ExchangeType.Fanout); + rabbit.BindExchange(orderEvents, ExchangeType.Fanout).ToQueue("payment-orders"); + opts.PublishMessage().ToRabbitExchange(paymentEvents); + opts.PublishMessage().ToRabbitExchange(paymentEvents); + opts.ListenToRabbitQueue("payment-orders"); // Transactional outbox: persist outgoing messages to SQL Server in the same // transaction as the entity write, then dispatch via background flush. diff --git a/README.md b/README.md index 4f8944ae..3ae842d3 100644 --- a/README.md +++ b/README.md @@ -17,12 +17,12 @@ NextAurora demonstrates a production-style distributed system with event-driven > **How it was built — AI-assisted, multi-model review, verification at every layer.** > - **Two AI reviewers, not one.** [Claude Code](https://claude.com/claude-code) (Opus 4.7) is the primary pair-programmer — reads [`CLAUDE.md`](CLAUDE.md) plus the [`.claude/`](.claude/) folder beside it (agents, [skills](.claude/skills/), [slash commands](.claude/commands/), hook scripts, hook + permission wiring) every session, and a persistent project memory. **GitHub Copilot (GPT-5)** sits in-editor for second-opinion diff review, with project conventions encoded in [`.github/copilot-instructions.md`](.github/copilot-instructions.md). Disagreement between the two is treated as a signal to dig deeper, not pick the louder voice. The principle is not "AI wrote it" — it's *two models + a human author + automated checks all sign off before merge*. The working loop: implement → run unit + integration tests → cross-model review → fix → commit. > - **Continuous Rule Encoding — the compounding feedback loop.** Every meaningful finding (CodeRabbit catch, test failure, architecture-reviewer agent flag, manual code review, prod incident, even community articles audited via [`/article-audit`](.claude/commands/article-audit.md)) gets the question: *"could the next person repeat this?"* If yes, the rule gets written down at the right tier of an enforcement spectrum — **Convention** (CLAUDE.md + `.claude/`), **PR-review automation** ([`.coderabbit.yaml`](.coderabbit.yaml) + the [architecture-reviewer agent](.claude/agents/architecture-reviewer.md)), or **Mechanical** (build-fail / hook-block / CI-grep). Rules move down the spectrum as they prove their value — the looser the tier, the more you rely on noticing, and noticing always degrades first. CLAUDE.md itself stays lean (~300 lines; CI-guarded soft cap at 400, hard fail at 500); deep dives live in [`docs/`](docs/) and the [`dotnet-performance` skill](.claude/skills/dotnet-performance/SKILL.md). Six **disciplines** sit alongside the surfaces — *cross-reference convention, file-move, doc-and-diagram, lean-CLAUDE.md, presence-in-the-loop, continue-is-the-verb* — the first four with mechanical floors (hooks, CI guards, size budget), the last two convention-tier (judgment calls about presence and stopping). New features run through [`/feature-spec`](.claude/commands/feature-spec.md), which opens with a value gate (who needs this, would we still build it at engineering-time cost, who owns saying no), then drafts a structured handoff (goal + acceptance + upstream dependencies + auto-referenced CLAUDE.md constraints) and closes with a hole-test (*imagine handing this to someone not in your head — where would they have to guess?*). Full mechanics + the 5 encoding surfaces + the disciplines catalog: [`docs/dev-loop.md`](docs/dev-loop.md); scaffolding diagram: [`docs/dev-loop-scaffolding.svg`](docs/dev-loop-scaffolding.svg). -> - **Verification at every layer.** Build: `TreatWarningsAsErrors` + four build-time analyzers (Meziantou, SonarAnalyzer.CSharp, Roslynator, BannedApiAnalyzers — the last rejects concrete concurrency hazards at compile). Tests: 100+ unit + integration tests, with integration slices for all four DB-touching services via [Testcontainers](https://dotnet.testcontainers.org/) — Catalog (Postgres + Redis), Order (SQL Server + stubbed Wolverine transport), Payment (SQL Server), Shipping (Postgres). Each slice runs as its own step in CI so a single-slice failure doesn't mask the rest. CI: GitHub Actions runs build + tests + Coverlet/Cobertura coverage on every PR. Security: CodeQL on every push + weekly schedule. Dependencies: Dependabot weekly NuGet bumps (grouped per ecosystem). Local dev: Aspire orchestrates Postgres / SQL Server / Service Bus emulator / Redis / Keycloak in Docker so integration issues surface *before* push. -> - **Testing strategy was decided upfront, not retrofitted.** Unit tests cover handlers + domain rules with mocked infrastructure (NSubstitute, FakeTimeProvider). Integration tests use live containers to prove what mocks can't: EF migrations apply cleanly to fresh DBs, HybridCache actually invalidates on write, the `xmin` / `RowVersion` concurrency tokens actually fire under racing writes, Wolverine's transactional outbox + saga handlers actually persist and dispatch. Cross-service E2E over a live Azure Service Bus wire is tracked as deferred in [STATUS.md](docs/STATUS.md). Open issues and other deferred cleanups live there too. +> - **Verification at every layer.** Build: `TreatWarningsAsErrors` + four build-time analyzers (Meziantou, SonarAnalyzer.CSharp, Roslynator, BannedApiAnalyzers — the last rejects concrete concurrency hazards at compile). Tests: 100+ unit + integration tests, with integration slices for all four DB-touching services via [Testcontainers](https://dotnet.testcontainers.org/) — Catalog (Postgres + Redis), Order (SQL Server + stubbed Wolverine transport), Payment (SQL Server), Shipping (Postgres). Each slice runs as its own step in CI so a single-slice failure doesn't mask the rest. CI: GitHub Actions runs build + tests + Coverlet/Cobertura coverage on every PR. Security: CodeQL on every push + weekly schedule. Dependencies: Dependabot weekly NuGet bumps (grouped per ecosystem). Local dev: Aspire orchestrates Postgres / SQL Server / RabbitMQ / Redis / Keycloak in Docker so integration issues surface *before* push. +> - **Testing strategy was decided upfront, not retrofitted.** Unit tests cover handlers + domain rules with mocked infrastructure (NSubstitute, FakeTimeProvider). Integration tests use live containers to prove what mocks can't: EF migrations apply cleanly to fresh DBs, HybridCache actually invalidates on write, the `xmin` / `RowVersion` concurrency tokens actually fire under racing writes, Wolverine's transactional outbox + saga handlers actually persist and dispatch. Cross-service E2E over a live RabbitMQ wire is tracked as deferred in [STATUS.md](docs/STATUS.md). Open issues and other deferred cleanups live there too. [![NextAurora architecture — full system in one view](docs/nextaurora-architecture.svg)](docs/nextaurora-architecture.svg) -*Full system in one view — services, Service Bus topology, databases, and the 10-step order-placement saga. Click to view full-size.* +*Full system in one view — services, RabbitMQ messaging topology, databases, and the 10-step order-placement saga. Click to view full-size.* **Drill down into specific subsystems:** [service request lifecycle](#service-request-lifecycle) · [HybridCache flow](#hybridcache-flow) · [transactional outbox](#transactional-outbox) · [EF Core read and write](#ef-core-read-and-write) · [EF Core migrations](#ef-core-migrations) — all six diagrams in the [Reference diagrams](#reference-diagrams) section below. @@ -56,17 +56,17 @@ NextAurora demonstrates a production-style distributed system with event-driven +-----------------------------------------------------+ | MESSAGING LAYER | | | -| Async Messaging (Topics & Subscriptions) | -| Local/CI: Azure Service Bus emulator | -| AWS prod: Amazon SNS + SQS | +| Async Messaging (Fanout Exchanges & Queues) | +| RabbitMQ via Wolverine — same broker in | +| every environment (local dev, CI, deployment) | | | -| Topics: | +| Fanout exchanges (queue per consumer): | | order-events -----> PaymentSvc, NotificationSvc | | payment-events ---> OrderSvc, ShippingSvc, | | NotificationSvc | | shipping-events --> OrderSvc, NotificationSvc | | | -| Queue: | +| Direct queue: | | send-notification -> NotificationSvc | +-----------------------------------------------------+ | | | @@ -101,7 +101,7 @@ Orchestrated by .NET Aspire (service discovery, health checks, OpenTelemetry) - **Blazor WebAssembly** (Storefront, scaffolded — no business logic yet) - **ASP.NET Core static-file host** (SellerPortal scaffold — no UI framework chosen yet, currently serves a placeholder `index.html`) - **Entity Framework Core 10** (PostgreSQL + SQL Server) with EF migrations -- **Azure Service Bus** for async event-driven messaging +- **RabbitMQ** for async event-driven messaging (fanout exchanges + queue per consumer, via Wolverine's `WolverineFx.RabbitMQ` transport) - **Wolverine** for command/query dispatch, message handling, and the transactional outbox - **gRPC** for synchronous inter-service communication - **Keycloak + JWT Bearer** for authentication and authorization @@ -113,7 +113,7 @@ Orchestrated by .NET Aspire (service discovery, health checks, OpenTelemetry) ## Prerequisites - [.NET 10 SDK](https://dotnet.microsoft.com/download) -- [Docker Desktop](https://www.docker.com/products/docker-desktop/) — running, not just installed (Aspire spins up Postgres/SQL Server/Service Bus emulator/Keycloak/Redis as containers) +- [Docker Desktop](https://www.docker.com/products/docker-desktop/) — running, not just installed (Aspire spins up Postgres/SQL Server/RabbitMQ/Keycloak/Redis as containers) - [Aspire CLI](https://learn.microsoft.com/en-us/dotnet/aspire/) - **ASP.NET Core dev certificate** — required by the Aspire dashboard's HTTPS endpoint. One-time per machine: @@ -146,7 +146,7 @@ dotnet restore dotnet run --project NextAurora.AppHost ``` -This starts all services, databases (PostgreSQL, SQL Server), Redis, and Azure Service Bus emulator in Docker containers. The Aspire dashboard opens automatically showing all services, health status, logs, and distributed traces. +This starts all services, databases (PostgreSQL, SQL Server), Redis, and RabbitMQ (management UI on :15672) in Docker containers. The Aspire dashboard opens automatically showing all services, health status, logs, and distributed traces. 4. **Access the applications** @@ -158,7 +158,7 @@ This starts all services, databases (PostgreSQL, SQL Server), Redis, and Azure S | CatalogService API | Shown in Aspire Dashboard | | OrderService API | Shown in Aspire Dashboard | -5. **Verify it's working** — once every resource in the Aspire dashboard reaches `Running` (first boot is slow; SQL Server + Service Bus emulator take 60–90s to be healthy on cold runs): +5. **Verify it's working** — once every resource in the Aspire dashboard reaches `Running` (first boot is slow; SQL Server takes 60–90s to be healthy on cold runs): - Click `catalog-service` in the Resources tab and open its `/scalar/v1` URL - Run `GET /api/v1/products` — you should see 7 seeded products @@ -270,13 +270,13 @@ NotificationService receives ShipmentDispatchedEvent ## Observability -Every request and Service Bus message carries three identifiers through the entire chain: +Every request and RabbitMQ message carries three identifiers through the entire chain: | Field | Source | Propagated Via | |-------|--------|---------------| -| `CorrelationId` | `X-Correlation-Id` header (generated if absent) | Activity baggage → Service Bus `ApplicationProperties` | -| `UserId` | JWT `sub` claim | Activity baggage → Service Bus `ApplicationProperties` | -| `SessionId` | `X-Session-Id` header | Activity baggage → Service Bus `ApplicationProperties` | +| `CorrelationId` | `X-Correlation-Id` header (generated if absent) | Activity baggage → message envelope headers | +| `UserId` | JWT `sub` claim | Activity baggage → message envelope headers | +| `SessionId` | `X-Session-Id` header | Activity baggage → message envelope headers | These appear on **every structured log line** in every service, making it possible to search for a single `CorrelationId` and see the complete transaction timeline across all five services. @@ -286,7 +286,7 @@ Key components: - **`OutgoingContextMiddleware`** — Wolverine middleware on the outgoing side; stamps the three IDs onto outgoing message envelopes - **`WolverineEventPublisher`** — thin pass-through to `IMessageBus.PublishAsync` so domain code stays infrastructure-agnostic -Order, Payment, and Shipping run **Wolverine's transactional outbox**: outgoing events persist to a `wolverine` schema in each service's database in the same DB transaction as the entity write, then dispatch to Service Bus via a background flush. See [`docs/context-propagation.md`](docs/context-propagation.md) and [`docs/performance-and-data-correctness.md`](docs/performance-and-data-correctness.md) for full details. +Order, Payment, and Shipping run **Wolverine's transactional outbox**: outgoing events persist to a `wolverine` schema in each service's database in the same DB transaction as the entity write, then dispatch to RabbitMQ via a background flush. See [`docs/context-propagation.md`](docs/context-propagation.md) and [`docs/performance-and-data-correctness.md`](docs/performance-and-data-correctness.md) for full details. ## Performance Testing @@ -349,7 +349,7 @@ If Keycloak isn't configured (no `Authentication:Authority` and no `Keycloak:Url | Pattern | Technology | Use Case | |---------|-----------|----------| -| **Event-Driven (Async)** | Azure Service Bus | Order workflows, payment processing, shipping, notifications | +| **Event-Driven (Async)** | RabbitMQ (via Wolverine) | Order workflows, payment processing, shipping, notifications | | **gRPC (Sync)** | Protocol Buffers | Product validation during order placement | | **REST (External)** | ASP.NET Core Minimal APIs | Frontend-to-service communication | @@ -389,7 +389,7 @@ This is documented as a hard rule in [CLAUDE.md "Communication Patterns → Wolv | [Modern .NET 10 / C# 13 Features in Use](docs/dotnet-10-features.md) | Reference of the modern .NET features actively used in NextAurora — HybridCache, primary constructors, collection expressions, Asp.Versioning.Http, IExceptionHandler, Wolverine over MediatR+MassTransit, etc. Anchored in file:line. | | [Project Decisions — API, Libraries, Architecture](docs/project-decisions.md) | Reference guide: cross-cutting decisions — Minimal APIs, URL versioning, Wolverine vs MediatR, HybridCache, Keycloak, observability, every library pick + alternative considered | | [VSA vs. Clean Architecture](docs/vsa-vs-clean-architecture.md) | Portable decision guide (reusable across systems): the dependency rule vs the 4-project structure, the enforcement spectrum (convention → architecture tests → project split), how Testcontainers shifted the testing calculus, the duplication tradeoff, when to use which — NextAurora's VSA-everywhere choice as worked example | -| [Messaging Transport Selection](docs/messaging-transport-selection.md) | Portable decision guide (reusable across systems): when to use Redis Pub/Sub vs Redis Streams vs RabbitMQ vs Azure Service Bus vs AWS SNS+SQS vs Kafka/Event Hubs/Kinesis — decision axes, comparison matrix, common mistakes, NextAurora's choice as worked example | +| [Messaging Transport Selection](docs/messaging-transport-selection.md) | Portable decision guide (reusable across systems): when to use Redis Pub/Sub vs Redis Streams vs RabbitMQ vs cloud brokers (ASB, AWS SNS+SQS) vs Kafka/Event Hubs/Kinesis — decision axes, comparison matrix, common mistakes, NextAurora's RabbitMQ choice as worked example | | [Observability](docs/observability.md) | Correlation/user/session ID propagation, distributed tracing, Wolverine handler logging, DLQ handling, metrics | | [Event Replay](docs/event-replay.md) | Wolverine outbox state, where to inspect outgoing/dead-letter envelopes, `IMessageStore` API | | [Business Requirements](docs/BRD.md) | Functional requirements, implementation status, business processes, glossary | @@ -403,7 +403,7 @@ Six diagrams break the system down — one concept per visual. Each is self-cont #### Full system architecture -5 services, Service Bus topology, databases, 10-step order-placement saga, cache + outbox callouts — embedded above in the overview. +5 services, RabbitMQ messaging topology, databases, 10-step order-placement saga, cache + outbox callouts — embedded above in the overview. #### Service request lifecycle @@ -419,7 +419,7 @@ Catalog's cache flow: GetOrLoadAsync → L1 (μs) → L2 (ms) → factory (once #### Transactional outbox -The load-bearing reliability mechanism behind every cross-service event. Entity write + outbox-row write committed in ONE transaction (visual: dotted "TRANSACTION BOUNDARY" wrapping both). Background dispatcher → Service Bus → delete envelope. All failure modes spelled out. +The load-bearing reliability mechanism behind every cross-service event. Entity write + outbox-row write committed in ONE transaction (visual: dotted "TRANSACTION BOUNDARY" wrapping both). Background dispatcher → RabbitMQ → delete envelope. All failure modes spelled out. [![Transactional outbox](docs/transactional-outbox.svg)](docs/transactional-outbox.svg) diff --git a/ShippingService/Program.cs b/ShippingService/Program.cs index be6d9f81..c2041390 100644 --- a/ShippingService/Program.cs +++ b/ShippingService/Program.cs @@ -6,7 +6,7 @@ using ShippingService.Infrastructure.Data; using Scalar.AspNetCore; using Wolverine; -using Wolverine.AzureServiceBus; +using Wolverine.RabbitMQ; using Wolverine.EntityFrameworkCore; using Wolverine.Postgresql; @@ -17,23 +17,22 @@ builder.Host.UseWolverine(opts => { var connectionString = builder.Configuration.GetConnectionString("messaging")!; - var azureServiceBus = opts.UseAzureServiceBus(connectionString); - // AutoProvision creates topics/subscriptions via the Service Bus management API at host - // startup. Disabled in two environments: integration tests (fake ASB string hangs) and - // local dev (the emulator has no management API → BrokerInitializationException). The - // AppHost injects Wolverine__AutoProvision=false for the emulator. Gate on a config flag - // (defaults true) so real Azure still provisions. See OrderService/Program.cs + CLAUDE.md. + // Channel names (RabbitMQ fanout exchanges). + const string shippingEvents = "shipping-events"; + const string paymentEvents = "payment-events"; + + // RabbitMQ transport. Wolverine declares the exchange/queue/binding via AutoProvision (gated so + // it's off for transport-stubbed integration tests). See OrderService/Program.cs + CLAUDE.md. + var rabbit = opts.UseRabbitMq(factory => factory.Uri = new Uri(connectionString)); if (builder.Configuration.GetValue("Wolverine:AutoProvision", defaultValue: true)) { - azureServiceBus.AutoProvision(); + rabbit.AutoProvision(); } - - // Publish outgoing events to their topics - opts.PublishMessage().ToAzureServiceBusTopic("shipping-events"); - - // Listen to incoming events from other services - opts.ListenToAzureServiceBusSubscription("payment-events/shipping-payments-sub"); + rabbit.DeclareExchange(shippingEvents, e => e.ExchangeType = ExchangeType.Fanout); + rabbit.BindExchange(paymentEvents, ExchangeType.Fanout).ToQueue("shipping-payments"); + opts.PublishMessage().ToRabbitExchange(shippingEvents); + opts.ListenToRabbitQueue("shipping-payments"); // Transactional outbox: persist outgoing messages to Postgres in the same // transaction as the entity write, then dispatch via background flush. diff --git a/ShippingService/ShippingService.csproj b/ShippingService/ShippingService.csproj index cbe58c5b..daf450e4 100644 --- a/ShippingService/ShippingService.csproj +++ b/ShippingService/ShippingService.csproj @@ -12,7 +12,7 @@ - + diff --git a/docs/BRD.md b/docs/BRD.md index b0f2b1a5..93514994 100644 --- a/docs/BRD.md +++ b/docs/BRD.md @@ -137,15 +137,15 @@ The system is built as a set of independently deployable microservices to suppor | PERF-01 | API response time for product queries | < 200ms (p95) | | PERF-02 | Order placement end-to-end | < 500ms | | PERF-03 | gRPC product validation latency | < 50ms | -| PERF-04 | Event processing latency (Service Bus) | < 2 seconds | +| PERF-04 | Event processing latency (RabbitMQ) | < 2 seconds | ### 5.2 Reliability | ID | Requirement | Target | |----|-------------|--------| | REL-01 | Service availability | 99.9% per service | -| REL-02 | No data loss on service failure | Event-driven with at-least-once delivery | -| REL-03 | Independent service failure isolation | Service Bus decouples services | +| REL-02 | No data loss on service failure | Transactional outbox (publish side) + durable RabbitMQ queues with at-least-once redelivery (consume side; hardening tracked in #168/#169) | +| REL-03 | Independent service failure isolation | RabbitMQ decouples services | | REL-04 | Health check endpoints on all services | Implemented (/health, /alive) | ### 5.3 Scalability @@ -162,7 +162,7 @@ The system is built as a set of independently deployable microservices to suppor | ID | Requirement | Priority | Status | |----|-------------|----------|--------| | SEC-01 | API authentication (JWT/OAuth2) | High | Implemented (JWT Bearer + Keycloak realm via Aspire; `.RequireAuthorization()` on protected endpoints; buyer-scope checks on order endpoints) | -| SEC-02 | Service-to-service authentication | High | Not implemented (gRPC + Service Bus calls are inside the Aspire-managed mesh; mTLS / per-service tokens not yet configured) | +| SEC-02 | Service-to-service authentication | High | Not implemented (gRPC + RabbitMQ traffic is inside the Aspire-managed mesh; mTLS / per-service tokens not yet configured) | | SEC-03 | Input validation on all endpoints | High | Implemented (FluentValidation + Wolverine pipeline + domain guard clauses) | | SEC-04 | Secrets management | Medium | Aspire User Secrets (dev) | | SEC-05 | HTTPS enforcement | Medium | Implemented (production redirection) | @@ -265,7 +265,7 @@ The system is built as a set of independently deployable microservices to suppor ### Data Consistency Model - **Within a service:** Strong consistency (ACID transactions via EF Core) -- **Across services:** Eventual consistency (Azure Service Bus at-least-once delivery) +- **Across services:** Eventual consistency (RabbitMQ at-least-once delivery) - **Product validation at order time:** Strong consistency via synchronous gRPC call --- @@ -277,7 +277,7 @@ The system is built as a set of independently deployable microservices to suppor | Stripe (Payment Gateway) | External API | Simulated | | Email Provider (SendGrid/Twilio) | External API | Console placeholder | | Carrier APIs (FedEx/UPS/USPS/DHL) | External API | Simulated | -| Azure Service Bus | Managed Service | Implemented (emulator for dev) | +| RabbitMQ | Message Broker | Implemented (Docker container via Aspire, same broker in every environment) | | PostgreSQL | Database | Implemented (Docker) | | SQL Server | Database | Implemented (Docker) | | Redis | Cache (L2 tier) | Implemented (CatalogService via HybridCache; other services pending need) | @@ -289,7 +289,7 @@ The system is built as a set of independently deployable microservices to suppor ### Assumptions 1. Customers have a stable internet connection for the Blazor WASM storefront -2. Azure Service Bus (or emulator) is available for event processing +2. RabbitMQ is available for event processing (Aspire-managed container in every environment) 3. Payment gateway is reachable for order processing 4. Single currency per order (multi-currency across orders is supported) @@ -305,10 +305,10 @@ The system is built as a set of independently deployable microservices to suppor | Risk | Impact | Mitigation | |------|--------|-----------| -| Payment service failure | Orders placed but not processed | Service Bus queues messages; payment processes when service recovers | -| Stock oversold (concurrent orders) | Customer dissatisfaction | Optimistic concurrency tokens (`xmin`/`RowVersion`) on all aggregates; `DbUpdateConcurrencyException` → 409 on HTTP, 3-attempt Wolverine retry on Service Bus. Future: stock reservation in Catalog. | -| Event message loss | Incomplete order lifecycle | Wolverine transactional outbox in Order/Payment/Shipping (events persist to a `wolverine` schema in the same DB transaction as the entity write); Azure Service Bus at-least-once delivery + DLQ for downstream consumers. | -| Service Bus unavailable | Order pipeline halts | Wolverine outbox dispatcher retries with backoff; events stay durable on disk until the bus recovers. Aspire resilience handlers cover gRPC/HTTP. | +| Payment service failure | Orders placed but not processed | RabbitMQ queues messages; payment processes when service recovers | +| Stock oversold (concurrent orders) | Customer dissatisfaction | Optimistic concurrency tokens (`xmin`/`RowVersion`) on all aggregates; `DbUpdateConcurrencyException` → 409 on HTTP, 3-attempt Wolverine retry on the message path. Future: stock reservation in Catalog. | +| Event message loss | Incomplete order lifecycle | Wolverine transactional outbox in Order/Payment/Shipping (events persist to a `wolverine` schema in the same DB transaction as the entity write); RabbitMQ at-least-once delivery + DLQ for downstream consumers. | +| RabbitMQ unavailable | Order pipeline halts | Wolverine outbox dispatcher retries with backoff; events stay durable on disk until the broker recovers. Aspire resilience handlers cover gRPC/HTTP. | | gRPC catalog call failure | Order placement fails | HTTP resilience handler retries; future: circuit breaker with cached fallback | --- @@ -333,7 +333,7 @@ The system is built as a set of independently deployable microservices to suppor | **CQRS** | Command Query Responsibility Segregation - separating read and write operations | | **Choreography Saga** | A distributed transaction pattern where services react to events independently | | **gRPC** | A high-performance RPC framework using Protocol Buffers | -| **Service Bus** | Azure messaging service for pub/sub and queue-based communication | +| **RabbitMQ** | Open-source message broker used for pub/sub (fanout exchanges) and queue-based communication | | **Eventual Consistency** | Data across services will become consistent over time, not immediately | | **Dead Letter Queue** | A queue for messages that cannot be processed after multiple attempts | | **Idempotent** | An operation that produces the same result regardless of how many times it is executed | diff --git a/docs/STATUS.md b/docs/STATUS.md index 41f90465..d40c73c8 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -2,7 +2,7 @@ > **Read this first when picking up work.** Entry-point doc: where the project is, how to run it, what's source-of-truth where. Keep it short (~100 lines). **Open work lives in [GitHub Issues](https://github.com/emeraldleaf/NextAurora/issues)**, not here. -**Last updated:** 2026-06-14 (Wolverine 5→6 upgrade in review on branch `chore/wolverine-6-upgrade`) +**Last updated:** 2026-07-02 (RabbitMQ transport swap #159 + frontend saga narrator #167 in the merge train; auth hardening #166 merged) --- @@ -14,7 +14,9 @@ Full microservices architecture (.NET 10, Aspire, Wolverine, EF Core, choreograp **Code-side loop encoded.** CLAUDE.md is the canonical rule set; `.coderabbit.yaml` mirrors it at PR-review time; `.claude/agents/architecture-reviewer.md` Pattern Checklist applies at local review time; `.claude/skills/` holds procedures; GitHub Issues holds deferred work. Continuous Rule Encoding (CLAUDE.md) is the reflexive step that makes the loop compound. -**In-flight: Wolverine 5.39.3 → 6.8.0** (branch `chore/wolverine-6-upgrade`, all tests green). Three runtime breaking changes handled — RuntimeCompilation package split, `ServiceLocationPolicy` default flip, and the `IMessageContext` outbox-enlistment trap that broke PaymentService's Acceptor→Gateway split. Full detail in [docs/project-decisions.md "Wolverine 5→6 upgrade notes"](project-decisions.md). **Open follow-up:** verify outbox atomicity of OrderService/ShippingService external publishes (same constructor-`IMessageBus` non-enlistment) — tracked in Issues. +**In-flight: the merge train.** #166 (Keycloak token policy + fail-closed HTTPS metadata + NU1903 pin) **merged**; next #159 (RabbitMQ transport, ASB removed — full saga verified live: order → `Shipped` in seconds), then #167 (frontend saga timeline + narrator, verified in-browser against the merged stack). Wolverine is on 6.8.0 (upgrade landed). + +**Durability caveats (known, tracked, encoded in CLAUDE.md traps):** fresh-broker first-boot topology race can drop fanout publishes (#168); listeners are buffered, not durable-inbox (#169) — until both land, the no-loss guarantee is publish-side-only. Also open from the #159 ultracode review: dead-end `send-notification` queue (#170), dead observability artifacts `messages.abandoned` / `NextAurora.Messaging` (#171), real-wire failure-injection tests (#68). Drift RCA + new controls (tombstone CI audit, upgraded paraphrase hook): `.claude/audits/2026-07-02-ultracode-review-pr159.md`. --- diff --git a/docs/architecture.md b/docs/architecture.md index cff8f223..1d631d4b 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -1,6 +1,6 @@ # NextAurora Architecture -[![NextAurora architecture — full system, services, Service Bus topology, databases, and the 10-step order-placement saga](nextaurora-architecture.svg)](nextaurora-architecture.svg) +[![NextAurora architecture — full system, services, RabbitMQ messaging topology, databases, and the 10-step order-placement saga](nextaurora-architecture.svg)](nextaurora-architecture.svg) *Full system in one view. Click for full-size. Source: [`nextaurora-architecture.excalidraw`](nextaurora-architecture.excalidraw) — edit with the [VS Code Excalidraw extension](https://marketplace.visualstudio.com/items?itemName=pomdtr.excalidraw-editor) or [excalidraw.com](https://excalidraw.com).* @@ -43,7 +43,8 @@ NextAurora is a distributed e-commerce platform built as a microservices archite | +-------------v--------------+ | Async Messaging | - | (Topics & Subscriptions) | + | (RabbitMQ: fanout exchanges| + | + queue per consumer) | +---+------+------+------+---+ | | | | +-----v-+ +--v---+ +v------v+ +--------+ @@ -54,7 +55,7 @@ NextAurora is a distributed e-commerce platform built as a microservices archite SQL Server SQL Svr PostgreSQL Stateless ``` -> **Transport choice is environmental, not architectural.** Locally and in CI, the async-messaging layer is Azure Service Bus run as an Aspire-managed emulator. In production on AWS, it's Amazon SNS + SQS. Handlers and domain code don't change — see the [Deployment](#deployment) section. +> **Transport choice is environmental, not architectural.** The async-messaging broker is **RabbitMQ** in every environment — local dev, CI, and the Hetzner deployment, so dev matches prod. Wolverine abstracts the broker (handlers and domain code don't change, only a small per-service transport block), so the transport stays swappable — Azure Service Bus was evaluated and removed. See the [Messaging transport](#messaging-transport--rabbitmq) and [Deployment](#deployment) sections. ## Service Architecture @@ -104,7 +105,7 @@ See [CLAUDE.md](../CLAUDE.md#project-structure) for the canonical rule. - **Purpose:** Order lifecycle management - **Database:** SQL Server - **Exposes:** REST API (external) -- **Consumes:** CatalogService via gRPC, PaymentService and ShippingService events via Service Bus +- **Consumes:** CatalogService via gRPC, PaymentService and ShippingService events via RabbitMQ - **Entities:** Order, OrderLine - **Key Feature:** Orchestrates order state through event-driven saga @@ -112,7 +113,7 @@ See [CLAUDE.md](../CLAUDE.md#project-structure) for the canonical rule. - **Purpose:** Payment processing - **Database:** SQL Server - **Exposes:** REST API (external) -- **Consumes:** OrderService events via Service Bus +- **Consumes:** OrderService events via RabbitMQ - **Entities:** Payment, Refund - **Key Feature:** Stripe gateway integration (anti-corruption layer) @@ -120,14 +121,14 @@ See [CLAUDE.md](../CLAUDE.md#project-structure) for the canonical rule. - **Purpose:** Shipment creation and tracking - **Database:** PostgreSQL - **Exposes:** REST API (external) -- **Consumes:** PaymentService events via Service Bus +- **Consumes:** PaymentService events via RabbitMQ - **Entities:** Shipment, TrackingEvent - **Key Feature:** Auto-generates tracking numbers and assigns carriers #### NotificationService - **Purpose:** Customer notifications - **Database:** None (stateless) -- **Consumes:** OrderService and ShippingService events via Service Bus +- **Consumes:** OrderService and ShippingService events via RabbitMQ - **Entities:** NotificationRequest (in-memory) - **Key Feature:** Pluggable notification sender (console in dev, email/SMS in production) @@ -137,7 +138,7 @@ See [CLAUDE.md](../CLAUDE.md#project-structure) for the canonical rule. ### 1. Event-Driven Messaging (Async) -Used for all workflow/saga communication between services. Azure Service Bus provides at-least-once delivery with topic/subscription pub-sub model. +Used for all workflow/saga communication between services. RabbitMQ provides at-least-once delivery with a fanout-exchange / per-consumer-queue pub-sub model. **When to use:** State changes that trigger downstream workflows (order placed, payment completed, shipment dispatched). @@ -233,25 +234,25 @@ Each service owns its database. No service accesses another service's database d ### Message Topology ``` -Azure Service Bus +RabbitMQ (fanout exchanges -> bound queues) | - +-- Topic: order-events - | +-- Subscription: payment-orders-sub -> PaymentService - | +-- Subscription: notify-orders-sub -> NotificationService + +-- Exchange: order-events + | +-- Queue: payment-orders -> PaymentService + | +-- Queue: notify-orders -> NotificationService | - +-- Topic: payment-events - | +-- Subscription: order-payments-sub -> OrderService - | +-- Subscription: shipping-payments-sub -> ShippingService - | +-- Subscription: notify-payments-sub -> NotificationService + +-- Exchange: payment-events + | +-- Queue: order-payments -> OrderService + | +-- Queue: shipping-payments -> ShippingService + | +-- Queue: notify-payments -> NotificationService | - +-- Topic: shipping-events - | +-- Subscription: order-shipping-sub -> OrderService - | +-- Subscription: notify-shipping-sub -> NotificationService + +-- Exchange: shipping-events + | +-- Queue: order-shipping -> OrderService + | +-- Queue: notify-shipping -> NotificationService | - +-- Queue: send-notification -> NotificationService + +-- Queue: send-notification -> NotificationService (direct) ``` -**Subscription naming convention: `{consumer}-{source-events}-sub`.** Aspire 13 enforces globally unique subscription names within a bus namespace (the per-topic scoping behavior of Aspire 9 was dropped). Including the source-events suffix in the name keeps it readable and unique. The strings here must match the `ListenToAzureServiceBusSubscription("{topic}/{sub}")` calls in each service's `Program.cs`. +**Queue naming convention: `{consumer}-{source-events}`** (e.g. `payment-orders` = PaymentService consuming `order-events`). Each consumer gets its own queue bound to the source fanout exchange, so consumers scale and dead-letter independently. Wolverine declares the exchange, queue, and binding in each service's `Program.cs` via `DeclareExchange` / `BindExchange(...).ToQueue(...)` and provisions them with `AutoProvision()`. ### Event Contracts (NextAurora.Contracts) @@ -396,9 +397,11 @@ Storefront -> catalog-service, order-service SellerPortal -> catalog-service, order-service ``` -#### Service Bus emulator — Wolverine AutoProvision is disabled locally +#### Messaging transport — RabbitMQ -Locally the `messaging` resource runs as the Azure Service Bus **emulator** (`.RunAsEmulator()`), which implements only the AMQP data plane — **not** the management/administration HTTP API. Wolverine's `AutoProvision()` (on by default) provisions topics/subscriptions via that management API at host startup, so against the emulator it fails, retries, and the host dies with `BrokerInitializationException: Unable to initialize the Broker asb in time` — a deterministic startup crash for every Wolverine service (CatalogService has no Service Bus, so it's unaffected). The AppHost already declares the full topology (`AddServiceBusTopic` / `AddServiceBusSubscription`), so provisioning is both impossible and redundant locally. The AppHost therefore injects `Wolverine__AutoProvision=false` into the four Wolverine services (Order/Payment/Shipping/Notification); in Publish mode against real Azure the flag stays `true` so provisioning creates the entities. This mirrors the integration-test harnesses' `Wolverine:AutoProvision=false`. See CLAUDE.md "Package Management". +The `messaging` broker is **RabbitMQ**. Wolverine maps the saga's pub/sub onto **fanout exchanges** (one per event family: `order-events`, `payment-events`, `shipping-events`) with a **queue per consumer** bound to each. It runs as a single Aspire container (`AddRabbitMQ("messaging").WithManagementPlugin()`, management UI at `:15672`); `AutoProvision()` declares the exchanges/queues/bindings against the live broker (gated by `Wolverine:AutoProvision`, default on — off in integration tests, which stub the transport). RabbitMQ is the broker in **every** environment (local dev, CI, Hetzner deploy — [full-saga-deployment-plan.md](full-saga-deployment-plan.md) D3), so dev matches prod and the **full saga flows locally** — verified end-to-end (place an order → it reaches `Shipped` within seconds as Payment and Shipping consume and re-publish). + +**Why not Azure Service Bus?** It was evaluated and removed. Wolverine abstracts the transport, so swapping back is a ~5-line block per service — but carrying a second wiring earned nothing today: there's no Azure deployment, and the local ASB **emulator can't run the saga**. The 2.0.0 emulator *does* expose the management API (port 5300), but its **subscription** admin endpoints return **HTTP 500** (topics/queues return 200; verified by `curl`), so Wolverine's `AutoProvision` crashes the host; and even with AutoProvision off, Wolverine's system queues (`wolverine.retries.*`/`response.*`) can't auto-provision against the emulator while `BrokerResource.Setup` calls the management API for every endpoint regardless — so the saga never flows on it. Full investigation: [#148](https://github.com/emeraldleaf/NextAurora/issues/148). Re-add ASB (or another Wolverine-supported broker) if a cloud-managed target becomes real. ### Service Defaults (NextAurora.ServiceDefaults) @@ -428,8 +431,8 @@ All services inherit shared infrastructure configuration: ## Cross-Cutting Concerns ### Observability -- **Tracing:** OpenTelemetry distributed traces across all services (ASP.NET Core, HTTP client, gRPC client, `Azure.Messaging.ServiceBus`). Service Bus processors create consumer spans via `ActivitySource("NextAurora.Messaging")` so the full event chain is visible in the Aspire dashboard and any OTLP backend. -- **Context Propagation:** Every HTTP request and Service Bus message carries three identifiers — `CorrelationId`, `UserId`, `SessionId` — stamped by `CorrelationIdMiddleware` (HTTP) or each processor (Service Bus) into `Activity` baggage and `logger.BeginScope()`. All log lines produced by any handler automatically include these fields. See [docs/context-propagation.md](context-propagation.md). +- **Tracing:** OpenTelemetry distributed traces across all services (ASP.NET Core, HTTP client, gRPC client, `Wolverine`). Wolverine's own ActivitySource emits the message send/receive/handle spans for the saga — transport-agnostic — so the full event chain is visible in the Aspire dashboard and any OTLP backend. +- **Context Propagation:** Every HTTP request and RabbitMQ message carries three identifiers — `CorrelationId`, `UserId`, `SessionId` — stamped by `CorrelationIdMiddleware` (HTTP) or each processor (Service Bus) into `Activity` baggage and `logger.BeginScope()`. All log lines produced by any handler automatically include these fields. See [docs/context-propagation.md](context-propagation.md). - **Wolverine Pipeline Logging:** Wolverine's built-in `Policies.LogMessageStarting()` logs handler name and elapsed time. `ContextPropagationMiddleware` (in ServiceDefaults) opens a `logger.BeginScope()` so all handler log lines carry `CorrelationId`/`UserId`/`SessionId`. - **Metrics:** Business counters via `Meter("NextAurora")` in `NextAuroraMetrics`: `orders.placed`, `payments.processed` (tag: `outcome`), `shipments.dispatched`, `notifications.sent` (tag: `channel`), `messages.abandoned` (tags: `subject`, `service`). Exported via OTLP; visible in Aspire Metrics dashboard. - **Logging:** Structured logging with OpenTelemetry export @@ -517,7 +520,7 @@ Read paths never load tracked entities (would over-read columns + materialize en | **Repository (built-in)** | `DbContext` IS Unit-of-Work; `DbSet` IS Repository. No `I*Repository` wrappers. | | **Domain-Driven Design** | Aggregates with factory methods, guard clauses, encapsulated collections (`IReadOnlyList`), no public setters | | **Validation Pipeline** | FluentValidation + Wolverine `opts.UseFluentValidation()` for pre-handler validation | -| **Event-Driven Architecture** | Azure Service Bus pub/sub with topic/subscription model | +| **Event-Driven Architecture** | RabbitMQ pub/sub with fanout-exchange / per-consumer-queue model | | **Choreography Saga** | Order lifecycle managed through event chain across services | | **Anti-Corruption Layer** | StripePaymentGateway isolates domain from external payment API | | **Service Discovery** | Aspire-based automatic service resolution | @@ -530,7 +533,7 @@ Read paths never load tracked entities (would over-read columns + materialize en ### Implemented - **Input Validation** - FluentValidation on all commands via `opts.UseFluentValidation()` in Wolverine pipeline - **Wolverine Pipeline Logging** - Wolverine built-in `LogMessageStarting` + `ContextPropagationMiddleware` scope covers timing, correlation ID, elapsed time, and outcome -- **Context Propagation** - `CorrelationId`, `UserId`, `SessionId` flow through HTTP and Service Bus; see [docs/context-propagation.md](context-propagation.md) +- **Context Propagation** - `CorrelationId`, `UserId`, `SessionId` flow through HTTP and RabbitMQ; see [docs/context-propagation.md](context-propagation.md) - **Domain Invariants** - Guard clauses in all entity factory methods - **Global Exception Handling** - ProblemDetails responses, no internal state leakage - **Encapsulated Aggregates** - `IReadOnlyList` collections, private backing fields @@ -549,7 +552,7 @@ Read paths never load tracked entities (would over-read columns + materialize en - **API Gateway** - Centralized routing, rate limiting, auth - **Saga Compensation** - Rollback logic for failed payments/shipments - **Frontend Implementation** - Storefront and SellerPortal business logic -- **Cross-service integration tests over the real wire** - Single-service slices exist for all four DB-touching services (`tests/{CatalogService,OrderService,PaymentService,ShippingService}.Tests.Integration` — Testcontainers Postgres+Redis for Catalog, SQL Server for Order + Payment, Postgres for Shipping, Wolverine transports stubbed in each). The remaining gap is an end-to-end `OrderPlacedEvent → PaymentService → PaymentCompletedEvent` test over the real Azure Service Bus emulator container — wire-level cross-service coverage is still pending. See [docs/STATUS.md](STATUS.md) "After the smoke run." +- **Cross-service integration tests over the real wire** - Single-service slices exist for all four DB-touching services (`tests/{CatalogService,OrderService,PaymentService,ShippingService}.Tests.Integration` — Testcontainers Postgres+Redis for Catalog, SQL Server for Order + Payment, Postgres for Shipping, Wolverine transports stubbed in each). The remaining gap is an end-to-end `OrderPlacedEvent → PaymentService → PaymentCompletedEvent` test over a real RabbitMQ Testcontainer — wire-level cross-service coverage is still pending (the saga is verified manually end-to-end on the live RabbitMQ stack). See [docs/dev-loop.md Gap 1](dev-loop.md) + issue #68. - **Order Cancellation Flow** - Cancel event and compensation logic - **Production migration deployment step** - In dev, `MigrateDatabaseAsync()` runs at startup; production should run migrations as a separate deploy step (not in-process) to avoid races between replicas. Tooling exists; deploy automation does not. @@ -557,38 +560,38 @@ Read paths never load tracked entities (would over-read columns + materialize en ## Deployment -> **Status: planning, not implemented.** The codebase currently runs on **Azure Service Bus** in every environment (locally via the Aspire emulator). All five services use `WolverineFx.AzureServiceBus`. AppHost wires the messaging through `AddAzureServiceBus("messaging").RunAsEmulator()`. **No AWS code is in the repo yet.** This section describes the AWS migration target so the work has a plan when it's prioritized — it is not a record of work done. +> **Status: planning, not implemented.** The codebase currently runs on **RabbitMQ** in every environment (local dev, CI, and the active Hetzner deployment plan — see [full-saga-deployment-plan.md](full-saga-deployment-plan.md)). All five services use `WolverineFx.RabbitMQ`; AppHost wires `AddRabbitMQ("messaging")`. **No AWS code is in the repo yet.** This section sketches an *alternative* AWS (SNS + SQS) target; the active plan is Hetzner + RabbitMQ. ASB-era swap examples below are illustrative of Wolverine's transport-agnosticism (the "from" is RabbitMQ today). -NextAurora is built transport-agnostic via Wolverine — handlers depend on `IMessageBus` and `Envelope`, not on transport-specific types. That means the local-dev choice of Azure Service Bus (run as an Aspire emulator) does not lock the app into Azure. The intended production deployment target is **Amazon Web Services**, with **SNS + SQS** as the messaging backbone. +NextAurora is built transport-agnostic via Wolverine — handlers depend on `IMessageBus` and `Envelope`, not on transport-specific types. So the broker choice (RabbitMQ today) does not lock the app in. One possible production target is **Amazon Web Services**, with **SNS + SQS** as the messaging backbone (the active plan, however, is Hetzner + RabbitMQ). -### Why SNS + SQS over RabbitMQ +### Why SNS + SQS (not self-managed RabbitMQ) on AWS -A common confusion: "AWS deployment" doesn't imply RabbitMQ. The AWS-native equivalent of Azure Service Bus is the SNS + SQS pair — SNS provides the publish-subscribe topic surface; SQS provides per-subscriber queues. +If the target is AWS, the cloud-native equivalent of the fanout-exchange / per-consumer-queue model is the SNS + SQS pair — SNS provides the publish-subscribe topic surface; SQS provides per-subscriber queues. | Concern | Recommended | Alternative | Notes | |---|---|---|---| | Topic + per-subscriber queues | SNS topic → SQS queues | RabbitMQ on Amazon MQ | SNS+SQS is fully managed, IAM/VPC/CloudWatch native, pay-per-message. | -| Cross-cloud portability | — | RabbitMQ on Amazon MQ | Pick this only if running the same broker on AWS, Azure, GCP, on-prem matters more than first-class AWS integration. | +| Cross-cloud portability / dev-prod parity | — | RabbitMQ on Amazon MQ | Pick this to keep the exact broker the app runs today (managed RabbitMQ — the smallest possible migration). | | Event-streaming workloads | Amazon MSK (Kafka) | — | Out of scope for our saga model — different mental model (log-based). | -| Higher-level event routing | Amazon EventBridge | — | Wolverine support is limited; not a fit for the per-subscription idempotency model we use. | +| Higher-level event routing | Amazon EventBridge | — | Wolverine support is limited; not a fit for the per-consumer idempotency model we use. | ### 1:1 topology mapping -| NextAurora today (Azure) | AWS equivalent | +| NextAurora today (RabbitMQ) | AWS equivalent | |---|---| -| Topic `order-events` | SNS topic `order-events` | -| Subscription `payment-orders-sub` | SQS queue `payment-orders-sub` (subscribed to the topic) | -| Subscription `notify-orders-sub` | SQS queue `notify-orders-sub` (subscribed to the topic) | -| Topic `payment-events` + 3 subs | SNS topic + 3 SQS queues | -| Topic `shipping-events` + 2 subs | SNS topic + 2 SQS queues | +| Fanout exchange `order-events` | SNS topic `order-events` | +| Queue `payment-orders` (bound to it) | SQS queue `payment-orders` (subscribed to the topic) | +| Queue `notify-orders` (bound to it) | SQS queue `notify-orders` (subscribed to the topic) | +| Exchange `payment-events` + 3 queues | SNS topic + 3 SQS queues | +| Exchange `shipping-events` + 2 queues | SNS topic + 2 SQS queues | | Queue `send-notification` | SQS queue `send-notification` (no SNS needed; direct send) | Idempotency, dead-letter handling, and the transactional outbox all behave the same — Wolverine implements them at the framework level. ### What changes during the swap -- **`AppHost.cs`** — In production, infrastructure usually lives outside Aspire (Terraform/CDK/CloudFormation). The Aspire-driven `AddAzureServiceBus(...).RunAsEmulator()` exists only for local dev. -- **Each service's `Program.cs`** — `opts.UseAzureServiceBus(...)` becomes `opts.UseAmazonSqs(...)` (and SNS publishing config). The package reference swap is `WolverineFx.AzureServiceBus` → `WolverineFx.AmazonSqs`. +- **`AppHost.cs`** — In production, infrastructure usually lives outside Aspire (Terraform/CDK/CloudFormation). The Aspire-driven `AddRabbitMQ("messaging")` exists only for local dev. +- **Each service's `Program.cs`** — `opts.UseRabbitMq(...)` becomes `opts.UseAmazonSqs(...)` (and SNS publishing config). The package reference swap is `WolverineFx.RabbitMQ` → `WolverineFx.AmazonSqs`. - **Handlers and domain code** — zero changes. They read/write the same `Envelope.Headers`, the same `IMessageBus.PublishAsync(...)`. The `WolverineEventPublisher` adapter we already have is the seam. - **Database hosting** — Postgres → Amazon RDS for PostgreSQL or Aurora; SQL Server → RDS for SQL Server. EF Core providers stay the same. - **Identity** — Keycloak can run on ECS/EKS/EC2, or swap for **Amazon Cognito**. JWT validation in `ServiceDefaults` doesn't care which IdP issued the token, only that audience/issuer/signing match. @@ -603,14 +606,14 @@ The entire Domain layer, the entire Application layer (handlers + commands + que Phased plan, smallest blast radius first. Each phase is independently shippable. **Phase 0 — Code swap (~half day, app-level changes only).** No infra yet; just prove the codebase compiles and tests pass against the Wolverine SQS transport. -- Bump packages: `WolverineFx.AzureServiceBus` → `WolverineFx.AmazonSqs` in [Directory.Packages.props](../Directory.Packages.props) and the four service Api csprojs. -- In each service's `Program.cs`: `opts.UseAzureServiceBus(connectionString)` → `opts.UseAmazonSqs(...)`. Topic publish: `PublishMessage().ToAzureServiceBusTopic("foo")` → `.ToSnsTopic("foo")`. Subscription listen: `ListenToAzureServiceBusSubscription("foo/bar")` → `ListenToSqsQueue("bar")` (SQS doesn't have the topic-prefix path). -- Handlers, domain entities, DTOs, middleware, the `WolverineEventPublisher` adapter — all unchanged. +- Bump packages: `WolverineFx.RabbitMQ` → `WolverineFx.AmazonSqs` in [Directory.Packages.props](../Directory.Packages.props) and the four service csprojs. +- In each service's `Program.cs`: `opts.UseRabbitMq(...)` → `opts.UseAmazonSqs(...)`. Publish: `PublishMessage().ToRabbitExchange("foo")` → `.ToSnsTopic("foo")`. Listen: `ListenToRabbitQueue("bar")` → `ListenToSqsQueue("bar")` (drop the `BindExchange(...).ToQueue(...)` declarations — SNS subscriptions replace the bindings). +- Handlers, domain entities, DTOs, middleware — all unchanged. - Tests stay unit-level (no transport in unit tests). Add at least one integration test using LocalStack to exercise the new transport before going further. **Phase 1 — AWS infrastructure as code (~2-3 days).** Pure Terraform/CDK work, no app changes. - 3 SNS topics: `order-events`, `payment-events`, `shipping-events`. -- 7 SQS queues mapping to today's subscriptions (`payment-orders-sub`, `notify-orders-sub`, `order-payments-sub`, `shipping-payments-sub`, `notify-payments-sub`, `order-shipping-sub`, `notify-shipping-sub`), each subscribed to its source SNS topic. +- 7 SQS queues mapping to today's consumer queues (`payment-orders`, `notify-orders`, `order-payments`, `shipping-payments`, `notify-payments`, `order-shipping`, `notify-shipping`), each subscribed to its source SNS topic. - 1 standalone SQS queue: `send-notification` (NotificationService's direct queue, no SNS). - DLQ per queue with `maxReceiveCount` (~5). - IAM policies — each service gets a role with minimum-privilege publish/subscribe for the topics/queues it touches. @@ -647,4 +650,4 @@ Phased plan, smallest blast radius first. Each phase is independently shippable. ### Aspire's role after the migration -The Aspire AppHost stays — for local development. It continues to spin up Postgres, SQL Server, Service Bus emulator, Redis, Keycloak as containers. Production never sees it. Local dev keeps the fast inner loop; AWS deploy is a separate set of artifacts. +The Aspire AppHost stays — for local development. It continues to spin up Postgres, SQL Server, RabbitMQ, Redis, Keycloak as containers. Production never sees it. Local dev keeps the fast inner loop; AWS deploy is a separate set of artifacts. diff --git a/docs/code-flows.md b/docs/code-flows.md index 837259b9..e08b58a3 100644 --- a/docs/code-flows.md +++ b/docs/code-flows.md @@ -49,7 +49,7 @@ sequenceDiagram Order->>Order: persist Order + stage
OrderPlacedEvent in outbox
(same DB tx) Order-->>Buyer: 202 Accepted - Note over Order,Notif: From here on, async over Azure Service Bus + Note over Order,Notif: From here on, async over RabbitMQ (fanout exchange) Order--)Pay: OrderPlacedEvent Order--)Notif: OrderPlacedEvent (parallel) @@ -74,7 +74,7 @@ sequenceDiagram end ``` -The dashed (`--)`) arrows are over Service Bus; the solid arrows inside Phase 1 are synchronous HTTP/gRPC. +The dashed (`--)`) arrows are over RabbitMQ; the solid arrows are synchronous steps — HTTP/gRPC calls, in-process handler calls, and database operations. --- diff --git a/docs/code-flows/notificationservice.md b/docs/code-flows/notificationservice.md index 6b920e2f..14e1b519 100644 --- a/docs/code-flows/notificationservice.md +++ b/docs/code-flows/notificationservice.md @@ -13,14 +13,14 @@ ```mermaid sequenceDiagram autonumber - participant ASB as Azure Service Bus
(orders / payments / shipping topics) + participant MQ as RabbitMQ
(notify-orders / notify-payments /
notify-shipping queues, each bound to
its event family's fanout exchange) participant W as Wolverine consumer +
ContextPropagation middleware participant EH as NotificationEventHandlers
Features/NotificationEventHandlers.cs
(3 static overloads, return commands) participant SH as SendNotificationHandler
Features/SendNotification.cs participant Send as INotificationSender
Features/SendNotification.cs
(port) participant CS as ConsoleNotificationSender
Infrastructure/
(dev impl —
SendGrid/Twilio/SES swap in prod) - ASB->>W: OrderPlacedEvent
OR PaymentFailedEvent
OR ShipmentDispatchedEvent + MQ->>W: OrderPlacedEvent
OR PaymentFailedEvent
OR ShipmentDispatchedEvent Note over W: ContextPropagation restores
logger scope from envelope headers W->>EH: Handle(@event) Note over EH: pure event-to-command mapping —
no I/O, no state, just string formatting
(see "Why merged into one class" below) @@ -107,7 +107,7 @@ Adding a `Notification` entity with `Create()`, status enum, `private set` prope | [Features/SendNotification.cs](../../NotificationService/Features/SendNotification.cs) | The request record + `INotificationSender` port + `SendNotificationHandler` (all in one file — VSA) | | [Infrastructure/ConsoleNotificationSender.cs](../../NotificationService/Infrastructure/ConsoleNotificationSender.cs) | Dev-time `INotificationSender` impl — logs instead of sending | | [Infrastructure/DependencyInjection.cs](../../NotificationService/Infrastructure/DependencyInjection.cs) | DI wiring (registers the sender impl) | -| [Program.cs](../../NotificationService/Program.cs) | Composition root — Wolverine + ASB subscriptions + sender | +| [Program.cs](../../NotificationService/Program.cs) | Composition root — Wolverine + RabbitMQ queue bindings (notify-orders / notify-payments / notify-shipping, plus the direct send-notification queue) + sender | --- diff --git a/docs/code-flows/orderservice.md b/docs/code-flows/orderservice.md index e3c31f95..c0849360 100644 --- a/docs/code-flows/orderservice.md +++ b/docs/code-flows/orderservice.md @@ -1,12 +1,12 @@ # OrderService — code flow walkthrough -> **What this is.** A walk through the code paths a new contributor will hit first in [OrderService](../../OrderService/). OrderService is the **saga orchestrator** — every order placed here triggers a multi-step workflow that fans out to PaymentService, ShippingService, and NotificationService over Azure Service Bus, then comes back through three event handlers that mutate the Order aggregate. The diagrams below show *which files, classes, and interfaces* get touched at each step, in the order they actually execute. +> **What this is.** A walk through the code paths a new contributor will hit first in [OrderService](../../OrderService/). OrderService is the **saga orchestrator** — every order placed here triggers a multi-step workflow that fans out to PaymentService, ShippingService, and NotificationService over RabbitMQ, then comes back through three event handlers that mutate the Order aggregate. The diagrams below show *which files, classes, and interfaces* get touched at each step, in the order they actually execute. > > **Architecture style:** Vertical Slice Architecture (single csproj). Folders: [`Endpoints/`](../../OrderService/Endpoints), [`Features/`](../../OrderService/Features), [`Domain/`](../../OrderService/Domain), [`Infrastructure/`](../../OrderService/Infrastructure). Composition root: [`Program.cs`](../../OrderService/Program.cs). > > **Two flows to understand:** > 1. **Phase 1 — Request-driven (PlaceOrder):** buyer POSTs an order, OrderService validates against Catalog over gRPC, persists, and publishes `OrderPlacedEvent` via the transactional outbox. -> 2. **Phase 2 — Event-driven (saga consume):** three events come back over Service Bus — `PaymentCompletedEvent`, `PaymentFailedEvent`, `ShipmentDispatchedEvent` — each one transitions the Order through its state machine. +> 2. **Phase 2 — Event-driven (saga consume):** three events come back over RabbitMQ — `PaymentCompletedEvent`, `PaymentFailedEvent`, `ShipmentDispatchedEvent` — each one transitions the Order through its state machine. --- @@ -26,7 +26,7 @@ sequenceDiagram participant Ctx as OrderDbContext
Infrastructure/Data/OrderDbContext.cs participant Pub as IMessageContext
(Wolverine, method-injected) participant DB as SQL Server +
wolverine.outgoing_envelopes - participant ASB as Azure Service Bus
(orders topic) + participant MQ as RabbitMQ
(order-events fanout exchange) Buyer->>EP: POST /api/v1/orders
{ BuyerId, Currency, Lines[] } Note over EP: JWT sub == command.BuyerId?
else 403 Forbid @@ -61,8 +61,8 @@ sequenceDiagram Bus-->>EP: order.Id EP-->>Buyer: 202 Accepted
Location: /api/v1/orders/{id} - Note over DB,ASB: Wolverine background flush
dispatches envelope to ASB - DB->>ASB: OrderPlacedEvent + Note over DB,MQ: Wolverine background flush
dispatches envelope to RabbitMQ + DB->>MQ: OrderPlacedEvent ``` **Key wiring (in [`Program.cs`](../../OrderService/Program.cs)):** @@ -83,19 +83,19 @@ opts.AddConcurrencyRetry(); // OnException(payments topic, shipping topic) + participant MQ as RabbitMQ
(order-payments + order-shipping queues,
bound to the payment-events +
shipping-events fanout exchanges) participant W as Wolverine consumer +
ContextPropagation middleware participant H as Saga handler
(one of 3 below) participant Ctx as OrderDbContext participant Agg as Order aggregate
Domain/Order.cs participant DB as SQL Server
(orders + wolverine schema) - ASB->>W: PaymentCompletedEvent
(or PaymentFailedEvent /
ShipmentDispatchedEvent) + MQ->>W: PaymentCompletedEvent
(or PaymentFailedEvent /
ShipmentDispatchedEvent) Note over W: reads X-Correlation-Id,
X-User-Id, X-Session-Id
from envelope headers,
opens logger scope W->>H: HandleAsync(@event, ct)
(AutoApplyTransactions wraps) @@ -213,9 +213,9 @@ The handler's *code shape* is the contract — load-then-mutate-then-save is a w ## Open questions -**Per-aggregate ordering is handled via handler-level status checks + aggregate-level invariant throws + RowVersion retry, not via bus-level sessions.** Wolverine consumers on the same subscription compete, so two events for the same `OrderId` *can* be processed simultaneously by different replicas. Our defense is layered: each handler pre-checks `Status` and returns early on duplicate (idempotency); the aggregate's `MarkAsX` methods throw on invalid transitions (invariant); the `RowVersion` token rejects the stale writer (`DbUpdateConcurrencyException`); Wolverine's `AddConcurrencyRetry` policy retries 3× with backoff against the now-fresh state; the message lands in the DLQ only if all retries fail. That works in principle, and matches the "model the workflow, don't fight the queue" pattern from [Milan Jovanović's *Solving message ordering from first principles*](https://www.milanjovanovic.tech/blog/solving-message-ordering-from-first-principles). The alternative — Azure Service Bus sessions keyed on `OrderId`, with Wolverine's session-aware consumers — would give us a hard ordering guarantee but doesn't replace any of the above (sessions fix ordering, not duplicate delivery), so it's additive insurance rather than a replacement. +**Per-aggregate ordering is handled via handler-level status checks + aggregate-level invariant throws + RowVersion retry, not via broker-level ordering.** Wolverine consumers on the same queue compete, so two events for the same `OrderId` *can* be processed simultaneously by different replicas. Our defense is layered: each handler pre-checks `Status` and returns early on duplicate (idempotency); the aggregate's `MarkAsX` methods throw on invalid transitions (invariant); the `RowVersion` token rejects the stale writer (`DbUpdateConcurrencyException`); Wolverine's `AddConcurrencyRetry` policy retries 3× with backoff against the now-fresh state; the message lands in the DLQ only if all retries fail. That works in principle, and matches the "model the workflow, don't fight the queue" pattern from [Milan Jovanović's *Solving message ordering from first principles*](https://www.milanjovanovic.tech/blog/solving-message-ordering-from-first-principles). The alternative — partitioning by `OrderId` at the broker (e.g. RabbitMQ's consistent-hash exchange feeding single-active-consumer queues) — would give us a hard ordering guarantee but doesn't replace any of the above (partitioning fixes ordering, not duplicate delivery), so it's additive insurance rather than a replacement. -**The validation is undertested.** Our integration tests each create their own order, so the *concurrent same-aggregate* path the post warns about ("a subtle bug that only appears under load") is exactly the path with zero coverage. Two cheap things would change that without committing to bus sessions: (1) an integration test that fires `PaymentCompletedEvent` and `ShipmentDispatchedEvent` against the same `Order` simultaneously and asserts the final state lands at `Shipped` (not `PaymentFailed` or stuck at `Placed`); (2) a `payments_concurrency_retries_exhausted` / `orders_concurrency_retries_exhausted` counter so DLQ-bound retry exhaustion is observable in production, not invisible. If those metrics stay near zero, the state-guard pattern is validated and bus sessions are unnecessary. If they spike, that's the trigger to add sessions — evidence-driven, not architecture-astronaut-driven. There's no Inbox pattern (processed-message-ID table) today either; state guards catch most duplicates because aggregates have few valid transitions, but a proper Inbox would catch any duplicate before it reaches the handler. Add it if duplicates start appearing outside the state-guard-protected windows. +**The validation is undertested.** Our integration tests each create their own order, so the *concurrent same-aggregate* path the post warns about ("a subtle bug that only appears under load") is exactly the path with zero coverage. Two cheap things would change that without committing to broker-level partitioning: (1) an integration test that fires `PaymentCompletedEvent` and `ShipmentDispatchedEvent` against the same `Order` simultaneously and asserts the final state lands at `Shipped` (not `PaymentFailed` or stuck at `Placed`); (2) a `payments_concurrency_retries_exhausted` / `orders_concurrency_retries_exhausted` counter so DLQ-bound retry exhaustion is observable in production, not invisible. If those metrics stay near zero, the state-guard pattern is validated and broker-level partitioning is unnecessary. If they spike, that's the trigger to add it — evidence-driven, not architecture-astronaut-driven. There's no Inbox pattern (processed-message-ID table) today either; state guards catch most duplicates because aggregates have few valid transitions, but a proper Inbox would catch any duplicate before it reaches the handler. Add it if duplicates start appearing outside the state-guard-protected windows. --- diff --git a/docs/code-flows/paymentservice.md b/docs/code-flows/paymentservice.md index 23ecc03c..4bdbe557 100644 --- a/docs/code-flows/paymentservice.md +++ b/docs/code-flows/paymentservice.md @@ -1,6 +1,6 @@ # PaymentService — code flow walkthrough -> **What this is.** A walk through the code paths a new contributor will hit first in [PaymentService](../../PaymentService/). PaymentService is the **saga middle step** — it receives `OrderPlacedEvent` from OrderService over Service Bus, charges via a payment gateway (Stripe), and publishes `PaymentCompletedEvent` or `PaymentFailedEvent`. A `BackgroundService` recovery sweeper handles the "stuck in Pending" case where the gateway hangs. +> **What this is.** A walk through the code paths a new contributor will hit first in [PaymentService](../../PaymentService/). PaymentService is the **saga middle step** — it receives `OrderPlacedEvent` from OrderService over RabbitMQ, charges via a payment gateway (Stripe), and publishes `PaymentCompletedEvent` or `PaymentFailedEvent`. A `BackgroundService` recovery sweeper handles the "stuck in Pending" case where the gateway hangs. > > **Architecture style:** Vertical Slice Architecture (single csproj). Folders: [`Endpoints/`](../../PaymentService/Endpoints), [`Features/`](../../PaymentService/Features), [`Domain/`](../../PaymentService/Domain), [`Infrastructure/`](../../PaymentService/Infrastructure). Composition root: [`Program.cs`](../../PaymentService/Program.cs). > @@ -16,7 +16,7 @@ ```mermaid sequenceDiagram autonumber - participant ASB1 as Azure Service Bus
(orders topic) + participant MQ1 as RabbitMQ
(payment-orders queue, bound to the
order-events fanout exchange) participant W1 as Wolverine consumer +
ContextPropagation middleware participant OPH as OrderPlacedHandler
Features/OrderPlacedHandler.cs
(static, returns command) participant Val as ProcessPaymentCommandValidator
Features/ProcessPayment.cs
(FluentValidation) @@ -26,9 +26,9 @@ sequenceDiagram participant GW as IPaymentGateway
Infrastructure/Gateway/
StripePaymentGateway.cs participant Pub as IEventPublisher
Infrastructure/WolverineEventPublisher.cs participant DB as SQL Server +
wolverine.outgoing_envelopes - participant ASB2 as Azure Service Bus
(payments topic) + participant MQ2 as RabbitMQ
(payment-events fanout exchange) - ASB1->>W1: OrderPlacedEvent + MQ1->>W1: OrderPlacedEvent Note over W1: restores logger scope from
envelope headers (CorrelationId,
UserId, SessionId) W1->>OPH: Handle(@event) OPH-->>W1: returns ProcessPaymentCommand
(Wolverine cascading message —
no IMessageBus call needed) @@ -66,7 +66,7 @@ sequenceDiagram H->>Ctx: context.SaveChangesAsync(ct) Note over Ctx,DB: AutoApplyTransactions wraps —
UPDATE payments + outbox envelope
in ONE DB tx DB-->>H: tx commit - DB->>ASB2: dispatched to ASB
(payments topic) + DB->>MQ2: dispatched to RabbitMQ
(payment-events fanout exchange) end ``` @@ -97,7 +97,7 @@ sequenceDiagram EP-->>Admin: 200 OK + { Id } ``` -The admin endpoint matters because it's how an operator manually triggers a payment when the saga is unavailable (e.g. Service Bus outage). Same code path; same outbox guarantees. +The admin endpoint matters because it's how an operator manually triggers a payment when the saga is unavailable (e.g. RabbitMQ outage). Same code path; same outbox guarantees. --- @@ -115,7 +115,7 @@ sequenceDiagram participant Agg as Payment aggregate participant Pub as IEventPublisher participant DB as SQL Server +
wolverine.outgoing_envelopes - participant ASB as Azure Service Bus + participant MQ as RabbitMQ
(payment-events fanout exchange) loop every SweepInterval (default 60s) Tick->>Lock: TryAcquireLockAsync("payments-recovery",
TimeSpan.Zero) @@ -145,7 +145,7 @@ sequenceDiagram Note over Ctx,DB: flushes BOTH the MarkAsFailed mutation
AND the staged outbox envelope into
the ambient transaction Tick->>Ctx: tx.CommitAsync(ct) DB-->>Tick: tx commit (both rows or neither) - DB->>ASB: PaymentFailedEvent dispatched + DB->>MQ: PaymentFailedEvent dispatched end alt DbUpdateConcurrencyException diff --git a/docs/code-flows/shippingservice.md b/docs/code-flows/shippingservice.md index ccde3ae2..4552066c 100644 --- a/docs/code-flows/shippingservice.md +++ b/docs/code-flows/shippingservice.md @@ -1,6 +1,6 @@ # ShippingService — code flow walkthrough -> **What this is.** A walk through the code paths a new contributor will hit first in [ShippingService](../../ShippingService/). ShippingService is the **saga last step** — it receives `PaymentCompletedEvent` from PaymentService over Service Bus, creates a Shipment + immediately dispatches it (simulation), and publishes `ShipmentDispatchedEvent` for OrderService and NotificationService. Buyers can then query their shipment via a buyer-scoped HTTP endpoint with anti-enumeration IDOR protection. +> **What this is.** A walk through the code paths a new contributor will hit first in [ShippingService](../../ShippingService/). ShippingService is the **saga last step** — it receives `PaymentCompletedEvent` from PaymentService over RabbitMQ, creates a Shipment + immediately dispatches it (simulation), and publishes `ShipmentDispatchedEvent` for OrderService and NotificationService. Buyers can then query their shipment via a buyer-scoped HTTP endpoint with anti-enumeration IDOR protection. > > **Architecture style:** Vertical Slice Architecture (single csproj). Folders: [`Endpoints/`](../../ShippingService/Endpoints), [`Features/`](../../ShippingService/Features), [`Domain/`](../../ShippingService/Domain), [`Infrastructure/`](../../ShippingService/Infrastructure). Composition root: [`Program.cs`](../../ShippingService/Program.cs). > @@ -15,7 +15,7 @@ ```mermaid sequenceDiagram autonumber - participant ASB1 as Azure Service Bus
(payments topic) + participant MQ1 as RabbitMQ
(shipping-payments queue, bound to the
payment-events fanout exchange) participant W as Wolverine consumer +
ContextPropagation middleware participant PCH as PaymentCompletedHandler
Features/PaymentCompletedHandler.cs
(static, returns command) participant H as CreateShipmentHandler
Features/CreateShipment.cs @@ -23,9 +23,9 @@ sequenceDiagram participant Agg as Shipment aggregate
Domain/Shipment.cs participant Pub as IMessageContext
(Wolverine, method-injected) participant DB as Postgres +
wolverine.outgoing_envelopes - participant ASB2 as Azure Service Bus
(shipping topic) + participant MQ2 as RabbitMQ
(shipping-events fanout exchange) - ASB1->>W: PaymentCompletedEvent + MQ1->>W: PaymentCompletedEvent Note over W: restores logger scope from
envelope headers (CorrelationId,
UserId, SessionId) W->>PCH: Handle(@event) PCH-->>W: returns CreateShipmentCommand
(Wolverine cascading message —
no IMessageBus call needed) @@ -50,7 +50,7 @@ sequenceDiagram H->>Ctx: context.SaveChangesAsync(ct) Note over Ctx,DB: AutoApplyTransactions wraps —
INSERT shipments + INSERT tracking_events
+ outbox envelope all in ONE tx DB-->>H: tx commit - DB->>ASB2: ShipmentDispatchedEvent dispatched + DB->>MQ2: ShipmentDispatchedEvent dispatched H-->>W: shipment.Id end ``` diff --git a/docs/context-propagation.md b/docs/context-propagation.md index 25fc6d08..f018c519 100644 --- a/docs/context-propagation.md +++ b/docs/context-propagation.md @@ -73,15 +73,17 @@ Browser / App Client │ → Every log line in the handler now carries all three IDs │ ▼ - WolverineEventPublisher (outgoing async message) + OutgoingContextMiddleware (outgoing Wolverine envelope) │ - │ Reads baggage, writes to message ApplicationProperties: + │ Reads baggage, writes onto envelope headers + │ (RabbitMQ message headers on the wire): │ X-Correlation-Id, X-User-Id, X-Session-Id │ ▼ - Next Service's Processor (incoming async message) + Next Service's Wolverine Handler (incoming async message) │ - │ Reads ApplicationProperties back into Activity baggage + │ ContextPropagationMiddleware reads envelope headers + │ back into Activity baggage │ Opens logger scope — same three IDs continue │ ▼ @@ -149,8 +151,8 @@ When a service publishes an event, context would normally be lost — the messag ```csharp // Conceptual view — the middleware writes onto Envelope.Headers, which -// Wolverine maps to the underlying transport (Azure Service Bus -// ApplicationProperties for Service Bus, headers for other transports). +// Wolverine maps to the underlying transport (RabbitMQ message headers +// today; each transport maps the envelope headers to its native equivalent). envelope.Headers["X-Correlation-Id"] = correlationId; if (userId is not null) envelope.Headers["X-User-Id"] = userId; if (sessionId is not null) envelope.Headers["X-Session-Id"] = sessionId; @@ -164,7 +166,7 @@ The headers ride along with the message and are available to the receiving servi ### 4. Wolverine Message Handlers — The Entry Point for Async Messages -Wolverine discovers and invokes handler methods (`Handle(TEvent e)`) automatically for every incoming Service Bus message. `ContextPropagationMiddleware` runs before each handler, mirroring what `CorrelationIdMiddleware` does for HTTP — it reads the three IDs from `Envelope.Headers`, restores them into Activity baggage, and opens a logger scope. +Wolverine discovers and invokes handler methods (`Handle(TEvent e)`) automatically for every incoming RabbitMQ message. `ContextPropagationMiddleware` runs before each handler, mirroring what `CorrelationIdMiddleware` does for HTTP — it reads the three IDs from `Envelope.Headers`, restores them into Activity baggage, and opens a logger scope. ```csharp // Conceptual view of what ContextPropagationMiddleware.Before() does. @@ -215,7 +217,7 @@ If you add a fifth or sixth service, here is the checklist: **Outgoing events:** - Register `WolverineEventPublisher` as `IEventPublisher` in Infrastructure DI. `OutgoingContextMiddleware` stamps the three IDs onto every outgoing envelope automatically. -**Incoming Service Bus messages:** +**Incoming RabbitMQ messages:** - Wolverine + `ContextPropagationMiddleware` handles context extraction for all async message handlers automatically. No per-handler boilerplate needed. --- @@ -244,7 +246,7 @@ SessionId = "sess-abc123" CorrelationId = "a1b2c3d4e5f6" | sort by timestamp asc ``` -This last query gives you every log line — from the initial HTTP request through every Service Bus hop — in chronological order, across all five services. +This last query gives you every log line — from the initial HTTP request through every RabbitMQ hop — in chronological order, across all five services. --- diff --git a/docs/demo-deployment.md b/docs/demo-deployment.md index f778eec8..e4834cd5 100644 --- a/docs/demo-deployment.md +++ b/docs/demo-deployment.md @@ -30,14 +30,14 @@ The demo scaffolding is fully additive. Local Aspire development, the test suite **Watch-out**: if you ever export `ConnectionStrings__catalog-db=` in your local shell, a bare `dotnet run --project CatalogService` would try to talk to the remote DB. This is self-inflicted-only — `dotnet run --project NextAurora.AppHost` overrides connection strings before child processes inherit them, so Aspire-driven local runs are immune. -The [Dockerfile.catalog](../Dockerfile.catalog) and [.dockerignore](../.dockerignore) at the repo root are pure opt-in — Aspire runs the .NET services as `dotnet` processes (only infra deps like Postgres/SQL/Redis/Keycloak/ASB-emulator run in containers), so nothing in the local workflow invokes `docker build`. +The [Dockerfile.catalog](../Dockerfile.catalog) and [.dockerignore](../.dockerignore) at the repo root are pure opt-in — Aspire runs the .NET services as `dotnet` processes (only infra deps like Postgres/SQL/Redis/Keycloak/RabbitMQ run in containers), so nothing in the local workflow invokes `docker build`. ## What gets deployed (either path) - **CatalogService** as a single replica, scale-to-zero when idle - **Managed Postgres** for product/stock data (Fly Postgres or AWS RDS depending on path) - **No Redis** — HybridCache degrades to L1-only (in-process MemoryCache). Real prod would add a managed Redis for L2. -- **No Service Bus / no other services** — single-service demo. Cross-service choreography (Order → Payment → Shipping saga) doesn't fit a free-tier budget; flag this as a "would need ASB + ≥2 services" caveat when walking through the deployment. +- **No RabbitMQ / no other services** — single-service demo. Cross-service choreography (Order → Payment → Shipping saga) doesn't fit a free-tier budget; flag this as a "would need RabbitMQ + ≥2 services" caveat when walking through the deployment. --- @@ -337,7 +337,7 @@ The IAM role and OIDC provider are free to leave in place for next time. Useful when walking someone through the deployment, or as a refresher when you come back to this later. -- **"Cheap single-service demo — the production plan in [architecture.md](architecture.md) targets AWS SNS+SQS for the messaging fabric replacing Azure Service Bus, but that's a 2-3 service deployment that doesn't fit a free-tier budget."** +- **"Cheap single-service demo — the full multi-service plan in [full-saga-deployment-plan.md](full-saga-deployment-plan.md) runs the saga over RabbitMQ (same broker as local dev and CI), but that's a multi-service deployment that doesn't fit a free-tier budget."** - **"Scalar exposed via a `DemoMode` flag — normally dev-only in `Program.cs` because OpenAPI specs are reconnaissance gold. The flag default is off, so production posture is unchanged."** - **"HybridCache degrades to L1-only (in-process MemoryCache) when no `cache` connection string is set — registration is conditional. Real prod would add a managed Redis for L2 and pick up the same code path."** - **"Same `Dockerfile.catalog` deploys to either Fly.io or AWS App Runner — the only thing that varies is the orchestration layer, which is exactly the abstraction containers buy you."** diff --git a/docs/dev-loop.md b/docs/dev-loop.md index da2b2f84..35b6203e 100644 --- a/docs/dev-loop.md +++ b/docs/dev-loop.md @@ -387,12 +387,12 @@ different cadence, different lens.** | Tool | Role | |---|---| -| **.NET Aspire** | Local dev orchestration. `dotnet run --project NextAurora.AppHost` brings up all services + Postgres + SQL Server + Service Bus emulator + Redis + Keycloak in one command. Aspire dashboard at http://localhost:18888. | +| **.NET Aspire** | Local dev orchestration. `dotnet run --project NextAurora.AppHost` brings up all services + Postgres + SQL Server + RabbitMQ + Redis + Keycloak in one command. Aspire dashboard at http://localhost:18888. | | **OpenTelemetry** | Traces + metrics + logs throughout. Aspire ingests in dev; Application Insights ingests in prod. | -| **Wolverine** | In-process message bus + transactional outbox. Adapter for Azure Service Bus. | +| **Wolverine** | In-process message bus + transactional outbox. RabbitMQ transport for cross-service events. | | **Scalar UI** | Interactive API docs at `/scalar/v1` per service (dev-only). | | **Fly.io** | CatalogService demo at https://catalog-api-demo.fly.dev. Single Machine, auto-stops when idle. | -| **CorrelationId middleware** (in [NextAurora.ServiceDefaults](../NextAurora.ServiceDefaults/)) | Correlation/User/Session ID propagation across HTTP + Service Bus boundaries. | +| **CorrelationId middleware** (in [NextAurora.ServiceDefaults](../NextAurora.ServiceDefaults/)) | Correlation/User/Session ID propagation across HTTP + RabbitMQ boundaries. | --- @@ -438,20 +438,22 @@ before the first endpoint lands. That belongs in Gaps, not here. The gaps below are real. Each one is sized for how much the *actual* problem warrants — not how much could theoretically be done. -### Gap 1 — Cross-service E2E over the real Azure Service Bus wire is not tested +### Gap 1 — Cross-service E2E over the real RabbitMQ wire is not tested **What's missing:** All four integration slices use a stubbed Wolverine -transport. The actual `OrderPlacedEvent` → ASB → PaymentService consumer -round-trip is uncovered. +transport. The actual `OrderPlacedEvent` → RabbitMQ → PaymentService consumer +round-trip is uncovered in CI (it IS verified manually — the live saga runs +end-to-end on the local stack, order → Shipped in seconds). **Pragmatic solution:** Defer until needed. The stubbed-transport tests cover the load-bearing correctness (handler logic, outbox staging, EF + concurrency -tokens, idempotency); the wire itself mostly exercises Microsoft's ASB -emulator + Wolverine's adapter — the fragile last mile, not the architecture. -When this slice does land, gate it as a **manual nightly job** -(`workflow_dispatch:` or `schedule:` once a day), not every PR — the ASB -emulator container wants an MSSQL sidecar and adds ~3 minutes per run. Not -worth that tax per-PR. +tokens, idempotency); the wire itself mostly exercises RabbitMQ + Wolverine's +adapter — the fragile last mile, not the architecture. When this slice does +land, a **RabbitMQ Testcontainer** makes it far cheaper than the old +ASB-emulator plan: the broker is a single self-contained container, so it drops +the emulator's required MSSQL sidecar (the service DBs the saga harness needs +are unchanged), and AutoProvision works against it. Still gate it as a +nightly/manual job rather than per-PR until its runtime cost is measured. ### Gap 2 — No production performance baselines diff --git a/docs/dev-loop.svg b/docs/dev-loop.svg index 1215394e..b35b613f 100644 --- a/docs/dev-loop.svg +++ b/docs/dev-loop.svg @@ -1,276 +1,2 @@ - - - - - - - - Dev loop & tooling - Five stages from edit to runtime. Bold = fires autonomously. Italic = invoked manually. - - - - 1. Edit-time - IDE + Claude Code - - - - 2. Build-time - dotnet build - - - - 3. Test-time - dotnet test - - - - 4. PR-time - GitHub Actions + reviewers - - - - 5. Merge / Runtime - Aspire + Fly.io - - - - - - - - - - - - - - - CodeRabbit / arch-reviewer findings → fix → encode rule in .claude/ → push → re-review - - - - - - Edit-time - - Canonical rules - CLAUDE.md (25 KB) - .editorconfig - BannedSymbols.txt - - Hooks (autonomous) - PreToolUse: block-sync-over-async.sh - SessionStart: inject-status.sh - PostToolUse: check-claude-md-refs.sh - PostToolUse: check-file-moves.sh - - Slash commands (manual) - /new-feature-slice - /feature-spec - /article-audit - /sync-status - /check-rules - - Agents (manual) - architecture-reviewer - - Skills (auto-trigger on intent) - verification-before-completion - systematic-debugging - test-driven-development - variant-analysis - writing-plans + executing-plans - using-git-worktrees - skill-security-auditor - dotnet-performance, excalidraw-diagram - - Secondary reviewer (in-editor) - GitHub Copilot (GPT-5) - - - - - - Build-time - - Analyzers (zero warnings allowed) - Meziantou.Analyzer - SonarAnalyzer.CSharp - Roslynator.Analyzers - BannedApiAnalyzers - + BannedSymbols.txt - C# nullability (NRT) - - Build config - TreatWarningsAsErrors - Directory.Build.props - Directory.Packages.props - (central package management) - - - - - - Test-time - - Unit tests - xunit - AwesomeAssertions - NSubstitute - - Integration tests - WebApplicationFactory<Program> - Testcontainers (Docker) - Catalog: Postgres + Redis - Order: SQL Server + Wolverine - - Coverage + benchmarks - Coverlet (Cobertura XML) - reportgenerator - BenchmarkDotNet - k6 (load smoke) - - - - - - PR-time - - GitHub Actions workflows - BUILD_AND_TEST - build + unit + integration + Codecov - CodeQL (SAST) - Dependabot - NuGet weekly + Actions monthly - deploy-catalog-demo-fly.yml - - PR-side config - PULL_REQUEST_TEMPLATE.md - AI_WORKFLOW.md - copilot-instructions.md - .coderabbit.yaml - - Reviewers - CodeRabbit (LLM, autonomous) - Codecov (coverage trend) - CodeQL findings - dorny/test-reporter - architecture-reviewer agent - (manual invocation) - - - - - - Merge / Runtime - - Local dev orchestration - .NET Aspire - AppHost wires everything - Aspire dashboard (:18888) - - Messaging + data - Wolverine (outbox + saga) - Azure Service Bus emulator - Postgres + SQL Server + Redis - Keycloak (auth) - - Observability - OpenTelemetry (traces/metrics/logs) - CorrelationId / UserId / SessionId - propagated HTTP + Service Bus - Application Insights (prod) - - API surface + deploy - Scalar UI (/scalar/v1) - Fly.io (CatalogService demo) - catalog-api-demo.fly.dev - - - ━━━ flow from edit to runtime ╌╌╌ iteration loop (review findings) Bold = autonomous Italic = manual ◯ = gap with sized fix - - - - - - Gaps — and the pragmatic fix for each - Each gap is sized to the actual problem, not maximum theoretical coverage. Full rationale in dev-loop.md. - - 1. Cross-service E2E over real Azure Service Bus wire not tested - Fix: defer. Stubbed-transport tests cover load-bearing correctness. When this lands, gate as manual nightly (workflow_dispatch:), not per-PR. - - 2. No production performance baselines - Fix: baseline exactly 2 endpoints (GET /products/{id}, POST /orders) at 100 concurrent. Capture P50/P95/P99 + GC + cache hit ratio. Re-measure quarterly. - - 3. .claude/settings.json accumulates session cruft - Fix: don't build a hook. `git restore .claude/settings.json` periodically. If friction grows, add an 8-line Stop hook that strips non-whitelist entries. - - 4. GitHub Actions version-pinned, not SHA-pinned - Fix: stick with @vN + Dependabot Actions weekly (tag-move detected within ~24h). If higher assurance needed, run pin-github-action on ALL actions in one PR. - - 5. No coverage gate - Fix: codecov.yml with `coverage.status.project: target: auto, threshold: 1%`. Relative gate prevents regressions without perverse incentives of absolute thresholds. - - 6. AppHost smoke run is manual - Fix: workflow_dispatch: against the Fly demo OR self-hosted Aspire. Skip per-PR; Aspire boot is 60+ seconds and most PRs don't change smoke surface. - - 7. No secret scanning beyond CodeQL - Fix: add gitleaks/gitleaks-action@v2 (5 lines, free for public). Pair with quarterly `dotnet list package --vulnerable`. - - 8. Production migration deploy not automated - Fix: separate deploy-migrate job with `environment: production-migration` requiring manual approval. Auto-migrating on prod startup is dangerous. - - 9. CodeRabbit + architecture-reviewer feel redundant - Fix: keep both — complementary. CodeRabbit = rules-based, every PR. arch-reviewer = rule-application using CLAUDE.md, manual for non-trivial architectural changes. - - 10. No "AI-generated" commit tagging - Fix: don't add a tag. The PR template covers the disclosure layer; Co-Authored-By line that Claude Code adds is sufficient signal where it's used. - - - - - - Continuous Rule Encoding — the feedback loop - When a review surface (CodeRabbit, arch-reviewer agent, test failure, incident) finds a pattern, encode it the same session. Per CLAUDE.md "a merged fix PR without the corresponding rule is a half-finished job." - - Finding - → - Encode in 1+ of 5 surfaces: - CLAUDE.md - + - .coderabbit.yaml - + - arch-reviewer agent - + - skill/command - + - docs + paired diagrams - - Disciplines - → - cross-reference · file-move · doc-and-diagram · lean-CLAUDE.md · presence-in-loop · continue-is-the-verb - First 4 have mechanical floors (hooks / CI guards / size budget); last 2 are convention-tier (judgment about presence and stopping) - - Next cycle - → - Edit-time auto-prevented from repeating the antipattern. CodeRabbit catches future violations at PR-time. Agent surfaces them at local review. - - For when to use agent vs slash command vs skill (and how each is invoked), see .claude/README.md. - - - Full source in docs/dev-loop.md. Source .excalidraw at docs/dev-loop.excalidraw (open in Excalidraw to edit). - +Dev loop & toolingFive stages from edit to runtime. Tools that fire autonomously are filled; tools invoked manually are outlined.1. Edit-timeIDE + Claude Code2. Build-timedotnet build3. Test-timedotnet test4. PR-timeGitHub Actions + reviewers5. Merge / RuntimeAspire + Fly.io \ No newline at end of file diff --git a/docs/dotnet-10-features.md b/docs/dotnet-10-features.md index af62fb84..009237a0 100644 --- a/docs/dotnet-10-features.md +++ b/docs/dotnet-10-features.md @@ -185,7 +185,7 @@ Lets log aggregators index by `UserId` and `OrderId` — searchable, not text-gr ## 11. Wolverine — covers MediatR + MassTransit (both gone commercial) -**What it is**: Single library that handles in-process CQRS (the MediatR slot), Service Bus message dispatch (the MassTransit slot), AND the transactional outbox pattern across both. +**What it is**: Single library that handles in-process CQRS (the MediatR slot), message-broker dispatch (the MassTransit slot), AND the transactional outbox pattern across both. **Where**: Every service's `Program.cs` has `builder.Host.UseWolverine(opts => ...)`. Example: [CatalogService.Api/Program.cs:29-35](../CatalogService/Program.cs#L29-L35). diff --git a/docs/ef-core.md b/docs/ef-core.md index e2b3e708..d9d4696a 100644 --- a/docs/ef-core.md +++ b/docs/ef-core.md @@ -849,7 +849,7 @@ await eventPublisher.PublishAsync(@event, cancellationToken); return order.Id; ``` -After the handler returns: the transaction commits, both rows persist atomically, a background dispatcher reads from `wolverine.outgoing_envelopes` and sends to Service Bus. +After the handler returns: the transaction commits, both rows persist atomically, a background dispatcher reads from `wolverine.outgoing_envelopes` and sends to RabbitMQ. ### 15.4 Failure modes — all handled @@ -883,7 +883,7 @@ DbUpdateConcurrencyException => new ProblemDetails The caller refetches and decides what to do. -### 16.2 Service Bus path → Wolverine retry +### 16.2 Message path → Wolverine retry For event handlers, retry is correct: the event is still valid, the handler just needs to reload state and reapply. Wolverine policy in [NextAurora.ServiceDefaults](../NextAurora.ServiceDefaults/Extensions.cs): @@ -1141,7 +1141,7 @@ A condensed walkthrough of the key EF Core decisions in this codebase, each mapp ### "How do you handle the dual-write problem?" -> Wolverine's transactional outbox. The entity write and the outgoing message persist to a `wolverine` schema in the same DB, in the same EF transaction. After the handler returns, both commit together — neither happens without the other. A background dispatcher reads from `wolverine.outgoing_envelopes` and sends to Service Bus with retry. So "order saved but event lost" can't happen. +> Wolverine's transactional outbox. The entity write and the outgoing message persist to a `wolverine` schema in the same DB, in the same EF transaction. After the handler returns, both commit together — neither happens without the other. A background dispatcher reads from `wolverine.outgoing_envelopes` and sends to RabbitMQ with retry. So "order saved but event lost" can't happen. ### "Repository pattern over EF Core — isn't that redundant?" diff --git a/docs/event-catalog.md b/docs/event-catalog.md index 2277f09c..7133a0e8 100644 --- a/docs/event-catalog.md +++ b/docs/event-catalog.md @@ -4,19 +4,20 @@ This catalog documents every domain event in NextAurora: who publishes it, who c --- -## Topic / Subscription Matrix +## Exchange / Queue Matrix (RabbitMQ) -| Topic | Publisher | Subscription | Subscriber | +One **fanout exchange** per event family; each consumer binds its own queue (naming: `{consumer}-{source-events}`). Wolverine declares and AutoProvisions all of it from each service's `Program.cs`. + +| Exchange | Publisher | Bound queue | Consumer | |---|---|---|---| -| `order-events` | OrderService | `payment-sub` | PaymentService | -| `order-events` | OrderService | `shipping-sub` | *(reserved — ShippingService subscribes via PaymentService cascade)* | -| `order-events` | OrderService | `notify-sub` | NotificationService | -| `payment-events` | PaymentService | `order-sub` | OrderService | -| `payment-events` | PaymentService | `shipping-sub` | ShippingService | -| `payment-events` | PaymentService | `notify-sub` | NotificationService | -| `shipping-events` | ShippingService | `order-sub` | OrderService | -| `shipping-events` | ShippingService | `notify-sub` | NotificationService | -| `send-notification` *(queue)* | Any service | *(queue)* | NotificationService | +| `order-events` | OrderService | `payment-orders` | PaymentService | +| `order-events` | OrderService | `notify-orders` | NotificationService | +| `payment-events` | PaymentService | `order-payments` | OrderService | +| `payment-events` | PaymentService | `shipping-payments` | ShippingService | +| `payment-events` | PaymentService | `notify-payments` | NotificationService | +| `shipping-events` | ShippingService | `order-shipping` | OrderService | +| `shipping-events` | ShippingService | `notify-shipping` | NotificationService | +| `send-notification` *(direct queue, no exchange)* | Any service | `send-notification` | NotificationService | --- @@ -24,7 +25,7 @@ This catalog documents every domain event in NextAurora: who publishes it, who c ### `OrderPlacedEvent` -**Topic:** `order-events` +**Exchange:** `order-events` **Subject header:** `OrderPlacedEvent` **Producer:** OrderService (`PlaceOrderHandler`) **Consumers:** PaymentService → triggers payment processing; NotificationService → sends "Order Received" email @@ -42,7 +43,7 @@ This catalog documents every domain event in NextAurora: who publishes it, who c ### `PaymentCompletedEvent` -**Topic:** `payment-events` +**Exchange:** `payment-events` **Subject header:** `PaymentCompletedEvent` **Producer:** PaymentService (`ProcessPaymentHandler`) **Consumers:** OrderService → marks order as `Paid`; ShippingService → creates shipment @@ -59,7 +60,7 @@ This catalog documents every domain event in NextAurora: who publishes it, who c ### `PaymentFailedEvent` -**Topic:** `payment-events` +**Exchange:** `payment-events` **Subject header:** `PaymentFailedEvent` **Producer:** PaymentService (`ProcessPaymentHandler`) **Consumers:** OrderService → marks order as `PaymentFailed`; NotificationService → sends "Payment Failed" email @@ -76,7 +77,7 @@ This catalog documents every domain event in NextAurora: who publishes it, who c ### `ShipmentDispatchedEvent` -**Topic:** `shipping-events` +**Exchange:** `shipping-events` **Subject header:** `ShipmentDispatchedEvent` **Producer:** ShippingService (`CreateShipmentHandler`) **Consumers:** OrderService → marks order as `Shipped`; NotificationService → sends "Order Shipped" email @@ -91,7 +92,7 @@ This catalog documents every domain event in NextAurora: who publishes it, who c --- -## Commands (Service Bus Queue) +## Commands (RabbitMQ Queue) ### `SendNotificationCommand` @@ -111,7 +112,7 @@ This catalog documents every domain event in NextAurora: who publishes it, who c ## Observability Headers -All messages carry these `ApplicationProperties` on every Service Bus message: +All messages carry these headers on every RabbitMQ message envelope: | Property | Description | |---|---| @@ -128,15 +129,14 @@ These are stamped by `OutgoingContextMiddleware` (in `ServiceDefaults`) onto eve 1. **Adding a new field** — safe. All consumers ignore unknown fields during JSON deserialization. Add the field as optional (with a default) in the C# record. 2. **Renaming a field** — breaking change. Coordinate a dual-publish/dual-read migration window, or use a new event type name. 3. **Removing a field** — breaking change for any consumer that depends on it. Check all subscribers in this catalog before removing. -4. **New event type** — add it to this catalog. Add a handler in every subscriber listed in the topic matrix. Update the processor's Subject dispatch switch. +4. **New event type** — add it to this catalog. Add a Wolverine handler in every consumer listed in the exchange/queue matrix. --- ## Dead Letter Queues -Each subscription has a dead-letter sub-queue at: -`{topic}/{subscription}/$deadletterqueue` +Wolverine's RabbitMQ transport dead-letters exhausted messages to a Wolverine-managed dead-letter queue on the broker (visible alongside the consumer queues in the RabbitMQ management UI at `:15672`). -Messages land there after exceeding `MaxDeliveryCount` retries. The `messages.abandoned` OTel counter (tagged with `subject` and `service`) rises as messages approach the DLQ. Alert when this counter crosses your threshold. +Messages land there after exhausting Wolverine's retry policy. The `messages.abandoned` OTel counter (tagged with `subject` and `service`) rises as messages approach the DLQ. Alert when this counter crosses your threshold. Replay is available through Wolverine's transactional outbox tables (`wolverine` schema in each service's database) and its `IMessageStore` API. See [docs/event-replay.md](event-replay.md). diff --git a/docs/event-driven-observability.md b/docs/event-driven-observability.md index 4cba375c..30c7193a 100644 --- a/docs/event-driven-observability.md +++ b/docs/event-driven-observability.md @@ -21,37 +21,29 @@ The features below give every team member a clear view of what happened, when, a Every request, message, and log line carries three identifiers that link the entire transaction chain: -| Identifier | HTTP Header | Service Bus Property | Logger Scope Key | +| Identifier | HTTP Header | Wolverine Envelope Header | Logger Scope Key | |---|---|---|---| | Correlation ID | `X-Correlation-Id` | `X-Correlation-Id` | `CorrelationId` | | User ID | `X-User-Id` | `X-User-Id` | `UserId` | | Session ID | `X-Session-Id` | `X-Session-Id` | `SessionId` | -`CorrelationIdMiddleware` → `ContextPropagationMiddleware` → `WolverineEventPublisher` → each handler restores all three from `ApplicationProperties` into `Activity` baggage and a `logger.BeginScope()`. See **[docs/context-propagation.md](context-propagation.md)** for the full developer guide (per-component breakdown, new-service checklist, pitfalls) and **[docs/observability.md](observability.md)** for the technical reference and code patterns. +`CorrelationIdMiddleware` (HTTP entry) stamps all three into `Activity` baggage; `OutgoingContextMiddleware` writes them onto outgoing Wolverine envelope headers (RabbitMQ message headers on the wire); `ContextPropagationMiddleware` restores them from the envelope headers into `Activity` baggage and a `logger.BeginScope()` on the consuming side. See **[docs/context-propagation.md](context-propagation.md)** for the full developer guide (per-component breakdown, new-service checklist, pitfalls) and **[docs/observability.md](observability.md)** for the technical reference and code patterns. --- ## Distributed Tracing -All Service Bus processors create consumer spans via `ActivitySource("NextAurora.Messaging")`, registered in `Extensions.cs` alongside `"Azure.Messaging.ServiceBus"`. Combined with the `logger.BeginScope()` that every processor opens (carrying `CorrelationId`, `UserId`, `SessionId`, `MessageId`, `Subject`, and `DeliveryCount`), every handler log line carries full context automatically. `DeliveryCount > 1` signals a retry. See **[docs/observability.md](observability.md)** for the full OTel configuration, registered sources, and trace span diagram. +Saga message spans (send/receive/handle) come from Wolverine's own `ActivitySource("Wolverine")`, registered in `Extensions.cs` — transport-agnostic, so the full event chain is visible in the Aspire dashboard and any OTLP backend regardless of broker (RabbitMQ today). Combined with the `logger.BeginScope()` that `ContextPropagationMiddleware` opens (carrying `CorrelationId`, `UserId`, `SessionId`), every handler log line carries full context automatically. See **[docs/observability.md](observability.md)** for the full OTel configuration, registered sources, and trace span diagram. --- ## Dead Letter Queue (DLQ) Alerting -When a message handler throws, the processor calls `AbandonMessageAsync`, incrementing the message's `DeliveryCount`. Once that exceeds `MaxDeliveryCount`, Service Bus moves the message to the Dead Letter Queue. See **[docs/observability.md#dead-letter-queue-dlq-handling](observability.md)** for the full DLQ path table, investigation steps, and transport error logging. +When a message handler throws, Wolverine applies its retry/error policy; a message that exhausts its retries is dead-lettered by Wolverine's RabbitMQ transport to a Wolverine-managed dead-letter queue on the broker. Dead-lettered messages are visible in the RabbitMQ management UI (`:15672`) and in Wolverine's message store (the `wolverine` schema in each service's database). See **[docs/observability.md#dead-letter-queue-dlq-handling](observability.md)** for the topology table and investigation steps. ### The `messages.abandoned` Metric -Every processor increments the `messages.abandoned` counter when abandoning: - -```csharp -_messagesAbandoned.Add(1, - new KeyValuePair("subject", args.Message.Subject), - new KeyValuePair("service", "OrderService")); -``` - -The counter is defined in `NextAuroraMetrics` (`"NextAurora"` meter) and appears in the Aspire metrics dashboard. Configure an alert when this counter rises above your threshold to catch DLQ pile-ups before they cause user-visible outages. +The `messages.abandoned` counter is defined in `NextAuroraMetrics` (`"NextAurora"` meter) but is **not currently incremented by anything** — the processors that incremented it were deleted in the RabbitMQ/Wolverine migration. Re-wiring it into the Wolverine pipeline is a tracked follow-up; until then, monitor DLQ depth via the RabbitMQ management UI or the `wolverine` schema. --- @@ -67,22 +59,16 @@ Previously, when payment failed: ### What's Implemented Now -**OrderService** dispatches `payment-events` messages by `Subject`: +**OrderService** handles both event types via Wolverine's type-based dispatch — each event has its own handler class in `Features/`: -```csharp -if (string.Equals(subject, nameof(PaymentCompletedEvent), StringComparison.Ordinal)) - // → PaymentCompletedHandler → order.MarkAsPaid() -else if (string.Equals(subject, nameof(PaymentFailedEvent), StringComparison.Ordinal)) - // → PaymentFailedHandler → order.MarkAsPaymentFailed() -else - logger.LogWarning("Unrecognised subject '{Subject}' — completing without processing", subject); -``` +- `PaymentCompletedEvent` → `PaymentCompletedHandler` → `order.MarkAsPaid()` +- `PaymentFailedEvent` → `PaymentFailedHandler` → `order.MarkAsPaymentFailed()` **OrderService domain** gained a new status and method: - `OrderStatus.PaymentFailed` — terminal status for orders where payment was rejected - `Order.MarkAsPaymentFailed()` — enforces the invariant that only `Placed` orders can transition -**NotificationService** subscribes to `payment-events / notify-sub` and sends a "Payment Failed" email to the buyer when `PaymentFailedEvent` arrives. +**NotificationService** consumes `payment-events` via its `notify-payments` queue and sends a "Payment Failed" email to the buyer when `PaymentFailedEvent` arrives. **`PaymentFailedEvent`** now carries `BuyerId` so NotificationService can resolve the buyer's contact details without a cross-service call to OrderService. diff --git a/docs/event-replay.md b/docs/event-replay.md index 1e9e071b..b6de58a3 100644 --- a/docs/event-replay.md +++ b/docs/event-replay.md @@ -8,7 +8,7 @@ | Table | What it holds | |---|---| -| `wolverine.outgoing_envelopes` | Messages staged in a DB transaction but not yet flushed to Service Bus. The "in-flight" queue. | +| `wolverine.outgoing_envelopes` | Messages staged in a DB transaction but not yet flushed to the broker. The "in-flight" queue. | | `wolverine.incoming_envelopes` | Inbox: deduplication record for received messages. | | `wolverine.dead_letters` | Messages that exhausted retries and are no longer being processed. | diff --git a/docs/full-saga-deployment-plan.md b/docs/full-saga-deployment-plan.md index 81e705f0..7b1f881a 100644 --- a/docs/full-saga-deployment-plan.md +++ b/docs/full-saga-deployment-plan.md @@ -148,7 +148,7 @@ internal Docker network as the services. its Traefik-routed hostname for the browser-facing auth-code flow). Zero service code change. -### D3 — Messaging transport → **RabbitMQ container in deployed; dev keeps the ASB emulator (config-driven, swappable)** +### D3 — Messaging transport → **RabbitMQ in every environment (Azure Service Bus evaluated and removed)** **Changed from the Fly plan, sub-point resolved 2026-05-27.** On Fly, D3 was AWS SQS+SNS (free tier, but cross-cloud). On one box, run **RabbitMQ** as a @@ -157,15 +157,19 @@ egress. RabbitMQ over NATS because it maps cleanly onto the existing Azure Service Bus topic/subscription topology (exchanges + queues), Wolverine has a first-class RabbitMQ transport, and the management UI is a nice demo artifact. -**Resolved: RabbitMQ in the deployed environment only; dev keeps the Azure -Service Bus emulator already wired in the Aspire AppHost.** The transport is -selected by environment config so the two don't fight. Chose deployed-only over -RabbitMQ-everywhere because it leaves the working dev setup untouched, and the -dev/prod transport difference costs almost nothing here (see the -Wolverine-abstraction note below — handlers, outbox, saga are identical -regardless of transport). Can flip to RabbitMQ-everywhere later for ~free if -the ASB emulator's flakiness (see STATUS.md's smoke-test debugging arc) becomes -annoying in dev. +**Resolved (2026-06-17): RabbitMQ in *every* environment — dev, CI, and deployed (no config +switch; services call `UseRabbitMq(...)` unconditionally).** The original plan kept the ASB +emulator in dev "to leave the working dev setup untouched" — but that premise proved false: the +ASB **emulator never actually ran the saga** (its subscription admin endpoints return HTTP 500, +and Wolverine's system queues can't auto-provision against it; full arc in #148). RabbitMQ, by +contrast, runs as one clean container, AutoProvisions against the live broker, and the **full saga +flows locally** — verified end-to-end (order → `Shipped` in seconds), giving a working local saga +**and** dev/prod parity. An interim step made the transport config-selectable to prove RabbitMQ +out; the ASB wiring was then **removed entirely**, not kept as a dormant option — per the +codebase's anti-carry-debt rule, a second transport that runs nowhere is speculative coupling, not +optionality. Wolverine still abstracts the broker, so the transport-agnostic claim holds: re-adding +ASB is the same ~5-line block per service (shown below + in git history). Re-add it the day Azure +becomes a real target. **RabbitMQ licensing (verified 2026-05-27):** the core broker is **MPL 2.0, free and open-source, self-host at no cost, no vendor lock-in.** Broadcom's @@ -183,12 +187,11 @@ choice nor a future move to Azure Service Bus (e.g. if NextAurora ever goes all-Azure) is a lock-in — it's a localized config swap, not a rewrite: ```csharp -// What changes per service (Order, Payment, Shipping, Notification): -opts.UseAzureServiceBus(conn); // ← UseRabbitMq(conn) +// What changes per service (Order, Payment, Shipping, Notification) — current = RabbitMQ: +opts.UseRabbitMq(f => f.Uri = new Uri(conn)); // ← was UseAzureServiceBus(conn) opts.PublishMessage() - .ToAzureServiceBusTopic("payment-events"); // ← .ToRabbitExchange("payment-events") -opts.ListenToAzureServiceBusSubscription("order-events/payment-orders-sub"); - // ← .ListenToRabbitQueue("payment-orders") + .ToRabbitExchange("payment-events"); // ← was .ToAzureServiceBusTopic("payment-events") +opts.ListenToRabbitQueue("payment-orders"); // ← was ListenToAzureServiceBusSubscription("payment-orders-sub", c => c.TopicName = "order-events") // What does NOT change — transport-agnostic: opts.PersistMessagesWithSqlServer(db, "wolverine"); // outbox = DB concern @@ -199,10 +202,11 @@ opts.AddNextAuroraContextPropagation(); // correlation/user/session ``` **Implications:** -- `Wolverine.RabbitMQ` added to `Directory.Packages.props`. -- Each event-publishing service's `Program.cs` transport block branches on - environment: ASB emulator in `Development`, RabbitMQ in deployed. Topic→ - exchange, subscription→queue mapping. +- `WolverineFx.RabbitMQ` + `Aspire.Hosting.RabbitMQ` added; `WolverineFx.AzureServiceBus` + + `Aspire.Hosting.Azure.ServiceBus` removed. +- Each service's `Program.cs` uses RabbitMQ unconditionally (topic→exchange, + subscription→queue mapping); the AppHost provisions a single RabbitMQ container with the + management UI. Same `messaging` connection-string name. - The transactional outbox is **unaffected** — it's a DB concern (`PersistMessagesWith{SqlServer|Postgresql}`, which per D1 is SqlServer for Order+Payment, Postgresql for Catalog+Shipping), independent of the wire @@ -315,8 +319,9 @@ with stubbed Stripe. - [ ] Banner in Storefront: *"Payments are stubbed for demo safety"* **Risk callouts.** -- RabbitMQ topology — confirm exchanges/queues match the saga's event routing - (the old ASB topic/subscription names map to RabbitMQ exchanges/queues). +- RabbitMQ topology — verified locally (live saga: order → Shipped in seconds); + confirm the deployed broker gets the same exchanges/queues via Wolverine + AutoProvision (the old ASB topic/subscription names map to exchanges/queues). - Box load with all 5 services + infra running — watch RAM/CPU headroom. **Definition of done.** Place an order, watch it flow Payment (stubbed) → @@ -448,8 +453,8 @@ Existing CatalogService Fly demo (separate ledger): ~$0–$5/mo (scale-to-zero, ## Prerequisites before any phase starts -- [x] D3 sub-point confirmed (2026-05-27): RabbitMQ-deployed-only — dev keeps - the ASB emulator, transport branches on environment config +- [x] D3 resolved (2026-06-17): RabbitMQ in every environment (dev/CI/Hetzner); + Azure Service Bus evaluated and removed - [ ] Hetzner account + billing alert set - [ ] GHCR access token for Dokploy's image pulls - [ ] Domain/subdomain for the demo (for Traefik routing + Let's Encrypt) diff --git a/docs/how-it-works.md b/docs/how-it-works.md index 45047f6b..a376fe1c 100644 --- a/docs/how-it-works.md +++ b/docs/how-it-works.md @@ -341,7 +341,7 @@ return new HandlerResult(order.Id, new OrderPlacedEvent }); ``` -**Why no `bus.PublishAsync(...)` call** — `opts.Policies.AutoApplyTransactions()` wraps the handler chain in an EF transaction, and `opts.Policies.UseDurableOutboxOnAllSendingEndpoints()` makes Wolverine stage outgoing messages to the `wolverine.outgoing_envelopes` table. The entity write and the outbox row commit *together*. A background dispatcher then forwards the staged messages to Azure Service Bus with retry. This eliminates the dual-write problem (entity saved but event publish crashed, or vice versa). Full rationale: [docs/performance-and-data-correctness.md "Resolved: transactional outbox via Wolverine"](performance-and-data-correctness.md#resolved-transactional-outbox-via-wolverine). +**Why no `bus.PublishAsync(...)` call** — `opts.Policies.AutoApplyTransactions()` wraps the handler chain in an EF transaction, and `opts.Policies.UseDurableOutboxOnAllSendingEndpoints()` makes Wolverine stage outgoing messages to the `wolverine.outgoing_envelopes` table. The entity write and the outbox row commit *together*. A background dispatcher then forwards the staged messages to RabbitMQ with retry. This eliminates the dual-write problem (entity saved but event publish crashed, or vice versa). Full rationale: [docs/performance-and-data-correctness.md "Resolved: transactional outbox via Wolverine"](performance-and-data-correctness.md#resolved-transactional-outbox-via-wolverine). ### Step 6 — HTTP Response @@ -386,22 +386,23 @@ builder.Services.AddScoped(); Aspire resolves `catalog-service` to the running instance automatically — no hardcoded URLs. -### Asynchronous: Azure Service Bus via Wolverine (all workflow events) +### Asynchronous: RabbitMQ via Wolverine (all workflow events) -Used for the order fulfillment pipeline where immediate response isn't required. **Wolverine handles everything** — there is no hand-rolled `ServiceBusMessage` construction, no `ProcessMessageAsync` event handler, no manual `CompleteMessage` / `AbandonMessage` ack logic. Every concern below is configured once in `Program.cs` and the handler code is just a class with `HandleAsync`. +Used for the order fulfillment pipeline where immediate response isn't required. **Wolverine handles everything** — there is no hand-rolled `BasicPublish` call, no `EventingBasicConsumer` callback, no manual `BasicAck` / `BasicNack` logic. Every concern below is configured once in `Program.cs` and the handler code is just a class with `HandleAsync`. -**Publishing.** A handler returns the event (cascading message) or calls `bus.PublishAsync(@event)`. The outbox-aware sending endpoint stages it into `wolverine.outgoing_envelopes` in the same DB transaction as the entity write; a background dispatcher forwards it to Azure Service Bus with retry. Headers (`X-Correlation-Id`, `X-User-Id`, `X-Session-Id`) are stamped onto outgoing envelopes by `OutgoingContextMiddleware` reading from `Activity` baggage — handler code stays clean. +**Publishing.** A handler returns the event (cascading message) or calls `bus.PublishAsync(@event)`. The outbox-aware sending endpoint stages it into `wolverine.outgoing_envelopes` in the same DB transaction as the entity write; a background dispatcher forwards it to RabbitMQ with retry. Each event family publishes to a **fanout exchange** (`order-events`, `payment-events`, `shipping-events`) via `opts.PublishMessage().ToRabbitExchange("order-events")`. Headers (`X-Correlation-Id`, `X-User-Id`, `X-Session-Id`) are stamped onto outgoing envelopes by `OutgoingContextMiddleware` reading from `Activity` baggage — handler code stays clean. -**Consuming.** Wolverine subscribes to topics declared in `Program.cs`: +**Consuming.** Each consumer binds its own queue to the fanout exchange and listens to it, declared in `Program.cs`: ```csharp -opts.ListenToAzureServiceBusSubscription("order-events/payment-orders-sub") - .FromTopic("order-events"); +var rabbit = opts.UseRabbitMq(f => f.Uri = new Uri(conn)); +rabbit.BindExchange("order-events", ExchangeType.Fanout).ToQueue("payment-orders"); +opts.ListenToRabbitQueue("payment-orders"); ``` -Wolverine then discovers handler classes for the message types and dispatches each incoming envelope to the right one. The pipeline around each consumer is the same as the HTTP-side one: FluentValidation (rare for events) → `ContextPropagationMiddleware` (restores the correlation/user/session scope from envelope headers) → `AutoApplyTransactions` → handler. Idempotency guards inside handlers (status checks, "already processed" lookups) handle Service Bus's at-least-once delivery. +Wolverine's `AutoProvision()` declares the exchanges, queues, and bindings against the live broker at each service's startup (gated by `Wolverine:AutoProvision` — default on; off in integration tests, which stub the transport). Wolverine then discovers handler classes for the message types and dispatches each incoming envelope to the right one. The pipeline around each consumer is the same as the HTTP-side one: FluentValidation (rare for events) → `ContextPropagationMiddleware` (restores the correlation/user/session scope from envelope headers) → `AutoApplyTransactions` → handler. Idempotency guards inside handlers (status checks, "already processed" lookups) handle RabbitMQ's at-least-once delivery. -**Retries and DLQ.** `opts.AddConcurrencyRetry()` retries `DbUpdateConcurrencyException` 3 times with 50/100/250ms cooldowns; transient transport failures use Wolverine's defaults. After retries are exhausted, the message goes to the Service Bus dead-letter queue and surfaces as the `messages.abandoned` metric. +**Retries and DLQ.** `opts.AddConcurrencyRetry()` retries `DbUpdateConcurrencyException` 3 times with 50/100/250ms cooldowns; transient transport failures use Wolverine's defaults. After retries are exhausted, the message goes to the dead-letter queue — inspect it via the RabbitMQ management UI (`:15672`) or the `wolverine.dead_letters` table (metric wiring for DLQ alerting is tracked in #171). --- @@ -414,21 +415,21 @@ The full order lifecycle is driven by a choreography-based saga — no central o ↓ OrderService creates Order (status: Placed) ↓ - publishes OrderPlacedEvent → "order-events" topic + publishes OrderPlacedEvent → "order-events" exchange ↓ ┌───────────┴──────────┐ ↓ ↓ PaymentService NotificationService processes payment sends "Order Received" email ↓ -publishes PaymentCompletedEvent → "payment-events" topic +publishes PaymentCompletedEvent → "payment-events" exchange ↓ ┌───────────┴──────────┐ ↓ ↓ OrderService ShippingService marks Order as Paid creates Shipment, assigns carrier + tracking ↓ - publishes ShipmentDispatchedEvent → "shipping-events" topic + publishes ShipmentDispatchedEvent → "shipping-events" exchange ↓ ┌───────────┴──────────┐ ↓ ↓ @@ -455,7 +456,7 @@ Using a shared contracts project ensures all services agree on the same message ### Idempotent Event Handlers -Because Service Bus delivers messages at-least-once, handlers guard against processing the same event twice: +Because RabbitMQ delivers messages at-least-once, handlers guard against processing the same event twice: ```csharp // PaymentCompletedHandler — idempotency guard @@ -506,7 +507,7 @@ Every error response includes a `traceId` that links to the full server-side log Three context identifiers flow through every request — HTTP and async — automatically. There are no per-handler reads or writes; the middleware does it all. -| Concept | Source | HTTP / Service Bus header | Logger scope key | +| Concept | Source | HTTP / message header | Logger scope key | |---------|--------|---------------------------|------------------| | Correlation | `X-Correlation-Id` header, or generated from trace ID | `X-Correlation-Id` | `CorrelationId` | | User | JWT `sub` claim (`ClaimTypes.NameIdentifier`) | `X-User-Id` | `UserId` | @@ -539,9 +540,9 @@ var ordersDb = builder.AddSqlServer("orders-sql").AddDatabase("orders-db"); var paymentsDb = builder.AddSqlServer("payments-sql").AddDatabase("payments-db"); var shippingDb = builder.AddPostgres("shipping-pg").AddDatabase("shipping-db"); -// L2 cache + messaging — Aspire 13+ requires explicit local-dev fallbacks -var redis = builder.AddRedis("cache"); -var serviceBus = builder.AddAzureServiceBus("messaging").RunAsEmulator(); // mandatory in Aspire 13+ +// L2 cache + messaging — a real RabbitMQ container, same broker in every environment +var redis = builder.AddRedis("cache"); +var messaging = builder.AddRabbitMQ("messaging").WithManagementPlugin(); // management UI on :15672 // App Insights only when publishing — no local emulator exists IResourceBuilder? insights = null; @@ -550,22 +551,20 @@ if (builder.ExecutionContext.IsPublishMode) insights = builder.AddAzureApplicationInsights("insights"); } -// Service Bus topology — subscription names are GLOBALLY UNIQUE in the namespace -var orderTopic = serviceBus.AddServiceBusTopic("order-events"); -orderTopic.AddServiceBusSubscription("payment-orders-sub"); -orderTopic.AddServiceBusSubscription("notify-orders-sub"); -// ... payment-events and shipping-events topics, each with consumer-prefixed subs +// No broker topology declared here — each service's Wolverine setup AutoProvisions +// its own fanout exchanges (order-events, payment-events, shipping-events) and +// consumer queues at startup (gated by Wolverine:AutoProvision; off in integration tests) // Services with their dependencies — every WithReference gets a matching WaitFor // because Aspire 13's WithReference no longer waits for healthy builder.AddProject("order-service") .WithReference(ordersDb).WaitFor(ordersDb) - .WithReference(serviceBus).WaitFor(serviceBus) + .WithReference(messaging).WaitFor(messaging) .WithReference(catalogService).WaitFor(catalogService); // gRPC service discovery ``` Aspire handles: -- Spinning up Docker containers for each database, Redis, Keycloak, and the Service Bus emulator +- Spinning up Docker containers for each database, Redis, Keycloak, and RabbitMQ - Injecting connection strings into each service automatically - Resolving service names (`catalog-service`) to the correct URL - Health-check aggregation in the dashboard @@ -574,8 +573,7 @@ Every service calls `builder.AddServiceDefaults()` in `Program.cs` to register s **Aspire 13 gotchas the AppHost has to handle** (each captured in CLAUDE.md after surfacing): - Aspire SDK and runtime package versions must match exactly (or SDK ≥ packages). -- Service Bus subscription names are globally unique in the namespace — convention is `{consumer}-{source}-sub` (e.g., `payment-orders-sub`). -- `AddAzureServiceBus(...)` requires a chained `.RunAsEmulator()` for local runs. +- RabbitMQ queue names follow the `{consumer}-{source}` convention (e.g., `payment-orders` = PaymentService's queue bound to the `order-events` exchange); Wolverine AutoProvisions them at startup. - `AddAzureApplicationInsights(...)` has no local emulator — gate it on `IsPublishMode`. - Every `.WithReference(x)` on a non-trivial dependency needs a matching `.WaitFor(x)` since `WithReference` no longer waits for healthy in Aspire 13. @@ -661,7 +659,7 @@ All tests in the solution run. Each test project targets the unit tests for one | Change a domain business rule | `{Service}.Domain/Entities/` | | Add a new event type | `NextAurora.Contracts/Events/` | | Change which events a service publishes | Return them as cascading messages from the handler, or `bus.PublishAsync` | -| Change which events a service consumes | Add a handler class for the event in `{Service}.Application/Handlers/`, plus an `opts.ListenToAzureServiceBusSubscription(...)` line in `{Service}.Api/Program.cs` | +| Change which events a service consumes | Add a handler class for the event in the service's `Features/` folder, plus an `opts.ListenToRabbitQueue(...)` + `rabbit.BindExchange(...).ToQueue(...)` line in `{Service}.Api/Program.cs` | | Inspect outgoing events / outbox state | Each event-publishing service's DB has a `wolverine` schema; `outgoing_envelopes` is the staged-but-not-yet-flushed queue, `dead_letters` the DLQ. See [event-replay.md](./event-replay.md) | | Add a new gRPC method to CatalogService | `CatalogService.Api/Protos/catalog.proto` + `CatalogService.Api/Services/CatalogGrpcService.cs` (regenerate clients in OrderService) | | Add a cached read query in Catalog | `IProductCache.GetOrLoadAsync(id, factory)` — see [HybridProductCache.cs](../CatalogService/Infrastructure/Caching/HybridProductCache.cs) | diff --git a/docs/nextaurora-architecture.excalidraw b/docs/nextaurora-architecture.excalidraw index c5a9d5fe..d07dd08a 100644 --- a/docs/nextaurora-architecture.excalidraw +++ b/docs/nextaurora-architecture.excalidraw @@ -10,8 +10,8 @@ "y": 22, "width": 1810, "height": 44, - "text": "NextAurora \u2014 Saga Workflow & Architecture", - "originalText": "NextAurora \u2014 Saga Workflow & Architecture", + "text": "NextAurora — Saga Workflow & Architecture", + "originalText": "NextAurora — Saga Workflow & Architecture", "fontSize": 36, "fontFamily": 3, "textAlign": "center", @@ -47,8 +47,8 @@ "y": 70, "width": 1810, "height": 24, - "text": "5 backend services \u00b7 Wolverine transactional outbox \u00b7 Azure Service Bus (local) \u2192 AWS SNS+SQS (planned) \u00b7 Keycloak / JWT \u00b7 Aspire 13.3.0", - "originalText": "5 backend services \u00b7 Wolverine transactional outbox \u00b7 Azure Service Bus (local) \u2192 AWS SNS+SQS (planned) \u00b7 Keycloak / JWT \u00b7 Aspire 13.3.0", + "text": "5 backend services · Wolverine transactional outbox · RabbitMQ via Wolverine (local + deployed) · Keycloak / JWT · Aspire 13.3.0", + "originalText": "5 backend services · Wolverine transactional outbox · RabbitMQ via Wolverine (local + deployed) · Keycloak / JWT · Aspire 13.3.0", "fontSize": 17, "fontFamily": 3, "textAlign": "center", @@ -748,8 +748,8 @@ "y": 130, "width": 280, "height": 20, - "text": "OAuth password grant \u2192 JWT", - "originalText": "OAuth password grant \u2192 JWT", + "text": "OAuth password grant → JWT", + "originalText": "OAuth password grant → JWT", "fontSize": 12, "fontFamily": 3, "textAlign": "center", @@ -852,8 +852,8 @@ "y": 335, "width": 316.40625, "height": 156, - "text": "Postgres + Redis + gRPC server\n\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\nREST /api/v1/products (paginated)\nREST /api/v1/products/search (rate-limited)\nREST POST/PUT (seller-only, JWT)\ngRPC GetProduct, ReserveStock\n\nHandlers: Create / Update / GetById /\n GetAll / Search / ReserveStock\nIProductCache (HybridCache L1+L2 on GetById)", - "originalText": "Postgres + Redis + gRPC server\n\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\nREST /api/v1/products (paginated)\nREST /api/v1/products/search (rate-limited)\nREST POST/PUT (seller-only, JWT)\ngRPC GetProduct, ReserveStock\n\nHandlers: Create / Update / GetById /\n GetAll / Search / ReserveStock\nIProductCache (HybridCache L1+L2 on GetById)", + "text": "Postgres + Redis + gRPC server\n──────────────────────\nREST /api/v1/products (paginated)\nREST /api/v1/products/search (rate-limited)\nREST POST/PUT (seller-only, JWT)\ngRPC GetProduct, ReserveStock\n\nHandlers: Create / Update / GetById /\n GetAll / Search / ReserveStock\nIProductCache (HybridCache L1+L2 on GetById)", + "originalText": "Postgres + Redis + gRPC server\n──────────────────────\nREST /api/v1/products (paginated)\nREST /api/v1/products/search (rate-limited)\nREST POST/PUT (seller-only, JWT)\ngRPC GetProduct, ReserveStock\n\nHandlers: Create / Update / GetById /\n GetAll / Search / ReserveStock\nIProductCache (HybridCache L1+L2 on GetById)", "fontSize": 12, "fontFamily": 3, "textAlign": "left", @@ -961,8 +961,8 @@ "y": 335, "width": 316.40625, "height": 171.60000000000002, - "text": "SQL Server + Wolverine outbox\n\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\nREST POST /api/v1/orders (saga entry)\nREST GET /api/v1/orders/{id}\nREST GET /api/v1/orders/buyer/{buyerId}\nbuyerId must match JWT sub \u2192 403\n\nPlaceOrder \u2192 gRPC validate + reserve\n on Catalog \u2192 Order.Create \u2192\n AddAsync + PublishAsync (one tx)\nReacts: PaymentCompleted, ShipmentDispatched", - "originalText": "SQL Server + Wolverine outbox\n\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\nREST POST /api/v1/orders (saga entry)\nREST GET /api/v1/orders/{id}\nREST GET /api/v1/orders/buyer/{buyerId}\nbuyerId must match JWT sub \u2192 403\n\nPlaceOrder \u2192 gRPC validate + reserve\n on Catalog \u2192 Order.Create \u2192\n AddAsync + PublishAsync (one tx)\nReacts: PaymentCompleted, ShipmentDispatched", + "text": "SQL Server + Wolverine outbox\n──────────────────────\nREST POST /api/v1/orders (saga entry)\nREST GET /api/v1/orders/{id}\nREST GET /api/v1/orders/buyer/{buyerId}\nbuyerId must match JWT sub → 403\n\nPlaceOrder → gRPC validate + reserve\n on Catalog → Order.Create →\n AddAsync + PublishAsync (one tx)\nReacts: PaymentCompleted, ShipmentDispatched", + "originalText": "SQL Server + Wolverine outbox\n──────────────────────\nREST POST /api/v1/orders (saga entry)\nREST GET /api/v1/orders/{id}\nREST GET /api/v1/orders/buyer/{buyerId}\nbuyerId must match JWT sub → 403\n\nPlaceOrder → gRPC validate + reserve\n on Catalog → Order.Create →\n AddAsync + PublishAsync (one tx)\nReacts: PaymentCompleted, ShipmentDispatched", "fontSize": 12, "fontFamily": 3, "textAlign": "left", @@ -1079,8 +1079,8 @@ "y": 334.6891096348655, "width": 302.34375, "height": 171.60000000000002, - "text": "SQL Server + Wolverine outbox\n\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\nREST POST /api/v1/payments/process (admin)\nUNIQUE INDEX on OrderId (db backstop)\n\nOrderPlacedHandler (cascade msg)\n \u2192 ProcessPaymentCommand\n \u2192 Stripe gateway (anti-corruption)\n \u2192 Mark Completed/Failed\n \u2192 Publish PaymentCompleted /\n PaymentFailedEvent", - "originalText": "SQL Server + Wolverine outbox\n\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\nREST POST /api/v1/payments/process (admin)\nUNIQUE INDEX on OrderId (db backstop)\n\nOrderPlacedHandler (cascade msg)\n \u2192 ProcessPaymentCommand\n \u2192 Stripe gateway (anti-corruption)\n \u2192 Mark Completed/Failed\n \u2192 Publish PaymentCompleted /\n PaymentFailedEvent", + "text": "SQL Server + Wolverine outbox\n──────────────────────\nREST POST /api/v1/payments/process (admin)\nUNIQUE INDEX on OrderId (db backstop)\n\nOrderPlacedHandler (cascade msg)\n → ProcessPaymentCommand\n → Stripe gateway (anti-corruption)\n → Mark Completed/Failed\n → Publish PaymentCompleted /\n PaymentFailedEvent", + "originalText": "SQL Server + Wolverine outbox\n──────────────────────\nREST POST /api/v1/payments/process (admin)\nUNIQUE INDEX on OrderId (db backstop)\n\nOrderPlacedHandler (cascade msg)\n → ProcessPaymentCommand\n → Stripe gateway (anti-corruption)\n → Mark Completed/Failed\n → Publish PaymentCompleted /\n PaymentFailedEvent", "fontSize": 12, "fontFamily": 3, "textAlign": "left", @@ -1201,8 +1201,8 @@ "y": 335, "width": 302.34375, "height": 156, - "text": "Postgres + Wolverine outbox\n\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\nREST GET /api/v1/shipments/order/{orderId}\nUNIQUE INDEX on OrderId (db backstop)\n\nPaymentCompletedHandler (cascade)\n \u2192 CreateShipmentCommand\n \u2192 Shipment.Create + Dispatch\n \u2192 TrackingNumber generated\n \u2192 Publish ShipmentDispatchedEvent", - "originalText": "Postgres + Wolverine outbox\n\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\nREST GET /api/v1/shipments/order/{orderId}\nUNIQUE INDEX on OrderId (db backstop)\n\nPaymentCompletedHandler (cascade)\n \u2192 CreateShipmentCommand\n \u2192 Shipment.Create + Dispatch\n \u2192 TrackingNumber generated\n \u2192 Publish ShipmentDispatchedEvent", + "text": "Postgres + Wolverine outbox\n──────────────────────\nREST GET /api/v1/shipments/order/{orderId}\nUNIQUE INDEX on OrderId (db backstop)\n\nPaymentCompletedHandler (cascade)\n → CreateShipmentCommand\n → Shipment.Create + Dispatch\n → TrackingNumber generated\n → Publish ShipmentDispatchedEvent", + "originalText": "Postgres + Wolverine outbox\n──────────────────────\nREST GET /api/v1/shipments/order/{orderId}\nUNIQUE INDEX on OrderId (db backstop)\n\nPaymentCompletedHandler (cascade)\n → CreateShipmentCommand\n → Shipment.Create + Dispatch\n → TrackingNumber generated\n → Publish ShipmentDispatchedEvent", "fontSize": 12, "fontFamily": 3, "textAlign": "left", @@ -1319,8 +1319,8 @@ "y": 335, "width": 246.09375, "height": 202.8, - "text": "Stateless \u00b7 no DB \u00b7 no outbox\n\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\nConsumes 3 topics + 1 queue:\n order-events\n payment-events\n shipping-events\n send-notification\n\nEach event handler returns\nSendNotificationRequest \u2192\nWolverine cascades to\nSendNotificationHandler \u2192\nINotificationSender (console / SES)", - "originalText": "Stateless \u00b7 no DB \u00b7 no outbox\n\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\nConsumes 3 topics + 1 queue:\n order-events\n payment-events\n shipping-events\n send-notification\n\nEach event handler returns\nSendNotificationRequest \u2192\nWolverine cascades to\nSendNotificationHandler \u2192\nINotificationSender (console / SES)", + "text": "Stateless · no DB · no outbox\n──────────────────────\nConsumes 3 exchanges + 1 queue:\n order-events\n payment-events\n shipping-events\n send-notification\n\nEach event handler returns\nSendNotificationRequest →\nWolverine cascades to\nSendNotificationHandler →\nINotificationSender (console / SES)", + "originalText": "Stateless · no DB · no outbox\n──────────────────────\nConsumes 3 exchanges + 1 queue:\n order-events\n payment-events\n shipping-events\n send-notification\n\nEach event handler returns\nSendNotificationRequest →\nWolverine cascades to\nSendNotificationHandler →\nINotificationSender (console / SES)", "fontSize": 12, "fontFamily": 3, "textAlign": "left", @@ -1391,8 +1391,8 @@ "y": 850.1639108992975, "width": 1740, "height": 25, - "text": "Azure Service Bus \u00b7 topics & subscriptions \u00b7 Wolverine outbox dispatcher publishes here", - "originalText": "Azure Service Bus \u00b7 topics & subscriptions \u00b7 Wolverine outbox dispatcher publishes here", + "text": "RabbitMQ · fanout exchanges & queues · Wolverine outbox dispatcher publishes here", + "originalText": "RabbitMQ · fanout exchanges & queues · Wolverine outbox dispatcher publishes here", "fontSize": 17, "fontFamily": 3, "textAlign": "center", @@ -1428,8 +1428,8 @@ "y": 876.6948346162736, "width": 1740, "height": 22, - "text": "(local: Aspire emulator container \u00b7 prod target: Amazon SNS topics + SQS queues \u2014 see Deployment section)", - "originalText": "(local: Aspire emulator container \u00b7 prod target: Amazon SNS topics + SQS queues \u2014 see Deployment section)", + "text": "(local + deployed: RabbitMQ container · alt cloud target: Amazon SNS+SQS — see Deployment section)", + "originalText": "(local + deployed: RabbitMQ container · alt cloud target: Amazon SNS+SQS — see Deployment section)", "fontSize": 13, "fontFamily": 3, "textAlign": "center", @@ -1500,8 +1500,8 @@ "y": 632.9243123875752, "width": 300, "height": 28, - "text": "topic: order-events", - "originalText": "topic: order-events", + "text": "exchange: order-events", + "originalText": "exchange: order-events", "fontSize": 16, "fontFamily": 3, "textAlign": "center", @@ -1583,8 +1583,8 @@ "y": 687.3256819597833, "width": 198.046875, "height": 118.30000000000001, - "text": "Event: OrderPlacedEvent\n\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\nsub: payment-orders-sub\n \u2192 PaymentService\n\nsub: notify-orders-sub\n \u2192 NotificationService", - "originalText": "Event: OrderPlacedEvent\n\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\nsub: payment-orders-sub\n \u2192 PaymentService\n\nsub: notify-orders-sub\n \u2192 NotificationService", + "text": "Event: OrderPlacedEvent\n──────────────────────────\nqueue: payment-orders\n → PaymentService\n\nqueue: notify-orders\n → NotificationService", + "originalText": "Event: OrderPlacedEvent\n──────────────────────────\nqueue: payment-orders\n → PaymentService\n\nqueue: notify-orders\n → NotificationService", "fontSize": 13, "fontFamily": 3, "textAlign": "left", @@ -1655,8 +1655,8 @@ "y": 631.9920190983912, "width": 360, "height": 28, - "text": "topic: payment-events", - "originalText": "topic: payment-events", + "text": "exchange: payment-events", + "originalText": "exchange: payment-events", "fontSize": 16, "fontFamily": 3, "textAlign": "center", @@ -1738,8 +1738,8 @@ "y": 686.3933886705993, "width": 388.4765625, "height": 84.50000000000001, - "text": "Events: PaymentCompleted, PaymentFailed\n\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\nsub: order-payments-sub \u2192 OrderService\nsub: shipping-payments-sub \u2192 ShippingService\nsub: notify-payments-sub \u2192 NotificationService", - "originalText": "Events: PaymentCompleted, PaymentFailed\n\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\nsub: order-payments-sub \u2192 OrderService\nsub: shipping-payments-sub \u2192 ShippingService\nsub: notify-payments-sub \u2192 NotificationService", + "text": "Events: PaymentCompleted, PaymentFailed\n──────────────────────────────────\nqueue: order-payments → OrderService\nqueue: shipping-payments → ShippingService\nqueue: notify-payments → NotificationService", + "originalText": "Events: PaymentCompleted, PaymentFailed\n──────────────────────────────────\nqueue: order-payments → OrderService\nqueue: shipping-payments → ShippingService\nqueue: notify-payments → NotificationService", "fontSize": 13, "fontFamily": 3, "textAlign": "left", @@ -1814,8 +1814,8 @@ "y": 632.9341260011456, "width": 330, "height": 28, - "text": "topic: shipping-events", - "originalText": "topic: shipping-events", + "text": "exchange: shipping-events", + "originalText": "exchange: shipping-events", "fontSize": 16, "fontFamily": 3, "textAlign": "center", @@ -1897,8 +1897,8 @@ "y": 687.648316327861, "width": 358.0078125, "height": 67.60000000000001, - "text": "Event: ShipmentDispatchedEvent\n\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\nsub: order-shipping-sub \u2192 OrderService\nsub: notify-shipping-sub \u2192 NotificationService", - "originalText": "Event: ShipmentDispatchedEvent\n\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\nsub: order-shipping-sub \u2192 OrderService\nsub: notify-shipping-sub \u2192 NotificationService", + "text": "Event: ShipmentDispatchedEvent\n──────────────────────────────────\nqueue: order-shipping → OrderService\nqueue: notify-shipping → NotificationService", + "originalText": "Event: ShipmentDispatchedEvent\n──────────────────────────────────\nqueue: order-shipping → OrderService\nqueue: notify-shipping → NotificationService", "fontSize": 13, "fontFamily": 3, "textAlign": "left", @@ -2001,8 +2001,8 @@ "y": 668.983171527933, "width": 225, "height": 156, - "text": "Direct queue (not topic).\nFan-in only \u2014 single consumer.\n\nNotificationService consumes\nin addition to its 3 topic\nsubscriptions.\n\nUse: ad-hoc \"send a notification\nright now\" requests from any\nservice or admin tool.", - "originalText": "Direct queue (not topic).\nFan-in only \u2014 single consumer.\n\nNotificationService consumes\nin addition to its 3 topic\nsubscriptions.\n\nUse: ad-hoc \"send a notification\nright now\" requests from any\nservice or admin tool.", + "text": "Direct queue (not exchange).\nFan-in only — single consumer.\n\nNotificationService consumes\nin addition to its 3 exchange\nbindings.\n\nUse: ad-hoc \"send a notification\nright now\" requests from any\nservice or admin tool.", + "originalText": "Direct queue (not exchange).\nFan-in only — single consumer.\n\nNotificationService consumes\nin addition to its 3 exchange\nbindings.\n\nUse: ad-hoc \"send a notification\nright now\" requests from any\nservice or admin tool.", "fontSize": 12, "fontFamily": 3, "textAlign": "left", @@ -2073,8 +2073,8 @@ "y": 942.5, "width": 147.65625, "height": 75, - "text": "catalog-db\n(PostgreSQL 17)\n\nProducts \u00b7 Categories\nxmin concurrency", - "originalText": "catalog-db\n(PostgreSQL 17)\n\nProducts \u00b7 Categories\nxmin concurrency", + "text": "catalog-db\n(PostgreSQL 17)\n\nProducts · Categories\nxmin concurrency", + "originalText": "catalog-db\n(PostgreSQL 17)\n\nProducts · Categories\nxmin concurrency", "fontSize": 12, "fontFamily": 3, "textAlign": "center", @@ -2217,8 +2217,8 @@ "y": 942.1957779793188, "width": 309.375, "height": 75, - "text": "orders-db (SQL Server 2022)\n\nOrders \u00b7 OrderLines \u00b7 RowVersion concurrency\n+ wolverine schema (outgoing_envelopes,\n scheduled_envelopes, dead_letters)", - "originalText": "orders-db (SQL Server 2022)\n\nOrders \u00b7 OrderLines \u00b7 RowVersion concurrency\n+ wolverine schema (outgoing_envelopes,\n scheduled_envelopes, dead_letters)", + "text": "orders-db (SQL Server 2022)\n\nOrders · OrderLines · RowVersion concurrency\n+ wolverine schema (outgoing_envelopes,\n scheduled_envelopes, dead_letters)", + "originalText": "orders-db (SQL Server 2022)\n\nOrders · OrderLines · RowVersion concurrency\n+ wolverine schema (outgoing_envelopes,\n scheduled_envelopes, dead_letters)", "fontSize": 12, "fontFamily": 3, "textAlign": "center", @@ -2289,8 +2289,8 @@ "y": 942.5, "width": 253.125, "height": 75, - "text": "payments-db (SQL Server 2022)\n\nPayments \u00b7 Refunds \u00b7 UNIQUE(OrderId)\nRowVersion concurrency\n+ wolverine schema", - "originalText": "payments-db (SQL Server 2022)\n\nPayments \u00b7 Refunds \u00b7 UNIQUE(OrderId)\nRowVersion concurrency\n+ wolverine schema", + "text": "payments-db (SQL Server 2022)\n\nPayments · Refunds · UNIQUE(OrderId)\nRowVersion concurrency\n+ wolverine schema", + "originalText": "payments-db (SQL Server 2022)\n\nPayments · Refunds · UNIQUE(OrderId)\nRowVersion concurrency\n+ wolverine schema", "fontSize": 12, "fontFamily": 3, "textAlign": "center", @@ -2361,8 +2361,8 @@ "y": 942.5, "width": 309.375, "height": 75, - "text": "shipping-db (PostgreSQL 17)\n\nShipments \u00b7 TrackingEvents \u00b7 UNIQUE(OrderId)\nxmin concurrency\n+ wolverine schema", - "originalText": "shipping-db (PostgreSQL 17)\n\nShipments \u00b7 TrackingEvents \u00b7 UNIQUE(OrderId)\nxmin concurrency\n+ wolverine schema", + "text": "shipping-db (PostgreSQL 17)\n\nShipments · TrackingEvents · UNIQUE(OrderId)\nxmin concurrency\n+ wolverine schema", + "originalText": "shipping-db (PostgreSQL 17)\n\nShipments · TrackingEvents · UNIQUE(OrderId)\nxmin concurrency\n+ wolverine schema", "fontSize": 12, "fontFamily": 3, "textAlign": "center", @@ -2437,8 +2437,8 @@ "y": 942.5, "width": 260.15625, "height": 75, - "text": "External: Stripe API\n(via IPaymentGateway anti-corruption)\n\nNotificationService is stateless \u2014\nno DB, no outbox.", - "originalText": "External: Stripe API\n(via IPaymentGateway anti-corruption)\n\nNotificationService is stateless \u2014\nno DB, no outbox.", + "text": "External: Stripe API\n(via IPaymentGateway anti-corruption)\n\nNotificationService is stateless —\nno DB, no outbox.", + "originalText": "External: Stripe API\n(via IPaymentGateway anti-corruption)\n\nNotificationService is stateless —\nno DB, no outbox.", "fontSize": 12, "fontFamily": 3, "textAlign": "center", @@ -3019,8 +3019,8 @@ "y": 1080, "width": 1780, "height": 30, - "text": "Order placement saga \u00b7 end-to-end walk", - "originalText": "Order placement saga \u00b7 end-to-end walk", + "text": "Order placement saga · end-to-end walk", + "originalText": "Order placement saga · end-to-end walk", "fontSize": 22, "fontFamily": 3, "textAlign": "center", @@ -3056,8 +3056,8 @@ "y": 1130, "width": 858.921875, "height": 39.199999999999996, - "text": "\u2460 Buyer (with JWT bearer) submits POST /api/v1/orders to OrderService. Endpoint validates buyer-scope: \n JWT sub claim must match command.BuyerId, else 403.", - "originalText": "\u2460 Buyer (with JWT bearer) submits POST /api/v1/orders to OrderService. Endpoint validates buyer-scope: \n JWT sub claim must match command.BuyerId, else 403.", + "text": "① Buyer (with JWT bearer) submits POST /api/v1/orders to OrderService. Endpoint validates buyer-scope: \n JWT sub claim must match command.BuyerId, else 403.", + "originalText": "① Buyer (with JWT bearer) submits POST /api/v1/orders to OrderService. Endpoint validates buyer-scope: \n JWT sub claim must match command.BuyerId, else 403.", "fontSize": 14, "fontFamily": 3, "textAlign": "left", @@ -3093,8 +3093,8 @@ "y": 1184.701109978775, "width": 891.734375, "height": 39.199999999999996, - "text": "\u2461 PlaceOrderHandler calls CatalogService over gRPC, once per line: GetProduct (validate exists + available \n + stock) then ReserveStock (xmin token enforces atomicity).", - "originalText": "\u2461 PlaceOrderHandler calls CatalogService over gRPC, once per line: GetProduct (validate exists + available \n + stock) then ReserveStock (xmin token enforces atomicity).", + "text": "② PlaceOrderHandler calls CatalogService over gRPC, once per line: GetProduct (validate exists + available \n + stock) then ReserveStock (xmin token enforces atomicity).", + "originalText": "② PlaceOrderHandler calls CatalogService over gRPC, once per line: GetProduct (validate exists + available \n + stock) then ReserveStock (xmin token enforces atomicity).", "fontSize": 14, "fontFamily": 3, "textAlign": "left", @@ -3130,8 +3130,8 @@ "y": 1240, "width": 842.515625, "height": 39.199999999999996, - "text": "\u2462 Order.Create() validates invariants. Order + lines persisted to orders-db. OrderPlacedEvent staged \n into wolverine.outgoing_envelopes \u2014 same DB transaction. Atomic.", - "originalText": "\u2462 Order.Create() validates invariants. Order + lines persisted to orders-db. OrderPlacedEvent staged \n into wolverine.outgoing_envelopes \u2014 same DB transaction. Atomic.", + "text": "③ Order.Create() validates invariants. Order + lines persisted to orders-db. OrderPlacedEvent staged \n into wolverine.outgoing_envelopes — same DB transaction. Atomic.", + "originalText": "③ Order.Create() validates invariants. Order + lines persisted to orders-db. OrderPlacedEvent staged \n into wolverine.outgoing_envelopes — same DB transaction. Atomic.", "fontSize": 14, "fontFamily": 3, "textAlign": "left", @@ -3167,8 +3167,8 @@ "y": 1294.8700478168587, "width": 842.515625, "height": 39.199999999999996, - "text": "\u2463 Wolverine background dispatcher reads from outgoing_envelopes, publishes OrderPlacedEvent to topic \n order-events on the Service Bus. Marks PublishedAt.", - "originalText": "\u2463 Wolverine background dispatcher reads from outgoing_envelopes, publishes OrderPlacedEvent to topic \n order-events on the Service Bus. Marks PublishedAt.", + "text": "④ Wolverine background dispatcher reads from outgoing_envelopes, publishes OrderPlacedEvent to exchange \n order-events on RabbitMQ. Marks PublishedAt.", + "originalText": "④ Wolverine background dispatcher reads from outgoing_envelopes, publishes OrderPlacedEvent to exchange \n order-events on RabbitMQ. Marks PublishedAt.", "fontSize": 14, "fontFamily": 3, "textAlign": "left", @@ -3204,8 +3204,8 @@ "y": 1350, "width": 727.671875, "height": 39.199999999999996, - "text": "\u2464 Two consumers receive the event in parallel: PaymentService (payment-orders-sub) and \n NotificationService (notify-orders-sub) \u2014 \"order received\" email.", - "originalText": "\u2464 Two consumers receive the event in parallel: PaymentService (payment-orders-sub) and \n NotificationService (notify-orders-sub) \u2014 \"order received\" email.", + "text": "⑤ Two consumers receive the event in parallel: PaymentService (payment-orders) and \n NotificationService (notify-orders) — \"order received\" email.", + "originalText": "⑤ Two consumers receive the event in parallel: PaymentService (payment-orders) and \n NotificationService (notify-orders) — \"order received\" email.", "fontSize": 14, "fontFamily": 3, "textAlign": "left", @@ -3241,8 +3241,8 @@ "y": 1130, "width": 795.703125, "height": 117.6, - "text": "\u2465 PaymentService OrderPlacedHandler is a cascade \u2014 returns ProcessPaymentCommand. \n Handler checks for existing Payment by OrderId (idempotency), creates Pending Payment, calls \n Stripe gateway via IPaymentGateway.\n\n\n", - "originalText": "\u2465 PaymentService OrderPlacedHandler is a cascade \u2014 returns ProcessPaymentCommand. \n Handler checks for existing Payment by OrderId (idempotency), creates Pending Payment, calls \n Stripe gateway via IPaymentGateway.\n\n\n", + "text": "⑥ PaymentService OrderPlacedHandler is a cascade — returns ProcessPaymentCommand. \n Handler checks for existing Payment by OrderId (idempotency), creates Pending Payment, calls \n Stripe gateway via IPaymentGateway.\n\n\n", + "originalText": "⑥ PaymentService OrderPlacedHandler is a cascade — returns ProcessPaymentCommand. \n Handler checks for existing Payment by OrderId (idempotency), creates Pending Payment, calls \n Stripe gateway via IPaymentGateway.\n\n\n", "fontSize": 14, "fontFamily": 3, "textAlign": "left", @@ -3278,8 +3278,8 @@ "y": 1202.3694423287736, "width": 826.109375, "height": 39.199999999999996, - "text": "\u2466 Gateway returns success or failure. Payment.MarkAsCompleted/Failed (status guard = idempotency). \n Publish PaymentCompletedEvent or PaymentFailedEvent \u2192 payment-events.", - "originalText": "\u2466 Gateway returns success or failure. Payment.MarkAsCompleted/Failed (status guard = idempotency). \n Publish PaymentCompletedEvent or PaymentFailedEvent \u2192 payment-events.", + "text": "⑦ Gateway returns success or failure. Payment.MarkAsCompleted/Failed (status guard = idempotency). \n Publish PaymentCompletedEvent or PaymentFailedEvent → payment-events.", + "originalText": "⑦ Gateway returns success or failure. Payment.MarkAsCompleted/Failed (status guard = idempotency). \n Publish PaymentCompletedEvent or PaymentFailedEvent → payment-events.", "fontSize": 14, "fontFamily": 3, "textAlign": "left", @@ -3315,8 +3315,8 @@ "y": 1257.7972464195939, "width": 850.71875, "height": 39.199999999999996, - "text": "\u2467 payment-events fans out to 3 subs: order-payments-sub (OrderService transitions Order Placed\u2192Paid), \n shipping-payments-sub (creates shipment), notify-payments-sub.", - "originalText": "\u2467 payment-events fans out to 3 subs: order-payments-sub (OrderService transitions Order Placed\u2192Paid), \n shipping-payments-sub (creates shipment), notify-payments-sub.", + "text": "⑧ payment-events fans out to 3 queues: order-payments (OrderService transitions Order Placed→Paid), \n shipping-payments (creates shipment), notify-payments.", + "originalText": "⑧ payment-events fans out to 3 queues: order-payments (OrderService transitions Order Placed→Paid), \n shipping-payments (creates shipment), notify-payments.", "fontSize": 14, "fontFamily": 3, "textAlign": "left", @@ -3352,8 +3352,8 @@ "y": 1312.6360030773835, "width": 809.703125, "height": 39.199999999999996, - "text": "\u2468 ShippingService PaymentCompletedHandler cascades CreateShipmentCommand. Creates Shipment, runs \n Dispatch() (Created\u2192Dispatched), generates tracking number. Publish ShipmentDispatchedEvent.", - "originalText": "\u2468 ShippingService PaymentCompletedHandler cascades CreateShipmentCommand. Creates Shipment, runs \n Dispatch() (Created\u2192Dispatched), generates tracking number. Publish ShipmentDispatchedEvent.", + "text": "⑨ ShippingService PaymentCompletedHandler cascades CreateShipmentCommand. Creates Shipment, runs \n Dispatch() (Created→Dispatched), generates tracking number. Publish ShipmentDispatchedEvent.", + "originalText": "⑨ ShippingService PaymentCompletedHandler cascades CreateShipmentCommand. Creates Shipment, runs \n Dispatch() (Created→Dispatched), generates tracking number. Publish ShipmentDispatchedEvent.", "fontSize": 14, "fontFamily": 3, "textAlign": "left", @@ -3389,8 +3389,8 @@ "y": 1367.6360030773835, "width": 826.109375, "height": 39.199999999999996, - "text": "\u2469 shipping-events fans out: order-shipping-sub (OrderService Paid\u2192Shipped) and notify-shipping-sub \n (\"shipped\" email with tracking number). Saga complete.", - "originalText": "\u2469 shipping-events fans out: order-shipping-sub (OrderService Paid\u2192Shipped) and notify-shipping-sub \n (\"shipped\" email with tracking number). Saga complete.", + "text": "⑩ shipping-events fans out: order-shipping (OrderService Paid→Shipped) and notify-shipping \n (\"shipped\" email with tracking number). Saga complete.", + "originalText": "⑩ shipping-events fans out: order-shipping (OrderService Paid→Shipped) and notify-shipping \n (\"shipped\" email with tracking number). Saga complete.", "fontSize": 14, "fontFamily": 3, "textAlign": "left", @@ -3456,8 +3456,8 @@ "y": 1463, "width": 840, "height": 28, - "text": "HybridCache (L1+L2) \u2014 IProductCache (CatalogService)", - "originalText": "HybridCache (L1+L2) \u2014 IProductCache (CatalogService)", + "text": "HybridCache (L1+L2) — IProductCache (CatalogService)", + "originalText": "HybridCache (L1+L2) — IProductCache (CatalogService)", "fontSize": 17, "fontFamily": 3, "textAlign": "center", @@ -3493,8 +3493,8 @@ "y": 1497.0097153239103, "width": 857.8125, "height": 218.39999999999998, - "text": "Read path (GetProductByIdHandler):\n cache.GetOrLoadAsync(productId, factory)\n L1 (MemoryCache, in-proc) HIT \u2192 return ProductDto (microseconds, no network)\n L1 MISS \u2192 L2 (Redis) HIT \u2192 promote to L1, return\n L2 MISS \u2192 factory runs ONCE under stampede \u2192 repo.GetByIdAsync \u2192 project DTO \u2192 store L1+L2 \u2192 return\n\nWrite path (UpdateProductHandler / ReserveStockHandler):\n mutate aggregate \u2192 repository.UpdateAsync \u2192 cache.InvalidateAsync(id) \u2190 tag-based; clears L2 + this replica's L1.\n\nKeys: catalog:product:{guid} \u00b7 Tag: product:{guid} \u00b7 TTL: 5 min absolute on both tiers (safety net).\nCached unit: ProductDto (immutable projection), not the EF entity \u2014 caching tracked entities is a footgun.\nNot cached: GetAllProducts / SearchProducts \u2014 long-tail keys + cross-page invalidation kill the hit ratio.\nSingle-replica today. Multi-replica L1 invalidation needs FusionCache backplane \u2014 see perf doc.", - "originalText": "Read path (GetProductByIdHandler):\n cache.GetOrLoadAsync(productId, factory)\n L1 (MemoryCache, in-proc) HIT \u2192 return ProductDto (microseconds, no network)\n L1 MISS \u2192 L2 (Redis) HIT \u2192 promote to L1, return\n L2 MISS \u2192 factory runs ONCE under stampede \u2192 repo.GetByIdAsync \u2192 project DTO \u2192 store L1+L2 \u2192 return\n\nWrite path (UpdateProductHandler / ReserveStockHandler):\n mutate aggregate \u2192 repository.UpdateAsync \u2192 cache.InvalidateAsync(id) \u2190 tag-based; clears L2 + this replica's L1.\n\nKeys: catalog:product:{guid} \u00b7 Tag: product:{guid} \u00b7 TTL: 5 min absolute on both tiers (safety net).\nCached unit: ProductDto (immutable projection), not the EF entity \u2014 caching tracked entities is a footgun.\nNot cached: GetAllProducts / SearchProducts \u2014 long-tail keys + cross-page invalidation kill the hit ratio.\nSingle-replica today. Multi-replica L1 invalidation needs FusionCache backplane \u2014 see perf doc.", + "text": "Read path (GetProductByIdHandler):\n cache.GetOrLoadAsync(productId, factory)\n L1 (MemoryCache, in-proc) HIT → return ProductDto (microseconds, no network)\n L1 MISS → L2 (Redis) HIT → promote to L1, return\n L2 MISS → factory runs ONCE under stampede → repo.GetByIdAsync → project DTO → store L1+L2 → return\n\nWrite path (UpdateProductHandler / ReserveStockHandler):\n mutate aggregate → repository.UpdateAsync → cache.InvalidateAsync(id) ← tag-based; clears L2 + this replica's L1.\n\nKeys: catalog:product:{guid} · Tag: product:{guid} · TTL: 5 min absolute on both tiers (safety net).\nCached unit: ProductDto (immutable projection), not the EF entity — caching tracked entities is a footgun.\nNot cached: GetAllProducts / SearchProducts — long-tail keys + cross-page invalidation kill the hit ratio.\nSingle-replica today. Multi-replica L1 invalidation needs FusionCache backplane — see perf doc.", + "originalText": "Read path (GetProductByIdHandler):\n cache.GetOrLoadAsync(productId, factory)\n L1 (MemoryCache, in-proc) HIT → return ProductDto (microseconds, no network)\n L1 MISS → L2 (Redis) HIT → promote to L1, return\n L2 MISS → factory runs ONCE under stampede → repo.GetByIdAsync → project DTO → store L1+L2 → return\n\nWrite path (UpdateProductHandler / ReserveStockHandler):\n mutate aggregate → repository.UpdateAsync → cache.InvalidateAsync(id) ← tag-based; clears L2 + this replica's L1.\n\nKeys: catalog:product:{guid} · Tag: product:{guid} · TTL: 5 min absolute on both tiers (safety net).\nCached unit: ProductDto (immutable projection), not the EF entity — caching tracked entities is a footgun.\nNot cached: GetAllProducts / SearchProducts — long-tail keys + cross-page invalidation kill the hit ratio.\nSingle-replica today. Multi-replica L1 invalidation needs FusionCache backplane — see perf doc.", "fontSize": 12, "fontFamily": 3, "textAlign": "left", @@ -3597,8 +3597,8 @@ "y": 1497, "width": 786.328125, "height": 200.2, - "text": "Per-service config in Program.cs:\n opts.PersistMessagesWith{SqlServer | Postgresql}(connStr, \"wolverine\");\n opts.UseEntityFrameworkCoreTransactions();\n opts.Policies.AutoApplyTransactions();\n opts.Policies.UseDurableOutboxOnAllSendingEndpoints();\n\nGuarantee: entity write + IMessageBus.PublishAsync commit in ONE DB tx.\n If the bus publish fails later, the entity write rolls back. No \"order saved but PaymentService never heard.\"\n\nConcurrency-retry: AddConcurrencyRetry() \u2192 3 attempts \u00b7 50ms / 100ms / 250ms backoff on DbUpdateConcurrencyException.\n After exhaustion, message \u2192 DLQ (messages.abandoned metric tags it).\n\nReplay/audit: Wolverine IMessageStore or SELECT TOP 50 * FROM wolverine.outgoing_envelopes ORDER BY received_at DESC.", - "originalText": "Per-service config in Program.cs:\n opts.PersistMessagesWith{SqlServer | Postgresql}(connStr, \"wolverine\");\n opts.UseEntityFrameworkCoreTransactions();\n opts.Policies.AutoApplyTransactions();\n opts.Policies.UseDurableOutboxOnAllSendingEndpoints();\n\nGuarantee: entity write + IMessageBus.PublishAsync commit in ONE DB tx.\n If the bus publish fails later, the entity write rolls back. No \"order saved but PaymentService never heard.\"\n\nConcurrency-retry: AddConcurrencyRetry() \u2192 3 attempts \u00b7 50ms / 100ms / 250ms backoff on DbUpdateConcurrencyException.\n After exhaustion, message \u2192 DLQ (messages.abandoned metric tags it).\n\nReplay/audit: Wolverine IMessageStore or SELECT TOP 50 * FROM wolverine.outgoing_envelopes ORDER BY received_at DESC.", + "text": "Per-service config in Program.cs:\n opts.PersistMessagesWith{SqlServer | Postgresql}(connStr, \"wolverine\");\n opts.UseEntityFrameworkCoreTransactions();\n opts.Policies.AutoApplyTransactions();\n opts.Policies.UseDurableOutboxOnAllSendingEndpoints();\n\nGuarantee: entity write + IMessageBus.PublishAsync commit in ONE DB tx.\n If the bus publish fails later, the entity write rolls back. No \"order saved but PaymentService never heard.\"\n\nConcurrency-retry: AddConcurrencyRetry() → 3 attempts · 50ms / 100ms / 250ms backoff on DbUpdateConcurrencyException.\n After exhaustion, message → DLQ (messages.abandoned metric tags it).\n\nReplay/audit: Wolverine IMessageStore or SELECT TOP 50 * FROM wolverine.outgoing_envelopes ORDER BY received_at DESC.", + "originalText": "Per-service config in Program.cs:\n opts.PersistMessagesWith{SqlServer | Postgresql}(connStr, \"wolverine\");\n opts.UseEntityFrameworkCoreTransactions();\n opts.Policies.AutoApplyTransactions();\n opts.Policies.UseDurableOutboxOnAllSendingEndpoints();\n\nGuarantee: entity write + IMessageBus.PublishAsync commit in ONE DB tx.\n If the bus publish fails later, the entity write rolls back. No \"order saved but PaymentService never heard.\"\n\nConcurrency-retry: AddConcurrencyRetry() → 3 attempts · 50ms / 100ms / 250ms backoff on DbUpdateConcurrencyException.\n After exhaustion, message → DLQ (messages.abandoned metric tags it).\n\nReplay/audit: Wolverine IMessageStore or SELECT TOP 50 * FROM wolverine.outgoing_envelopes ORDER BY received_at DESC.", "fontSize": 11, "fontFamily": 3, "textAlign": "left", @@ -3781,8 +3781,8 @@ "y": 200, "width": 114.2578125, "height": 1571.7000000000003, - "text": "Aspire\nDashboard\n:17222\n\nResources\nLogs\nTraces\nMetrics\n\n\u2500\u2500\u2500\u2500\u2500\u2500\n\nOpenTele-\nmetry\nOTLP \u2192\nAspire\nin-process\ncollector\n\nTracing:\n\u00b7 ASP.NET\n Core\n\u00b7 HttpClient\n\u00b7 gRPC\n\u00b7 Wolverine\n\u00b7 Service Bus\n\nMetrics\n(Meter\n\"NextAurora\"):\n\u00b7 orders.\n placed\n\u00b7 payments.\n processed\n\u00b7 shipments.\n dispatched\n\u00b7 notifications\n .sent\n\u00b7 messages.\n abandoned\n\n\u2500\u2500\u2500\u2500\u2500\u2500\n\nContext\npropag-\nation\n\nCorrelationId\nUserId\nSessionId\n\nflow via:\n\u00b7 HTTP\n X-* headers\n\u00b7 Wolverine\n Envelope\n .Headers\n\u00b7 Activity\n baggage\n\nHandlers\nnever read\nthese\ndirectly \u2014\nmiddleware\nopens scope\n\n\u2500\u2500\u2500\u2500\u2500\u2500\n\nWolverine\nOUTBOX\n(per service)\n\nschema:\nwolverine\n.outgoing_\nenvelopes\n\nentity write\n+ event\nstage same\nDB tx\n\ndispatcher\nflushes to\nbus async\n\n3-attempt\nretry on\nDbUpdate-\nConcurrency-\nException\n50/100/250ms\n\u2192 DLQ", - "originalText": "Aspire\nDashboard\n:17222\n\nResources\nLogs\nTraces\nMetrics\n\n\u2500\u2500\u2500\u2500\u2500\u2500\n\nOpenTele-\nmetry\nOTLP \u2192\nAspire\nin-process\ncollector\n\nTracing:\n\u00b7 ASP.NET\n Core\n\u00b7 HttpClient\n\u00b7 gRPC\n\u00b7 Wolverine\n\u00b7 Service Bus\n\nMetrics\n(Meter\n\"NextAurora\"):\n\u00b7 orders.\n placed\n\u00b7 payments.\n processed\n\u00b7 shipments.\n dispatched\n\u00b7 notifications\n .sent\n\u00b7 messages.\n abandoned\n\n\u2500\u2500\u2500\u2500\u2500\u2500\n\nContext\npropag-\nation\n\nCorrelationId\nUserId\nSessionId\n\nflow via:\n\u00b7 HTTP\n X-* headers\n\u00b7 Wolverine\n Envelope\n .Headers\n\u00b7 Activity\n baggage\n\nHandlers\nnever read\nthese\ndirectly \u2014\nmiddleware\nopens scope\n\n\u2500\u2500\u2500\u2500\u2500\u2500\n\nWolverine\nOUTBOX\n(per service)\n\nschema:\nwolverine\n.outgoing_\nenvelopes\n\nentity write\n+ event\nstage same\nDB tx\n\ndispatcher\nflushes to\nbus async\n\n3-attempt\nretry on\nDbUpdate-\nConcurrency-\nException\n50/100/250ms\n\u2192 DLQ", + "text": "Aspire\nDashboard\n:17222\n\nResources\nLogs\nTraces\nMetrics\n\n──────\n\nOpenTele-\nmetry\nOTLP →\nAspire\nin-process\ncollector\n\nTracing:\n· ASP.NET\n Core\n· HttpClient\n· gRPC\n· Wolverine\n· RabbitMQ\n\nMetrics\n(Meter\n\"NextAurora\"):\n· orders.\n placed\n· payments.\n processed\n· shipments.\n dispatched\n· notifications\n .sent\n· messages.\n abandoned\n\n──────\n\nContext\npropag-\nation\n\nCorrelationId\nUserId\nSessionId\n\nflow via:\n· HTTP\n X-* headers\n· Wolverine\n Envelope\n .Headers\n· Activity\n baggage\n\nHandlers\nnever read\nthese\ndirectly —\nmiddleware\nopens scope\n\n──────\n\nWolverine\nOUTBOX\n(per service)\n\nschema:\nwolverine\n.outgoing_\nenvelopes\n\nentity write\n+ event\nstage same\nDB tx\n\ndispatcher\nflushes to\nbus async\n\n3-attempt\nretry on\nDbUpdate-\nConcurrency-\nException\n50/100/250ms\n→ DLQ", + "originalText": "Aspire\nDashboard\n:17222\n\nResources\nLogs\nTraces\nMetrics\n\n──────\n\nOpenTele-\nmetry\nOTLP →\nAspire\nin-process\ncollector\n\nTracing:\n· ASP.NET\n Core\n· HttpClient\n· gRPC\n· Wolverine\n· RabbitMQ\n\nMetrics\n(Meter\n\"NextAurora\"):\n· orders.\n placed\n· payments.\n processed\n· shipments.\n dispatched\n· notifications\n .sent\n· messages.\n abandoned\n\n──────\n\nContext\npropag-\nation\n\nCorrelationId\nUserId\nSessionId\n\nflow via:\n· HTTP\n X-* headers\n· Wolverine\n Envelope\n .Headers\n· Activity\n baggage\n\nHandlers\nnever read\nthese\ndirectly —\nmiddleware\nopens scope\n\n──────\n\nWolverine\nOUTBOX\n(per service)\n\nschema:\nwolverine\n.outgoing_\nenvelopes\n\nentity write\n+ event\nstage same\nDB tx\n\ndispatcher\nflushes to\nbus async\n\n3-attempt\nretry on\nDbUpdate-\nConcurrency-\nException\n50/100/250ms\n→ DLQ", "fontSize": 13, "fontFamily": 3, "textAlign": "left", @@ -3818,8 +3818,8 @@ "y": 200, "width": 114.2578125, "height": 1250.6000000000001, - "text": "Keycloak\nrealm:\nnextaurora\n\n3 clients:\n\u00b7 storefront\n (public)\n\u00b7 seller-\n portal\n (public)\n\u00b7 nextaurora\n -api\n (bearer)\n\nTest users:\n\u00b7 buyer1\n\u00b7 seller1\n\u00b7 admin\n\n\u2500\u2500\u2500\u2500\u2500\u2500\n\nJWT Bearer\nin every\nservice via\nServiceDefaults\n.AddJwtBearer\n\nValidates:\n\u00b7 issuer\n\u00b7 audience\n\u00b7 lifetime\n\u00b7 signature\n\nClaims:\nsub \u2192\n user.id\nrealm_access\n .roles \u2192\n Role\npreferred_\n username \u2192\n Name\n\n\u2500\u2500\u2500\u2500\u2500\u2500\n\nProtected:\n\u00b7 POST/PUT\n products\n\u00b7 all of\n /orders\n\u00b7 /payments\n /process\n\u00b7 all of\n /shipments\n\nBuyer-\nscope:\nsub claim\nmust match\nbody/route\nbuyerId\n\u2192 403 if\nnot\n\n\u2500\u2500\u2500\u2500\u2500\u2500\n\nNo-op\nfallback if\nKeycloak\nconfig\nabsent \u2192\n401 on every\nprotected\nendpoint", - "originalText": "Keycloak\nrealm:\nnextaurora\n\n3 clients:\n\u00b7 storefront\n (public)\n\u00b7 seller-\n portal\n (public)\n\u00b7 nextaurora\n -api\n (bearer)\n\nTest users:\n\u00b7 buyer1\n\u00b7 seller1\n\u00b7 admin\n\n\u2500\u2500\u2500\u2500\u2500\u2500\n\nJWT Bearer\nin every\nservice via\nServiceDefaults\n.AddJwtBearer\n\nValidates:\n\u00b7 issuer\n\u00b7 audience\n\u00b7 lifetime\n\u00b7 signature\n\nClaims:\nsub \u2192\n user.id\nrealm_access\n .roles \u2192\n Role\npreferred_\n username \u2192\n Name\n\n\u2500\u2500\u2500\u2500\u2500\u2500\n\nProtected:\n\u00b7 POST/PUT\n products\n\u00b7 all of\n /orders\n\u00b7 /payments\n /process\n\u00b7 all of\n /shipments\n\nBuyer-\nscope:\nsub claim\nmust match\nbody/route\nbuyerId\n\u2192 403 if\nnot\n\n\u2500\u2500\u2500\u2500\u2500\u2500\n\nNo-op\nfallback if\nKeycloak\nconfig\nabsent \u2192\n401 on every\nprotected\nendpoint", + "text": "Keycloak\nrealm:\nnextaurora\n\n3 clients:\n· storefront\n (public)\n· seller-\n portal\n (public)\n· nextaurora\n -api\n (bearer)\n\nTest users:\n· buyer1\n· seller1\n· admin\n\n──────\n\nJWT Bearer\nin every\nservice via\nServiceDefaults\n.AddJwtBearer\n\nValidates:\n· issuer\n· audience\n· lifetime\n· signature\n\nClaims:\nsub →\n user.id\nrealm_access\n .roles →\n Role\npreferred_\n username →\n Name\n\n──────\n\nProtected:\n· POST/PUT\n products\n· all of\n /orders\n· /payments\n /process\n· all of\n /shipments\n\nBuyer-\nscope:\nsub claim\nmust match\nbody/route\nbuyerId\n→ 403 if\nnot\n\n──────\n\nNo-op\nfallback if\nKeycloak\nconfig\nabsent →\n401 on every\nprotected\nendpoint", + "originalText": "Keycloak\nrealm:\nnextaurora\n\n3 clients:\n· storefront\n (public)\n· seller-\n portal\n (public)\n· nextaurora\n -api\n (bearer)\n\nTest users:\n· buyer1\n· seller1\n· admin\n\n──────\n\nJWT Bearer\nin every\nservice via\nServiceDefaults\n.AddJwtBearer\n\nValidates:\n· issuer\n· audience\n· lifetime\n· signature\n\nClaims:\nsub →\n user.id\nrealm_access\n .roles →\n Role\npreferred_\n username →\n Name\n\n──────\n\nProtected:\n· POST/PUT\n products\n· all of\n /orders\n· /payments\n /process\n· all of\n /shipments\n\nBuyer-\nscope:\nsub claim\nmust match\nbody/route\nbuyerId\n→ 403 if\nnot\n\n──────\n\nNo-op\nfallback if\nKeycloak\nconfig\nabsent →\n401 on every\nprotected\nendpoint", "fontSize": 13, "fontFamily": 3, "textAlign": "left", @@ -3897,4 +3897,4 @@ "viewBackgroundColor": "#ffffff" }, "files": {} -} \ No newline at end of file +} diff --git a/docs/nextaurora-architecture.svg b/docs/nextaurora-architecture.svg index b13347cf..d1810b79 100644 --- a/docs/nextaurora-architecture.svg +++ b/docs/nextaurora-architecture.svg @@ -1,2 +1,2 @@ -NextAurora — Saga Workflow & Architecture5 backend services · Wolverine transactional outbox · Azure Service Bus (local) → AWS SNS+SQS (planned) · Keycloak / JWT · Aspire 13.3.0SECURITY& AUTHBuyerStorefront(Blazor WASM)scaffoldSellerPortal(static scaffold)UI framework TBDSellerKeycloak (Aspire container)realm: nextauroraimported from realms/nextaurora-realm.jsonissuer for every JWTOAuth password grant → JWTCatalogServicePostgres + Redis + gRPC server──────────────────────REST /api/v1/products (paginated)REST /api/v1/products/search (rate-limited)REST POST/PUT (seller-only, JWT)gRPC GetProduct, ReserveStockHandlers: Create / Update / GetById / GetAll / Search / ReserveStockIProductCache (HybridCache L1+L2 on GetById)OrderServiceSQL Server + Wolverine outbox──────────────────────REST POST /api/v1/orders (saga entry)REST GET /api/v1/orders/{id}REST GET /api/v1/orders/buyer/{buyerId}buyerId must match JWT sub → 403PlaceOrder → gRPC validate + reserve on Catalog → Order.Create → AddAsync + PublishAsync (one tx)Reacts: PaymentCompleted, ShipmentDispatchedPaymentServiceSQL Server + Wolverine outbox──────────────────────REST POST /api/v1/payments/process (admin)UNIQUE INDEX on OrderId (db backstop)OrderPlacedHandler (cascade msg) → ProcessPaymentCommand → Stripe gateway (anti-corruption) → Mark Completed/Failed → Publish PaymentCompleted / PaymentFailedEventShippingServicePostgres + Wolverine outbox──────────────────────REST GET /api/v1/shipments/order/{orderId}UNIQUE INDEX on OrderId (db backstop)PaymentCompletedHandler (cascade) → CreateShipmentCommand → Shipment.Create + Dispatch → TrackingNumber generated → Publish ShipmentDispatchedEventNotificationServiceStateless · no DB · no outbox──────────────────────Consumes 3 topics + 1 queue: order-events payment-events shipping-events send-notificationEach event handler returnsSendNotificationRequest →Wolverine cascades toSendNotificationHandler →INotificationSender (console / SES)Azure Service Bus · topics & subscriptions · Wolverine outbox dispatcher publishes here(local: Aspire emulator container · prod target: Amazon SNS topics + SQS queues — see Deployment section)topic: order-eventspublisher: OrderServiceEvent: OrderPlacedEvent──────────────────────────sub: payment-orders-sub → PaymentServicesub: notify-orders-sub → NotificationServicetopic: payment-eventspublisher: PaymentServiceEvents: PaymentCompleted, PaymentFailed──────────────────────────────────sub: order-payments-sub → OrderServicesub: shipping-payments-sub → ShippingServicesub: notify-payments-sub → NotificationServicetopic: shipping-eventspublisher: ShippingServiceEvent: ShipmentDispatchedEvent──────────────────────────────────sub: order-shipping-sub → OrderServicesub: notify-shipping-sub → NotificationServicequeue: send-notificationDirect queue (not topic).Fan-in only — single consumer.NotificationService consumesin addition to its 3 topicsubscriptions.Use: ad-hoc "send a notificationright now" requests from anyservice or admin tool.catalog-db(PostgreSQL 17)Products · Categoriesxmin concurrencyRedis 8 (L2)HybridCacheL1: in-proc perreplicaL2: this Rediscatalog:product: {guid}5 min TTLorders-db (SQL Server 2022)Orders · OrderLines · RowVersion concurrency+ wolverine schema (outgoing_envelopes, scheduled_envelopes, dead_letters)payments-db (SQL Server 2022)Payments · Refunds · UNIQUE(OrderId)RowVersion concurrency+ wolverine schemashipping-db (PostgreSQL 17)Shipments · TrackingEvents · UNIQUE(OrderId)xmin concurrency+ wolverine schemaExternal: Stripe API(via IPaymentGateway anti-corruption)NotificationService is stateless —no DB, no outbox.gRPCPOST /api/v1/ordersOrder placement saga · end-to-end walk① Buyer (with JWT bearer) submits POST /api/v1/orders to OrderService. Endpoint validates buyer-scope: JWT sub claim must match command.BuyerId, else 403.② PlaceOrderHandler calls CatalogService over gRPC, once per line: GetProduct (validate exists + available + stock) then ReserveStock (xmin token enforces atomicity).③ Order.Create() validates invariants. Order + lines persisted to orders-db. OrderPlacedEvent staged into wolverine.outgoing_envelopes — same DB transaction. Atomic.④ Wolverine background dispatcher reads from outgoing_envelopes, publishes OrderPlacedEvent to topic order-events on the Service Bus. Marks PublishedAt.⑤ Two consumers receive the event in parallel: PaymentService (payment-orders-sub) and NotificationService (notify-orders-sub) — "order received" email.⑥ PaymentService OrderPlacedHandler is a cascade — returns ProcessPaymentCommand. Handler checks for existing Payment by OrderId (idempotency), creates Pending Payment, calls Stripe gateway via IPaymentGateway.⑦ Gateway returns success or failure. Payment.MarkAsCompleted/Failed (status guard = idempotency). Publish PaymentCompletedEvent or PaymentFailedEvent → payment-events.⑧ payment-events fans out to 3 subs: order-payments-sub (OrderService transitions Order Placed→Paid), shipping-payments-sub (creates shipment), notify-payments-sub.⑨ ShippingService PaymentCompletedHandler cascades CreateShipmentCommand. Creates Shipment, runs Dispatch() (Created→Dispatched), generates tracking number. Publish ShipmentDispatchedEvent.⑩ shipping-events fans out: order-shipping-sub (OrderService Paid→Shipped) and notify-shipping-sub ("shipped" email with tracking number). Saga complete.HybridCache (L1+L2) — IProductCache (CatalogService)Read path (GetProductByIdHandler): cache.GetOrLoadAsync(productId, factory) L1 (MemoryCache, in-proc) HIT → return ProductDto (microseconds, no network) L1 MISS → L2 (Redis) HIT → promote to L1, return L2 MISS → factory runs ONCE under stampede → repo.GetByIdAsync → project DTO → store L1+L2 → returnWrite path (UpdateProductHandler / ReserveStockHandler): mutate aggregate → repository.UpdateAsync → cache.InvalidateAsync(id) ← tag-based; clears L2 + this replica's L1.Keys: catalog:product:{guid} · Tag: product:{guid} · TTL: 5 min absolute on both tiers (safety net).Cached unit: ProductDto (immutable projection), not the EF entity — caching tracked entities is a footgun.Not cached: GetAllProducts / SearchProducts — long-tail keys + cross-page invalidation kill the hit ratio.Single-replica today. Multi-replica L1 invalidation needs FusionCache backplane — see perf doc.Wolverine transactional outbox (Order, Payment, Shipping)Per-service config in Program.cs: opts.PersistMessagesWith{SqlServer | Postgresql}(connStr, "wolverine"); opts.UseEntityFrameworkCoreTransactions(); opts.Policies.AutoApplyTransactions(); opts.Policies.UseDurableOutboxOnAllSendingEndpoints();Guarantee: entity write + IMessageBus.PublishAsync commit in ONE DB tx. If the bus publish fails later, the entity write rolls back. No "order saved but PaymentService never heard."Concurrency-retry: AddConcurrencyRetry() → 3 attempts · 50ms / 100ms / 250ms backoff on DbUpdateConcurrencyException. After exhaustion, message → DLQ (messages.abandoned metric tags it).Replay/audit: Wolverine IMessageStore or SELECT TOP 50 * FROM wolverine.outgoing_envelopes ORDER BY received_at DESC.OBSERV-ABILITYAspireDashboard:17222ResourcesLogsTracesMetrics──────OpenTele-metryOTLP →Aspirein-processcollectorTracing:· ASP.NET Core· HttpClient· gRPC· Wolverine· Service BusMetrics(Meter"NextAurora"):· orders. placed· payments. processed· shipments. dispatched· notifications .sent· messages. abandoned──────Contextpropag-ationCorrelationIdUserIdSessionIdflow via:· HTTP X-* headers· Wolverine Envelope .Headers· Activity baggageHandlersnever readthesedirectly —middlewareopens scope──────WolverineOUTBOX(per service)schema:wolverine.outgoing_envelopesentity write+ eventstage sameDB txdispatcherflushes tobus async3-attemptretry onDbUpdate-Concurrency-Exception50/100/250ms→ DLQKeycloakrealm:nextaurora3 clients:· storefront (public)· seller- portal (public)· nextaurora -api (bearer)Test users:· buyer1· seller1· admin──────JWT Bearerin everyservice viaServiceDefaults.AddJwtBearerValidates:· issuer· audience· lifetime· signatureClaims:sub → user.idrealm_access .roles → Rolepreferred_ username → Name──────Protected:· POST/PUT products· all of /orders· /payments /process· all of /shipmentsBuyer-scope:sub claimmust matchbody/routebuyerId→ 403 ifnot──────No-opfallback ifKeycloakconfigabsent →401 on everyprotectedendpoint \ No newline at end of file +NextAurora — Saga Workflow & Architecture5 backend services · Wolverine transactional outbox · RabbitMQ via Wolverine (local + deployed) · Keycloak / JWT · Aspire 13.3.0SECURITY& AUTHBuyerStorefront(Blazor WASM)scaffoldSellerPortal(static scaffold)UI framework TBDSellerKeycloak (Aspire container)realm: nextauroraimported from realms/nextaurora-realm.jsonissuer for every JWTOAuth password grant → JWTCatalogServicePostgres + Redis + gRPC server──────────────────────REST /api/v1/products (paginated)REST /api/v1/products/search (rate-limited)REST POST/PUT (seller-only, JWT)gRPC GetProduct, ReserveStockHandlers: Create / Update / GetById / GetAll / Search / ReserveStockIProductCache (HybridCache L1+L2 on GetById)OrderServiceSQL Server + Wolverine outbox──────────────────────REST POST /api/v1/orders (saga entry)REST GET /api/v1/orders/{id}REST GET /api/v1/orders/buyer/{buyerId}buyerId must match JWT sub → 403PlaceOrder → gRPC validate + reserve on Catalog → Order.Create → AddAsync + PublishAsync (one tx)Reacts: PaymentCompleted, ShipmentDispatchedPaymentServiceSQL Server + Wolverine outbox──────────────────────REST POST /api/v1/payments/process (admin)UNIQUE INDEX on OrderId (db backstop)OrderPlacedHandler (cascade msg) → ProcessPaymentCommand → Stripe gateway (anti-corruption) → Mark Completed/Failed → Publish PaymentCompleted / PaymentFailedEventShippingServicePostgres + Wolverine outbox──────────────────────REST GET /api/v1/shipments/order/{orderId}UNIQUE INDEX on OrderId (db backstop)PaymentCompletedHandler (cascade) → CreateShipmentCommand → Shipment.Create + Dispatch → TrackingNumber generated → Publish ShipmentDispatchedEventNotificationServiceStateless · no DB · no outbox──────────────────────Consumes 3 exchanges + 1 queue: order-events payment-events shipping-events send-notificationEach event handler returnsSendNotificationRequest →Wolverine cascades toSendNotificationHandler →INotificationSender (console / SES)RabbitMQ · fanout exchanges & queues · Wolverine outbox dispatcher publishes here(local + deployed: RabbitMQ container · alt cloud target: Amazon SNS+SQS — see Deployment section)exchange: order-eventspublisher: OrderServiceEvent: OrderPlacedEvent──────────────────────────queue: payment-orders → PaymentServicequeue: notify-orders → NotificationServiceexchange: payment-eventspublisher: PaymentServiceEvents: PaymentCompleted, PaymentFailed──────────────────────────────────queue: order-payments → OrderServicequeue: shipping-payments → ShippingServicequeue: notify-payments → NotificationServiceexchange: shipping-eventspublisher: ShippingServiceEvent: ShipmentDispatchedEvent──────────────────────────────────queue: order-shipping → OrderServicequeue: notify-shipping → NotificationServicequeue: send-notificationDirect queue (not exchange).Fan-in only — single consumer.NotificationService consumesin addition to its 3 exchangebindings.Use: ad-hoc "send a notificationright now" requests from anyservice or admin tool.catalog-db(PostgreSQL 17)Products · Categoriesxmin concurrencyRedis 8 (L2)HybridCacheL1: in-proc perreplicaL2: this Rediscatalog:product: {guid}5 min TTLorders-db (SQL Server 2022)Orders · OrderLines · RowVersion concurrency+ wolverine schema (outgoing_envelopes, scheduled_envelopes, dead_letters)payments-db (SQL Server 2022)Payments · Refunds · UNIQUE(OrderId)RowVersion concurrency+ wolverine schemashipping-db (PostgreSQL 17)Shipments · TrackingEvents · UNIQUE(OrderId)xmin concurrency+ wolverine schemaExternal: Stripe API(via IPaymentGateway anti-corruption)NotificationService is stateless —no DB, no outbox.gRPCPOST /api/v1/ordersOrder placement saga · end-to-end walk① Buyer (with JWT bearer) submits POST /api/v1/orders to OrderService. Endpoint validates buyer-scope: JWT sub claim must match command.BuyerId, else 403.② PlaceOrderHandler calls CatalogService over gRPC, once per line: GetProduct (validate exists + available + stock) then ReserveStock (xmin token enforces atomicity).③ Order.Create() validates invariants. Order + lines persisted to orders-db. OrderPlacedEvent staged into wolverine.outgoing_envelopes — same DB transaction. Atomic.④ Wolverine background dispatcher reads from outgoing_envelopes, publishes OrderPlacedEvent to exchange order-events on RabbitMQ. Marks PublishedAt.⑤ Two consumers receive the event in parallel: PaymentService (payment-orders) and NotificationService (notify-orders) — "order received" email.⑥ PaymentService OrderPlacedHandler is a cascade — returns ProcessPaymentCommand. Handler checks for existing Payment by OrderId (idempotency), creates Pending Payment, calls Stripe gateway via IPaymentGateway.⑦ Gateway returns success or failure. Payment.MarkAsCompleted/Failed (status guard = idempotency). Publish PaymentCompletedEvent or PaymentFailedEvent → payment-events.⑧ payment-events fans out to 3 queues: order-payments (OrderService transitions Order Placed→Paid), shipping-payments (creates shipment), notify-payments.⑨ ShippingService PaymentCompletedHandler cascades CreateShipmentCommand. Creates Shipment, runs Dispatch() (Created→Dispatched), generates tracking number. Publish ShipmentDispatchedEvent.⑩ shipping-events fans out: order-shipping (OrderService Paid→Shipped) and notify-shipping ("shipped" email with tracking number). Saga complete.HybridCache (L1+L2) — IProductCache (CatalogService)Read path (GetProductByIdHandler): cache.GetOrLoadAsync(productId, factory) L1 (MemoryCache, in-proc) HIT → return ProductDto (microseconds, no network) L1 MISS → L2 (Redis) HIT → promote to L1, return L2 MISS → factory runs ONCE under stampede → repo.GetByIdAsync → project DTO → store L1+L2 → returnWrite path (UpdateProductHandler / ReserveStockHandler): mutate aggregate → repository.UpdateAsync → cache.InvalidateAsync(id) ← tag-based; clears L2 + this replica's L1.Keys: catalog:product:{guid} · Tag: product:{guid} · TTL: 5 min absolute on both tiers (safety net).Cached unit: ProductDto (immutable projection), not the EF entity — caching tracked entities is a footgun.Not cached: GetAllProducts / SearchProducts — long-tail keys + cross-page invalidation kill the hit ratio.Single-replica today. Multi-replica L1 invalidation needs FusionCache backplane — see perf doc.Wolverine transactional outbox (Order, Payment, Shipping)Per-service config in Program.cs: opts.PersistMessagesWith{SqlServer | Postgresql}(connStr, "wolverine"); opts.UseEntityFrameworkCoreTransactions(); opts.Policies.AutoApplyTransactions(); opts.Policies.UseDurableOutboxOnAllSendingEndpoints();Guarantee: entity write + IMessageBus.PublishAsync commit in ONE DB tx. If the bus publish fails later, the entity write rolls back. No "order saved but PaymentService never heard."Concurrency-retry: AddConcurrencyRetry() → 3 attempts · 50ms / 100ms / 250ms backoff on DbUpdateConcurrencyException. After exhaustion, message → DLQ (messages.abandoned metric tags it).Replay/audit: Wolverine IMessageStore or SELECT TOP 50 * FROM wolverine.outgoing_envelopes ORDER BY received_at DESC.OBSERV-ABILITYAspireDashboard:17222ResourcesLogsTracesMetrics──────OpenTele-metryOTLP →Aspirein-processcollectorTracing:· ASP.NET Core· HttpClient· gRPC· Wolverine· RabbitMQMetrics(Meter"NextAurora"):· orders. placed· payments. processed· shipments. dispatched· notifications .sent· messages. abandoned──────Contextpropag-ationCorrelationIdUserIdSessionIdflow via:· HTTP X-* headers· Wolverine Envelope .Headers· Activity baggageHandlersnever readthesedirectly —middlewareopens scope──────WolverineOUTBOX(per service)schema:wolverine.outgoing_envelopesentity write+ eventstage sameDB txdispatcherflushes tobus async3-attemptretry onDbUpdate-Concurrency-Exception50/100/250ms→ DLQKeycloakrealm:nextaurora3 clients:· storefront (public)· seller- portal (public)· nextaurora -api (bearer)Test users:· buyer1· seller1· admin──────JWT Bearerin everyservice viaServiceDefaults.AddJwtBearerValidates:· issuer· audience· lifetime· signatureClaims:sub → user.idrealm_access .roles → Rolepreferred_ username → Name──────Protected:· POST/PUT products· all of /orders· /payments /process· all of /shipmentsBuyer-scope:sub claimmust matchbody/routebuyerId→ 403 ifnot──────No-opfallback ifKeycloakconfigabsent →401 on everyprotectedendpoint \ No newline at end of file diff --git a/docs/observability-and-context-propagation.md b/docs/observability-and-context-propagation.md index 3a1d6fd3..249c5e44 100644 --- a/docs/observability-and-context-propagation.md +++ b/docs/observability-and-context-propagation.md @@ -2,11 +2,11 @@ Deep-dive companion to [CLAUDE.md "Observability & Context Propagation"](../CLAUDE.md#observability--context-propagation), which keeps the always-on traps and points here for mechanism + wiring detail. -NextAurora propagates three context identifiers across HTTP and Service Bus -boundaries so every log line, span, and event in a request's lifecycle can -be correlated: +NextAurora propagates three context identifiers across HTTP and RabbitMQ +(Wolverine) boundaries so every log line, span, and event in a request's +lifecycle can be correlated: -| Concept | Activity Baggage Key | HTTP / SB Property | Logger Scope Key | +| Concept | Activity Baggage Key | HTTP Header / Envelope Header | Logger Scope Key | |---|---|---|---| | Correlation | `correlation.id` | `X-Correlation-Id` | `CorrelationId` | | User | `user.id` | `X-User-Id` | `UserId` | @@ -52,7 +52,7 @@ Order in the Wolverine pipeline: ## Wolverine envelope context extraction -Handlers don't extract context manually — `ContextPropagationMiddleware` does it for them. The middleware reads `Envelope.Headers["X-Correlation-Id" | "X-User-Id" | "X-Session-Id"]` (Wolverine's transport-agnostic header bag, mapped to Service Bus `ApplicationProperties` over the wire), restores them into Activity baggage, and opens a `logger.BeginScope()`. After the handler runs, `Finally()` disposes the scope. +Handlers don't extract context manually — `ContextPropagationMiddleware` does it for them. The middleware reads `Envelope.Headers["X-Correlation-Id" | "X-User-Id" | "X-Session-Id"]` (Wolverine's transport-agnostic header bag, mapped to RabbitMQ message headers over the wire), restores them into Activity baggage, and opens a `logger.BeginScope()`. After the handler runs, `Finally()` disposes the scope. Outgoing context is stamped by `OutgoingContextMiddleware`, which reads Activity baggage and writes the same headers onto outgoing envelopes. The full mechanism is registered via `opts.AddNextAuroraContextPropagation()` in each service's `Program.cs`. @@ -60,7 +60,7 @@ Outgoing context is stamped by `OutgoingContextMiddleware`, which reads Activity ## Transactional Outbox (Wolverine) -Each event-publishing service (Order, Payment, Shipping) runs Wolverine's transactional outbox. Outgoing events are persisted to a `wolverine.*` schema in the same DB transaction as the entity write, then dispatched to Azure Service Bus by Wolverine's background flush. Configuration lives in each service's `Program.cs`: +Each event-publishing service (Order, Payment, Shipping) runs Wolverine's transactional outbox. Outgoing events are persisted to a `wolverine.*` schema in the same DB transaction as the entity write, then dispatched to RabbitMQ by Wolverine's background flush. Configuration lives in each service's `Program.cs`: ```csharp opts.PersistMessagesWithSqlServer(connectionString, "wolverine"); // or PersistMessagesWithPostgresql diff --git a/docs/observability.md b/docs/observability.md index e3f7f47c..72980cfa 100644 --- a/docs/observability.md +++ b/docs/observability.md @@ -6,7 +6,7 @@ This document describes the observability features added to NextAurora, how they ## Overview -Every request or event in NextAurora now carries a **Correlation ID** that flows from the initial HTTP call through every Service Bus message and log line across all services. Combined with OpenTelemetry distributed tracing, structured logging, Wolverine pipeline telemetry, business metrics, and Dead Letter Queue (DLQ) handling, this gives you a complete picture of any transaction — even when it spans five microservices. +Every request or event in NextAurora now carries a **Correlation ID** that flows from the initial HTTP call through every RabbitMQ message and log line across all services. Combined with OpenTelemetry distributed tracing, structured logging, Wolverine pipeline telemetry, business metrics, and Dead Letter Queue (DLQ) handling, this gives you a complete picture of any transaction — even when it spans five microservices. --- @@ -21,28 +21,28 @@ Every request or event in NextAurora now carries a **Correlation ID** that flows 3. Opens an `ILogger` scope enriched with `CorrelationId`, so every log line written during that request automatically includes the value. 4. Echoes the ID in the `X-Correlation-Id` response header so clients can record it. -### Propagation Through Service Bus +### Propagation Through RabbitMQ -When a service publishes an event, the publisher injects the correlation ID into the message: +When a service publishes an event, `OutgoingContextMiddleware` (Wolverine outgoing-envelope middleware) reads the IDs from `Activity` baggage and stamps them onto the Wolverine envelope, which the RabbitMQ transport carries as message headers: ```csharp -message.ApplicationProperties["X-Correlation-Id"] = correlationId; -message.CorrelationId = correlationId; // also visible in Azure portal +envelope.Headers["X-Correlation-Id"] = correlationId; +envelope.Headers["X-User-Id"] = userId; // only when present +envelope.Headers["X-Session-Id"] = sessionId; // only when present ``` -When a processor receives the message, it extracts the correlation ID and opens a logging scope before dispatching: +When a message arrives, `ContextPropagationMiddleware` (Wolverine incoming middleware) reads the envelope headers back into `Activity` baggage and opens a logging scope before the handler runs: ```csharp using var scope = logger.BeginScope(new Dictionary(StringComparer.Ordinal) { ["CorrelationId"] = correlationId, - ["MessageId"] = args.Message.MessageId, - ["Subject"] = args.Message.Subject, - ["DeliveryCount"] = args.Message.DeliveryCount + ["UserId"] = userId, // added only when present + ["SessionId"] = sessionId // added only when present }); ``` -Every log line written by any handler invoked from that processor will carry all four fields. +Every log line written by the handler (and anything it calls transitively) will carry those fields. ### Finding a Transaction @@ -52,7 +52,7 @@ Given a correlation ID (from a client error report or response header), you can CorrelationId = "a3f1b2c4..." ``` -This returns every log line — HTTP request, Wolverine handler, Service Bus publish, Service Bus receive, and notification send — for that single transaction. +This returns every log line — HTTP request, Wolverine handler, RabbitMQ publish, RabbitMQ receive, and notification send — for that single transaction. For the full three-identifier guide (UserId, SessionId, new-service checklist, common pitfalls), see **[docs/context-propagation.md](context-propagation.md)**. @@ -67,7 +67,8 @@ For the full three-identifier guide (UserId, SessionId, new-service checklist, c | Source | What It Covers | |--------|----------------| | `{ServiceName}` (application name) | Custom spans per service | -| `Azure.Messaging.ServiceBus` | Service Bus send/receive/complete/abandon operations | +| `Wolverine` | Message send/receive/handle spans for the saga — transport-agnostic (RabbitMQ today) | +| `NextAurora.Messaging` | Registered but currently dormant — no code emits spans under this name today; saga message spans come from the `Wolverine` source | | ASP.NET Core instrumentation | Inbound HTTP requests | | gRPC client instrumentation | OrderService → CatalogService gRPC calls | | HTTP client instrumentation | All outbound HTTP calls | @@ -84,12 +85,13 @@ A single trace for an order placement will show spans across: [OrderService] POST /orders └─ [OrderService] PlaceOrderCommand handler └─ [CatalogService gRPC] GetProduct / ReserveStock - └─ [Azure.Messaging.ServiceBus] Send → order-events - └─ [PaymentService] OrderPlaced processor + └─ [Wolverine] Send → order-events + ├─ [NotificationService] OrderPlaced handler + └─ [PaymentService] OrderPlaced handler └─ [PaymentService] ProcessPayment handler - └─ [Azure.Messaging.ServiceBus] Send → payment-events - └─ [ShippingService] PaymentCompleted processor - └─ [NotificationService] OrderPlaced processor + └─ [Wolverine] Send → payment-events + ├─ [ShippingService] PaymentCompleted handler + └─ [NotificationService] PaymentCompleted handler ``` --- @@ -126,49 +128,25 @@ The exception itself is handled and logged by `GlobalExceptionHandler` in `Servi ## Dead Letter Queue (DLQ) Handling -Previously, all Service Bus processors silently discarded failures after logging. Now, on any unhandled exception during message processing, the processor calls: +When a message handler throws, Wolverine applies the configured error policy (e.g. the `AddConcurrencyRetry` cooldown retries for `DbUpdateConcurrencyException`). A message that exhausts its retries is dead-lettered by Wolverine's RabbitMQ transport to a Wolverine-managed dead-letter queue on the broker. -```csharp -await args.AbandonMessageAsync(args.Message, cancellationToken: stoppingToken); -``` - -Azure Service Bus then increments the message's **DeliveryCount**. Once `DeliveryCount` reaches the queue/subscription's configured `MaxDeliveryCount`, the message is automatically moved to the **Dead Letter Queue** for that entity. - -The `DeliveryCount` is always logged in the structured scope, so you can see retry progress: - -``` -[ERR] Failed to process OrderPlaced event. Abandoning for retry/DLQ - CorrelationId=a3f1b2c4, MessageId=abc-123, Subject=OrderPlacedEvent, DeliveryCount=2 -``` - -### DLQ Entities - -| Service Bus Entity | DLQ Path | -|--------------------|----------| -| `order-events / payment-sub` | `order-events/Subscriptions/payment-sub/$deadletterqueue` | -| `order-events / notify-sub` | `order-events/Subscriptions/notify-sub/$deadletterqueue` | -| `payment-events / order-sub` | `payment-events/Subscriptions/order-sub/$deadletterqueue` | -| `payment-events / shipping-sub` | `payment-events/Subscriptions/shipping-sub/$deadletterqueue` | -| `shipping-events / order-sub` | `shipping-events/Subscriptions/order-sub/$deadletterqueue` | -| `shipping-events / notify-sub` | `shipping-events/Subscriptions/notify-sub/$deadletterqueue` | -| `send-notification` (queue) | `send-notification/$deadletterqueue` | +### The RabbitMQ Topology -### Investigating a DLQ Message +Each event family has a fanout exchange with one queue per consumer bound to it: -1. In the Azure portal, navigate to the Service Bus namespace → topic/queue → subscription → Dead-letter. -2. Peek or receive the message. -3. Check `ApplicationProperties["X-Correlation-Id"]` to retrieve the original correlation ID. -4. Search your log sink with that ID to see the full history of attempts. -5. Fix the root cause, then replay the message by receiving it from the DLQ and re-publishing it to the original topic/queue. +| Fanout Exchange | Consumer Queues | +|-----------------|-----------------| +| `order-events` | `payment-orders`, `notify-orders` | +| `payment-events` | `order-payments`, `shipping-payments`, `notify-payments` | +| `shipping-events` | `order-shipping`, `notify-shipping` | +| — (direct send) | `send-notification` | -### Transport Errors +### Investigating a Dead-Lettered Message -`ProcessErrorAsync` on each processor now logs structured fields for infrastructure-level errors (disconnects, auth failures): - -``` -[ERR] Service Bus transport error on order-events/Subscriptions/payment-sub - ErrorSource=Receive, FullyQualifiedNamespace=nextaurora.servicebus.windows.net -``` +1. Open the RabbitMQ management UI (`http://localhost:15672` in local dev) and inspect the dead-letter queue, or query Wolverine's message store (the `wolverine` schema in each service's database). +2. Check the `X-Correlation-Id` message header to retrieve the original correlation ID. +3. Search your log sink with that ID to see the full history of attempts. +4. Fix the root cause, then replay via Wolverine's `IMessageStore` / DLQ tooling (see [Event Replay](#event-replay) below). --- @@ -182,7 +160,7 @@ A `Meter("NextAurora")` is registered in `ServiceDefaults` and collected by the | `payments.processed` | `ProcessPaymentHandler` | `outcome=success\|failed` | | `shipments.dispatched` | `CreateShipmentHandler` | — | | `notifications.sent` | `SendNotificationHandler` | `channel=Email\|…` | -| `messages.abandoned` | All service processors | `subject=`, `service=` | +| `messages.abandoned` | Nothing currently — declared in `NextAuroraMetrics`, but the processors that incremented it were deleted in the RabbitMQ/Wolverine migration; re-wiring it is a tracked follow-up. Monitor DLQ depth via the RabbitMQ management UI (`:15672`) or the `wolverine` schema instead | `subject=`, `service=` | These are available in the Aspire dashboard under **Metrics** in development. In production, they are exported via OTLP to your metrics backend (Prometheus, Azure Monitor, etc.). @@ -222,10 +200,10 @@ A failing database health check returns HTTP 503, allowing Kubernetes or Aspire | File | Change | |------|--------| -| `NextAurora.ServiceDefaults/Extensions.cs` | Register middleware; add Azure SB + NextAurora meter sources; enable health checks in all environments | +| `NextAurora.ServiceDefaults/Extensions.cs` | Register middleware; add `Wolverine` trace source + NextAurora meter; enable health checks in all environments | | `Directory.Packages.props` | Added `Microsoft.Extensions.Diagnostics.HealthChecks.EntityFrameworkCore 10.0.2` | | `OutgoingContextMiddleware` (Wolverine) + handler publishing via the enlisted `IMessageContext` / `IDbContextOutbox` | Context propagation on outgoing messages; Wolverine EF Core outbox for delivery guarantees (publish enlisted in the entity transaction — see the Wolverine 5→6 upgrade notes) | -| `{Order,Payment,Shipping,Notification}Service` (Wolverine handlers) | Context extraction + structured logging scope via `ContextPropagationMiddleware`; `AbandonMessageAsync` on failure handled by Wolverine dead-letter config | +| `{Order,Payment,Shipping,Notification}Service` (Wolverine handlers) | Context extraction + structured logging scope via `ContextPropagationMiddleware`; failed messages dead-lettered by Wolverine's retry/error policy | | `{Order,Payment,Catalog,Shipping}Service.Infrastructure/DependencyInjection.cs` | Added `AddDbContextCheck()` | | `{Order,Payment,Catalog,Shipping}Service.Infrastructure/*.csproj` | Added EF Core health checks package reference | | `{Payment,Catalog,Shipping}Service.Application/*.csproj` | Added `Microsoft.Extensions.Logging.Abstractions` | diff --git a/docs/performance-and-data-correctness.md b/docs/performance-and-data-correctness.md index fe32fe2b..81b855b8 100644 --- a/docs/performance-and-data-correctness.md +++ b/docs/performance-and-data-correctness.md @@ -774,9 +774,9 @@ DbUpdateConcurrencyException => new ProblemDetails The caller refetches and decides what to do. This is the right response for HTTP commands (admin-initiated updates, etc.) where the user can react. -### Service Bus path → Wolverine retry +### Message path → Wolverine retry -For event handlers (driven by Azure Service Bus), retry is correct: the event is still valid, the handler just needs to read the latest state and reapply. We added a Wolverine error policy in `NextAurora.ServiceDefaults`: +For event handlers (driven by RabbitMQ via Wolverine), retry is correct: the event is still valid, the handler just needs to read the latest state and reapply. We added a Wolverine error policy in `NextAurora.ServiceDefaults`: ```csharp public static WolverineOptions AddConcurrencyRetry(this WolverineOptions opts) diff --git a/docs/project-decisions.md b/docs/project-decisions.md index 2c9613b7..772ed21e 100644 --- a/docs/project-decisions.md +++ b/docs/project-decisions.md @@ -45,7 +45,7 @@ This doc is the **map of the technical decisions** with the *rationale* for each ## 2. Architectural style — microservices over modular monolith -NextAurora is **microservices, not modular monolith**. Five backend services, each independently deployable, each owning its own database, each communicating with peers via gRPC (sync) or Azure Service Bus (async). +NextAurora is **microservices, not modular monolith**. Five backend services, each independently deployable, each owning its own database, each communicating with peers via gRPC (sync) or RabbitMQ (async, via Wolverine). ``` NextAurora/ @@ -777,7 +777,7 @@ builder.Services.AddOpenTelemetry() .AddMeter("NextAurora")) .WithTracing(t => t .AddSource(builder.Environment.ApplicationName) - .AddSource("Azure.Messaging.ServiceBus") + .AddSource("Wolverine") .AddSource("NextAurora.Messaging") .AddAspNetCoreInstrumentation(opts => opts.Filter = ctx => @@ -975,7 +975,7 @@ Every project references packages **without versions**. Versions live in [Direct ```xml - + ``` ```xml @@ -1036,9 +1036,9 @@ The full inventory of significant non-Microsoft libraries, what they do, why we | Package | Version | Role | Why this, not [X] | |---|---|---|---| -| **WolverineFx** | 5.36.2 | In-process CQRS dispatch + distributed async messaging + transactional outbox | Covers what **MediatR** (in-process CQRS — commercial since 2024) and **MassTransit** (distributed messaging — commercial in v9, GA Q1 2026) together do, in one MIT-licensed framework. The combined library + license story is the load-bearing reason | -| **WolverineFx.AzureServiceBus** | 5.36.2 | Wolverine transport for Azure Service Bus | Production target; swappable for `WolverineFx.AmazonSqs` in AWS deploy | -| **WolverineFx.SqlServer / .Postgresql** | 5.36.2 | Wolverine outbox persistence | Same DB as the service, same transaction as the entity write | +| **WolverineFx** | 6.8.0 | In-process CQRS dispatch + distributed async messaging + transactional outbox | Covers what **MediatR** (in-process CQRS — commercial since 2024) and **MassTransit** (distributed messaging — commercial in v9, GA Q1 2026) together do, in one MIT-licensed framework. The combined library + license story is the load-bearing reason | +| **WolverineFx.RabbitMQ** | 6.8.0 | Wolverine transport for RabbitMQ | Broker in every environment (local/CI/Hetzner); swappable for another Wolverine transport if a cloud-managed target lands | +| **WolverineFx.SqlServer / .Postgresql** | 6.8.0 | Wolverine outbox persistence | Same DB as the service, same transaction as the entity write | | **Microsoft.EntityFrameworkCore** | 10.0.2 | ORM | Standard .NET ORM. See [ef-core.md](ef-core.md) for the full decision | | **Npgsql.EntityFrameworkCore.PostgreSQL** | 10.0.0 | Postgres EF provider | Only viable Postgres provider for EF Core | | **Microsoft.EntityFrameworkCore.SqlServer** | 10.0.2 | SQL Server EF provider | Microsoft's first-party SQL Server provider | @@ -1047,7 +1047,7 @@ The full inventory of significant non-Microsoft libraries, what they do, why we | **Microsoft.Extensions.Caching.StackExchangeRedis** | 10.0.2 | Redis L2 backend for HybridCache | Standard ASP.NET Core Redis integration | | **Microsoft.Extensions.Http.Resilience** | 10.1.0 | Wraps Polly v8 with curated defaults | vs custom Polly pipelines — one line gives the full pattern | | **FluentValidation.DependencyInjectionExtensions** | (latest) | DI integration for FluentValidation validators | Standard FluentValidation auto-discovery | -| **WolverineFx.FluentValidation** | 5.36.2 | Wolverine pipeline integration | Runs validators before handlers | +| **WolverineFx.FluentValidation** | 6.8.0 | Wolverine pipeline integration | Runs validators before handlers | | **Asp.Versioning.Http** | 10.0.0 | URL-segment API versioning | vs header versioning — see §5 | | **Asp.Versioning.Mvc.ApiExplorer** | 10.0.0 | OpenAPI integration for versioned routes | Required for `MapV1ApiGroup` helper | | **Microsoft.AspNetCore.OpenApi** | 10.0.2 | OpenAPI emission | First-party, replaces Swashbuckle for new projects | @@ -1080,7 +1080,7 @@ A condensed walkthrough of the key decisions, each mapped to a section above. Us ### "Walk me through the architecture." -> NextAurora is a .NET 10 microservices platform with 5 backend services — Catalog, Order, Payment, Shipping, Notification. Each is independently deployable with its own database. Catalog and Shipping run on Postgres; Order and Payment on SQL Server; Notification is stateless. Cross-service communication is gRPC for synchronous queries (Order calls Catalog to validate products) and Azure Service Bus for asynchronous workflow events. **The per-service shape varies by complexity**: CatalogService — the largest — uses Clean Architecture (Domain/Application/Infrastructure/Api as four csprojs). The other four are smaller (≤2 aggregates each) and use Vertical Slice Architecture: a single project with feature folders, Domain/, Infrastructure/, Endpoints/. The cross-service diff is intentional and documented in CLAUDE.md. +> NextAurora is a .NET 10 microservices platform with 5 backend services — Catalog, Order, Payment, Shipping, Notification. Each is independently deployable with its own database. Catalog and Shipping run on Postgres; Order and Payment on SQL Server; Notification is stateless. Cross-service communication is gRPC for synchronous queries (Order calls Catalog to validate products) and RabbitMQ for asynchronous workflow events. **The per-service shape varies by complexity**: CatalogService — the largest — uses Clean Architecture (Domain/Application/Infrastructure/Api as four csprojs). The other four are smaller (≤2 aggregates each) and use Vertical Slice Architecture: a single project with feature folders, Domain/, Infrastructure/, Endpoints/. The cross-service diff is intentional and documented in CLAUDE.md. ### "Why microservices instead of modular monolith?" @@ -1144,7 +1144,7 @@ A condensed walkthrough of the key decisions, each mapped to a section above. Us ### "How would you scale this?" -> Three layers. **Vertically:** each service can scale to a larger VM. **Horizontally:** add replicas — but Catalog needs FusionCache before we deploy multi-replica because HybridCache 10.x lacks a backplane. **Database:** move from Aspire-managed local containers to RDS / managed Postgres / Azure SQL. The whole deployment story (AWS via SNS+SQS replacing Azure Service Bus) is laid out in [architecture.md "Deployment"](architecture.md). Wolverine's transport-agnostic design means swapping `WolverineFx.AzureServiceBus` for `WolverineFx.AmazonSqs` is a Program.cs change — handlers, contracts, the outbox all stay the same. +> Three layers. **Vertically:** each service can scale to a larger VM. **Horizontally:** add replicas — but Catalog needs FusionCache before we deploy multi-replica because HybridCache 10.x lacks a backplane. **Database:** move from Aspire-managed local containers to RDS / managed Postgres / Azure SQL. The whole deployment story (AWS via SNS+SQS replacing RabbitMQ) is laid out in [architecture.md "Deployment"](architecture.md). Wolverine's transport-agnostic design means swapping `WolverineFx.RabbitMQ` for `WolverineFx.AmazonSqs` is a Program.cs change — handlers, contracts, the outbox all stay the same. --- @@ -1163,7 +1163,7 @@ The classic five, plus **Workflow** — the durable-orchestration block that new | Dapr building block | What NextAurora uses today | Verdict | |---|---|---| | **Service invocation** | gRPC client factory + `.proto`-defined contracts (Order → Catalog product validation) | Covered with typed contracts | -| **Pub/sub** | Wolverine + Azure Service Bus + transactional outbox (§13) | Covered — and *better-integrated* (outbox in the EF `SaveChanges`) | +| **Pub/sub** | Wolverine + RabbitMQ + transactional outbox (§13) | Covered — and *better-integrated* (outbox in the EF `SaveChanges`) | | **State store** | EF Core (aggregates with concurrency tokens) + HybridCache (L1+L2 with stampede protection, §16) | Covered | | **Secrets** | Standard `IConfiguration` provider chain — env vars locally, Azure Key Vault in prod | Covered | | **Distributed locks** | None today | Not needed today — see below | @@ -1172,7 +1172,7 @@ The classic five, plus **Workflow** — the durable-orchestration block that new ### Why Dapr would *regress* what we have 1. **The transactional outbox is better-integrated in Wolverine.** Dapr *does* have a transactional outbox (added v1.12) — earlier versions of this doc claimed it didn't; that's now wrong and the correction matters. But Dapr's outbox is coupled to its **state-store** model: the atomic unit is "Dapr state write + Dapr pub/sub publish," routed through a configured state component. Ours is coupled to **EF Core**: the entity write and the staged message commit in the *same* `SaveChangesAsync` against the service's own DbContext (`PersistMessagesWithSqlServer` + `AutoApplyTransactions`), with the message store living in the same database as the aggregates. For a stack where the source of truth is EF aggregates (not a Dapr state component), Wolverine's outbox is the natural fit and Dapr's would mean routing writes through Dapr's state abstraction to get the atomicity — adopting Dapr's data model, not just its messaging. So: not "Dapr can't," but "Dapr's version assumes a Dapr-shaped persistence layer we don't have." -2. **The "swap brokers via YAML" claim oversells portability.** Broker semantics differ — Service Bus topic+subscription with sessions, FIFO, dead-lettering, and the globally-unique-subscription-name constraint doesn't map to a Kafka swap by editing one YAML line. The portability is real only for trivial fire-and-forget publishes. The real cross-cloud swap (ASB → SQS) is already in scope and handled by switching `WolverineFx.AzureServiceBus` to `WolverineFx.AmazonSqs` in `Program.cs` — same handler shape, same outbox guarantees. +2. **The "swap brokers via YAML" claim oversells portability.** Broker semantics differ — Service Bus topic+subscription with sessions, FIFO, dead-lettering, and the globally-unique-subscription-name constraint doesn't map to a Kafka swap by editing one YAML line. The portability is real only for trivial fire-and-forget publishes. The real cross-cloud swap (ASB → SQS) is already in scope and handled by switching `WolverineFx.RabbitMQ` to `WolverineFx.AmazonSqs` in `Program.cs` — same handler shape, same outbox guarantees. 3. **Typed contracts become stringly-typed Dapr invocations.** gRPC's compile-time safety and `.proto`-based versioning would disappear behind generic `InvokeMethodAsync(appId, method, payload)` calls. 4. **Sidecar adds a hop on every call.** localhost → sidecar → network → sidecar → service. For gRPC product validation on the order hot path, measurable cost we don't pay today. 5. **Speculative coupling at a runtime level.** The CLAUDE.md "interfaces earn their keep through consumer substitution" rule applies to runtimes too. Dapr adds an abstraction layer to enable swaps we've never needed and aren't planning. The five SDKs Dapr "replaces" in the marketing pitch are largely a strawman for our stack: Wolverine is *one* SDK covering messaging + outbox + middleware; secrets are stock .NET config; caching is HybridCache; service-to-service is gRPC. That's a coherent .NET-native stack, not five disconnected concerns. diff --git a/docs/transactional-outbox.excalidraw b/docs/transactional-outbox.excalidraw index 4040a75d..bbaf0dd8 100644 --- a/docs/transactional-outbox.excalidraw +++ b/docs/transactional-outbox.excalidraw @@ -47,8 +47,8 @@ "y": 80, "width": 799.8046875, "height": 84.50000000000001, - "text": "Solves the DUAL-WRITE PROBLEM: entity save + message publish must either BOTH happen or NEITHER. \n\\Wolverine stages outgoing messages into a wolverine.outgoing_envelopes table in the SAME EF transaction \nas the entity write. A background dispatcher then flushes to Service Bus with retry.\nSource: each service's Api/Program.cs (UseDurableOutboxOnAllSendingEndpoints) \n· WolverineFx.SqlServer / .Postgresql packages · docs/performance-and-data-correctness.md", - "originalText": "Solves the DUAL-WRITE PROBLEM: entity save + message publish must either BOTH happen or NEITHER. \n\\Wolverine stages outgoing messages into a wolverine.outgoing_envelopes table in the SAME EF transaction \nas the entity write. A background dispatcher then flushes to Service Bus with retry.\nSource: each service's Api/Program.cs (UseDurableOutboxOnAllSendingEndpoints) \n· WolverineFx.SqlServer / .Postgresql packages · docs/performance-and-data-correctness.md", + "text": "Solves the DUAL-WRITE PROBLEM: entity save + message publish must either BOTH happen or NEITHER. \n\\Wolverine stages outgoing messages into a wolverine.outgoing_envelopes table in the SAME EF transaction \nas the entity write. A background dispatcher then flushes to RabbitMQ with retry.\nSource: each service's Api/Program.cs (UseDurableOutboxOnAllSendingEndpoints) \n· WolverineFx.SqlServer / .Postgresql packages · docs/performance-and-data-correctness.md", + "originalText": "Solves the DUAL-WRITE PROBLEM: entity save + message publish must either BOTH happen or NEITHER. \n\\Wolverine stages outgoing messages into a wolverine.outgoing_envelopes table in the SAME EF transaction \nas the entity write. A background dispatcher then flushes to RabbitMQ with retry.\nSource: each service's Api/Program.cs (UseDurableOutboxOnAllSendingEndpoints) \n· WolverineFx.SqlServer / .Postgresql packages · docs/performance-and-data-correctness.md", "fontSize": 13, "fontFamily": 3, "textAlign": "left", @@ -523,8 +523,8 @@ "y": 440, "width": 367.3828125, "height": 71.5, - "text": "WolverineEventPublisher is a thin pass-through to \nIMessageBus.PublishAsync. \nUseDurableOutboxOnAllSendingEndpoints policy \nintercepts — instead of sending to Service Bus \ndirectly, it STAGES the envelope in the SAME transaction.", - "originalText": "WolverineEventPublisher is a thin pass-through to \nIMessageBus.PublishAsync. \nUseDurableOutboxOnAllSendingEndpoints policy \nintercepts — instead of sending to Service Bus \ndirectly, it STAGES the envelope in the SAME transaction.", + "text": "WolverineEventPublisher is a thin pass-through to \nIMessageBus.PublishAsync. \nUseDurableOutboxOnAllSendingEndpoints policy \nintercepts — instead of sending to RabbitMQ \ndirectly, it STAGES the envelope in the SAME transaction.", + "originalText": "WolverineEventPublisher is a thin pass-through to \nIMessageBus.PublishAsync. \nUseDurableOutboxOnAllSendingEndpoints policy \nintercepts — instead of sending to RabbitMQ \ndirectly, it STAGES the envelope in the SAME transaction.", "fontSize": 11, "fontFamily": 3, "textAlign": "left", @@ -1105,8 +1105,8 @@ "y": 794, "width": 260.15625, "height": 30, - "text": "Azure Service Bus topic\n(order-events / payment-events / ...)", - "originalText": "Azure Service Bus topic\n(order-events / payment-events / ...)", + "text": "RabbitMQ fanout exchange\n(order-events / payment-events / ...)", + "originalText": "RabbitMQ fanout exchange\n(order-events / payment-events / ...)", "fontSize": 12, "fontFamily": 3, "textAlign": "center", @@ -1451,4 +1451,4 @@ "viewBackgroundColor": "#ffffff" }, "files": {} -} \ No newline at end of file +} diff --git a/docs/transactional-outbox.svg b/docs/transactional-outbox.svg index 7641453e..8ca3f26b 100644 --- a/docs/transactional-outbox.svg +++ b/docs/transactional-outbox.svg @@ -1,2 +1,2 @@ -NextAurora — Transactional outbox (same-tx staging)Solves the DUAL-WRITE PROBLEM: entity save + message publish must either BOTH happen or NEITHER. \Wolverine stages outgoing messages into a wolverine.outgoing_envelopes table in the SAME EF transaction as the entity write. A background dispatcher then flushes to Service Bus with retry.Source: each service's Api/Program.cs (UseDurableOutboxOnAllSendingEndpoints) · WolverineFx.SqlServer / .Postgresql packages · docs/performance-and-data-correctness.mdPlaceOrderHandler.HandleAsync(cmd, ct)═══ EF DATABASE TRANSACTION (opened by AutoApplyTransactions middleware) ═══1. await orderRepository.AddAsync(order) → ctx.Orders.Add(order) →SaveChangesAsync → INSERT INTO Orders ...Repository writes the entity. EF Core executes the INSERT against the open transaction — not committed yet. The change tracker marks the entity as Unchanged after the write.2. await eventPublisher.PublishAsync( new OrderPlacedEvent { OrderId = … }) → bus.PublishAsync(@event)WolverineEventPublisher is a thin pass-through to IMessageBus.PublishAsync. UseDurableOutboxOnAllSendingEndpoints policy intercepts — instead of sending to Service Bus directly, it STAGES the envelope in the SAME transaction.3. INSERT INTO wolverine.outgoing_envelopes (id, message_type, body, destination, ...) VALUES (...); -- same DB transaction as step 1wolverine schema in the SAME database as the entity. Auto-created at startup by AddResourceSetupOnStartup().KEY INVARIANT: entity row and envelope row are in ONE transaction. Either both commit or neither does.✓ COMMIT (both rows persisted)background dispatcher reads outboxWolverine Background DispatcherSELECT * FROM wolverine.outgoing_envelopesWHERE owner_id = … ORDER BY idAzure Service Bus topic(order-events / payment-events / ...)On ack: DELETE FROMwolverine.outgoing_envelopes WHERE id = …Failure modes — all handled❌ BAD (without outbox): await SaveChangesAsync(); await bus.PublishAsync(...); ← two ops, can fail independently.Failures the outbox makes impossible: ● Bus publish fails mid-handler → tx hasn't committed yet → ROLLBACK on exception → neither row persists → handler errors out, client retries cleanly. ● Process crashes between SaveChanges and bus.PublishAsync → impossible: there IS no separate publish step. Wolverine staged the envelope inside the same tx. Both rows are there OR neither is. ● Bus publish succeeds, save commit fails → impossible: there's only one commit. If it fails, no envelope exists. If it succeeds, both rows are durable. ● Bus is down when dispatcher tries to flush → envelope stays in wolverine.outgoing_envelopes. Dispatcher retries with backoff. As soon as bus is back, drains. ● Dispatcher process crashes after sending but before deleting envelope → at-least-once: a duplicate delivery. Consumer must be idempotent (status guard pattern). See OrderService/Features/.// each service's Api/Program.cs — the configuration that makes it workbuilder.Host.UseWolverine(opts => { opts.PersistMessagesWithSqlServer(connStr, "wolverine"); // or .PersistMessagesWithPostgresql for Shipping opts.UseEntityFrameworkCoreTransactions(); // bridge to EF's tx opts.Policies.AutoApplyTransactions(); // wrap handlers in EF tx opts.Policies.UseDurableOutboxOnAllSendingEndpoints(); // stage outgoing into wolverine.outgoing_envelopes});builder.Services.AddResourceSetupOnStartup(); // auto-create wolverine schema + tables on app boot \ No newline at end of file +NextAurora — Transactional outbox (same-tx staging)Solves the DUAL-WRITE PROBLEM: entity save + message publish must either BOTH happen or NEITHER. \Wolverine stages outgoing messages into a wolverine.outgoing_envelopes table in the SAME EF transaction as the entity write. A background dispatcher then flushes to RabbitMQ with retry.Source: each service's Api/Program.cs (UseDurableOutboxOnAllSendingEndpoints) · WolverineFx.SqlServer / .Postgresql packages · docs/performance-and-data-correctness.mdPlaceOrderHandler.HandleAsync(cmd, ct)═══ EF DATABASE TRANSACTION (opened by AutoApplyTransactions middleware) ═══1. await orderRepository.AddAsync(order) → ctx.Orders.Add(order) →SaveChangesAsync → INSERT INTO Orders ...Repository writes the entity. EF Core executes the INSERT against the open transaction — not committed yet. The change tracker marks the entity as Unchanged after the write.2. await eventPublisher.PublishAsync( new OrderPlacedEvent { OrderId = … }) → bus.PublishAsync(@event)WolverineEventPublisher is a thin pass-through to IMessageBus.PublishAsync. UseDurableOutboxOnAllSendingEndpoints policy intercepts — instead of sending to RabbitMQ directly, it STAGES the envelope in the SAME transaction.3. INSERT INTO wolverine.outgoing_envelopes (id, message_type, body, destination, ...) VALUES (...); -- same DB transaction as step 1wolverine schema in the SAME database as the entity. Auto-created at startup by AddResourceSetupOnStartup().KEY INVARIANT: entity row and envelope row are in ONE transaction. Either both commit or neither does.✓ COMMIT (both rows persisted)background dispatcher reads outboxWolverine Background DispatcherSELECT * FROM wolverine.outgoing_envelopesWHERE owner_id = … ORDER BY idRabbitMQ fanout exchange(order-events / payment-events / ...)On ack: DELETE FROMwolverine.outgoing_envelopes WHERE id = …Failure modes — all handled❌ BAD (without outbox): await SaveChangesAsync(); await bus.PublishAsync(...); ← two ops, can fail independently.Failures the outbox makes impossible: ● Bus publish fails mid-handler → tx hasn't committed yet → ROLLBACK on exception → neither row persists → handler errors out, client retries cleanly. ● Process crashes between SaveChanges and bus.PublishAsync → impossible: there IS no separate publish step. Wolverine staged the envelope inside the same tx. Both rows are there OR neither is. ● Bus publish succeeds, save commit fails → impossible: there's only one commit. If it fails, no envelope exists. If it succeeds, both rows are durable. ● Bus is down when dispatcher tries to flush → envelope stays in wolverine.outgoing_envelopes. Dispatcher retries with backoff. As soon as bus is back, drains. ● Dispatcher process crashes after sending but before deleting envelope → at-least-once: a duplicate delivery. Consumer must be idempotent (status guard pattern). See OrderService/Features/.// each service's Api/Program.cs — the configuration that makes it workbuilder.Host.UseWolverine(opts => { opts.PersistMessagesWithSqlServer(connStr, "wolverine"); // or .PersistMessagesWithPostgresql for Shipping opts.UseEntityFrameworkCoreTransactions(); // bridge to EF's tx opts.Policies.AutoApplyTransactions(); // wrap handlers in EF tx opts.Policies.UseDurableOutboxOnAllSendingEndpoints(); // stage outgoing into wolverine.outgoing_envelopes});builder.Services.AddResourceSetupOnStartup(); // auto-create wolverine schema + tables on app boot \ No newline at end of file diff --git a/tests/CatalogService.Tests.Integration/README.md b/tests/CatalogService.Tests.Integration/README.md index 15ac05d5..2baa632f 100644 --- a/tests/CatalogService.Tests.Integration/README.md +++ b/tests/CatalogService.Tests.Integration/README.md @@ -57,6 +57,6 @@ CI (`ubuntu-latest`) ships Docker at the standard path, so the `integration-test ## Adding more This is the CatalogService slice — the proven harness pattern. The heavier saga/messaging -integration tests (Wolverine outbox staging, cross-service choreography) need the Azure -Service Bus emulator container and are a separate, larger effort. See +integration tests (Wolverine outbox staging, cross-service choreography) need a RabbitMQ +broker (Testcontainer) and are a separate, larger effort. See [docs/STATUS.md](../../docs/STATUS.md). diff --git a/tests/OrderService.Tests.Integration/OrderApiFactory.cs b/tests/OrderService.Tests.Integration/OrderApiFactory.cs index 7e93e380..ba1a5b85 100644 --- a/tests/OrderService.Tests.Integration/OrderApiFactory.cs +++ b/tests/OrderService.Tests.Integration/OrderApiFactory.cs @@ -25,23 +25,24 @@ namespace OrderService.Tests.Integration; /// over an actual database. /// /// -/// Why stub the transport instead of using the Azure Service Bus emulator container: the +/// Why stub the transport instead of spinning up a real RabbitMQ broker: the /// outbox-staging guarantee (entity-write + envelope-write same transaction) and the -/// handler/saga logic are what the unit tests can't reach. The ASB wire path itself mostly -/// exercises Microsoft's emulator + Wolverine's transport adapter — it's the fragile last mile, -/// not the load-bearing correctness piece. See docs/STATUS.md for the deferred follow-up. +/// handler/saga logic are what the unit tests can't reach. The broker wire path itself mostly +/// exercises RabbitMQ + Wolverine's transport adapter — it's the fragile last mile, not the +/// load-bearing correctness piece. (A RabbitMQ Testcontainer for real-wire saga coverage is the +/// deferred follow-up — see docs/dev-loop.md Gap 1 + issue #68.) /// /// /// Why a fake "messaging" connection string: Program.cs does -/// GetConnectionString("messaging")! then UseAzureServiceBus(connectionString). -/// Even with external transports disabled, the parsing happens at registration time, so the -/// string has to be syntactically valid ASB. It's never used over the wire. +/// GetConnectionString("messaging")! then UseRabbitMq(...). Even with external +/// transports disabled, the connection is parsed at registration time, so the string has to be a +/// syntactically valid AMQP URI. It's never used over the wire. /// /// /// Why stub ICatalogClient: PlaceOrderHandler validates products + reserves /// stock over gRPC to CatalogService. We're testing OrderService in isolation, so the stub /// returns valid products with enough stock. Cross-service choreography is the heavier slice -/// tracked in STATUS.md. +/// tracked in docs/dev-loop.md Gap 1 + issue #68. /// /// public sealed class OrderApiFactory : WebApplicationFactory, IAsyncLifetime @@ -95,28 +96,19 @@ protected override void ConfigureWebHost(IWebHostBuilder builder) { ArgumentNullException.ThrowIfNull(builder); - // Disable Wolverine's .AutoProvision() in Program.cs. AutoProvision tries to - // create topics/subscriptions on the configured Service Bus namespace at host - // startup; against our fake "sb://fake.servicebus.windows.net/..." connection - // string it hangs trying to resolve/connect, eventually killing the test job - // at the runner's job limit. DisableAllExternalWolverineTransports() below - // handles the message-routing path; this handles broker-provisioning, which - // runs *before* ConfigureTestServices fires. + // Disable Wolverine's .AutoProvision() in Program.cs. AutoProvision connects to the broker + // at host startup to declare exchanges/queues; against our fake connection string it would + // hang. DisableAllExternalWolverineTransports() below handles message routing, but + // provisioning runs *before* ConfigureTestServices fires, so it's gated off here. builder.UseSetting("Wolverine:AutoProvision", "false"); // Real SQL Server for the order DB + Wolverine outbox tables. builder.UseSetting("ConnectionStrings:orders-db", _sqlServer.GetConnectionString()); - // Syntactically-valid Azure Service Bus connection string. Never used over the wire — - // DisableAllExternalWolverineTransports below routes outgoing messages to local stubs — - // but Wolverine's UseAzureServiceBus(...) registration parses it eagerly. SharedAccessKey - // base64-decodes to "fake-shared-key-for-testing-only". The inline `gitleaks:allow` - // marker on the literal line is the suppressor. There is no project-level gitleaks - // config (global [[allowlists]] needs gitleaks 8.25+, runner ships 8.24.x); the inline - // marker is the load-bearing mechanism. See CLAUDE.md. - builder.UseSetting( - "ConnectionStrings:messaging", - "Endpoint=sb://fake.servicebus.windows.net/;SharedAccessKeyName=fake;SharedAccessKey=ZmFrZS1zaGFyZWQta2V5LWZvci10ZXN0aW5nLW9ubHk="); // gitleaks:allow + // Syntactically-valid RabbitMQ (AMQP) connection string. Never used over the wire — + // DisableAllExternalWolverineTransports() below routes messages to local stubs — but + // Wolverine's UseRabbitMq(...) registration parses it eagerly. + builder.UseSetting("ConnectionStrings:messaging", "amqp://guest:guest@localhost:5672"); builder.ConfigureTestServices(services => { @@ -130,7 +122,7 @@ protected override void ConfigureWebHost(IWebHostBuilder builder) // this; the stub keeps OrderService boot-able without standing up CatalogService. services.AddSingleton(Catalog); - // Disable the Azure Service Bus listeners + senders Wolverine registered in + // Disable the RabbitMQ listeners + senders Wolverine registered in // Program.cs. Outgoing messages route to in-process stubs, which still flow through // Wolverine's middleware chain (FluentValidation, AutoApplyTransactions, the // durable outbox, ContextPropagation) — so the outbox-staging guarantee is what diff --git a/tests/OrderService.Tests.Integration/OrderSagaTests.cs b/tests/OrderService.Tests.Integration/OrderSagaTests.cs index c27811ef..683fd054 100644 --- a/tests/OrderService.Tests.Integration/OrderSagaTests.cs +++ b/tests/OrderService.Tests.Integration/OrderSagaTests.cs @@ -234,7 +234,7 @@ public async Task PaymentCompletedEvent_transitions_Placed_to_Paid_and_is_idempo // ARRANGE — Seed a Placed order directly via the DbContext (faster than going // through the full PlaceOrder flow). The PaymentCompletedEvent simulates what // PaymentService publishes after a successful charge. We use the same event - // twice to verify idempotency under Service Bus at-least-once delivery. + // twice to verify idempotency under the broker's at-least-once delivery. var orderId = await SeedOrderAsync(status: OrderStatus.Placed); var paymentEvent = new PaymentCompletedEvent { @@ -249,7 +249,7 @@ public async Task PaymentCompletedEvent_transitions_Placed_to_Paid_and_is_idempo // ACT — First dispatch: the handler should run and the Order transitions // Placed → Paid. PublishMessageAndWaitAsync invokes the consumer-side pipeline - // exactly as Wolverine would on a real Service Bus message. + // exactly as Wolverine would on a real RabbitMQ message. await host.TrackActivity() .Timeout(TimeSpan.FromSeconds(30)) .PublishMessageAndWaitAsync(paymentEvent); @@ -257,7 +257,7 @@ await host.TrackActivity() // ASSERT (intermediate) — After the first dispatch, status is Paid. (await GetOrderStatusAsync(orderId)).Should().Be(OrderStatus.Paid); - // ACT — Second dispatch (Service Bus redelivery simulation). The handler's + // ACT — Second dispatch (broker redelivery simulation). The handler's // status-guard MUST short-circuit cleanly — no exception, no extra mutation. await host.TrackActivity() .Timeout(TimeSpan.FromSeconds(30)) diff --git a/tests/OrderService.Tests.Integration/README.md b/tests/OrderService.Tests.Integration/README.md index 87be5bf5..ffe20202 100644 --- a/tests/OrderService.Tests.Integration/README.md +++ b/tests/OrderService.Tests.Integration/README.md @@ -46,8 +46,12 @@ CI (`ubuntu-latest`) ships Docker at the standard path, so the `integration-test - **`OrderApiFactory`** — `WebApplicationFactory` + `IAsyncLifetime`. Starts the SQL Server container, injects its connection string as both `orders-db` and the persistence endpoint for Wolverine's outbox tables (`wolverine` schema, auto-created at startup via - `AddResourceSetupOnStartup`). A syntactically-valid Azure Service Bus connection string is - also injected — never used over the wire, but `UseAzureServiceBus(...)` parses it eagerly. + `AddResourceSetupOnStartup`). A syntactically-valid low-entropy AMQP connection string + (`amqp://guest:guest@localhost:5672`) is also injected — never used over the wire, but + `UseRabbitMq(...)` in `Program.cs` parses it eagerly at registration time, before + `DisableAllExternalWolverineTransports()` stubs the transport. `Wolverine:AutoProvision` + is set to `false` via `UseSetting` — AutoProvision connects at host startup to declare + exchanges/queues, and against the fake connection string it would hang. Calls `services.DisableAllExternalWolverineTransports()` so outgoing messages route to in-process stubs while the durable outbox still wraps them. - **`TestAuthHandler`** — always-succeeds auth. The handler stamps a fixed buyer Guid into the @@ -57,17 +61,17 @@ CI (`ubuntu-latest`) ships Docker at the standard path, so the `integration-test `PublishMessageAndWaitAsync` to drive the saga consume-side. Test 5 hits the EF concurrency path directly via two DbContext scopes. -## Why stubbed transport instead of the Azure Service Bus emulator +## Why stubbed transport instead of a real RabbitMQ broker The outbox-staging guarantee (entity-write + envelope-write same transaction) and the saga -consume-side handlers are what this slice proves — the ASB wire path itself mostly exercises -Microsoft's emulator + Wolverine's transport adapter, which is the fragile last mile and -lower-value per unit of effort. The ASB-emulator-based wire test is filed as a separate -follow-up in [docs/STATUS.md](../../docs/STATUS.md). +consume-side handlers are what this slice proves — the broker wire path itself mostly +exercises RabbitMQ + Wolverine's transport adapter, which is the fragile last mile and +lower-value per unit of effort. A RabbitMQ Testcontainer for real-wire saga coverage is +filed as a separate follow-up in [docs/STATUS.md](../../docs/STATUS.md). ## Adding more This is the second integration slice (after CatalogService) and the proven pattern for any service that has a DB + Wolverine handlers. Payment and Shipping would follow the same shape; the saga *across* services (real cross-service choreography) needs all services booted plus -either the ASB emulator or coordinated in-process Wolverine — a separate, heavier project. +either a RabbitMQ Testcontainer or coordinated in-process Wolverine — a separate, heavier project. diff --git a/tests/OrderService.Tests.Unit/Domain/OrderTests.cs b/tests/OrderService.Tests.Unit/Domain/OrderTests.cs index 2edd9a95..fb8535a5 100644 --- a/tests/OrderService.Tests.Unit/Domain/OrderTests.cs +++ b/tests/OrderService.Tests.Unit/Domain/OrderTests.cs @@ -111,7 +111,7 @@ public void Create_SetsPlacedAtToUtcNow() public void MarkAsPaid_WhenPlaced_SetsStatusToPaid() { // ARRANGE — Happy-path saga transition: PaymentCompletedHandler calls MarkAsPaid - // after Service Bus delivers PaymentCompletedEvent. + // after the broker delivers PaymentCompletedEvent. var order = OrderBuilder.Default().Build(); // ACT — Transition Placed → Paid. @@ -128,7 +128,7 @@ public void MarkAsPaid_WhenPlaced_SetsStatusToPaid() public void MarkAsPaid_WhenNotPlaced_ThrowsInvalidOperationException() { // ARRANGE — Calling MarkAsPaid on an order that's already Paid (or any non-Placed - // state). The status guard is what makes the handler idempotent under Service Bus + // state). The status guard is what makes the handler idempotent under RabbitMQ // at-least-once delivery — a redelivered PaymentCompletedEvent must NOT corrupt // state. The HANDLER catches the throw and treats it as a no-op (see // PaymentCompletedHandlerTests); here we verify the DOMAIN-level guard exists. diff --git a/tests/PaymentService.Tests.Integration/PaymentApiFactory.cs b/tests/PaymentService.Tests.Integration/PaymentApiFactory.cs index c80a8053..fb77007d 100644 --- a/tests/PaymentService.Tests.Integration/PaymentApiFactory.cs +++ b/tests/PaymentService.Tests.Integration/PaymentApiFactory.cs @@ -25,16 +25,16 @@ namespace PaymentService.Tests.Integration; /// RowVersion concurrency token, and the OrderId-uniqueness idempotency guard. /// /// -/// Why stub the transport instead of the ASB emulator: the outbox-staging guarantee and +/// Why stub the transport instead of a real RabbitMQ broker: the outbox-staging guarantee and /// the handler logic are what unit tests can't reach. The internal PaymentProcessingRequested /// message has no external routing, so it stays on the in-process local queue and is fully /// exercised here; only the outbound PaymentCompletedEvent/PaymentFailedEvent wire -/// hop is stubbed. See docs/STATUS.md. +/// hop is stubbed. See docs/dev-loop.md Gap 1 + issue #68. /// /// /// Why a fake "messaging" connection string: Program.cs calls -/// UseAzureServiceBus(GetConnectionString("messaging")!), parsed eagerly at registration -/// even with external transports disabled — so it must be syntactically valid ASB. +/// UseRabbitMq(GetConnectionString("messaging")!), parsed eagerly at registration even with +/// external transports disabled — so it must be a syntactically valid AMQP URI. /// /// /// Why stub IPaymentGateway: the Gateway handler calls the Stripe gateway. Tests @@ -81,22 +81,17 @@ protected override void ConfigureWebHost(IWebHostBuilder builder) { ArgumentNullException.ThrowIfNull(builder); - // Disable Wolverine AutoProvision — against the fake ASB string it would hang trying to - // provision topics/subscriptions at startup. DisableAllExternalWolverineTransports below + // Disable Wolverine AutoProvision — against the fake AMQP string it would hang trying to + // provision exchanges/queues at startup. DisableAllExternalWolverineTransports below // handles routing; this handles the broker-provisioning that runs earlier. builder.UseSetting("Wolverine:AutoProvision", "false"); // Real SQL Server for the payments DB + Wolverine outbox tables. builder.UseSetting("ConnectionStrings:payments-db", _sqlServer.GetConnectionString()); - // Syntactically-valid ASB connection string — parsed eagerly, never used over the wire. - // SharedAccessKey base64-decodes to "fake-shared-key-for-testing-only". The inline - // `gitleaks:allow` marker on the literal line is the suppressor. There is no project-level - // gitleaks config (global [[allowlists]] needs gitleaks 8.25+, runner ships 8.24.x); the - // inline marker is the load-bearing mechanism. See CLAUDE.md. - builder.UseSetting( - "ConnectionStrings:messaging", - "Endpoint=sb://fake.servicebus.windows.net/;SharedAccessKeyName=fake;SharedAccessKey=ZmFrZS1zaGFyZWQta2V5LWZvci10ZXN0aW5nLW9ubHk="); // gitleaks:allow + // Syntactically-valid RabbitMQ (AMQP) connection string — parsed eagerly by UseRabbitMq(...), + // never used over the wire (DisableAllExternalWolverineTransports() stubs the transport). + builder.UseSetting("ConnectionStrings:messaging", "amqp://guest:guest@localhost:5672"); builder.ConfigureTestServices(services => { @@ -107,7 +102,7 @@ protected override void ConfigureWebHost(IWebHostBuilder builder) // Stub the Stripe gateway — tests drive Completed/Failed via the configured result. services.AddSingleton(Gateway); - // Disable external ASB listeners/senders. Outgoing events route to in-process stubs + // Disable external RabbitMQ listeners/senders. Outgoing events route to in-process stubs // (still through the outbox + middleware chain); the internal PaymentProcessingRequested // message stays on the local queue and runs end-to-end. services.DisableAllExternalWolverineTransports(); diff --git a/tests/PaymentService.Tests.Unit/Application/OrderPlacedHandlerTests.cs b/tests/PaymentService.Tests.Unit/Application/OrderPlacedHandlerTests.cs index 0a487acf..94c58a2c 100644 --- a/tests/PaymentService.Tests.Unit/Application/OrderPlacedHandlerTests.cs +++ b/tests/PaymentService.Tests.Unit/Application/OrderPlacedHandlerTests.cs @@ -9,7 +9,7 @@ public class OrderPlacedHandlerTests [Fact] public void Handle_TranslatesEventIntoProcessPaymentCommand() { - // ARRANGE — Build an OrderPlacedEvent as it would arrive over Service Bus. + // ARRANGE — Build an OrderPlacedEvent as it would arrive over RabbitMQ. // OrderPlacedHandler is a static "Wolverine cascading message" — it returns the // next command, and Wolverine handles dispatch. The whole class exists for one // reason: ProcessPaymentCommand is also reachable from the HTTP endpoint, so we diff --git a/tests/ShippingService.Tests.Integration/ShippingApiFactory.cs b/tests/ShippingService.Tests.Integration/ShippingApiFactory.cs index cd55f92a..4db6ebfe 100644 --- a/tests/ShippingService.Tests.Integration/ShippingApiFactory.cs +++ b/tests/ShippingService.Tests.Integration/ShippingApiFactory.cs @@ -22,15 +22,15 @@ namespace ShippingService.Tests.Integration; /// xmin concurrency token; and EF migrations applying against a fresh Postgres. /// /// -/// Why stub the transport instead of the ASB emulator: outbox-staging atomicity and -/// handler logic are what unit tests can't reach. The ASB wire path mostly exercises Microsoft's -/// emulator + Wolverine's transport adapter — the fragile last mile, not the load-bearing -/// correctness piece. See docs/STATUS.md. +/// Why stub the transport instead of a real RabbitMQ broker: outbox-staging atomicity and +/// handler logic are what unit tests can't reach. The broker wire path mostly exercises RabbitMQ + +/// Wolverine's transport adapter — the fragile last mile, not the load-bearing correctness piece. +/// See docs/dev-loop.md Gap 1 + issue #68. /// /// /// Why a fake "messaging" connection string: Program.cs parses -/// UseAzureServiceBus(GetConnectionString("messaging")!) eagerly at registration, even -/// with external transports later disabled. The string has to be syntactically valid ASB. +/// UseRabbitMq(GetConnectionString("messaging")!) eagerly at registration, even with +/// external transports later disabled. The string has to be a syntactically valid AMQP URI. /// /// public sealed class ShippingApiFactory : WebApplicationFactory, IAsyncLifetime @@ -68,22 +68,17 @@ protected override void ConfigureWebHost(IWebHostBuilder builder) { ArgumentNullException.ThrowIfNull(builder); - // Disable Wolverine AutoProvision — against the fake ASB string it would hang trying to - // provision topics/subscriptions at startup. DisableAllExternalWolverineTransports + // Disable Wolverine AutoProvision — against the fake AMQP string it would hang trying to + // provision exchanges/queues at startup. DisableAllExternalWolverineTransports // handles routing; this handles broker-provisioning, which runs earlier. builder.UseSetting("Wolverine:AutoProvision", "false"); // Real Postgres for the shipping DB + Wolverine outbox tables. builder.UseSetting("ConnectionStrings:shipping-db", _postgres.GetConnectionString()); - // Syntactically-valid ASB connection string — parsed eagerly, never used over the wire. - // SharedAccessKey base64-decodes to "fake-shared-key-for-testing-only". The inline - // `gitleaks:allow` marker on the literal line is the suppressor. There is no project-level - // gitleaks config (global [[allowlists]] needs gitleaks 8.25+, runner ships 8.24.x); the - // inline marker is the load-bearing mechanism. See CLAUDE.md. - builder.UseSetting( - "ConnectionStrings:messaging", - "Endpoint=sb://fake.servicebus.windows.net/;SharedAccessKeyName=fake;SharedAccessKey=ZmFrZS1zaGFyZWQta2V5LWZvci10ZXN0aW5nLW9ubHk="); // gitleaks:allow + // Syntactically-valid RabbitMQ (AMQP) connection string — parsed eagerly by UseRabbitMq(...), + // never used over the wire (DisableAllExternalWolverineTransports() stubs the transport). + builder.UseSetting("ConnectionStrings:messaging", "amqp://guest:guest@localhost:5672"); builder.ConfigureTestServices(services => { @@ -93,7 +88,7 @@ protected override void ConfigureWebHost(IWebHostBuilder builder) services.AddAuthentication(TestAuthHandler.SchemeName) .AddScheme(TestAuthHandler.SchemeName, _ => { }); - // Disable external ASB listeners/senders. Saga events the handler publishes route to + // Disable external RabbitMQ listeners/senders. Saga events the handler publishes route to // in-process stubs (still through the outbox + middleware chain). services.DisableAllExternalWolverineTransports(); });