Skip to content

Report whether durability/projection affinity engaged (GH-3785) - #3854

Merged
jeremydmiller merged 1 commit into
mainfrom
gh-3785/affinity-diagnostic
Aug 5, 2026
Merged

Report whether durability/projection affinity engaged (GH-3785)#3854
jeremydmiller merged 1 commit into
mainfrom
gh-3785/affinity-diagnostic

Conversation

@jeremydmiller

Copy link
Copy Markdown
Member

The GH-3785 cross-family affinity join (#3805) is deliberately fail-silent — a durability agent that cannot be matched to its database's projection owner falls back to the even spread, because a miss is never wrong, only not-better. That is the right runtime behavior and an unusable diagnostic story: a join that never fires because the two descriptor pipelines spell the same database differently looks exactly like the feature working, minus the benefit.

Short of joining pg_stat_activity against the assignment table on a production cluster, there was no way to tell the two apart. That matters right now: #3785 is one confirming measurement away from closing, and the measurement it is waiting on is precisely "did the join engage against this cluster's 512 databases and its particular server naming."

What this adds

DurabilityAffinityPreference wraps the preference function and counts both outcomes:

KnownDatabases databases with event-subscription agents in the grid — something to follow
Considered durability agents whose URI actually names a database (the null store and the composite marker are not databases, and would otherwise read as permanent misses)
Matched of those, how many resolved to their database's projection owner

Logged only when the numbers move. Assignment is re-evaluated on every health check, and a settled cluster reports the same figures forever — an unconditional line would be noise at exactly the cadence that makes it unreadable. A change means a deploy, a rebalance, or a database arriving or leaving.

Three outcomes, three behaviors:

  • Matched > 0Information, once: co-located {Matched} of {Considered} durability agents ... across {KnownDatabases} databases
  • Matched == 0 with KnownDatabases and Considered both non-zero → Warning. There are projection agents, there are database-bearing durability agents, and not one joined. On a multi-database store that is a spelling divergence, not a coincidence — and it names the consequence (each of those databases attracts two nodes' connection pools instead of one).
  • KnownDatabases == 0 → silent. An application with no projections has nothing to co-locate with; that is not a finding.

Testing

7 new CoreTests, each mutation-tested against a matching defect — counting non-database URIs as considered, dropping the change gate, not escalating the fail-silent case, warning when there was nothing to follow. All four mutants killed, one test each, no overlap.

The real-stores Marten test now also asserts every database matched. That test exists because the join spans two descriptor pipelines (Weasel vs Marten), so it is the right place to pin the counters — it is the only test where a spelling divergence could actually occur.

  • CoreTests.Runtime.Agents — 293/293
  • durability_projection_affinity_real_stores — passes against real Postgres tenant databases, 3 of 3 matched
  • Full wolverine.slnx Release build clean

Behavior is unchanged: this is counting and logging only.

Related: #3785, #3805

🤖 Generated with Claude Code

https://claude.ai/code/session_01WHAuhdWS3XeAk16swV9G8m

The GH-3785 cross-family affinity join is deliberately fail-silent: a durability
agent that cannot be matched to its database's projection owner falls back to the
even spread, because a miss is never wrong, only not-better. That is the right
runtime behavior and an unusable diagnostic story -- a join that never fires
because the two descriptor pipelines spell the same database differently looks
exactly like the feature working, minus the benefit.

Short of joining pg_stat_activity against the assignment table on a production
cluster, there was no way to tell the two apart. So count both outcomes and say
so: matched, considered, and the number of databases there were to follow.

Logged only when the numbers move -- assignment is re-evaluated on every health
check and a settled cluster reports the same figures forever, so an unconditional
line would be noise at exactly the cadence that makes it unreadable. A change
means a deploy, a rebalance, or a database arriving or leaving.

The one case that escalates to a warning is Matched == 0 with both KnownDatabases
and Considered non-zero: there ARE projection agents, there ARE database-bearing
durability agents, and not one of them joined. On a multi-database store that is
a spelling divergence, not a coincidence. An application with no projections has
nothing to co-locate with and stays silent.

Seven new CoreTests, each mutation-tested against a matching defect (counting
non-database URIs, dropping the change gate, not escalating the fail-silent case,
warning when there was nothing to follow). The real-stores Marten test now also
asserts every database matched -- that test exists because the join spans two
descriptor pipelines, so it is the right place to pin the counters.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WHAuhdWS3XeAk16swV9G8m
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant