Skip to content

Keep the behind_threshold columns so upgrading does not change this schema (#5173) - #5188

Merged
jeremydmiller merged 1 commit into
masterfrom
fix/5173-keep-behind-threshold-columns
Aug 4, 2026
Merged

Keep the behind_threshold columns so upgrading does not change this schema (#5173)#5188
jeremydmiller merged 1 commit into
masterfrom
fix/5173-keep-behind-threshold-columns

Conversation

@jeremydmiller

Copy link
Copy Markdown
Member

Partially reverts #5184. The read-side half of that change stands — warning_behind_threshold / critical_behind_threshold are still not selected and not hydrated onto ShardState, and ShardStateSelector's ordinals still skip them. Only the DDL is restored, so the two columns exist and carry nothing.

Why

#5184's reasoning about the data holds: nothing in any repo ever wrote or read these, every read returned NULL, and every lag threshold lives in the monitoring tool. What it got wrong was the upgrade cost. Its commit message says existing deployments "get an alter table ... drop column on their next ApplyAllConfiguredChangesToDatabaseAsync(), which is lossless."

Measured against PostgreSQL — on a store with event sourcing registered, with an mt_event_progression carrying a column the configuration no longer declares:

CreateMigrationAsync().Difference           = Update   (mt_event_progression=Update)
AssertDatabaseMatchesConfigurationAsync()   -> throws
ApplyAllConfiguredChangesToDatabaseAsync()  -> drops the column

Two consequences for existing deployments, neither of them "lossless in passing":

  1. A host running AutoCreate.None with a schema assertion at boot — the standard production posture — does not migrate. It refuses to start until someone migrates it.
  2. A host that does apply runs alter table … drop column. In PostgreSQL that is O(1) metadata, but it needs ACCESS EXCLUSIVE on a small, hot table that every running daemon writes. The operation is instant; acquiring the lock is not, and while it waits it blocks every reader and writer queued behind it.

Two always-NULL columns are much cheaper than asking every deployment to absorb that. Keeping them also means a freshly created database and an upgraded one have the same shape, rather than diverging on a detail nothing reads.

If they are ever removed for real, it should be a documented migration step — not a silent consequence of upgrading a package.

A note on the tests

Both new tests fail with the columns taken back out. The AddEventType in each is load-bearing:

Without an event type registered, the event store's tables are not part of the schema migration at all — every delta comes back None and any assertion about mt_event_progression passes vacuously.

That is exactly how the first version of this test was green against both shapes, and it is the same feature-scoping trap that produced a wrong diagnosis on CritterWatch#889. Worth knowing about for anything else that asserts on schema deltas.

Verification

🤖 Generated with Claude Code

https://claude.ai/code/session_016v2Aijyo8MX2AdPUZL5VtG

…chema (#5173)

#5184 removed warning_behind_threshold / critical_behind_threshold from three places: the
DDL, the extended-tracking SELECT, and ShardStateSelector's ordinals. The reasoning holds
-- nothing in any repo ever wrote or read them, and every lag threshold lives in the
monitoring tool -- and the read-side half of it stands. Only the DDL is restored.

The cost of removing them was never the two always-NULL columns, it is the schema CHANGE.
Measured against PostgreSQL, on a store with event sourcing registered, an
mt_event_progression carrying a column the configuration no longer declares gives:

    CreateMigrationAsync().Difference   = Update   (mt_event_progression=Update)
    AssertDatabaseMatchesConfigurationAsync()      -> throws
    ApplyAllConfiguredChangesToDatabaseAsync()     -> drops the column

So an upgrade is not the no-op #5184's commit message implied. Two things happen to
existing deployments. A host running AutoCreate.None with a schema assertion at boot --
the standard production posture -- does not migrate, it refuses to start until someone
migrates it. And a host that does apply runs `alter table ... drop column`, which in
PostgreSQL is O(1) metadata but needs ACCESS EXCLUSIVE on a small, hot table that every
running daemon writes; while that lock waits it blocks every reader and writer behind it.

Two unused columns are much cheaper than asking every deployment to absorb that, and
keeping them also means a freshly created database and an upgraded one have the same
shape rather than diverging on a detail nothing reads.

If they are ever removed for real it has to be a documented migration step rather than a
silent consequence of upgrading a package.

The tests pin both halves and both fail with the columns taken back out. Note the
`AddEventType` in each: without an event type registered the event store's tables are not
part of the schema migration at all, every delta comes back None, and any assertion about
this table passes vacuously -- which is how the original version of this test was green
against both shapes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016v2Aijyo8MX2AdPUZL5VtG
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant