Skip to content

[FIXED] NRG: Truncate back to snapshot properly#7095

Merged
neilalexander merged 3 commits intomainfrom
maurice/truncate-to-snap
Jul 23, 2025
Merged

[FIXED] NRG: Truncate back to snapshot properly#7095
neilalexander merged 3 commits intomainfrom
maurice/truncate-to-snap

Conversation

@MauriceVanVeen
Copy link
Copy Markdown
Member

We would not truncate back to a snapshot properly if n.applied was not caught up or moved further ahead.

For example:

  • outdated server catches up and gets a snapshot at index 10 (n.commit = 10, but n.applied = 0 still)
  • server misses message at index 11
  • server receives message at index 12 and catches up between indices 10-12
  • server now receives message at index 11 with a delay (the ae.pindex will be 10, so it requires a consistency check on an append entry that doesn't exist because a snapshot is there)
  • we now need to truncate the log back to the snapshot (we would reset the WAL fully, which is bad)

We now check if we can truncate back to the snapshot successfully, otherwise we still reset the WAL as before.

Also improved some logs to make debug lines more helpful when debugging issues.

Signed-off-by: Maurice van Veen github@mauricevanveen.com

Signed-off-by: Maurice van Veen <github@mauricevanveen.com>
Signed-off-by: Maurice van Veen <github@mauricevanveen.com>
@MauriceVanVeen MauriceVanVeen requested a review from a team as a code owner July 23, 2025 10:03
Signed-off-by: Maurice van Veen <github@mauricevanveen.com>
Copy link
Copy Markdown
Member

@neilalexander neilalexander left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Copy link
Copy Markdown
Member

@derekcollison derekcollison left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@neilalexander neilalexander merged commit 7e0dddd into main Jul 23, 2025
132 of 136 checks passed
@neilalexander neilalexander deleted the maurice/truncate-to-snap branch July 23, 2025 12:17
@wallyqs wallyqs changed the title NRG: Truncate back to snapshot properly [FIXED] NRG: Truncate back to snapshot properly Jul 24, 2025
neilalexander added a commit that referenced this pull request Jul 25, 2025
Includes the following:

- #7031
- #7033
- #7034
- #7035
- #7036
- #7040
- #7043
- #7045
- #7047
- #7046
- #7050
- #7051
- #7052
- #7053
- #7061
- #7063
- #7064
- #7065
- #7066
- #7070
- #7072
- #7080
- #7026
- #6728
- #7074
- #7089
- #7095
- #7087
- #7094
- #7096
- #7099

Signed-off-by: Neil Twigg <neil@nats.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants