Skip to content

Migrate LogIndex block numbers from int to uint#10430

Closed
LukaszRozmej wants to merge 5 commits intofeature/log-index-2from
feature/log-index-2-uint
Closed

Migrate LogIndex block numbers from int to uint#10430
LukaszRozmej wants to merge 5 commits intofeature/log-index-2from
feature/log-index-2-uint

Conversation

@LukaszRozmej
Copy link
Copy Markdown
Member

@LukaszRozmej LukaszRozmej commented Feb 6, 2026

Based on #8464

Changes

  • Migrate all LogIndex block number types from int to uint, supporting block numbers up to ~4.3 billion
  • Fix IsCompressed detection: replace the sign-bit trick (len > 0 on a negated int32 marker) with key-based detection — transient keys (BackwardMerge/ForwardMerge postfixes) are never compressed, finalized keys are always compressed. The old approach broke for block numbers >= 2^31 because the compression marker (stored as a negative int) could collide with valid uint block numbers
  • Fix BinarySearch: replace MemoryMarshal.Cast<uint, int> + built-in BinarySearch with a proper uint binary search, since the cast produces incorrect ordering for values >= 2^31
  • Switch CompressionAlgorithm delegates from ReadOnlySpan<int>/Span<int> to ReadOnlySpan<uint>/Span<uint>, removing MemoryMarshal.Cast<uint, int> workarounds in compress/decompress paths
  • Add a TestFixtureData entry with startNum: int.MaxValue - 200 to the integration test suite, exercising block numbers crossing the 2^31 boundary across all existing test methods

How compression detection works now

Each filter (address/topic) maps to multiple DB keys. Two are transient (mutable, uncompressed):

Key Postfix Content
Backward merge filter || 0x00000000 Descending uint block numbers
Forward merge filter || 0xFFFFFFFF Ascending uint block numbers

Any number of finalized keys store immutable, compressed data:

Key Postfix Content
Finalized filter || (first_block + 1) TurboPFor-compressed block numbers

IsCompressed checks whether the key ends with the BackwardMerge or ForwardMerge postfix. If it does, the value is uncompressed raw uint sequences. Otherwise it is a finalized compressed value whose first 4 bytes encode the decompressed element count.

Postfix collisions (finalized key matching a transient postfix) occur at block 0xFFFFFFFF and 0xFFFFFFFE — roughly 1600 years away at 12s/block.

Types of changes

What types of changes does your code introduce?

  • Bugfix (a non-breaking change that fixes an issue)
  • Refactoring

Testing

Requires testing

  • Yes

If yes, did you write tests?

  • Yes

Notes on testing

The new TestFixtureData(5, 100, startNum: (uint)int.MaxValue - 200) creates 500 blocks spanning 2147483447-2147483946, crossing int.MaxValue at block 2147483647. All existing integration test methods (set/get, backwards sync, reorg, compaction, concurrent access, multi-instance, failure recovery) run automatically against this fixture.

Documentation

Requires documentation update

  • Yes
  • No

Requires explanation in Release Notes

  • Yes
  • No

@LukaszRozmej LukaszRozmej changed the base branch from master to feature/log-index-2 February 6, 2026 15:01
LukaszRozmej and others added 3 commits February 6, 2026 17:00
Use key-based IsCompressed detection (transient vs finalized postfix)
instead of sign-bit trick that breaks for block numbers >= 2^31.
Switch CompressionAlgorithm delegates from int to uint signatures,
removing MemoryMarshal.Cast<uint, int> workarounds. Replace unsafe
BinarySearch cast with proper uint binary search implementation.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add LogIndexHighBlockNumberTests verifying that block numbers crossing
the int.MaxValue boundary are correctly stored, compressed, and
retrieved. Fix collision comment to use 100ms/block timing (~13.6 years).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@LukaszRozmej LukaszRozmej changed the title log index uint Migrate LogIndex block numbers from int to uint Feb 7, 2026
@alexb5dh alexb5dh mentioned this pull request Feb 7, 2026
6 tasks
@LukaszRozmej
Copy link
Copy Markdown
Member Author

Can't make it stable

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant