Skip to content

feat: add storagechange object pool for better performance#3112

Merged
MatusKysel merged 2 commits intodevelopfrom
journal-mem-allocs
May 22, 2025
Merged

feat: add storagechange object pool for better performance#3112
MatusKysel merged 2 commits intodevelopfrom
journal-mem-allocs

Conversation

@MatusKysel
Copy link
Contributor

Description

This PR introduces an object pool for storageChange entries in the state journal to improve memory allocation performance. The changes include:

  1. Added a sync.Pool for storageChange objects to reduce GC pressure
  2. Modified storageChange methods to work with pointers instead of values
  3. Optimized memory allocations in journal operations

Changes

  • Introduced storageChangePool using sync.Pool for efficient object reuse
  • Changed storageChange methods to use pointer receivers
  • Optimized journal.copy() to pre-allocate slices with exact capacity
  • Improved memory management in storageChange operations

Performance Impact

The changes reduce memory allocations by:

  • Reusing storageChange objects instead of creating new ones
  • Reducing GC pressure through object pooling

Benchmark

Old profile (note % not total GB as I was running it for different blocks)
Screenshot 2025-05-19 at 14 12 51
New Profile after the change
Screenshot 2025-05-19 at 13 39 48

@MatusKysel MatusKysel requested a review from Copilot May 20, 2025 11:37
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds an object pool for storageChange entries to cut down on allocations and GC pressure when journalling state changes.

  • Introduces sync.Pool for reusable storageChange instances
  • Converts storageChange methods to pointer receivers and updates journal operations to track and reset pooled entries
  • Optimizes journal.copy() and slice pre-allocation

@MatusKysel MatusKysel force-pushed the journal-mem-allocs branch from da77009 to 4d6b5cb Compare May 20, 2025 11:41
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR improves memory allocation performance in the state journal by reusing storageChange objects via a sync.Pool.

  • Introduces a storageChangePool and updates storageChange methods to pointer receivers
  • Tracks pooled storageChange objects in journal.storageEntries and returns them in journal.reset
  • Optimizes journal.copy to pre-allocate slices and clone existing data
Comments suppressed due to low confidence (1)

core/state/journal.go:63

  • Add unit tests for the pooling behavior (e.g., verify that reset returns objects to the pool and that subsequent Get calls reuse instances) to catch regressions in memory reuse.
var storageChangePool = sync.Pool{

@zzzckck zzzckck changed the base branch from master to develop May 21, 2025 05:44
@MatusKysel MatusKysel force-pushed the journal-mem-allocs branch from 0627051 to 955773c Compare May 21, 2025 12:07
@MatusKysel MatusKysel merged commit a81e6fd into develop May 22, 2025
7 checks passed
@zzzckck zzzckck deleted the journal-mem-allocs branch May 22, 2025 14:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants