Skip to content

feat(cache): add a in-memory expiry map to package file cache - #42531

Merged
viceice merged 6 commits into
renovatebot:mainfrom
Thooms:paps/perf-cache-cleanup
Apr 14, 2026
Merged

feat(cache): add a in-memory expiry map to package file cache#42531
viceice merged 6 commits into
renovatebot:mainfrom
Thooms:paps/perf-cache-cleanup

Conversation

@Thooms

@Thooms Thooms commented Apr 10, 2026

Copy link
Copy Markdown
Contributor

Changes

This PR introduces a small in-memory map that stores expiry times for keys in the PackageCacheFile implementation.
We noticed in our deployment that it was starting to bottleneck on a non-small amount of repositories.

This allows for faster expiry lookup, which makes for the bulk of the loop on destroy(). Note that this doesn't remove the overhead of looping through all entries, but at least doesn't load the data from disk with a deserialization.

Result of the provided benchmark:

 ✓ lib/util/cache/package/impl/file.bench.ts > PackageCacheFile destroy() 8481ms
     name                                                           hz      min      max     mean      p75      p99     p995     p999     rme  samples
   · cold path — no expiryMap (disk read per entry)             2.8974   338.00   350.81   345.14   347.57   350.81   350.81   350.81  ±0.82%       10
   · warm path — expiryMap populated via set() (no disk read)  20.1797  48.4270  50.5818  49.5548  50.1291  50.5818  50.5818  50.5818  ±0.82%       11

 BENCH  Summary

  warm path — expiryMap populated via set() (no disk read) - lib/util/cache/package/impl/file.bench.ts > PackageCacheFile destroy()
    6.96x faster than cold path — no expiryMap (disk read per entry)

Context

Please select one of the following:

  • This closes an existing Issue, Closes: #
  • This doesn't close an Issue, but I accept the risk that this PR may be closed if maintainers disagree with its opening or implementation

AI assistance disclosure

Did you use AI tools to create any part of this pull request?

Please select one option and, if yes, briefly describe how AI was used (e.g., code, tests, docs) and which tool(s) you used.

  • No — I did not use AI for this contribution.
  • Yes — minimal assistance (e.g., IDE autocomplete, small code completions, grammar fixes).
  • Yes — substantive assistance (AI-generated non‑trivial portions of code, tests, or documentation).
  • Yes — other (please describe):

Documentation (please check one with an [x])

  • I have updated the documentation, or
  • No documentation update is required

How I've tested my work (please select one)

I have verified these changes via:

  • Code inspection only, or
  • Newly added/modified unit tests, or
  • No unit tests, but ran on a real repository, or
  • Both unit tests + ran on a real repository

Tested also on an internal repository with indeed a ~50% speedup on cache cleanup.

Loading
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants