Add pooling for arrays for deflate#1347
Merged
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR reduces allocation/GC pressure in the Deflate stack by switching several frequently-allocated buffers to ArrayPool<T>.Shared and adding explicit return/cleanup paths, improving throughput and memory efficiency during compression/decompression.
Changes:
- Rent/return large Deflate/Inflate working arrays (
window,prev,head,pending,hufts) fromArrayPool<T>.Shared. - Pool a reusable working buffer in
ZlibBaseStreamand return it during dispose paths. - Pool the temporary copy buffer in
CRC32.GetCrc32AndCopy()and ensure it is returned viafinally.
Issues to address before approval:
ZlibCodec.EndDeflate()discards the return value fromdstate.End()and always returnsZ_OK, which can maskZ_STREAM_ERROR/Z_DATA_ERRORresults now produced byDeflateManager.End().- In
ZlibBaseStream.Dispose(...)/DisposeAsync(),ReturnWorkingBuffer()is called afterend(). Ifend()throws, the working buffer will not be returned to the pool (and subsequent dispose steps may be skipped), causing resource leakage.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| src/SharpCompress/Compressors/Deflate/ZlibCodec.cs | Calls dstate.End() during EndDeflate(), but currently ignores its return value. |
| src/SharpCompress/Compressors/Deflate/ZlibBaseStream.cs | Pools the internal working buffer and adds a helper to return it during dispose paths. |
| src/SharpCompress/Compressors/Deflate/Inflate.cs | Pools the hufts table used by InflateBlocks and returns it in Free(). |
| src/SharpCompress/Compressors/Deflate/DeflateManager.cs | Pools core Deflate buffers and centralizes returning them in End(). |
| src/SharpCompress/Compressors/Deflate/CRC32.cs | Pools the copy buffer in GetCrc32AndCopy() and returns it in a finally block. |
This was referenced Jul 13, 2026
Merged
This was referenced Jul 16, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request introduces significant memory management improvements across the Deflate compression components by adopting array pooling for buffer allocations. This change aims to reduce memory allocations and garbage collection overhead, enhancing performance and resource efficiency. Key updates include replacing direct buffer allocations with pooled arrays and ensuring proper buffer return and cleanup in disposal and reset paths.
Memory management and array pooling improvements:
window,prev,head,pendinginDeflateManager,huftsinInflateBlocks, and working buffers inZlibBaseStreamandCRC32) with pooled arrays fromArrayPool<T>.Shared. This change reduces memory pressure and improves performance. [1] [2] [3] [4] [5]Dispose,DisposeAsync,Reset, andEndmethods, and introduced helper methods likeReturnBuffersandReturnWorkingBufferto centralize cleanup logic. [1] [2] [3] [4] [5] [6]System.Bufferswhere array pooling is now used. [1] [2] [3]Code consistency and minor refactoring:
These changes collectively make the codebase more efficient and robust in terms of memory usage, especially during high-throughput or large-scale compression operations.