Skip to content

More test fixes and some perf changes#1131

Merged
adamhathcock merged 9 commits intomasterfrom
adam/async-again
Jan 15, 2026
Merged

More test fixes and some perf changes#1131
adamhathcock merged 9 commits intomasterfrom
adam/async-again

Conversation

@adamhathcock
Copy link
Owner

This pull request introduces several improvements and refactorings to asynchronous ZIP archive handling and Deflate decompression. The main changes include refactoring buffer management for memory efficiency, updating async reading APIs for consistency, and improving resource cleanup. These updates enhance performance, reduce allocations, and make the codebase easier to maintain.

Async ZIP reading and buffer management improvements

  • Refactored SeekableZipHeaderFactory to use ArrayPool<byte> for buffer allocation in async header seeking, reducing memory allocations and improving performance. Also, updated the header search logic to use spans for efficient data access. [1] [2]
  • Updated async ZIP header reading methods to use explicit buffer arrays and offsets instead of returning new arrays, improving memory management and consistency across DirectoryEndHeader, DirectoryEntryHeader, LocalEntryHeader, and Zip64DirectoryEndHeader. [1] [2] [3] [4]
  • Added a SkipAsync method to AsyncBinaryReader and replaced unnecessary buffer reads with skips in ZIP header parsing, improving efficiency. [1] [2]

Deflate decompression memory optimization

  • Changed the Deflate decompression window from a raw byte[] to an IMemoryOwner<byte> from MemoryPool<byte>, ensuring pooled buffer usage and proper disposal for reduced GC pressure and memory leaks. All array copy operations were updated to use spans. [1] [2] [3] [4] [5] [6] [7] [8] [9] [10]

Resource cleanup and async disposal

  • Implemented DisposeAsync in RarArchive to support proper async resource cleanup, including disposal of unpacker resources.
  • Ensured the Deflate window buffer is disposed correctly in InflateBlocks.Free() to prevent memory leaks when decompressing data.

These changes collectively improve performance, resource management, and maintainability of the archive and compression codebase.

# Conflicts:
#	tests/SharpCompress.Test/GZip/GZipReaderAsyncTests.cs
#	tests/SharpCompress.Test/Rar/RarArchiveAsyncTests.cs
#	tests/SharpCompress.Test/SevenZip/SevenZipArchiveAsyncTests.cs
#	tests/SharpCompress.Test/Tar/TarArchiveAsyncTests.cs
#	tests/SharpCompress.Test/Tar/TarReaderAsyncTests.cs
#	tests/SharpCompress.Test/Zip/Zip64AsyncTests.cs
#	tests/SharpCompress.Test/Zip/ZipMemoryArchiveWithCrcAsyncTests.cs
Copilot AI review requested due to automatic review settings January 15, 2026 15:21
@adamhathcock adamhathcock mentioned this pull request Jan 15, 2026
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR refactors async ZIP archive handling and Deflate decompression to improve memory efficiency and resource management. The changes include replacing allocations with pooled buffers, updating async reading APIs for consistency, adding async disposal support to RarArchive, and fixing resource cleanup issues.

Changes:

  • Refactored buffer management to use ArrayPool<byte> and MemoryPool<byte> for reduced allocations
  • Updated async ZIP header reading methods to use explicit buffer arrays with offsets instead of returning new arrays
  • Added SkipAsync method to AsyncBinaryReader and replaced unnecessary buffer reads with skips
  • Changed Deflate decompression window from byte[] to IMemoryOwner<byte> for pooled buffer usage

Reviewed changes

Copilot reviewed 24 out of 24 changed files in this pull request and generated 13 comments.

Show a summary per file
File Description
tests/SharpCompress.Test/Zip/ZipMemoryArchiveWithCrcAsyncTests.cs Wraps zipStream with AsyncOnlyStream to test async-only code paths
tests/SharpCompress.Test/Zip/Zip64AsyncTests.cs Wraps file stream with AsyncOnlyStream for async testing
tests/SharpCompress.Test/Xz/XZStreamAsyncTests.cs Wraps XZ streams with AsyncOnlyStream for async testing
tests/SharpCompress.Test/Tar/TarReaderAsyncTests.cs Wraps file stream with AsyncOnlyStream for async testing
tests/SharpCompress.Test/Tar/TarArchiveAsyncTests.cs Wraps streams with AsyncOnlyStream for async testing
tests/SharpCompress.Test/SevenZip/SevenZipArchiveAsyncTests.cs Switches to async archive APIs with AsyncOnlyStream wrappers
tests/SharpCompress.Test/Rar/RarArchiveAsyncTests.cs Wraps file stream with AsyncOnlyStream for async testing
tests/SharpCompress.Test/GZip/GZipReaderAsyncTests.cs Wraps test stream with AsyncOnlyStream for async testing
tests/SharpCompress.Test/BZip2/BZip2StreamAsyncTests.cs Wraps streams with AsyncOnlyStream for async testing
tests/SharpCompress.Test/Arj/ArjReaderAsyncTests.cs New async test file for ARJ reader with AsyncOnlyStream wrappers
tests/SharpCompress.Test/Ace/AceReaderAsyncTests.cs New async test file for ACE reader with AsyncOnlyStream wrappers
src/SharpCompress/IO/SharpCompressStream.cs Removes unused using directives
src/SharpCompress/IO/BufferedSubStream.cs Changes cache allocation to use ArrayPool with proper disposal
src/SharpCompress/Compressors/Rar/UnpackV1/Unpack.cs Simplifies Init method by removing external window parameter
src/SharpCompress/Compressors/Rar/RarStream.cs Moves readStream disposal inside disposing check
src/SharpCompress/Compressors/Deflate/Inflate.cs Changes window to IMemoryOwner with proper disposal and span-based copies
src/SharpCompress/Common/Zip/ZipHeaderFactory.cs Replaces unnecessary buffer read with SkipAsync
src/SharpCompress/Common/Zip/SeekableZipHeaderFactory.cs Uses ArrayPool for buffer allocation in header seeking with proper disposal
src/SharpCompress/Common/Zip/Headers/Zip64DirectoryEndHeader.cs Pre-allocates byte array and uses ReadBytesAsync with explicit parameters
src/SharpCompress/Common/Zip/Headers/LocalEntryHeader.cs Pre-allocates byte arrays and uses ReadBytesAsync with explicit parameters
src/SharpCompress/Common/Zip/Headers/DirectoryEntryHeader.cs Pre-allocates byte arrays and uses ReadBytesAsync with explicit parameters
src/SharpCompress/Common/Zip/Headers/DirectoryEndHeader.cs Pre-allocates byte array and uses ReadBytesAsync with explicit parameters
src/SharpCompress/Common/AsyncBinaryReader.cs Changes ReadBytesAsync to write into provided buffer and adds SkipAsync method
src/SharpCompress/Archives/Rar/RarArchive.cs Implements DisposeAsync with proper unpacker disposal

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@adamhathcock adamhathcock merged commit 3e219fa into master Jan 15, 2026
11 checks passed
@adamhathcock adamhathcock deleted the adam/async-again branch January 15, 2026 16:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants