Release to master

[adamhathcock]

This pull request introduces a number of improvements focused on increasing the robustness and correctness of archive decompression, particularly for BZip2 and ArcLzw formats, as well as refactoring the Winzip AES crypto stream logic for better correctness and maintainability. It also increases the default rewindable buffer size to better handle large ZStandard blocks and silences a specific IDE warning. The most important changes are summarized below:

Robustness and Error Handling Improvements

• Added extensive input validation and error handling to BZip2 decompression, including checks for invalid Huffman trees, selector values, Huffman code lengths, block data bounds, and stream initialization, ensuring that malformed or corrupted archives are handled gracefully. [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15]
• Improved ArcLzw decompression by adding checks for empty input and out-of-range codes, raising clear exceptions for invalid data. [1] [2]

Winzip AES Crypto Stream Refactoring

• Refactored the Winzip AES decryption logic to use a rolling buffer offset (_counterOutOffset) instead of a final block flag, improving correctness and simplifying the block transformation logic for both sync and async code paths. [1] [2] [3]

Buffering and Performance

• Increased the default RewindableBufferSize from 81KB to 160KB to support ZStandard's worst-case block sizes, reducing the risk of failed format detection on large blocks. [1] [2]

Code Quality and Maintenance

• Suppressed IDE0051 warnings (unused private members) project-wide to reduce noise in build output.
• Minor cleanup in ZIP file part logic, simplifying stream handling conditions. [1] [2]

These changes collectively make the decompression codebase more robust, maintainable, and better suited for handling edge cases and corrupted input.

[kilo-code-bot]

Code Review Summary

Status: No Issues Found | Recommendation: Merge

Files Reviewed (19 files)

• Directory.Build.props - Added NoWarn for IDE0051
• src/SharpCompress/Common/Constants.cs - Increased RewindableBufferSize to 160KB
• src/SharpCompress/Common/Zip/WinzipAesCryptoStream.cs - Refactored AES crypto handling
• src/SharpCompress/Common/Zip/WinzipAesCryptoStream.Async.cs - Async refactoring
• src/SharpCompress/Common/Zip/ZipFilePart.cs - Simplified stream handling
• src/SharpCompress/Common/Zip/ZipFilePart.Async.cs - Simplified stream handling
• src/SharpCompress/Compressors/ArcLzw/ArcLzwStream.cs - Added input validation
• src/SharpCompress/Compressors/BZip2/Randoms.cs - Added Huffman validation
• src/SharpCompress/Compressors/Deflate64/HuffmanTree.cs - Added bounds checking
• src/SharpCompress/Compressors/Explode/ExplodeStream.cs - Added Huffman validation
• src/SharpCompress/Compressors/Explode/ExplodeStream.Async.cs - Added Huffman validation
• src/SharpCompress/Compressors/LZMA/LZ/LzOutWindow.cs - Added dictionary size validation
• src/SharpCompress/Compressors/Lzw/LzwStream.cs - Added header validation
• src/SharpCompress/Compressors/Lzw/LzwStream.Async.cs - Added header validation
• src/SharpCompress/Compressors/PPMd/I1/Model.cs - Added context validation
• src/SharpCompress/Compressors/Reduce/ReduceStream.cs - Added bounds checking
• src/SharpCompress/Compressors/Reduce/ReduceStream.Async.cs - Added bounds checking
• src/SharpCompress/Compressors/Squeezed/SqueezedStream.cs - Added node index validation
• src/SharpCompress/Compressors/Squeezed/SqueezedStream.Async.cs - Added node index validation
• tests/SharpCompress.Test/MalformedInputTests.cs - New tests
• tests/SharpCompress.Test/Streams/WinzipAesCryptoStreamTests.cs - New tests
• tests/SharpCompress.Test/Zip/ZipFilePartTests.cs - New tests

Overview

The PR adds comprehensive error handling improvements across multiple decompression formats (BZip2, ArcLZW, LZW, Deflate64, PPMd, LZMA, Reduce, Squeezed, Explode). It also refactors the Winzip AES crypto stream logic for better correctness and increases the default rewindable buffer size to handle large ZStandard blocks.

Key Improvements

1. Robustness: Added extensive input validation with proper exceptions (InvalidFormatException, ArchiveException)
2. Winzip AES: Refactored to use rolling buffer offset instead of final block flag
3. Buffer Size: Increased from 81KB to 160KB for ZStandard compatibility
4. Tests: Added comprehensive malformed input tests and Winzip AES crypto stream tests

Notes

• All new code follows existing patterns and conventions
• Error messages are descriptive and follow the existing style
• Tests verify the new error handling works correctly
• The ZipFilePart change removes an unnecessary Zip64 condition that was superseded by the CompressedSize check

[Copilot]

Pull request overview

This PR hardens SharpCompress decompression against malformed/corrupt inputs (notably BZip2 and ARC/LZW), refactors WinZip AES CTR handling for correctness across partial reads, and adjusts buffering defaults to improve format detection for large Zstandard blocks.

Changes:

• Added defensive bounds/validity checks across multiple decompressor implementations to ensure malformed inputs raise library exceptions instead of raw runtime exceptions.
• Refactored WinZip AES crypto stream to track a rolling keystream offset for correct behavior across non-aligned sync/async reads.
• Increased default rewindable buffer size and added/expanded regression tests (malformed input, Zip64 stream bounding, Zstd+AES archives).

Reviewed changes

Copilot reviewed 25 out of 26 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
tests/SharpCompress.Test/Zip/ZipFilePartTests.cs	Adds coverage to ensure ZIP entry streams are bounded when compressed size is known (incl. Zip64 markers).
tests/SharpCompress.Test/Zip/ZipArchiveTests.cs	Adds tests for Zstd+WinZip AES archives and a large Zip64 regression test (currently problematic).
tests/SharpCompress.Test/Streams/WinzipAesCryptoStreamTests.cs	Adds keystream/CTR regression tests for aligned and non-aligned sync/async reads.
tests/SharpCompress.Test/MalformedInputTests.cs	Adds “malformed input should throw SharpCompressException” regression tests across multiple formats.
src/SharpCompress/Readers/ReaderOptions.cs	Updates docs to reflect new default rewindable buffer sizing rationale for Zstandard.
src/SharpCompress/Compressors/Squeezed/SqueezedStream.cs	Adds Huffman tree index validation to avoid out-of-range access on corrupt inputs.
src/SharpCompress/Compressors/Squeezed/SqueezedStream.Async.cs	Async parity for Huffman tree index validation.
src/SharpCompress/Compressors/Reduce/ReduceStream.cs	Adds bounds check for reduce “next byte table” lookup indices.
src/SharpCompress/Compressors/Reduce/ReduceStream.Async.cs	Async parity for reduce table index bounds check.
src/SharpCompress/Compressors/PPMd/I1/Model.cs	Adds initialization validation to prevent use of uninitialized model context.
src/SharpCompress/Compressors/Lzw/LzwStream.cs	Adds header validation (min bits) and ensures header parse failures mark EOF state.
src/SharpCompress/Compressors/Lzw/LzwStream.Async.cs	Async parity for LZW header validation and failure handling.
src/SharpCompress/Compressors/LZMA/LZ/LzOutWindow.cs	Validates dictionary/window size to prevent invalid allocations/usage.
src/SharpCompress/Compressors/Explode/ExplodeStream.cs	Validates Huffman table setup result and throws a format exception on failure.
src/SharpCompress/Compressors/Explode/ExplodeStream.Async.cs	Async parity for explode table setup validation.
src/SharpCompress/Compressors/Deflate64/HuffmanTree.cs	Adds bounds checking during Huffman table construction to prevent invalid indexing.
src/SharpCompress/Compressors/BZip2/CBZip2InputStream.cs	Adds extensive BZip2 validation (stream init, selectors, Huffman decode bounds, block bounds).
src/SharpCompress/Compressors/BZip2/CBZip2InputStream.Async.cs	Async parity for BZip2 validation and stream init handling.
src/SharpCompress/Compressors/ArcLzw/ArcLzwStream.cs	Adds empty-input and code-range validation for ARC LZW decompression.
src/SharpCompress/Common/Zip/ZipFilePart.cs	Adjusts crypto stream bounding rules to avoid unbounded reads when compressed size is known.
src/SharpCompress/Common/Zip/ZipFilePart.Async.cs	Async parity for ZIP crypto stream bounding logic.
src/SharpCompress/Common/Zip/WinzipAesCryptoStream.cs	Refactors CTR keystream handling to use _counterOutOffset across partial reads.
src/SharpCompress/Common/Zip/WinzipAesCryptoStream.Async.cs	Async parity for CTR keystream offset handling.
src/SharpCompress/Common/Constants.cs	Increases default rewindable buffer size and updates documentation accordingly.
Directory.Build.props	Suppresses IDE0051 warnings repository-wide.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

[Fact(Explicit = true)] is not a supported parameter on xUnit's FactAttribute (xUnit typically uses Skip = "..." to disable tests). If this isn't recognized by the runner, this test will run by default and will attempt to generate/write a ~4GB ZIP on disk, which is not suitable for CI. Consider converting this to [Fact(Skip = "...")] (or an existing conditional/trait-based skip mechanism used in this repo) and keep the large-file generation behind an explicit opt-in.

Suggested change

	[Fact(Explicit = true)]
	[Fact(Skip = "Generates a very large Zip64 archive; run manually when explicitly needed.")]