more complete CRC checking#1357
Merged
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR expands SharpCompress checksum validation support by introducing an explicit ExtractionOptions.CheckCrc (default true), plumbing checksum validation into common extraction paths, and improving XZ/LZMA/LZip integrity behavior, alongside new tests and documentation clarifying format-specific expectations.
Changes:
- Add
ExtractionOptions.CheckCrc(defaulttrue) and integrate checksum validation into entry/reader extraction flows. - Implement/extend per-format checksum exposure/validation (ZIP/7z/ARJ/ARC/ACE, GZip trailer validation, LZip trailer validation, improved XZ block checks including CRC64/XZ).
- Add targeted regression tests and update user/API/format docs, plus new agent reference docs for XZ/LZMA.
Review Notes (issues to address)
- Breaking public API (critical):
src/SharpCompress/Compressors/Xz/XZBlock.cschangesXZBlock’s previously-public members (BlockHeaderSize,Filters,HeaderIsLoaded) to non-public/private. SinceXZBlockispublic, this is a source/binary breaking change for consumers who referenced these properties. Consider restoring the public surface (even as passthroughs) and, if desired, deprecating via[Obsolete]for a future major version. - Avoid per-byte allocations in checksum wrapper (performance/maintainability):
ChecksumValidationStream.ReadByte()updates viaUpdateChecksum([(byte)value]), which allocates for every byte read. Update the CRC32 seed directly via the existing single-byte CRC helper and update CRC16 similarly without allocating. - API completeness for
CheckCrc(api_design):IArchiveEntryExtensions.WriteTo(Stream, …)/WriteToAsync(Stream, …)do not expose anExtractionOptionsoverload, so callers usingWriteTocannot opt into (or out of) checksum validation even thoughCheckCrcis now a core extraction option. Consider adding overloads (non-breaking) that acceptExtractionOptions.
Reviewed changes
Copilot reviewed 55 out of 56 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| tests/SharpCompress.Test/Zip/ZipCrcExtractionTests.cs | New tests for ZIP CRC mismatch behavior and CheckCrc opt-out. |
| tests/SharpCompress.Test/Xz/XZStreamTests.cs | Add test asserting corrupt XZ block checks throw. |
| tests/SharpCompress.Test/Xz/XZBlockTests.cs | Fix assertion order; align indexed-stream check type/size. |
| tests/SharpCompress.Test/Xz/XZBlockAsyncTests.cs | Async equivalents of XZ block test updates. |
| tests/SharpCompress.Test/Xz/Crc64Tests.cs | Add CRC64/XZ known-vector test. |
| tests/SharpCompress.Test/Streams/LzmaStreamAsyncTests.cs | Add async test exercising corrupt LZMA2-in-XZ behavior. |
| tests/SharpCompress.Test/SevenZip/SevenZipCrcExtractionTests.cs | New tests for 7z CRC validation + opt-out. |
| tests/SharpCompress.Test/RemainingCrcExtractionTests.cs | New tests covering remaining formats (ARJ/ACE/ARC reader) and LZip trailer CRC. |
| tests/SharpCompress.Test/Rar/RarCrcExtractionTests.cs | New tests for RAR/RAR5 CRC mismatch on extraction. |
| tests/SharpCompress.Test/OptionsUsabilityTests.cs | Assert ExtractionOptions.CheckCrc default is true. |
| tests/SharpCompress.Test/GZip/GZipCrcExtractionTests.cs | New tests for GZip trailer CRC/size validation and non-seekable behavior. |
| tests/SharpCompress.Test/BZip2/BZip2StreamTests.cs | New test for BZip2 checksum failure behavior. |
| src/SharpCompress/Readers/IReaderExtensions.cs | Ensure reader extraction wraps entry stream with checksum validation when enabled. |
| src/SharpCompress/Readers/IAsyncReaderExtensions.cs | Async reader extraction now supports checksum validation via options. |
| src/SharpCompress/IO/CountingStream.cs | Extend counting to bytes read; add async read overrides. |
| src/SharpCompress/Crypto/Crc32Stream.cs | Widen CRC helper visibility for internal reuse. |
| src/SharpCompress/Compressors/Xz/XZBlock.cs | Implement XZ block check verification for CRC32/CRC64/XZ/SHA-256. |
| src/SharpCompress/Compressors/Xz/XZBlock.Async.cs | Async parity for padding/check reading and check verification. |
| src/SharpCompress/Compressors/Xz/Crc64.cs | Add CRC64/XZ polynomial/seed path (ComputeXz / UpdateXz). |
| src/SharpCompress/Compressors/Xz/Crc32.cs | Add incremental update helper for CRC32. |
| src/SharpCompress/Compressors/LZMA/LzmaStream.cs | Tighten LZMA2 corruption handling (EOPM/end-marker rejection). |
| src/SharpCompress/Compressors/LZMA/LzmaStream.Async.cs | Async parity for LZMA2 corruption handling updates. |
| src/SharpCompress/Compressors/LZMA/LzmaDecoder.cs | Expose internal end-marker detection signal. |
| src/SharpCompress/Compressors/LZMA/LZipStream.cs | Add LZip trailer CRC/size/member-size validation logic. |
| src/SharpCompress/Compressors/LZMA/LZipStream.Async.cs | Async read now validates trailer at EOF. |
| src/SharpCompress/Common/Zip/ZipHeaderFactory.cs | Track CRC availability when parsing ZIP headers. |
| src/SharpCompress/Common/Zip/ZipHeaderFactory.Async.cs | Async parity for CRC availability tracking. |
| src/SharpCompress/Common/Zip/ZipEntry.cs | Provide reliable Checksum descriptor for ZIP entries (incl. WinZip AES nuance). |
| src/SharpCompress/Common/Zip/StreamingZipHeaderFactory.cs | Mark CRC availability across streaming ZIP header paths. |
| src/SharpCompress/Common/Zip/StreamingZipHeaderFactory.Async.cs | Async parity for streaming ZIP CRC availability. |
| src/SharpCompress/Common/Zip/SeekableZipHeaderFactory.cs | Propagate CRC availability when using post-data descriptors. |
| src/SharpCompress/Common/Zip/Headers/ZipFileEntry.cs | Add IsCrcAvailable metadata field. |
| src/SharpCompress/Common/Zip/Headers/LocalEntryHeader.cs | Set IsCrcAvailable based on flags (post-data descriptor). |
| src/SharpCompress/Common/Zip/Headers/LocalEntryHeader.Async.cs | Async parity for IsCrcAvailable. |
| src/SharpCompress/Common/Zip/Headers/DirectoryEntryHeader.cs | Ensure directory CRC is marked available. |
| src/SharpCompress/Common/Zip/Headers/DirectoryEntryHeader.Async.cs | Async parity for directory CRC availability. |
| src/SharpCompress/Common/SevenZip/SevenZipEntry.cs | Expose Checksum descriptor when 7z CRC metadata is present. |
| src/SharpCompress/Common/IEntryExtensions.cs | Add helper to wrap streams with checksum validation based on options. |
| src/SharpCompress/Common/GZip/GZipFilePart.cs | Wrap raw stream and provide checksum-validation wrapper. |
| src/SharpCompress/Common/GZip/GZipEntry.cs | Hook gzip entry extraction into gzip trailer validation stream. |
| src/SharpCompress/Common/GZip/GZipChecksumValidationStream.cs | New stream wrapper to validate gzip CRC/ISIZE at EOF. |
| src/SharpCompress/Common/ExtractionOptions.cs | Add CheckCrc option (default true) with docs. |
| src/SharpCompress/Common/Entry.cs | Introduce Checksum descriptor + default checksum-validation wrapper. |
| src/SharpCompress/Common/ChecksumValidationStream.cs | New general-purpose entry checksum validation stream (CRC32/CRC16). |
| src/SharpCompress/Common/ChecksumDescriptor.cs | New internal checksum descriptor types used by entries. |
| src/SharpCompress/Common/Arj/ArjEntry.cs | Expose CRC32 checksum descriptor for ARJ entries where applicable. |
| src/SharpCompress/Common/Arc/ArcEntry.cs | Expose CRC16/ARC checksum descriptor for ARC entries. |
| src/SharpCompress/Common/Ace/AceEntry.cs | Expose CRC32 (no final XOR) checksum descriptor for eligible ACE entries. |
| src/SharpCompress/Archives/Rar/RarArchiveEntry.cs | Select correct RAR checksum stream wrapper (CRC vs BLAKE2sp). |
| src/SharpCompress/Archives/IArchiveEntryExtensions.cs | Plumb ExtractionOptions into entry extraction so CheckCrc is honored. |
| docs/USAGE.md | Document CheckCrc usage in extraction examples. |
| docs/FORMATS.md | Add XZ format notes emphasizing container integrity checks. |
| docs/API.md | Document CheckCrc semantics and defaults in API docs. |
| .agents/skills/xz-lzma-format/SKILL.md | New agent skill definition for XZ/LZMA format guidance. |
| .agents/skills/xz-lzma-format/references/xz-lzma-format.md | New detailed XZ/LZMA/LZMA2 reference notes for maintenance. |
This was referenced Jul 13, 2026
This was referenced Jul 13, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request introduces documentation and code changes to improve support for archive-level checksum validation, especially for the XZ and LZMA/LZMA2 formats, and to clarify the behavior and usage of the
CheckCrcextraction option across the library. It adds detailed skill and reference documents for XZ/LZMA, updates extraction APIs to support checksum validation, and enhances usage documentation for end users.Documentation and Reference Additions:
.agents/skills/xz-lzma-format/SKILL.mdand.agents/skills/xz-lzma-format/references/xz-lzma-format.mdto provide comprehensive guidance on the XZ container, LZMA/LZMA2 decoding, CRC64/XZ, block checks, and SharpCompress-specific implementation notes. These documents include workflow steps, upstream references, and guidance for handling corrupt files and test fixtures. [1] [2]docs/FORMATS.mdto include a new section on XZ format, emphasizing its container structure, integrity checks, and differences from raw LZMA/LZMA2 decoding.API and Usage Documentation Improvements:
docs/API.mdanddocs/USAGE.mdto document theCheckCrcoption inExtractionOptions. This clarifies its default behavior, when it is applicable, and how it affects checksum validation during extraction for supported formats. [1] [2] [3] [4] [5]Codebase Enhancements for Checksum Validation:
src/SharpCompress/Archives/IArchiveEntryExtensions.csto accept and utilize theExtractionOptionsparameter, ensuring that checksum validation is performed whenCheckCrcis enabled. This affects both synchronous and asynchronous extraction workflows, including file extraction. [1] [2] [3] [4] [5] [6] [7]Rar and Ace Format Checksum Handling:
src/SharpCompress/Archives/Rar/RarArchiveEntry.csto select the appropriate stream wrapper (RarCrcStreamorRarBLAKE2spStream) for RAR entries based on archive version and header properties, improving checksum validation logic.Checksumproperty insrc/SharpCompress/Common/Ace/AceEntry.csto expose CRC32 checksums for ACE entries when applicable, supporting the new validation workflow.