Release to Master#1274
Conversation
Agent-Logs-Url: https://github.com/adamhathcock/sharpcompress/sessions/84b02876-db79-41ed-875b-f1c7dbe5d8e2 Co-authored-by: adamhathcock <527620+adamhathcock@users.noreply.github.com>
…reams Agent-Logs-Url: https://github.com/adamhathcock/sharpcompress/sessions/b115976a-5d8a-412e-a462-72b4531f2d0a Co-authored-by: adamhathcock <527620+adamhathcock@users.noreply.github.com>
…reams Agent-Logs-Url: https://github.com/adamhathcock/sharpcompress/sessions/b115976a-5d8a-412e-a462-72b4531f2d0a Co-authored-by: adamhathcock <527620+adamhathcock@users.noreply.github.com>
…Async params Agent-Logs-Url: https://github.com/adamhathcock/sharpcompress/sessions/9ed10d73-ebf8-45d3-8d25-d5aa1f4989e7 Co-authored-by: adamhathcock <527620+adamhathcock@users.noreply.github.com>
…ngbuffer-bzip2 Dynamic ring buffer sizing for BZip2 and ZStandard on non-seekable streams
…bomb-and-crash-bugs Fix fuzzer-found decompression bomb and crash bugs
# Conflicts: # src/SharpCompress/packages.lock.json
There was a problem hiding this comment.
Pull request overview
This PR hardens archive decompression and tar format detection, focusing on safer handling of malformed/crafted inputs (to avoid OOB reads / OOM) and on ensuring non-seekable tar streams have adequate rewind buffering for formats like BZip2 and ZStandard.
Changes:
- Added bounds checks / clearer failures in Deflate64 and BZip2 decoding paths, plus improved ZIP streaming header EOF handling.
- Refactored Shrink decompression to avoid allocating based on untrusted “compressed size”, and added malformed-input regression tests (Reduce/Deflate64/BZip2/Shrink).
- Reworked tar detection buffering: per-wrapper minimum rewind sizes (BZip2/ZStandard) and a maximum used during tar probing; lowered the global default rewindable buffer size.
Reviewed changes
Copilot reviewed 20 out of 20 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/SharpCompress.Test/Tar/TarReaderTests.cs | Adds regression tests for non-seekable tar.bz2 reading and TarWrapper buffer sizing defaults. |
| tests/SharpCompress.Test/Streams/SharpCompressStreamSeekTest.cs | Adds tests covering StartRecording(minBufferSize) behavior and default buffer usage. |
| tests/SharpCompress.Test/MalformedInputTests.cs | Adds malformed-input tests covering Reduce “decompression bomb”, Deflate64 invalid Huffman data, BZip2 invalid symbol, and Shrink OOM attempt. |
| src/SharpCompress/Readers/Tar/TarReader.Factory.cs | Ensures non-seekable tar reader probing allocates a ring buffer sized to TarWrapper.MaximumRewindBufferSize. |
| src/SharpCompress/Providers/Default/ShrinkCompressionProvider.cs | Updates Shrink provider to use new ShrinkStream API (now only passes output size). |
| src/SharpCompress/packages.lock.json | Updates locked dependency resolutions for ILLink tasks. |
| src/SharpCompress/IO/SharpCompressStream.cs | Extends StartRecording to accept an optional minimum ring buffer size and documents the behavior. |
| src/SharpCompress/IO/SeekableSharpCompressStream.cs | Updates StartRecording override signature to match base (parameter ignored for seekable streams). |
| src/SharpCompress/Factories/TarWrapper.cs | Adds MinimumRewindBufferSize per wrapper, sets BZip2/ZStandard requirements, and exposes MaximumRewindBufferSize. |
| src/SharpCompress/Factories/TarFactory.cs | Uses TarWrapper.MaximumRewindBufferSize when starting recording for tar probing. |
| src/SharpCompress/Compressors/ZStandard/ZstandardConstants.cs | Adds constants for max block size and recommended streaming input buffer size. |
| src/SharpCompress/Compressors/Shrink/ShrinkStream.cs | Refactors ShrinkStream to avoid prealloc based on declared compressed size; reads actual compressed bytes instead; validates output size <= int.MaxValue. |
| src/SharpCompress/Compressors/Shrink/ShrinkStream.Async.cs | Mirrors ShrinkStream changes in async path (read actual compressed bytes, then decompress). |
| src/SharpCompress/Compressors/Reduce/ReduceStream.cs | Detects early exhaustion of compressed input and throws InvalidFormatException instead of producing unbounded output. |
| src/SharpCompress/Compressors/Reduce/ReduceStream.Async.cs | Async parity for ReduceStream early-exhaustion detection. |
| src/SharpCompress/Compressors/Deflate64/HuffmanTree.cs | Adds bounds check to prevent out-of-range table writes on malformed Huffman data. |
| src/SharpCompress/Compressors/BZip2/CBZip2InputStream.cs | Adds bounds checks for selector/symbol indices to prevent OOB on malformed BZip2 streams. |
| src/SharpCompress/Compressors/BZip2/CBZip2InputStream.Async.cs | Async parity for the new BZip2 bounds checks. |
| src/SharpCompress/Common/Zip/StreamingZipHeaderFactory.cs | Converts EndOfStreamException into InvalidFormatException with a clearer message during header reads. |
| src/SharpCompress/Common/Constants.cs | Reduces default RewindableBufferSize to 80KB and updates docs to point to per-format tar wrapper sizing. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| /// Shrink requires compressed and uncompressed sizes which must be provided via CompressionContext. | ||
| /// </remarks> | ||
| public sealed class ShrinkCompressionProvider : ContextRequiredDecompressionProviderBase | ||
| { | ||
| public override CompressionType CompressionType => CompressionType.Shrink; | ||
| protected override string CompressionNotSupportedMessage => | ||
| "Shrink compression is not supported by SharpCompress's internal implementation."; | ||
|
|
||
| protected override string DecompressionContextRequirementDescription => | ||
| "Shrink decompression requires compressed and uncompressed sizes"; | ||
|
|
||
| public override Stream CreateDecompressStream(Stream source, CompressionContext context) | ||
| { | ||
| ValidateRequiredSizes(context, "Shrink"); | ||
|
|
||
| return new ShrinkStream( | ||
| source, | ||
| CompressionMode.Decompress, | ||
| context.InputSize, | ||
| context.OutputSize | ||
| ); | ||
| return new ShrinkStream(source, context.OutputSize); |
There was a problem hiding this comment.
ValidateRequiredSizes enforces that CompressionContext.InputSize is provided, but the updated Shrink implementation no longer uses context.InputSize (only OutputSize) and reads the entire source stream. Either use InputSize to bound the compressed read (and keep the validation/messages accurate), or relax the validation + update the remarks/DecompressionContextRequirementDescription to reflect the real requirements.
There was a problem hiding this comment.
Fixed in 765f281. Added ValidateOutputSize to CompressionProviderBase that only checks OutputSize >= 0. ShrinkCompressionProvider now calls ValidateOutputSize instead of ValidateRequiredSizes, and the remarks and DecompressionContextRequirementDescription have been updated to reflect that only OutputSize (uncompressed size) is required — InputSize is not needed since ShrinkStream reads directly from the bounded source stream.
| // Allocate ring buffer with the requested minimum size (at least the global default). | ||
| if (_ringBuffer is null) | ||
| { | ||
| _ringBuffer = new RingBuffer(Constants.RewindableBufferSize); | ||
| var size = | ||
| minBufferSize.GetValueOrDefault() > Constants.RewindableBufferSize |
There was a problem hiding this comment.
minBufferSize is ignored when _ringBuffer was preallocated in the constructor (e.g., via SharpCompressStream.Create(stream, bufferSize: ...)). This makes the new StartRecording(minBufferSize) parameter ineffective in those cases; consider ensuring the existing ring buffer capacity is at least minBufferSize (or documenting that minBufferSize only applies when no ring buffer exists yet).
There was a problem hiding this comment.
Fixed in 765f281. StartRecording now calls EnsureMinimumRingBufferCapacity(minBufferSize.Value) when the ring buffer was pre-allocated but smaller than the requested minimum, making the minBufferSize parameter effective regardless of how the stream was constructed.
| stream, | ||
| bufferSize: readerOptions.RewindableBufferSize | ||
| bufferSize: Math.Max( | ||
| readerOptions.RewindableBufferSize ?? 0, | ||
| TarWrapper.MaximumRewindBufferSize | ||
| ) |
There was a problem hiding this comment.
For non-seekable inputs this forces the ring buffer to TarWrapper.MaximumRewindBufferSize (~900KB due to BZip2) even when the archive is e.g. tar/tar.gz. This can be a noticeable per-reader memory increase in high-concurrency scenarios; consider a strategy that allocates only the matched wrapper’s required size (or a growable ring buffer) after cheap magic detection, while still preserving the ability to rewind the initial probe bytes.
There was a problem hiding this comment.
Applied in eb2126e. The fix uses a two-phase approach:
TarReader.OpenReader(both sync and async) now creates the initialSharpCompressStreamwith only the default 80 KB ring buffer instead ofMaximumRewindBufferSize(~900 KB).- After the cheap magic-byte
IsMatchcheck identifies the compression format,EnsureMinimumRingBufferCapacityis called with the matched wrapper'sMinimumRewindBufferSize. This copies the small number of already-buffered bytes (at most a few hundred bytes from theNone-wrapper probe) into a new, larger ring buffer only when actually needed (i.e., only for BZip2 and ZStandard formats).
A plain tar or tar.gz stream now allocates ~80 KB; only tar.bz2 allocates ~900 KB. The EnsureMinimumRingBufferCapacity method is a no-op for seekable streams (SeekableSharpCompressStream) since they use native seeking with no ring buffer.
| var sharpCompressStream = SharpCompressStream.Create( | ||
| stream, | ||
| bufferSize: readerOptions.RewindableBufferSize | ||
| bufferSize: Math.Max( | ||
| readerOptions.RewindableBufferSize ?? 0, | ||
| TarWrapper.MaximumRewindBufferSize | ||
| ) |
There was a problem hiding this comment.
Same concern as the async path: on non-seekable streams this makes every TarReader allocate at least TarWrapper.MaximumRewindBufferSize (~900KB) regardless of actual compression type. If many readers are opened concurrently, this may increase memory usage significantly; consider allocating based on the matched wrapper’s minimum (or making the buffer growable) after initial detection.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 21 out of 21 changed files in this pull request and generated 8 comments.
Comments suppressed due to low confidence (1)
src/SharpCompress/Compressors/Shrink/ShrinkStream.cs:45
CanSeekreturns true, butPosition's setter throwsNotImplementedException(andSeek(...)is also unimplemented). This violates theStreamcontract and can break callers that branch onCanSeek. Since this stream is fully in-memory after decompression, consider implementingSeek/Positionproperly (or setCanSeekto false).
public override bool CanRead => true;
public override bool CanSeek => true;
public override bool CanWrite => false;
public override long Length => _uncompressedSize;
public override long Position
{
get => _position;
set => throw new NotImplementedException();
}
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| if (_ringBuffer is null) | ||
| { | ||
| _ringBuffer = new RingBuffer(Constants.RewindableBufferSize); | ||
| var size = | ||
| minBufferSize.GetValueOrDefault() > Constants.RewindableBufferSize | ||
| ? minBufferSize.GetValueOrDefault() | ||
| : Constants.RewindableBufferSize; | ||
| _ringBuffer = new RingBuffer(size); | ||
| } | ||
| else if (minBufferSize.HasValue && minBufferSize.Value > _ringBuffer.Capacity) |
There was a problem hiding this comment.
StartRecording(minBufferSize) only uses minBufferSize when _ringBuffer is null. If the stream was created via SharpCompressStream.Create(..., bufferSize: ...) (which pre-allocates _ringBuffer), a larger minBufferSize is silently ignored and a later rewind can still overflow. Consider ensuring the existing ring buffer capacity is at least the requested minimum (e.g., call EnsureMinimumRingBufferCapacity(size) here) or throwing when minBufferSize exceeds the current capacity.
| ) | ||
| ); | ||
| long pos = sharpCompressStream.Position; | ||
| foreach (var wrapper in TarWrapper.Wrappers) |
There was a problem hiding this comment.
The initial bufferSize is computed with Math.Max(readerOptions.RewindableBufferSize ?? 0, Constants.RewindableBufferSize), which makes it impossible to set a per-stream rewindable buffer smaller than the global default even though Constants.RewindableBufferSize is documented as overridable per stream. Consider using readerOptions.RewindableBufferSize ?? Constants.RewindableBufferSize for the initial buffer (and keep the later growth to wrapper.MinimumRewindBufferSize for formats that need it).
| ) | |
| ); | |
| long pos = sharpCompressStream.Position; | |
| foreach (var wrapper in TarWrapper.Wrappers) | |
| bufferSize: readerOptions.RewindableBufferSize ?? Common.Constants.RewindableBufferSize |
| /// <c>TarWrapper.MaximumRewindBufferSize</c> is used at stream construction | ||
| /// to ensure the correct capacity is allocated upfront. |
There was a problem hiding this comment.
These updated docs state that TarWrapper.MaximumRewindBufferSize is used “at stream construction” to ensure correct capacity is allocated upfront, but TarReader now grows the buffer lazily via EnsureMinimumRingBufferCapacity after wrapper matching (and TarFactory allocates during StartRecording, not construction). Consider adjusting the wording to reflect the actual behavior so consumers aren’t misled about when/where the max size is applied.
| /// <c>TarWrapper.MaximumRewindBufferSize</c> is used at stream construction | |
| /// to ensure the correct capacity is allocated upfront. | |
| /// <c>TarWrapper.MaximumRewindBufferSize</c> may be used to grow the rewind | |
| /// buffer as needed during format detection or recording rather than | |
| /// allocating the maximum capacity upfront at stream construction. |
| var providers = CompressionProviderRegistry.Default; | ||
| var sharpCompressStream = new SharpCompressStream(stream); | ||
| sharpCompressStream.StartRecording(); | ||
| sharpCompressStream.StartRecording(TarWrapper.MaximumRewindBufferSize); | ||
| foreach (var wrapper in TarWrapper.Wrappers) |
There was a problem hiding this comment.
Calling StartRecording(TarWrapper.MaximumRewindBufferSize) here forces allocating the maximum (~900KB due to BZip2) for every stream, even when the wrapper match fails early (e.g., plain tar/gzip) and even for seekable streams (since this path uses new SharpCompressStream(stream) rather than SharpCompressStream.Create). Consider the same two-phase strategy as TarReader.Factory: start with a small buffer, only grow when wrapper.IsMatch(...) succeeds (and/or use SharpCompressStream.Create to avoid ring buffering for seekable streams).
| var providers = CompressionProviderRegistry.Default; | ||
| var sharpCompressStream = new SharpCompressStream(stream); | ||
| sharpCompressStream.StartRecording(); | ||
| sharpCompressStream.StartRecording(TarWrapper.MaximumRewindBufferSize); | ||
| foreach (var wrapper in TarWrapper.Wrappers) |
There was a problem hiding this comment.
Calling StartRecording(TarWrapper.MaximumRewindBufferSize) here forces allocating the maximum (~900KB due to BZip2) for every stream, even when the wrapper match fails early and even for seekable streams (since this path uses new SharpCompressStream(stream) rather than SharpCompressStream.Create). Consider starting with a small buffer and only growing when IsMatchAsync succeeds (or switching to SharpCompressStream.Create to avoid buffering for seekable streams).
| options ??= new ReaderOptions(); | ||
| var sharpCompressStream = new SharpCompressStream(stream); | ||
| sharpCompressStream.StartRecording(); | ||
| sharpCompressStream.StartRecording(TarWrapper.MaximumRewindBufferSize); | ||
| foreach (var wrapper in TarWrapper.Wrappers) |
There was a problem hiding this comment.
Calling StartRecording(TarWrapper.MaximumRewindBufferSize) here forces allocating the maximum ring buffer for every stream opened via TarFactory.OpenReader, even when the archive isn’t BZip2/ZStd. Consider a lazy growth approach (small initial buffer, grow only after wrapper.IsMatch(...) succeeds) to preserve the memory savings introduced in TarReader.Factory.
| cancellationToken.ThrowIfCancellationRequested(); | ||
| options ??= new ReaderOptions(); | ||
| var sharpCompressStream = new SharpCompressStream(stream); | ||
| sharpCompressStream.StartRecording(); | ||
| sharpCompressStream.StartRecording(TarWrapper.MaximumRewindBufferSize); | ||
| foreach (var wrapper in TarWrapper.Wrappers) |
There was a problem hiding this comment.
Calling StartRecording(TarWrapper.MaximumRewindBufferSize) here forces allocating the maximum ring buffer for every stream opened via TarFactory.OpenAsyncReader, even when the archive isn’t BZip2/ZStd. Consider a lazy growth approach (small initial buffer, grow only after IsMatchAsync succeeds) to avoid the per-reader ~900KB allocation in the common case.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 21 out of 21 changed files in this pull request and generated 3 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
01c2067 to
e7cacdd
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 20 out of 20 changed files in this pull request and generated 5 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Updated [SharpCompress](https://github.com/adamhathcock/sharpcompress) from 0.48.1 to 0.49.1. <details> <summary>Release notes</summary> _Sourced from [SharpCompress's releases](https://github.com/adamhathcock/sharpcompress/releases)._ ## 0.49.1 ## What's Changed * Close writable entry streams during async archive disposal by @Copilot in adamhathcock/sharpcompress#1338 * Restore `WriteToDirectoryAsync` progress callbacks for solid 7z archives by @Copilot in adamhathcock/sharpcompress#1340 * Try to fix global.json to avoid churn in locks by @adamhathcock in adamhathcock/sharpcompress#1341 * Fix tar archive enumeration after fully reading entry streams by @adamhathcock in adamhathcock/sharpcompress#1342 **Full Changelog**: adamhathcock/sharpcompress@0.49.0...0.49.1 ## 0.49.0 This should contain a lot of write async fixes and some breaking API changes that fix previous broke `net48` usage ## What's Changed * Rename IWriteableArchiveFactory.cs to IWritableArchiveFactory.cs by @Copilot in adamhathcock/sharpcompress#1244 * Some API clean up from GPT 5.4 by @adamhathcock in adamhathcock/sharpcompress#1243 * Release to master by @adamhathcock in adamhathcock/sharpcompress#1267 * Fix three BLAKE2sp correctness bugs and eliminate allocations in hot path by @coderb in adamhathcock/sharpcompress#1266 * Corrected async examples. by @dlemstra in adamhathcock/sharpcompress#1277 * Fix setting invalid access time fails extraction by @aromaa in adamhathcock/sharpcompress#1279 * Fix incorrect code examples in docs for sync/async usage by @Copilot in adamhathcock/sharpcompress#1280 * Replace APPNOTE.TXT contents with reference link note by @puk06 in adamhathcock/sharpcompress#1286 * Release to Master by @adamhathcock in adamhathcock/sharpcompress#1274 * update docs for tar gap analysis and XZ usage by @adamhathcock in adamhathcock/sharpcompress#1288 * Add a PooledMemoryStream to avoid allocating by @adamhathcock in adamhathcock/sharpcompress#1275 * fix: Change LeaveStreamOpen default from true to false by @puk06 in adamhathcock/sharpcompress#1293 * Fix usage of ReaderOptions and pre-defined values by @adamhathcock in adamhathcock/sharpcompress#1295 * Enforce seekable, readable and writable on streams by @adamhathcock in adamhathcock/sharpcompress#1297 * Add ArchiveInformation record for consolidated archive detection and capability inspection by @Copilot in adamhathcock/sharpcompress#1299 * merge release to master by @adamhathcock in adamhathcock/sharpcompress#1314 * Some clean up and test clean up by @adamhathcock in adamhathcock/sharpcompress#1321 * Finish Write Async by @adamhathcock in adamhathcock/sharpcompress#1323 * More complete Tar implementation: USTAR, PAX, etc. by @adamhathcock in adamhathcock/sharpcompress#1289 * Add Polysharp and adjustments that do not break legacy frameworks by @adamhathcock in adamhathcock/sharpcompress#1330 * Fix null `IVolume.FileName` for single-volume file-based archives by @Copilot in adamhathcock/sharpcompress#1333 * Add skills by @adamhathcock in adamhathcock/sharpcompress#1332 * add AOT smoke and missing tests by @adamhathcock in adamhathcock/sharpcompress#1334 ## New Contributors * @dlemstra made their first contribution in adamhathcock/sharpcompress#1277 * @aromaa made their first contribution in adamhathcock/sharpcompress#1279 * @puk06 made their first contribution in adamhathcock/sharpcompress#1286 **Full Changelog**: adamhathcock/sharpcompress@0.48.1...0.49.0 ## 0.49.0-beta.140 ## What's Changed * Add Polysharp and adjustments that do not break legacy frameworks by @adamhathcock in adamhathcock/sharpcompress#1330 **Full Changelog**: adamhathcock/sharpcompress@0.49.0-beta.136...0.49.0-beta.140 ## 0.49.0-beta.136 ## What's Changed * Rename IWriteableArchiveFactory.cs to IWritableArchiveFactory.cs by @Copilot in adamhathcock/sharpcompress#1244 * Some API clean up from GPT 5.4 by @adamhathcock in adamhathcock/sharpcompress#1243 * Release to master by @adamhathcock in adamhathcock/sharpcompress#1267 * Fix three BLAKE2sp correctness bugs and eliminate allocations in hot path by @coderb in adamhathcock/sharpcompress#1266 * Corrected async examples. by @dlemstra in adamhathcock/sharpcompress#1277 * Fix setting invalid access time fails extraction by @aromaa in adamhathcock/sharpcompress#1279 * Fix incorrect code examples in docs for sync/async usage by @Copilot in adamhathcock/sharpcompress#1280 * Replace APPNOTE.TXT contents with reference link note by @puk06 in adamhathcock/sharpcompress#1286 * Release to Master by @adamhathcock in adamhathcock/sharpcompress#1274 * update docs for tar gap analysis and XZ usage by @adamhathcock in adamhathcock/sharpcompress#1288 * Add a PooledMemoryStream to avoid allocating by @adamhathcock in adamhathcock/sharpcompress#1275 * fix: Change LeaveStreamOpen default from true to false by @puk06 in adamhathcock/sharpcompress#1293 * Fix usage of ReaderOptions and pre-defined values by @adamhathcock in adamhathcock/sharpcompress#1295 * Enforce seekable, readable and writable on streams by @adamhathcock in adamhathcock/sharpcompress#1297 * Add ArchiveInformation record for consolidated archive detection and capability inspection by @Copilot in adamhathcock/sharpcompress#1299 * merge release to master by @adamhathcock in adamhathcock/sharpcompress#1314 * Some clean up and test clean up by @adamhathcock in adamhathcock/sharpcompress#1321 * Finish Write Async by @adamhathcock in adamhathcock/sharpcompress#1323 * More complete Tar implementation: USTAR, PAX, etc. by @adamhathcock in adamhathcock/sharpcompress#1289 ## New Contributors * @dlemstra made their first contribution in adamhathcock/sharpcompress#1277 * @aromaa made their first contribution in adamhathcock/sharpcompress#1279 * @puk06 made their first contribution in adamhathcock/sharpcompress#1286 **Full Changelog**: adamhathcock/sharpcompress@0.48.1...0.49.0-beta1 Commits viewable in [compare view](adamhathcock/sharpcompress@0.48.1...0.49.1). </details> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore <dependency name> major version` will close this group update PR and stop Dependabot creating any more for the specific dependency's major version (unless you unignore this specific dependency's major version or upgrade to it yourself) - `@dependabot ignore <dependency name> minor version` will close this group update PR and stop Dependabot creating any more for the specific dependency's minor version (unless you unignore this specific dependency's minor version or upgrade to it yourself) - `@dependabot ignore <dependency name>` will close this group update PR and stop Dependabot creating any more for the specific dependency (unless you unignore this specific dependency or upgrade to it yourself) - `@dependabot unignore <dependency name>` will remove all of the ignore conditions of the specified dependency - `@dependabot unignore <dependency name> <ignore condition>` will remove the ignore condition of the specified dependency and ignore conditions </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
This pull request introduces several important improvements to the robustness and security of archive decompression, especially for BZip2, Deflate64, Reduce, Shrink, and ZStandard formats. The changes focus on improved error handling, preventing out-of-memory issues, and ensuring buffer sizes are appropriate for the formats being processed.
Security and Robustness Improvements
EndOfStreamExceptionand throwing anInvalidFormatExceptionwith a descriptive message.InvalidFormatExceptionin such cases. [1] [2] [3] [4] [5]Shrink Decompression Hardening
ShrinkStreamto avoid pre-allocating buffers based on declared compressed size, instead reading actual compressed data from the stream. This prevents possibleOutOfMemoryExceptionattacks with crafted archives and ensures the stream is safely bounded. [1] [2] [3] [4]int.MaxValue, throwing an exception if it does.Buffer Size and Format Detection Adjustments
RewindableBufferSizefrom 160KB to 80KB, clarifying that formats needing larger buffers (e.g., BZip2, ZStandard) specify their own requirements. Updated documentation accordingly. [1] [2]TarReaderring buffer allocation strategy: instead of allocatingTarWrapper.MaximumRewindBufferSize(~900 KB) for every non-seekable stream regardless of format, the reader now starts with a small default buffer (80 KB) for cheap magic-byte detection, then grows the buffer lazily to the matched wrapper'sMinimumRewindBufferSizeonly when needed (e.g., ~900 KB for BZip2, smaller for GZip/plain tar). This reduces per-reader memory usage significantly in high-concurrency scenarios when most archives are not BZip2-compressed. [1] [2] [3] [4]SharpCompressStream.EnsureMinimumRingBufferCapacity(int)to support lazy ring buffer growth, preserving already-buffered bytes when the buffer is enlarged.