Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 12 additions & 12 deletions .agents/skills/tar-format/references/tar-format.md
Original file line number Diff line number Diff line change
Expand Up @@ -249,15 +249,15 @@ GNU tar uses base-256 binary fields for out-of-range numeric values. SharpCompre

Wrapper detection is defined in `TarWrapper.Wrappers`. Detection is content-based: wrapper detection is followed by a tar-header probe of the decompressed payload.

| Wrapper | Extensions | Read | Write |
| ------- | ---------- | ---- | ----- |
| Plain tar | `tar` | Yes | Yes |
| Tar + GZip | `tar.gz`, `taz`, `tgz` | Yes | Yes |
| Tar + BZip2 | `tar.bz2`, `tb2`, `tbz`, `tbz2`, `tz2` | Yes | Yes |
| Tar + LZip | `tar.lz` | Yes | Yes |
| Tar + XZ | `tar.xz`, `txz` | Yes | No |
| Tar + ZStandard | `tar.zst`, `tar.zstd`, `tzst`, `tzstd` | Yes | No |
| Tar + LZW compress | `tar.Z`, `tZ`, `taZ` | Yes | No |
| Wrapper | Extensions | Archive Read | Reader Read | Write |
| ------- | ---------- | ------------ | ----------- | ----- |
| Plain tar | `tar` | Yes | Yes | Yes |
| Tar + GZip | `tar.gz`, `taz`, `tgz` | No | Yes | Yes |
| Tar + BZip2 | `tar.bz2`, `tb2`, `tbz`, `tbz2`, `tz2` | No | Yes | Yes |
| Tar + LZip | `tar.lz` | No | Yes | Yes |
| Tar + XZ | `tar.xz`, `txz` | No | Yes | No |
| Tar + ZStandard | `tar.zst`, `tar.zstd`, `tzst`, `tzstd` | No | Yes | No |
| Tar + LZW compress | `tar.Z`, `tZ`, `taZ` | No | Yes | No |

Writer support currently accepts only these compression types:

Expand All @@ -278,9 +278,9 @@ Reader API:

Archive API:

- `TarArchive.OpenArchive(Stream)` and `TarArchive.OpenAsyncArchive(Stream)` require seekable streams.
- `TarArchive.OpenArchive(Stream)` and `TarArchive.OpenAsyncArchive(Stream)` require seekable raw tar streams.
- File/path overloads own the opened file stream.
- Compressed tar archive access follows streaming semantics over the decompressed stream rather than full random-access semantics.
- Compressed tar wrappers are supported through `TarReader`, not `TarArchive` or `ArchiveFactory`; archive detection blocks them instead of opening the outer compression wrapper as a standalone archive.

Parsed metadata surfaced through entries includes:

Expand Down Expand Up @@ -323,7 +323,7 @@ Keep these limitations explicit in code comments, docs, and tests:
- No write support for `tar.xz`, `tar.zst`, or `tar.Z`.
- PAX read support is limited to `path`, `linkpath`, `size`, `mtime`, `uid`, `gid`, and `mode`.
- Unknown PAX keys are ignored.
- Stream-based `TarArchive` open requires seekable input.
- Stream-based `TarArchive` open requires seekable raw tar input.

## Test Fixtures

Expand Down
2 changes: 1 addition & 1 deletion docs/API.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ using (var archive = ArchiveFactory.OpenArchive(parts))
}
```

`ArchiveInformation.SupportsRandomAccess` is `true` when the detected format supports `IArchive` random access. It is `false` for reader-only formats such as Ace, Arc, Arj, and standalone LZW, where `ReaderFactory.OpenReader` should be used instead.
`ArchiveInformation.SupportsRandomAccess` is `true` when the detected format supports `IArchive` random access. It is `false` for reader-only formats such as Ace, Arc, Arj, and standalone LZW, where `ReaderFactory.OpenReader` should be used instead. Compressed tar wrappers such as `.tar.gz` and `.tar.xz` are also reader-only; `ArchiveFactory.GetArchiveInformation` returns `null` for them and `ArchiveFactory.OpenArchive` does not open them as the outer compression wrapper. Use `ReaderFactory.OpenReader` or `TarReader.OpenReader` for those files.

### Creating Archives

Expand Down
14 changes: 7 additions & 7 deletions docs/FORMATS.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,12 +16,12 @@
| Rar | Rar | Decompress | RarArchive | RarReader | N/A |
| Zip (2) | None, Shrink, Reduce, Implode, DEFLATE, Deflate64, BZip2, LZMA, PPMd, ZStandard, XZ | Both | ZipArchive | ZipReader | ZipWriter |
| Tar | None | Both | TarArchive | TarReader | TarWriter (3) |
| Tar.GZip | DEFLATE | Both | TarArchive | TarReader | TarWriter (3) |
| Tar.BZip2 | BZip2 | Both | TarArchive | TarReader | TarWriter (3) |
| Tar.Zstandard | ZStandard | Decompress | TarArchive | TarReader | N/A |
| Tar.LZip | LZMA | Both | TarArchive | TarReader | TarWriter (3) |
| Tar.XZ | LZMA2 | Decompress | TarArchive | TarReader | N/A |
| Tar.LZW | LZW | Decompress | TarArchive | TarReader | N/A |
| Tar.GZip | DEFLATE | Both | N/A | TarReader | TarWriter (3) |
| Tar.BZip2 | BZip2 | Both | N/A | TarReader | TarWriter (3) |
| Tar.Zstandard | ZStandard | Decompress | N/A | TarReader | N/A |
| Tar.LZip | LZMA | Both | N/A | TarReader | TarWriter (3) |
| Tar.XZ | LZMA2 | Decompress | N/A | TarReader | N/A |
| Tar.LZW | LZW | Decompress | N/A | TarReader | N/A |
| GZip (single file) | DEFLATE | Both | GZipArchive | GZipReader | GZipWriter |
| 7Zip (4) | LZMA, LZMA2, BZip2, PPMd, BCJ, BCJ2, Deflate | Both | SevenZipArchive | N/A | SevenZipWriter |

Expand All @@ -31,7 +31,7 @@
4. The 7Zip format doesn't allow for reading as a forward-only stream, so 7Zip read support is only through the Archive API. Writing is supported through SevenZipWriter for non-solid archives with LZMA/LZMA2 and requires a seekable output stream. See [7Zip Format Notes](#7zip-format-notes) for details on async extraction behavior.
5. LZip has no support for extra data like the file name or timestamp. There is a default filename used when looking at the entry Key on the archive.

`ArchiveFactory.GetArchiveInformation(...).SupportsRandomAccess` is `true` when the detected format has an Archive API in this table. It is `false` for reader-only formats such as Ace, Arc, Arj, and standalone LZW.
`ArchiveFactory.GetArchiveInformation(...).SupportsRandomAccess` is `true` when the detected format has an Archive API in this table. It is `false` for reader-only formats such as Ace, Arc, Arj, and standalone LZW. Compressed tar wrappers are supported by `ReaderFactory`/`TarReader`, not by `ArchiveFactory`/`TarArchive`; ArchiveFactory detection blocks them instead of opening the outer compression wrapper as a standalone archive.

### Zip Format Notes

Expand Down
34 changes: 15 additions & 19 deletions docs/TAR_SPEC.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,18 +57,18 @@ Tar wrapper detection is defined by `TarWrapper.Wrappers` in `src/SharpCompress/
| Wrapper | Detection | `TarArchive` read | `TarReader` read | `TarWriter` write |
| ------- | --------- | ----------------- | ---------------- | ----------------- |
| Plain tar | Yes | Yes | Yes | Yes |
| Tar + GZip | Yes | Yes | Yes | Yes |
| Tar + BZip2 | Yes | Yes | Yes | Yes |
| Tar + LZip | Yes | Yes | Yes | Yes |
| Tar + XZ | Yes | Yes | Yes | No |
| Tar + ZStandard | Yes | Yes | Yes | No |
| Tar + LZW compress | Yes | Yes | Yes | No |
| Tar + GZip | Reader only | No | Yes | Yes |
| Tar + BZip2 | Reader only | No | Yes | Yes |
| Tar + LZip | Reader only | No | Yes | Yes |
| Tar + XZ | Reader only | No | Yes | No |
| Tar + ZStandard | Reader only | No | Yes | No |
| Tar + LZW compress | Reader only | No | Yes | No |

Write support is implemented in `src/SharpCompress/Writers/Tar/TarWriter.cs` and currently accepts only `CompressionType.None`, `CompressionType.GZip`, `CompressionType.BZip2`, and `CompressionType.LZip`.

## Detection Behavior

Tar detection is implemented in `TarFactory.IsArchive`, `TarFactory.IsArchiveAsync`, `TarFactory.GetCompressionType`, and `TarFactory.GetCompressionTypeAsync`.
Tar reader detection is implemented in `TarFactory.TryOpenReader`, `TarFactory.TryOpenReaderAsync`, `TarFactory.GetCompressionType`, and `TarFactory.GetCompressionTypeAsync`. Archive detection through `TarFactory.IsArchive` and `TarFactory.IsArchiveAsync` accepts raw tar only.

Detection behavior is:

Expand All @@ -77,13 +77,14 @@ Detection behavior is:
3. Probe each registered wrapper in order.
4. If a wrapper matches, create a decompression stream for that wrapper.
5. Call `TarArchive.IsTarFile` or `TarArchive.IsTarFileAsync` on the decompressed stream.
6. If the tar probe succeeds, treat the stream as tar with that wrapper compression.
6. If the tar probe succeeds, open `TarReader` with that wrapper compression.

Implications:

- Tar detection is content-based, not extension-based.
- Wrapper detection is not sufficient by itself. The decompressed payload must also parse as tar.
- Non-seekable detection is supported through the recording and rewind mechanism.
- Non-seekable wrapper detection is supported through the recording and rewind mechanism on the reader path.
- `ArchiveFactory`/`TarArchive` do not open compressed tar wrappers and do not fall through to the outer compression wrapper archive; use `ReaderFactory`/`TarReader` for `.tar.gz`, `.tar.bz2`, `.tar.xz`, `.tar.zst`, `.tar.lz`, and `.tar.Z`.
- The largest rewind requirement currently comes from BZip2, which declares a larger minimum probe buffer in `TarWrapper`.

`TarArchive.IsTarFile` and `TarArchive.IsTarFileAsync` attempt to read a single tar header and return `false` on any exception. They also treat an all-zero empty archive block as a valid empty tar archive when the entry type is defined.
Expand Down Expand Up @@ -151,20 +152,15 @@ Implementation files:

`TarArchive.OpenArchive(Stream)` and `TarArchive.OpenAsyncArchive(Stream)` require a seekable stream and throw `ArgumentException` when `CanSeek` is `false`.

`TarArchive.OpenArchive(FileInfo)` and the list-based overloads use `SourceStream` and determine wrapper compression by calling `TarFactory.GetCompressionType`.
`TarArchive.OpenArchive(FileInfo)` and the list-based overloads use `SourceStream` and validate the input as raw tar.

Asynchronous `OpenAsyncArchive` overloads use `TarFactory.GetCompressionTypeAsync` for wrapper detection.
Asynchronous `OpenAsyncArchive` overloads perform the same raw-tar validation.

### Entry Loading

`TarArchive.LoadEntries` and `LoadEntriesAsync` parse entries differently depending on wrapper compression:
`TarArchive.LoadEntries` and `LoadEntriesAsync` parse raw tar streams in `StreamingMode.Seekable`. The header stores `DataStartPosition`, and entries reopen data through `TarFilePart` by seeking back to the data position.

- Uncompressed tar uses `StreamingMode.Seekable`.
- Wrapped tar uses `StreamingMode.Streaming` because the decompressed stream is not treated as random-access.

When seekable mode is used, the header stores `DataStartPosition`, and entries reopen data through `TarFilePart` by seeking back to the data position.

When streaming mode is used, the header stores a `PackedStream`, and entry access follows streaming semantics over the decompressed stream.
Wrapped tar streams use `TarReader`, whose forward-only entry access follows streaming semantics over the decompressed stream.

### Archive Rewrite Behavior

Expand Down Expand Up @@ -418,7 +414,7 @@ This section documents current implementation limits, not desired future behavio
### Archive behavior limitations

- Stream-based archive open requires a seekable input stream
- Compressed tar archive access is not full random-access in the same sense as uncompressed seekable tar
- Compressed tar wrappers are supported through `TarReader`, not `TarArchive`

## Test Coverage Map

Expand Down
20 changes: 16 additions & 4 deletions src/SharpCompress/Archives/ArchiveFactory.Async.cs
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,11 @@ public static async ValueTask<IAsyncArchive> OpenAsyncArchive(
)
{
readerOptions ??= ReaderOptions.ForExternalStream;
var factory = await FindFactoryAsync<IArchiveFactory>(stream, cancellationToken)
var factory = await FindFactoryAsync<IArchiveFactory>(
stream,
readerOptions,
cancellationToken
)
.ConfigureAwait(false);
return await factory
.OpenAsyncArchive(stream, readerOptions, cancellationToken)
Expand Down Expand Up @@ -46,7 +50,7 @@ public static async ValueTask<IAsyncArchive> OpenAsyncArchive(
{
options ??= ReaderOptions.ForFilePath;

var factory = await FindFactoryAsync<IArchiveFactory>(fileInfo, cancellationToken)
var factory = await FindFactoryAsync<IArchiveFactory>(fileInfo, options, cancellationToken)
.ConfigureAwait(false);
return await factory
.OpenAsyncArchive(fileInfo, options, cancellationToken)
Expand Down Expand Up @@ -76,7 +80,11 @@ public static async ValueTask<IAsyncArchive> OpenAsyncArchive(
fileInfo.NotNull(nameof(fileInfo));
options ??= ReaderOptions.ForFilePath;

var factory = await FindFactoryAsync<IMultiArchiveFactory>(fileInfo, cancellationToken)
var factory = await FindFactoryAsync<IMultiArchiveFactory>(
fileInfo,
options,
cancellationToken
)
.ConfigureAwait(false);
return await factory
.OpenAsyncArchive(filesArray, options, cancellationToken)
Expand All @@ -101,7 +109,11 @@ public static async ValueTask<IAsyncArchive> OpenAsyncArchive(
firstStream.NotNull(nameof(firstStream));
options ??= ReaderOptions.ForExternalStream;

var factory = await FindFactoryAsync<IMultiArchiveFactory>(firstStream, cancellationToken)
var factory = await FindFactoryAsync<IMultiArchiveFactory>(
firstStream,
options,
cancellationToken
)
.ConfigureAwait(false);
return await factory
.OpenAsyncArchive(streamsArray, options, cancellationToken)
Expand Down
Loading