Skip to content

Commit

Permalink
Support xz compressed packages (#5513)
Browse files Browse the repository at this point in the history
## Summary

Closes #2187.

The [xz
backdoor](https://gist.github.com/thesamesam/223949d5a074ebc3dce9ee78baad9e27)
is still fairly recent, but luckily the [Rust `xz2` crate bundles
version 5.2.5 of the C `xz`
package](https://github.com/alexcrichton/xz2-rs/tree/main/lzma-sys),
which is before the backdoor was introduced.

It's worth noting that a security risk still exists if you have a
compromised version of `xz` installed on your system, but that risk is
not introduced by `uv` or the Rust packages in general.

## Test Plan

Tried installing the package mentioned in the linked issue: `python-apt
@
https://launchpad.net/ubuntu/+archive/primary/+sourcefiles/python-apt/2.7.6/python-apt_2.7.6.tar.xz`

(Note that this will only work on Ubuntu - I tried on a Mac and while
the archive was extracted properly, the package did not install because
of some missing files)

---------

Co-authored-by: Charlie Marsh <[email protected]>
  • Loading branch information
krishnan-chandra and charliermarsh authored Jul 28, 2024
1 parent caf0173 commit 4b41284
Show file tree
Hide file tree
Showing 3 changed files with 57 additions and 3 deletions.
23 changes: 22 additions & 1 deletion Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion crates/uv-extract/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ workspace = true
[dependencies]
pypi-types = { workspace = true }

async-compression = { workspace = true, features = ["bzip2", "gzip", "zstd"] }
async-compression = { workspace = true, features = ["bzip2", "gzip", "zstd", "xz"] }
async_zip = { workspace = true, features = ["tokio"] }
fs-err = { workspace = true, features = ["tokio"] }
futures = { workspace = true }
Expand Down
35 changes: 34 additions & 1 deletion crates/uv-extract/src/stream.rs
Original file line number Diff line number Diff line change
Expand Up @@ -198,7 +198,25 @@ pub async fn untar_zst<R: tokio::io::AsyncRead + Unpin>(
Ok(untar_in(&mut archive, target.as_ref()).await?)
}

/// Unzip a `.zip`, `.tar.gz`, or `.tar.bz2` archive into the target directory, without requiring `Seek`.
/// Unzip a `.tar.xz` archive into the target directory, without requiring `Seek`.
///
/// This is useful for unpacking files as they're being downloaded.
pub async fn untar_xz<R: tokio::io::AsyncRead + Unpin>(
reader: R,
target: impl AsRef<Path>,
) -> Result<(), Error> {
let reader = tokio::io::BufReader::new(reader);
let decompressed_bytes = async_compression::tokio::bufread::XzDecoder::new(reader);

let mut archive = tokio_tar::ArchiveBuilder::new(decompressed_bytes)
.set_preserve_mtime(false)
.build();
untar_in(&mut archive, target.as_ref()).await?;
Ok(())
}

/// Unzip a `.zip`, `.tar.gz`, `.tar.bz2`, `.tar.zst`, or `.tar.xz` archive into the target directory,
/// without requiring `Seek`.
pub async fn archive<R: tokio::io::AsyncRead + Unpin>(
reader: R,
source: impl AsRef<Path>,
Expand Down Expand Up @@ -258,5 +276,20 @@ pub async fn archive<R: tokio::io::AsyncRead + Unpin>(
return Ok(());
}

// `.tar.xz`
if source
.as_ref()
.extension()
.is_some_and(|ext| ext.eq_ignore_ascii_case("xz"))
&& source.as_ref().file_stem().is_some_and(|stem| {
Path::new(stem)
.extension()
.is_some_and(|ext| ext.eq_ignore_ascii_case("tar"))
})
{
untar_xz(reader, target).await?;
return Ok(());
}

Err(Error::UnsupportedArchive(source.as_ref().to_path_buf()))
}

0 comments on commit 4b41284

Please sign in to comment.