Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

docker: tarfile: improve auto-decompression handling #427

Closed
wants to merge 4 commits into from
Closed

docker: tarfile: improve auto-decompression handling #427

wants to merge 4 commits into from

Commits on Jun 6, 2018

  1. pkg: compression: support xz (lzma) decompression 

    For quite a while we were blocked on support xz decompression because it
effectively required shelling out to "unxz" (which is just bad in
general). However, there is now a library -- github.com/ulikunitz/xz --
which has implemented LZMA decompression in pure Go. It isn't as
featureful as liblzma (and only supports 1.0.4 of the specification) but
it is an improvement over not supporting xz at all. And since we aren't
using its writer implementation, we don't have to worry about sub-par
compression.

Signed-off-by: Aleksa Sarai <[email protected]>
    @cyphar
    cyphar committed Jun 6, 2018
    Configuration menu
    Copy the full SHA
    b9827eb View commit details
    Browse the repository at this point in the history

  2. pkg: compression: simplify DetectCompression boilerplate 

    Most users of DetectCompression will use the decompressorFunc
immediately afterwards (the only exception is the copy package). Wrap
up this boilerplate in AutoDecompress, so it can be used elsewhere.

Signed-off-by: Aleksa Sarai <[email protected]>
    @cyphar
    cyphar committed Jun 6, 2018
    Configuration menu
    Copy the full SHA
    246b92d View commit details
    Browse the repository at this point in the history

  3. docker tarfile: support auto-decompression of source 

    This matches how "docker load" deals with compressed images, as well as
being a general quality-of-life improvement over the previous error
messages we'd give. This also necessarily removes the previous
special-cased gzip handling, and adds support for auto-decompression for
streams as well.

Signed-off-by: Aleksa Sarai <[email protected]>
    @cyphar
    cyphar committed Jun 6, 2018
    Configuration menu
    Copy the full SHA
    97c59da View commit details
    Browse the repository at this point in the history

  4. docker tarfile: report correct size of compressed layers 

    Tools like umoci will always compress layers, and in order to work
around some *lovely* issues with DiffIDs, tarfile.GetBlob would always
decompress them. However the BlobInfo returned from tarfile.GetBlob
would incorrectly give the size of the *compressed* layer because the
size caching code didn't actually check the layer size, resulting in
"skopeo copy" failing whenever sourcing umoci images.

As an aside, for some reason the oci: transport doesn't report errors
when the size is wrong...

Signed-off-by: Aleksa Sarai <[email protected]>
    @cyphar
    cyphar committed Jun 6, 2018
    Configuration menu
    Copy the full SHA
    52c5d96 View commit details
    Browse the repository at this point in the history