Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lack of deflate backwards compatibility? #72

Open
Lupus opened this issue Dec 31, 2021 · 0 comments
Open

Lack of deflate backwards compatibility? #72

Lupus opened this issue Dec 31, 2021 · 0 comments

Comments

@Lupus
Copy link

Lupus commented Dec 31, 2021

I've faced issues with standard linux unzip being unable to unzip the file that I create programmatically with xflate compression. I decided to repro on a smaller case and got same results. I basically took the example from the documentation of xflate package and outputted the resulting zip to a file, source code follows:

package main

import (
	"archive/zip"
	"io"
	"io/ioutil"
	"log"
	"os"

	"github.com/dsnet/compress/xflate"
)

func init() { log.SetFlags(log.Lshortfile) }

// MustLoadFile must load a file or else panics.
func MustLoadFile(file string) []byte {
	b, err := ioutil.ReadFile(file)
	if err != nil {
		panic(err)
	}
	return b
}

func main() {
	// Test files of non-trivial sizes.
	files := map[string][]byte{
		"twain.txt":   MustLoadFile("testdata/twain.txt"),
		"digits.txt":  MustLoadFile("testdata/digits.txt"),
		"huffman.txt": MustLoadFile("testdata/huffman.txt"),
	}

	// Write the Zip archive.
	out, err := os.Create("output.zip")
	if err != nil {
		log.Fatal(err)
	}
	zw := zip.NewWriter(out)
	zw.RegisterCompressor(zip.Deflate, func(wr io.Writer) (io.WriteCloser, error) {
		// Instead of the default DEFLATE compressor, register one that uses
		// XFLATE instead. We choose a relative small chunk size of 64KiB for
		// better random access properties, at the expense of compression ratio.
		return xflate.NewWriter(wr, &xflate.WriterConfig{
			Level:     xflate.BestSpeed,
			ChunkSize: 1 << 16,
		})
	})
	for _, name := range []string{"twain.txt", "digits.txt", "huffman.txt"} {
		body := files[name]
		f, err := zw.Create(name)
		if err != nil {
			log.Fatal(err)
		}
		if _, err = f.Write(body); err != nil {
			log.Fatal(err)
		}
	}
	if err := zw.Close(); err != nil {
		log.Fatal(err)
	}
	err = out.Close()
	if err != nil {
		log.Fatal(err)
	}
}

Here's what's happening with output.zip that this program created:

$ unzip -l output.zip
Archive:  output.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
   387969  1980-00-00 00:00   twain.txt
   100003  1980-00-00 00:00   digits.txt
   262144  1980-00-00 00:00   huffman.txt
---------                     -------
   750116                     3 files

$ unzip -t output.zip
Archive:  output.zip
    testing: twain.txt               (incomplete d-tree)
  error:  invalid compressed data to inflate
    testing: digits.txt              (incomplete d-tree)
  error:  invalid compressed data to inflate
    testing: huffman.txt             (incomplete d-tree)
  error:  invalid compressed data to inflate
At least one error was detected in output.zip.

I'm not sure that's an expected behavior. To double check I've took the zip to my Windows system, and it was also unable to extract the files.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant