brotli: implement encoder

[jpillora]

Please close this if/when brotli encoder is implemented. Nice work btw!

[dsnet]

Thanks for the issue. I'll definitely try to get a working brotli encoder released soon. I'll give fair warning that the first cut might not perform much better than DEFLATE in terms of it's compression ratio. I plan on doing a iterative approach to implementing brotli, where each iteration improves it's compression performance.

[jpillora]

Cool that sounds fine to me +1
On Sat, 6 Feb 2016 at 7:05 AM Joe Tsai [email protected] wrote:

Thanks for the issue. I'll definitely try to get a working brotli encoder
released soon. I'll give fair warning that the first cut might not perform
much better than DEFLATE in terms of it's compression ratio. I plan on
doing a iterative approach to implementing brotli, where each iteration
improves it's compression performance.

—
Reply to this email directly or view it on GitHub
#42 (comment).

[dsnet]

Placing this here for reference: caddyserver/caddy#525

[akyoto]

Has this been abandoned or is there any chance somebody is still working on a native implementation?

[dsnet]

I wouldn't say "abandoned", but I'm not actively working on it at the present moment. I have a number of other projects currently occupying my time.

[akyoto]

I see, thank you for the reply!
Would have loved to see a native implementation in Go but I realize it's a very time-consuming task.

[dmitshur]

@dsnet In caddyserver/caddy#525 (comment), you said:

I'm actually currently working on finishing my own implementation of a RFC 1951 DEFLATE compressor. The components built to implement DEFLATE will make implementing Brotli significantly easier.

I had a question about that. Given that a pure Go DEFLATE decompressor already exists in Go standard library (compress/flate), what's the reason you're building your own? Is it because you have different objectives compared to the std lib one, or another reason?

Can the components of compress/flate compressor be used to help implement Brotli compressor, or is there a reason why compress/flate isn't helpful for that?

Just looking to understand this better, thanks.

[dsnet]

Obviously, the lack of traffic on this repo implies haven't done much in that regard 😞

The reason are the following:

• I already implemented huffman encoding (since it's the same for bzip2 and brotli), so the only other component left for DEFLATE is just LZ77. Implementing LZ77 would allow me to experiment with different algorithms and understand it better.
• There are low level functionality that I would want to be able to do on DEFLATE that I don't think they standard library should ever support. See the zlib manual for what features they provide.

[dsnet]

@blitzprog, I appreciate your enthusiasm, but implementation compression algorithms really does require specialized knowledge. You're welcome to do research on how LZ77 works and various approaches to compute it.

[faradaytrs]

Any progress on this?

[dmitshur]

@dsnet FYI, I was made aware (thanks to @mappu) that there is a pure Go implementation of a Brotli encoder that was created using a c2go-based approach by @andybalholm:

https://groups.google.com/forum/#!msg/golang-nuts/J5MwJnmDpOw/yQacKG2pBwAJ

I haven't had a chance to try it yet, but I wanted to share it with you (and others here).

[dsnet]

Thanks for the update @dmitshur! I've been too busy with my actual work to work on this implementation. I'm glad someone else is providing it.