Why does pyaff4 needs aff4-snappy instead of python-snappy?

[joachimmetz]

pyaff4 depends on aff4-snappy

• https://pypi.org/project/aff4-snappy/
• https://github.com/rekall-innovations/aff4-snappy

Which is forked from python-snappy:

• https://github.com/andrix/python-snappy
• https://pypi.org/project/python-snappy/

@blschatz @scudette I'm looking into building pyaff4 for testing purposes, some questions:

• What makes aff4-snappy different from python-snappy?
• Could the changes in aff4-snappy be merged into python-snappy?
• Could pyaff4 be changed to use python-snappy instead of aff4-snappy?

[scudette]

python snappy is not a hermetic distribution - it requires libsnappy to be installed on the system. It then builds the python binding with a runtime requirement for libsnappy. This is not an acceptable solution.

The main change is this commit aff4/aff4-snappy@45397f7

Modified to build snappy inline so we do not need to link against libsnappy.

For something as trivial as snappy it does not make sense to include a shared object - just build it statically into the binding.

[joachimmetz]

thx for the context

Let me know if you're OK with the following changes:

1. Move aff4-snappy under https://github.com/aff4?
2. Change aff4-snappy to have the libsnappy as a gitsubmodule (similar to pytsk)? So it easier to build against newer version of libsnappy
3. Update aff4-snappy to latest version of python-snappy. Push all non build related changes upstream
4. Include libsnappy and python-snappy licenses in sdist

[joachimmetz]

@scudette @blschatz friendly ping any feedback/input?

[scudette]

Thanks for the ping I totally missed it :-). Python-snappy builds a shared object which depends on the system libsnappy.so or libsnappy.dll. this makes it not hermetic so we can not use it to distribute on pipy. The aff4-snappy fork builds the bindings and the snappy code into a self contained module. This might have changed recently I have not looked at the package for a long time. But you can verify it using ldd on the produced .so file, or try to build on Windows which will fail unless you have snappy installed. Thanks Mike

…

On Mon, Feb 24, 2020, 19:15 Joachim Metz ***@***.***> wrote: @scudette <https://github.com/scudette> @blschatz <https://github.com/blschatz> friendly ping any feedback/input? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#9?email_source=notifications&email_token=AA5NRIXCXR3KY3HZYDTJTCLREN62VA5CNFSM4HHVERZ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMW5VLY#issuecomment-590207663>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA5NRIUWDZBSPJ2I64BJQD3REN62VANCNFSM4HHVERZQ> .

[joachimmetz]

My point was more to change pyaff4 to include snappy as is done for pytsk (as a git submodule) and create an hermetic build?

[scudette]

Ah sorry I don't read the entire thread (just send an email). Sure what you suggest sounds great. Will accept any pr to do this :-)

[joachimmetz]

Happy to help but could you maybe follow up on the steps proposed in #9 (comment)