Skip to content
This repository has been archived by the owner on Apr 19, 2023. It is now read-only.

[SUGGESTION] Add zlib-ng for speedup of gzip/zlib (de)compression #325

Closed
dweemx opened this issue Mar 22, 2021 · 1 comment
Closed

[SUGGESTION] Add zlib-ng for speedup of gzip/zlib (de)compression #325

dweemx opened this issue Mar 22, 2021 · 1 comment
Labels
enhancement New feature or request

Comments

@dweemx
Copy link
Contributor

dweemx commented Mar 22, 2021 •
edited
Loading

Is your feature request related to a problem? Please describe.

Improve parts of pipelines where compression/decompression is occurring.
From @ghuls: zlib-ng should speeding up some parts of the pipeline (gzip/zlib decompression speed x2, gzip/zlib compression speed x4).

Describe the solution you'd like

Use zlib-ng instead of zlib library
From @ghuls:

  • For tools using there own zlib implentation, use minigzip instead of zip.
  • For tools that use and link to zlib, adding the zlib-ng library path to LD_LIBRARY_PATH should be enough

Describe alternatives you've considered
/

Additional context
/
@dweemx dweemx added the enhancement New feature or request label Mar 22, 2021
cflerin added a commit that referenced this issue Mar 25, 2021
@cflerin
Add Dockerfiles with zlib-ng
6f9bad0 
- vibsinglecellnf/samtools:base-0.1 has zlib-ng and libdeflate on a
debian base image
- vibsinglecellnf/samtools:1.12 uses the base-0.1 image, and includes
htslib and samtools
- #325
@cflerin
Copy link
Member

cflerin commented Mar 25, 2021 •
edited
Loading

I added two Dockerfiles under /src/samtools/:

  • A base image that has zlib-ng and libdeflate. Can be used as a base for:
    • trimgalore
    • samtools
  • A samtools (1.12) image built on this base. Can be used as a base for:
    • popscle
    • bwamaptools
    • singlecelltoolkit

cflerin added a commit that referenced this issue Mar 28, 2021
@cflerin
Rebuild Docker image based on debian zlib-ng base
9e6a267 
- #325
cflerin added a commit that referenced this issue Apr 1, 2021
@cflerin
Update single_cell_toolkit Docker image
1282878 
- Use zlib-ng base image (#325)
- Include saturation script (#332), and python 3.7 install
cflerin added a commit that referenced this issue May 27, 2021
@cflerin
Update Dockerfile to use samtools zlibng image
5d4c4f6 
- Add bwa-mem2
- #325
@cflerin cflerin mentioned this issue Jun 29, 2021
Merged
@cflerin cflerin closed this as completed in 3b285b8 Jul 1, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@cflerin @dweemx