Download, compile and install commonly used software to a custom path.
- Star the repo before clone
- File issue if it does not work
- Unstar if you feel it is unmaintained
# configure setup path, you can put them into your `.bashrc` or `.zshrc`
export ZZROOT=$HOME/app
export PATH=$ZZROOT/bin:$PATH
export LD_LIBRARY_PATH=$ZZROOT/lib:$ZZROOT/lib64:$LD_LIBRARY_PATH
# e.g. install git
sh zzgit.sh
NOTE:
If the download speed of the machine is too slow, you can put the git.tar.gz
(using git as an example) into $ZZROOT/downloads
before running the script.
For the download link, check the DOWNLOADURL
variable in its script; for the exact name of the software, check the NAME
variable.
NOTE2: If you compile from source, please make sure that they are compiled using the same gcc version!
NOTE3:
If you DO have root privilate on your machine/server, then basic tools such as libtool
, ccache
, git
, etc. might be better installed by OS's package manager. Such as apt
in Ubuntu. Only install software that are not available on the OS's packaging system.
Verified on Ubuntu 18.04
Updated: July 8, 2020
- boost (1.71.0)
- install script:
./zzboost.sh
- post install:
export BOOST_ROOT=$ZZROOT
- install script:
- gmp (6.1.2)
- install script:
./zzgmp.sh
- install script:
- mpc (1.1.0)
- depends on:
gmp
,mpfr
- install script:
./zzmpc.sh
- depends on:
- mpfr (4.0.2)
- depends on:
gmp
- install script:
./zzmpfr.sh
- depends on:
- nasm (2.14.02)
- install script:
./zznasm.sh
- install script:
- yasm (1.3.0)
- install script:
./zzyasm.sh
- install script:
- ncurses (6.1)
- install script:
./zzncurses.sh
- post install:
export TERMINFO=$ZZROOT/share/terminfo
- install script:
- freetype (2.10.1)
- install script:
./zzfreetype.sh
- install script:
- opencv (4.1.2)
- note: this config is designed for denseflow. Please modify the cmake flags for your needs.
- depends on:
ccache
,ffmpeg
- install script:
./zzopencv.sh
- post install:
export OpenCV_DIR=$ZZROOT
- openssl (1.1.1d)
- note: both
PREFIX
andOPENSSLDIR
are set to$ZZROOT/ssl
- install script:
./zzopenssl.sh
- post install: for cmake
export OPENSSL_ROOT_DIR=$ZZROOT/ssl
, and for configure--with-ssl=$ZZROOT/ssl
(the flag may be different in different configure scripts)
- note: both
- libcurl (7.67.0)
- depends on:
autoconf
,automake
,libtool
,m4
,nroff
,perl
,openssl
- install script:
./zzlibcurl.sh
- depends on:
- libevent (2.1.11)
- install script:
./zzlibevent.sh
- install script:
- libvpx (master)
- install script:
./zzlibvpx.sh
- install script:
- libx264 (master)
- install script:
./zzlibx264.sh
- install script:
- libx265 (3.2.1)
- install script:
./zzlibx265.sh
- install script:
- libz (1.2.11)
- install script:
./zzlibz.sh
- install script:
- libzstd (1.4.4)
- install script:
./zzlibzstd.sh
- install script:
- libzip (1.5.2)
- install script:
./zzlibzip.sh
- install script:
- libtool (2.4.6)
- install script:
./zzlibtool.sh
- install script:
- libjpeg (6b)
- depends on:
libtool
- install script:
./zzlibjpeg.sh
- depends on:
- libjpeg-turbo (2.0.3)
- depends on:
nasm
,yasm
- install script:
./zzlibjpeg-turbo.sh
- depends on:
- libpng (1.6.37)
- install script:
./zzlibpng.sh
- install script:
- libtiff (4.1.0)
- install script:
./zzlibtiff.sh
- install script:
- libwebp (1.0.3)
- install script:
./zzlibwebp.sh
- install script:
- giflib (5.2.1)
- install script:
./zzgiflib.sh
- install script:
- libopenjp2 (2.3.1)
- install script:
./zzlibopenjp2.sh
- install script:
- autoconf (2.69)
- depends on:
m4
,perl
- install script:
./zzautoconf.sh
- depends on:
- ccache (3.7.6)
- install script:
./zzccache.sh
- post install: customize cache dir by
export CCACHE_DIR=~/.ccache
to avoid permission issue
- install script:
- cmake (3.16.0)
- install script:
./zzcmake.sh
- install script:
- anaconda3 (2019.10)
- note: anaconda is installed default on
$HOME/anaconda3
- install script:
./zzconda.sh
- post install:
export PATH=$HOME/anaconda3/bin:$PATH
- note: anaconda is installed default on
- ffmpeg (4.2.1)
- depends on:
nasm
,yasm
,libx264
,libx265
,libvpx
- install script:
./zzffmpeg.sh
- depends on:
- gcc (7.5.0)
- note: CUDA 10 support up to gcc 7, CUDA 9 support up to gcc 5.
- depends on:
gmp
,mpfr
,mpc
- install script:
./zzgcc.sh
- git (2.24.0)
- depends on:
libcurl
,libz
- install script:
./zzgit.sh
- depends on:
- hdf5 (1.10.5)
- install script:
./zzhdf5.sh
- install script:
- htop (2.2.0)
- depends on:
ncurses
- install script:
./zzhtop.sh
- depends on:
- julia (1.3.0)
- install script:
./zzjulia.sh
- install script:
- autojump (master)
- install script:
./zzautojump.sh
- post install: put lines of configs in
.zshrc
- install script:
- imagemagick (latest)
- depends on:
freetype
,libjpeg-turbo
,libpng
,libtool
,perl
- install script:
./zzimagemagick.sh
- depends on:
- nmon (16j)
- depends on:
ncurses
- install script:
./zznmon.sh
- depends on:
- zsh (5.7.1)
- depends on:
ncurses
- install script:
./zzzsh.sh
- post install: to set
zsh
as default shell without root, addexec $ZZROOT/bin/zsh -l
to.bashrc
(example)
- depends on:
- ohmyzsh (latest)
- depends on:
zsh
- install script:
./zzohmyzsh.sh
- depends on:
- perl (5.30.1)
- install script:
./zzperl.sh
- install script:
- ruby (2.7.0)
- depends on:
openssl (with libssl installed)
- install script:
./zzruby.sh
- depends on:
- tmux (3.0a)
- depends on:
libevent
- install script:
./zztmux.sh
- depends on:
- unrar (5.8.4)
- install script:
./zzunrar.sh
- install script:
- vmtouch (1.3.1)
- install script:
./zzvmtouch.sh
- install script:
- sshpass (support 2-step verification)
- depends on:
oathtool
,autoconf
- install script:
./zzsshpass.sh
- depends on:
- denseflow (extract frames, optical flow, resize, and more!)
- depends on:
boost
,opencv
,cmake
- install script:
./zzdenseflow.sh
- depends on:
- wslssh.bat (use ssh of wsl in win10, no installation,
.bat
script as is)- depends on:
sshpass
- you can set
vscode
ssh path to thiswslssh.bat
, work together withsshpass
(installed in wsl)
- depends on:
- sshkey.sh (gen an ssh key)
- run script:
./sshkey.sh
- run script:
- zshplugin.sh (completions, autosuggestions, syntax-highlighting)
- depends on:
git
- install script:
./zshplugin.sh
- post install: put lines of configs in
.zshrc
- depends on:
- nordtmux.sh (nord theme for tmux)
- depends on:
git
,tmux
- install script:
./nordtmux.sh
- depends on:
- dircolor.sh (nord theme for zsh)
- install script:
./dircolor.sh
- post install: put lines of configs in
.zshrc
- install script:
.tmux.conf
(tmux configuration).zshrc
(zsh configuration).netrc
(lftp password-less configuration).ssh/config
(suppress git clone warnings).lftp/rc
(suppress lftp ssl warning)
- opengl on remote machine
- useful Julia packages: ArgParse Glob Images FileIO StatsBase Shell UnicodePlots PyCall PyPlot Plots ImageFiltering ImageMagick Interpolations Revise OhMyREPL DataFrames ProgressMeter JLD2 Distances ImageTransformations JuMP HDF5 ImageInTerminal ImageView DataStructures JSON3 JuliaFormatter NNlib
- free JuMP solvers: Cbc Clp COSMO CSDP ECOS GLPK Juniper OSQP ProxSDP SCS SDPA
- good reference: http://www.linuxfromscratch.org/lfs/view/development/index.html and http://www.linuxfromscratch.org/blfs/view/svn/index.html and https://pkgs.org/
- CUDA arch and CUDA gencode list
- Instruction on how to load jpeg faster. https://github.com/innerlee/cvbenchmark#setup
- View image in terminal: viu and tiv
- image lossless optimization: jpegtran optipng
- image lossy optimization: pngquant
- ffmpeg tricks
- silent:
ffmpeg -hide_banner -loglevel panic
- get resolution
ffprobe -v error -select_streams v:0 -show_entries stream=width,height -of csv=p=0 input.mp4
- count frames
ffmpeg -i input.mp4 -map 0:v:0 -c copy -f null - 2>&1 | grep frame
- remove duplicated frames
ffmpeg -i input.mp4 -vf mpdecimate -vsync vfr out.mp4
- silent:
rsync -avzh --exclude='*.jpg' --exclude='*.png' lizz@lizz-dev:data .
- cool rust tools:
- bottom https://github.com/ClementTsang/bottom
- ripgrep https://github.com/BurntSushi/ripgrep
- fd https://github.com/sharkdp/fd
- dust https://github.com/bootandy/dust
- zenith https://github.com/bvaisvil/zenith
-
Q: My boost segfaluts :(
A: Make sure they the boost are comipiled and linked using the same gcc/g++ version -
Q:
Cert verify failed: BADCERT_NOT_TRUSTED
:(
A: Tryexport SSL_CERT_DIR=/etc/ssl/certs
-
Q:
warning: templates not found in /workspace/destdir/share/git-core/templates
:(
A: Add the following to$HOME/.gitconfig
[init]
templatedir = $HOME/app/share/git-core/templates
- Q:
fatal: unable to find remote helper for 'https'
:(
A: Add the following to$HOME/.zshrc
(or.bashrc
of course)
export GIT_EXEC_PATH=$ZZROOT/libexec/git-core
export GIT_SSL_CAINFO=/etc/ssl/certs/ca-bundle.crt
- Q:
Using the 'memory' GSettings backend. Your settings will not be saved or shared with other applications.
:(
A: Add the following to$HOME/.zshrc
(or.bashrc
of course)
export GIO_EXTRA_MODULES=/usr/lib/x86_64-linux-gnu/gio/modules/
- Q:
import pyvips
segfault :(
A: Make suregstreamer-orc <0.4.30
,imagemagick <7.0.9
.
conda install -c conda-forge imagemagick=7.0.8
conda install -c conda-forge gstreamer-orc=0.4.29
- Q: GitError(Code:ECERTIFICATE, Class:SSL, the SSL certificate is invalid: 0x08 - The certificate is not correctly signed by the trusted CA) :(
A: Run the following
julia> import LibGit2
julia> LibGit2.set_ssl_cert_locations("/etc/ssl/certs/ca-certificates.crt")
- Q: dotnet error :(
A: Add the following to$HOME/.zshrc
(or.bashrc
of course)
export DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=true
export MSBuildSDKsPath=$DOTNET_ROOT/sdk/3.1.102/Sdks
- Q: Install OpenCV: CUDA Version wrong:(
A: Add the path of the correct version CUDA tozzopencv.sh
when runing cmake
cmake ......
-DCUDA_TOOLKIT_ROOT_DIR=/mnt/lustre/share/cuda-9.0/ \
......
- Q: Install Denseflow: OpenCV Version wrong, can't find customized path for opencv:(
A: Add the path of the correct version opencv and corresponding cuda version tozzdenseflow.sh
when runing cmake
cmake -DCMAKE_INSTALL_PREFIX=$ROOTDIR \
-DOpenCV_DIR=$ROOT_DIR/lib64/cmake/opencv4 \
-gencode=arch=compute_61,code=sm_61 \
-DCUDA_TOOLKIT_ROOT_DIR=/mnt/lustre/share/cuda-9.0/ ..
- Q: Unzip filenames looks messy when there are unicodes :(
A: Specify the encoding of filenames.
$ unzip -h
UnZip 6.00 of 20 April 2009, by Debian. Original by Info-ZIP.
...
-O CHARSET specify a character encoding for DOS, Windows and OS/2 archives
-I CHARSET specify a character encoding for UNIX and other archives
...
unzip -O GB18030 gb18030.zip -d target_dir
- Q: Get locale warnings :(
A: Run
export LANGUAGE=en_US.UTF-8
export LANG=en_US.UTF-8
export LC_ALL=en_US.UTF-8
locale-gen en_US.UTF-8
sudo dpkg-reconfigure locales
-
Q: moov atom not found :(
A: Use https://github.com/anthwlock/untrunc to fix it. -
Q: you must add /ou=globalsign root ca ... :(
A: Set env var.
export SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt
jlinstall
is another way to install softwares.
It uses pre-built binaries so that no compilation is needed.
The default install path of jlinstall
is JLROOT=$ZZROOT/jl
.
export ZZROOT=$HOME/app
export JLROOT=$ZZROOT/jl
export PATH=$ZZROOT/bin:$JLROOT/bin:$PATH
export LD_LIBRARY_PATH=$ZZROOT/lib:$ZZROOT/lib64:$JLROOT/lib:$JLROOT/lib64:$LD_LIBRARY_PATH