Conda install and software versions

[hmontenegro]

Installing from conda (as of 2020-03-10) is resulting in some software version incompatibilities. Install commands:

. ~/bin/.miniconda3/etc/profile.d/conda.sh
conda create -n Nullarbor-2.0.20191013
conda activate Nullarbor-2.0.20191013
conda install -c conda-forge -c bioconda -c defaults nullarbor=2.0.20191013

I get a medal (yay!) when checking if everything is fine:

nullarbor.check.txt

Side comment: it seems nullarbor.pl --check doesn't check for KRAKEN2_DEFAULT_DB.

Then I run:

nullarbor.pl --cpus 12 --name S_S3 --mlst saureus --ref ./db/Saureus_FORC59.gbk --input S_sample.tsv --outdir S
make preview
nice make all -j 3 -l 188 -C S 2>&1 | tee -a S/nullarbor.log

The preview runs fine, but I get two issues with make all:

1. First issue is the same as issue conda package dependency issue - kraken & kraken2 #216

/home/dude/bin/.miniconda3/envs/Nullarbor-2.0.20191013/libexec/classify: invalid option -- 'd'

Solutions from the above issue didn't solve the problem. Issuing conda update --all results in the following changes:

The following packages will be REMOVED:

  libgomp-9.2.0-h24d8f2e_2
  openmp-8.0.1-0

The following packages will be UPDATED:

  certifi                         2019.11.28-py36h9f0ad1d_0 --> 2019.11.28-py36h9f0ad1d_1
  libblas                                 3.8.0-14_openblas --> 3.8.0-15_openblas
  libcblas                                3.8.0-14_openblas --> 3.8.0-15_openblas
  liblapack                               3.8.0-14_openblas --> 3.8.0-15_openblas
  libopenblas                              0.3.7-h5ec1e0e_6 --> 0.3.8-h5ec1e0e_0
  llvm-openmp                              8.0.1-hc9558a2_0 --> 9.0.1-hc9558a2_2

The following packages will be DOWNGRADED:

  _openmp_mutex                                   4.5-0_gnu --> 4.5-1_llvm
  cd-hit                                   4.8.1-h8b12597_3 --> 4.8.1-hdbcaa40_0
  spades                                  3.14.0-h2d02072_0 --> 3.13.0-0

Trying conda install -y -c bioconda kraken --force-reinstall results in:

The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    certifi-2019.11.28         |   py36h9f0ad1d_1         149 KB  conda-forge
    ------------------------------------------------------------
                                           Total:         149 KB

The following packages will be UPDATED:

  certifi                         2019.11.28-py36h9f0ad1d_0 --> 2019.11.28-py36h9f0ad1d_1

2. Second issue is related to SAMtools version parsing:

[00:51:00] Need samtools --version >= 1.7 but you have 1.10 - please upgrade it.

This causes snippy to fail.

Whole log file also attached.

nullarbor.log.txt

[tseemann]

Both these problems are old issues that I thought were solved by multiple conda patches since. Conda often has issues with this, i think mainly to do with python version issues in the background.

The versions of tools in your --check are quite old, which is weird because you created a new environment from scratch. Have you been keeping your base miniconda env up to to date too?
eg. conda activate base ; conda update --all --yes
And then create your env from scratch again.

Snippy was fixed a while back. Try force install snippy=4.6.0 for that.

The kraken issue was also solved a while back. Can you force reinstall the latest versions of both krakens.

And can you use k2 instead of k1? it's database is 30 times smaller and much faster.

[hmontenegro]

Base conda was a bit outdated, but not too much: it was at 4.8.0, current is 4.8.2. I just updated and will try again. I will report back as soon as I have something. I will check the tools installed version and I will try to force upgrade if needed.

Yes, I can (and prefer to) use kraken2. I was just doing a test run following the github README, and I didn't notice kraken was default over kraken2.

[tseemann]

I am going to try and update the conda recipe and force all the newer versions of things.

I think i forgot to make Kraken2 default because we use these settings on our system!

NULLARBOR_ASSEMBLER=skesa
NULLARBOR_PREFILL=1
NULLARBOR_TAXONER=kraken2
NULLARBOR_TREEBUILDER=iqtree
NULLARBOR_CPUS=32

[hmontenegro]

I am very confused right now, maybe my conda is somehow broken. Conda is latest (4.8.2), and I tried both upgrading snippy in an already installed env, and fresh installing snippy=4.6.0 along with Nullarbor. This results in unsolvable conflicts in both cases:

conda install -c conda-forge -c bioconda -c defaults nullarbor snippy=4.6.0
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: | 
Found conflicts! Looking for incompatible packages.
This can take several minutes.  Press CTRL-C to abort.
failed                                                                                                                                                                  

UnsatisfiableError: The following specifications were found to be incompatible with each other:

Output in format: Requested package -> Available versions

Package snippy conflicts for:
nullarbor -> snippy[version='>=4.2|>=4.4.3']
snippy=4.6.0

I will try with a fresh conda install now. Or maybe I will take the bite and manually install Nullarbor.

[tseemann]

That error is weird. It doesn't seem like a conflict at all. Hmmmm.

[sharbie88]

I am also trying to use snippy within Nullarbor and got:

Need samtools --version >= 1.7 but you have 1.10 - please upgrade it.

My conda was 4.8.3 and my nullarbor_env was freshly installed last December. Snippy is v4.4.3 and conda install snippy=4.6.0 gives me the same:

UnsatisfiableError: The following specifications were found to be incompatible with each other:

Output in format: Requested package -> Available versions

[alantsangmb]

I got the same error.
And I believe it is related to Perl version and the way how Perl compare the version number.
For this error, Perl interprets 1.10 is smaller than 1.7.
I have used a quick and dirty way to suppress this error by modifying the snippy script.
Add "use version;" to use version module to implement version objects
and change "$ver < $min" to "version->declare($ver) < version->declare($min)" in the parse_version() subroutine within the script.

[shlomobl]

Hi,
Has this been solved?
I did a fresh install (everything... Ubuntu 20.04, then conda, then nullarbor).
I thought nullarbor was working but then I cannot generate a full report.
I got an error message about samtools actually. But then I found this thread and thought about updating snippy too (there are some snippy messages too).
Now I did: conda install snippy=4.6.0 --force
And got:

UnsatisfiableError: The following specifications were found to be incompatible with each other:

Output in format: Requested package -> Available versionsThe following specifications were found to be incompatible with your system:

  - feature:/linux-64::__glibc==2.31=0
  - feature:|@/linux-64::__glibc==2.31=0

Your installed version is: 2.31

I have no idea what it means...

[DininduSenanayake]

@shlomobl Not sure whether this is resolve but I have custom environment
nullarbor_2.0.20191013.txt
file (attached) which we are using to build nullarbor when needed and it is working well as versions of individual applications were adjusted to avoid any conflicts.