initial benchmarking

[bluegenes]

10 threads were used for 0.2.0+ (prior versions could not use multithreading) However, given the 3 simultaneous download limit, I doubt we're effectively using more than 3 threads.

49, 243 acc sets contain a genome that is 1G!
https://www.ncbi.nlm.nih.gov/datasets/genome/GCA_014839935.2/

editable version, with full /usr/bin/time -v outputs: https://hackmd.io/JanbS4aITsKOcoKV3IXNTA?view

[bluegenes]

v0.2.2 -- 20 threads

95k microbial proteomes took 2h54mins and 12GB of RAM

cat /home/ntpierce/database-releases/gtdb-rs214.protein/logs/gtdb-rs214.protein.download_and_sketch_missing.benchmark

s	h:m:s	max_rss	max_vms	max_uss	max_pss	io_in	io_out	mean_load	cpu_time
10476.4868	2:54:36	445.03	12946.43	456.79	457.60	86.62	25.43	1.27	133.17

log:

== This is sourmash version 4.8.8. ==
== Please cite Brown and Irber (2016), doi:10.21105/joss.00027. ==

=> sourmash_plugin_directsketch 0.2.2
params: ['protein,k=7,k=10,scaled=200,abund']
downloading and sketching all accessions in 'gtdb-rs214.protein.downloadfile.csv using 20 threads