Source system: Xeon E3-1275, 64GB RAM, 2x SSD 480GB (for git dataset and local target), 2x4TB disks 7.2krpm (for bigger dataset), using XFS, running AlmaLinux 8.6
Target system: AMD Turion(tm) II Neo N54L Dual-Core Processor (yes, this is old), 6GB RAM, 2x4TB WD RE disks 7.2krpm, using ZFS 2.1.5, running AlmaLinux 8.6
Linux kernel sources, initial git checkout v5.19, then changed to v5.18, 4.18 and finally v3.10 for the last run.
Initial git directory totals 4.1GB, for 5039 directories and 76951 files. Using env GZIP=-9 tar cvzf kernel.tar.gz /opt/backup_test/linux
produced a 2.8GB file. Again, using "best" compression with tar cf - /opt/backup_test/linux | xz -9e -T4 -c - > kernel.tar.bz
produces a 2.6GB file, so there's probably big room for deduplication in the source files, even without running multiple consecutive backups on different points in time of the git repo.
Note: I removed restic_beta benchmark since restic 0.14.0 with compression support is officially released.
Numbers:
Operation | bupstash 0.11.0 | borg 1.2.2 | borg_beta 2.0.0b1 | kopia 0.11.3 | restic 0.14.0 | duplicacy 2.7.2 |
---|---|---|---|---|---|---|
backup 1st run | 9 | 38 | 54 | 9 | 23 | 30 |
backup 2nd run | 13 | 21 | 25 | 4 | 9 | 14 |
backup 3rd run | 9 | 29 | 39 | 7 | 18 | 24 |
backup 4th run | 6 | 21 | 29 | 5 | 13 | 16 |
restore | 3 | 17 | 16 | 6 | 10 | 16 |
size 1st run | 213220 | 257224 | 259512 | 259768 | 260588 | 360160 |
size 2nd run | 375712 | 338792 | 341096 | 341060 | 343356 | 480392 |
size 3rd run | 538768 | 527716 | 531980 | 529788 | 532216 | 722420 |
size 4th run | 655764 | 660808 | 665692 | 666396 | 668840 | 895192 |
Remarks:
- kopia was the best allround performer on local backups when it comes to speed, but is quite CPU intensive.
- bupstash was the most space efficient tool (beats borg beta by about 1MB), and is not CPU hungry.
- For the next instance, I'll need to post CPU / Memory / Disk IO usage graphs.
- Remote repositories are SSH (+ binary) for bupstash and burp.
- Remote repositories is SFTP for duplicacy.
- Remote repository is HTTPS for kopia (kopia server with 2048 bit RSA certificate)
- Remote repository is HTTP for restic (rest-server 0.11.0)
- [Update] I've also redone the same tests in HTTPS with
--insecure-tls
which is documented on restic docs but not visible when usingrestic --help
.
Numbers:
Operation | bupstash 0.11.0 | borg 1.2.2 | borg_beta 2.0.0b1 | kopia 0.11.3 | restic 0.14.0 | duplicacy 2.7.2 |
---|---|---|---|---|---|---|
backup 1st run | 23 | 62 | 64 | 1186 | 17 | 107 |
backup 2nd run | 16 | 25 | 29 | 292 | 9 | 44 |
backup 3rd run | 19 | 37 | 48 | 904 | 14 | 60 |
backup 4th run | 15 | 29 | 36 | 800 | 11 | 47 |
restore | 161 | 255 | 269 | 279 | 20 | 1217 |
size 1st run | 250012 | 257534 | 260090 | 257927 | 262816 | 382572 |
size 2nd run | 443008 | 339276 | 341704 | 339181 | 346264 | 508655 |
size 3rd run | 633083 | 528482 | 532710 | 526723 | 536403 | 761362 |
size 4th run | 769681 | 661720 | 666588 | 662558 | 673989 | 941247 |
Remarks:
- Very bad restore results can be observed across all backup solutions (except restic), we'll need to investigate this:
- Both links are monitored by dpinger, which shows no loss.
- Target server, although being (really) old, has no observed bottlenecks (monitored, no iowait, disk usage nor cpu is skyrocketing)
- Since last benchmark series, I changed Kopia's backend from SFTP to HTTPS. There must be a bottlebeck since backup times are really bad, but restore times improved.
- I opened an issue at kopia/kopia#2372 to see whether I configured kopia poorly.
- CPU usage on target is quite intensive when backing up via HTTPS contrary to SFTP backend. I need to investigate.
- Since last benchmark series, I changed restic's backend from SFTP to HTTP. There's a REALLY big speed improvement, and numbers are comparable to local repositories.
- I must add HTTPS encryption so we can compare what's comparable. [UPDATE]: Done, same results + or - a couple of seconds, table and image is updated
- Indeed I checked that those numbers are really bound to remote repository, I can confirm, restic with rest-server is an all over winner when dealing with remote repositories.
- Strangely, the repo sizes of bupstash and duplicacy are quite larger than local repos for the same data, I discussed the subject at andrewchambers/bupstash#26 .
- I think this might be ZFS related. The remote target has a default recordsize of 128KB. I think I need to redo a next series of benchmarks with XFS as remote filesystem for repositories.