branch-listing with gix

[Byron]

This PR adds brnach-listings using gix, but without branch details, to increase performance or at least make for more maintainable code.

Tasks

• basic benchmarking support using criterion
• baseline benchmark to capture git2 performance
  • needs with and without packed-refs, and also a version with a lot of references
• replace branch listing with gix
• fix tests (remote name from remote branch)
• assure that performance isn't getting worse.
• assure local overrides are removed and GitHub version is used.

Follow Ups

• check --all-targets assure benches do not rot #4651
• figure out how to build max-performance on CI - needs cmake which isn't installed - relevant for object database performance later.
  • Let's keep it for now but remember this might be an opportunity later. After all, benchmarks to check related performance are still missing.
• fix TODOs, additional refactors Follow-up cleanup of Branch-Listing with gix #4652
• normalization/sanitization built into gitoxide
  • It takes ~5% of the time, but the majority of that is the creation of regexes. Thus, creating these statically would already solve this problem even though it's still not necessarily correct.
  • Use gix for branch normalization #4665

Performance

When switching from git2 to gitx we see that git2 can be 7,5% faster on MacOS when traversing refs. But we also see that handling many tracked branches is super slow (20x to 200x!), and when dealing with tiny repositories, gix also is much faster as for git2 the time to open the repository dominates.
More testing showed that it's all about core.precomposeUnicode, which consumes a lot of power in gix and which seems to be ignored by git2. When turned off, gix is the clear winner - a win which didn't come easily by the way.

Note that the picture above was done with max-performance, something that I had to remove as it didn't work on CI just yet.

Git2 Performance Notes

• Instrumenting a git2 based run, one thing stuck out: In a repo with 300 tracked branches, getting the remote name is very slow
• Listing branches typically gets a 10x speedup when they are packed.
• Another hot function is normalize_branch_name - this will be faster when implemented in gitoxide as it won't use regex substitution, but edits in-line.
• Otherwise, traversing loose branches is quite time-consuming due to the amount of IO done here

Gix Performance Notes

When listing branches, gix isn't naturally faster. Major consumers overall are…

• Opening a repository in full and reading global configuration.
• Unicode precomposition/decomposition during iteration.
• References are eagerly read, which makes the name-filter pruning most of them very expensive. It looked like git2 is also eagerly loading though, so I guess it's OK to hope for packed refs to be present if there are too many refs.

Comparison

This is the criteron output (with the profiler attached) running the git2 after having recorded the gix version.
It's notable that some tests are from 4x to 20x faster in git2 initially.

It turned out all the time was spent reading and re-reading the same commit object, as object caches aren't enabled by default. The second factor was rechecking for packed-refs freshness too often - now it's done once for the operation.

list-branches[many local branches]/no filter
                        time:   [3.9939 ms 4.0238 ms 4.0560 ms]
                        thrpt:  [73.965 Kelem/s 74.556 Kelem/s 75.114 Kelem/s]
                 change:
                        time:   [-81.402% -81.120% -80.849%] (p = 0.00 < 0.05)
                        thrpt:  [+422.17% +429.67% +437.68%]
                        Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
  8 (8.00%) high mild
Benchmarking list-branches[many local branches]/name-filter rejecting all: Collecting 100 samples in estimated 5.0014 s (1200 iteratilist-branches[many local branches]/name-filter rejecting all
                        time:   [4.0703 ms 4.1143 ms 4.1633 ms]
                        thrpt:  [72.059 Kelem/s 72.916 Kelem/s 73.705 Kelem/s]
                 change:
                        time:   [-9.5855% -8.2445% -6.9103%] (p = 0.00 < 0.05)
                        thrpt:  [+7.4233% +8.9853% +10.602%]
                        Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
  1 (1.00%) high mild
  1 (1.00%) high severe

list-branches[tiny repo]/no filter
                        time:   [853.29 µs 857.06 µs 860.89 µs]
                        thrpt:  [3.4848 Kelem/s 3.5003 Kelem/s 3.5158 Kelem/s]
                 change:
                        time:   [+50.118% +52.354% +54.490%] (p = 0.00 < 0.05)
                        thrpt:  [-35.271% -34.364% -33.386%]
                        Performance has regressed.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild
list-branches[tiny repo]/name-filter rejecting all
                        time:   [544.09 µs 546.69 µs 549.36 µs]
                        thrpt:  [5.4609 Kelem/s 5.4876 Kelem/s 5.5138 Kelem/s]
                 change:
                        time:   [-0.9418% +0.0285% +1.0737%] (p = 0.96 > 0.05)
                        thrpt:  [-1.0623% -0.0285% +0.9507%]
                        No change in performance detected.
Found 6 outliers among 100 measurements (6.00%)
  2 (2.00%) low mild
  3 (3.00%) high mild
  1 (1.00%) high severe

list-branches[many local branches [packed]]/no filter
                        time:   [806.29 µs 816.63 µs 828.63 µs]
                        thrpt:  [362.04 Kelem/s 367.37 Kelem/s 372.08 Kelem/s]
                 change:
                        time:   [-95.403% -95.356% -95.306%] (p = 0.00 < 0.05)
                        thrpt:  [+2030.4% +2053.3% +2075.1%]
                        Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
  2 (2.00%) high mild
  2 (2.00%) high severe
Benchmarking list-branches[many local branches [packed]]/name-filter rejecting all: Collecting 100 samples in estimated 6.0408 s (10klist-branches[many local branches [packed]]/name-filter rejecting all
                        time:   [587.43 µs 590.73 µs 594.02 µs]
                        thrpt:  [505.03 Kelem/s 507.85 Kelem/s 510.70 Kelem/s]
                 change:
                        time:   [+16.923% +18.061% +19.170%] (p = 0.00 < 0.05)
                        thrpt:  [-16.086% -15.298% -14.473%]
                        Performance has regressed.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild

Benchmarking list-branches[many local branches [tracked]]/no filter: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 29.1s, or reduce sample count to 10.
list-branches[many local branches [tracked]]/no filter
                        time:   [285.91 ms 286.44 ms 287.06 ms]
                        thrpt:  [2.0901 Kelem/s 2.0947 Kelem/s 2.0985 Kelem/s]
                 change:
                        time:   [+1275.4% +1282.7% +1290.2%] (p = 0.00 < 0.05)
                        thrpt:  [-92.807% -92.768% -92.729%]
                        Performance has regressed.
Found 3 outliers among 100 measurements (3.00%)
  2 (2.00%) high mild
  1 (1.00%) high severe
Benchmarking list-branches[many local branches [tracked]]/name-filter rejecting all: Collecting 100 samples in estimated 5.5596 s (60list-branches[many local branches [tracked]]/name-filter rejecting all
                        time:   [9.1789 ms 9.2784 ms 9.3756 ms]
                        thrpt:  [63.996 Kelem/s 64.666 Kelem/s 65.367 Kelem/s]
                 change:
                        time:   [-6.3340% -5.0751% -3.7357%] (p = 0.00 < 0.05)
                        thrpt:  [+3.8806% +5.3464% +6.7624%]
                        Performance has improved.

Benchmarking list-branches[many local branches [tracked & packed]]/no filter: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 28.0s, or reduce sample count to 10.
Benchmarking list-branches[many local branches [tracked & packed]]/no filter: Collecting 100 samples in estimated 28.018 s (100 iteralist-branches[many local branches [tracked & packed]]/no filter
                        time:   [279.69 ms 280.71 ms 281.98 ms]
                        thrpt:  [2.1278 Kelem/s 2.1375 Kelem/s 2.1452 Kelem/s]
                 change:
                        time:   [+2244.0% +2258.0% +2272.5%] (p = 0.00 < 0.05)
                        thrpt:  [-95.785% -95.759% -95.734%]
                        Performance has regressed.
Found 9 outliers among 100 measurements (9.00%)
  5 (5.00%) high mild
  4 (4.00%) high severe
Benchmarking list-branches[many local branches [tracked & packed]]/name-filter rejecting all: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 7.7s, enable flat sampling, or reduce sample count to 50.
Benchmarking list-branches[many local branches [tracked & packed]]/name-filter rejecting all: Collecting 100 samples in estimated 7.6list-branches[many local branches [tracked & packed]]/name-filter rejecting all
                        time:   [1.5051 ms 1.5118 ms 1.5185 ms]
                        thrpt:  [395.13 Kelem/s 396.89 Kelem/s 398.65 Kelem/s]
                 change:
                        time:   [+58.172% +59.370% +60.574%] (p = 0.00 < 0.05)
                        thrpt:  [-37.723% -37.253% -36.778%]
                        Performance has regressed.
Found 4 outliers among 100 measurements (4.00%)
  2 (2.00%) low mild 
  2 (2.00%) high mild

Here is the 20x run in isolation and with instrumentation, coming from git2 and going to gix.

list-branches[many local branches [packed]]/no filter
                        time:   [18.303 ms 19.898 ms 22.018 ms]
                        thrpt:  [13.626 Kelem/s 15.077 Kelem/s 16.390 Kelem/s]
                 change:
                        time:   [+2190.2% +2371.2% +2669.1%] (p = 0.00 < 0.05)
                        thrpt:  [-96.389% -95.953% -95.634%]
                        Performance has regressed.
Found 13 outliers among 100 measurements (13.00%)
  9 (9.00%) high mild
  4 (4.00%) high severe
Benchmarking list-branches[many local branches [packed]]/name-filter rejecting all: Collecting 100 samples in estimated 5.0540 s (10klist-branches[many local branches [packed]]/name-filter rejecting all
                        time:   [493.38 µs 496.64 µs 499.71 µs]
                        thrpt:  [600.35 Kelem/s 604.06 Kelem/s 608.05 Kelem/s]
                 change:
                        time:   [-16.453% -15.824% -15.143%] (p = 0.00 < 0.05)
                        thrpt:  [+17.846% +18.799% +19.693%]
                        Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high severe

After adding an object cache and reusing packed refs buffers it's much better, but still up to 10% slower in some cases. Let's see if the performance of 20x in the bad git2 case gets worse for us when remote names can be extracted from local tracking branches.

list-branches[many local branches]/no filter
                        time:   [4.6951 ms 5.0242 ms 5.4801 ms]
                        thrpt:  [54.744 Kelem/s 59.711 Kelem/s 63.896 Kelem/s]
                 change:
                        time:   [+0.1315% +12.645% +27.003%] (p = 0.04 < 0.05)
                        thrpt:  [-21.262% -11.225% -0.1313%]
                        Change within noise threshold.
Found 4 outliers among 100 measurements (4.00%)
  2 (2.00%) high mild
  2 (2.00%) high severe
Benchmarking list-branches[many local branches]/name-filter rejecting all: Collecting 100 samples in estimated 5.3305 s (1300 iteratilist-branches[many local branches]/name-filter rejecting all
                        time:   [4.0972 ms 4.1088 ms 4.1217 ms]
                        thrpt:  [72.785 Kelem/s 73.014 Kelem/s 73.220 Kelem/s]
                 change:
                        time:   [+10.886% +11.347% +11.833%] (p = 0.00 < 0.05)
                        thrpt:  [-10.581% -10.190% -9.8176%]
                        Performance has regressed.
Found 14 outliers among 100 measurements (14.00%)
  9 (9.00%) high mild
  5 (5.00%) high severe

list-branches[tiny repo]/no filter
                        time:   [518.92 µs 520.24 µs 521.71 µs]
                        thrpt:  [5.7504 Kelem/s 5.7666 Kelem/s 5.7812 Kelem/s]
                 change:
                        time:   [-35.568% -35.326% -35.073%] (p = 0.00 < 0.05)
                        thrpt:  [+54.018% +54.622% +55.203%]
                        Performance has improved.
Found 6 outliers among 100 measurements (6.00%)
  1 (1.00%) high mild
  5 (5.00%) high severe
list-branches[tiny repo]/name-filter rejecting all
                        time:   [517.98 µs 519.15 µs 520.47 µs]
                        thrpt:  [5.7640 Kelem/s 5.7787 Kelem/s 5.7918 Kelem/s]
                 change:
                        time:   [+1.6697% +2.0104% +2.3665%] (p = 0.00 < 0.05)
                        thrpt:  [-2.3118% -1.9708% -1.6423%]
                        Performance has regressed.
Found 12 outliers among 100 measurements (12.00%)
  5 (5.00%) high mild
  7 (7.00%) high severe

list-branches[many local branches [packed]]/no filter
                        time:   [798.65 µs 801.38 µs 804.44 µs]
                        thrpt:  [372.93 Kelem/s 374.35 Kelem/s 375.64 Kelem/s]
                 change:
                        time:   [+7.3188% +8.1308% +8.8618%] (p = 0.00 < 0.05)
                        thrpt:  [-8.1404% -7.5194% -6.8197%]
                        Performance has regressed.
Found 11 outliers among 100 measurements (11.00%)
  1 (1.00%) low mild
  8 (8.00%) high mild
  2 (2.00%) high severe
Benchmarking list-branches[many local branches [packed]]/name-filter rejecting all: Collecting 100 samples in estimated 7.1028 s (15klist-branches[many local branches [packed]]/name-filter rejecting all
                        time:   [467.18 µs 468.33 µs 469.65 µs]
                        thrpt:  [638.77 Kelem/s 640.57 Kelem/s 642.15 Kelem/s]
                 change:
                        time:   [-16.433% -15.821% -15.217%] (p = 0.00 < 0.05)
                        thrpt:  [+17.948% +18.794% +19.665%]
                        Performance has improved.
Found 13 outliers among 100 measurements (13.00%)
  5 (5.00%) high mild
  8 (8.00%) high severe

list-branches[many local branches [tracked]]/no filter
                        time:   [8.9093 ms 8.9516 ms 8.9987 ms]
                        thrpt:  [66.676 Kelem/s 67.027 Kelem/s 67.345 Kelem/s]
                 change:
                        time:   [-96.796% -96.781% -96.764%] (p = 0.00 < 0.05)
                        thrpt:  [+2990.5% +3006.5% +3021.5%]
                        Performance has improved.
Found 14 outliers among 100 measurements (14.00%)
  4 (4.00%) high mild
  10 (10.00%) high severe
Benchmarking list-branches[many local branches [tracked]]/name-filter rejecting all: Collecting 100 samples in estimated 5.1016 s (60list-branches[many local branches [tracked]]/name-filter rejecting all
                        time:   [8.4425 ms 8.4848 ms 8.5339 ms]
                        thrpt:  [70.308 Kelem/s 70.715 Kelem/s 71.069 Kelem/s]
                 change:
                        time:   [+2.3070% +3.1059% +3.9172%] (p = 0.00 < 0.05)
                        thrpt:  [-3.7695% -3.0124% -2.2550%]
                        Performance has regressed.
Found 14 outliers among 100 measurements (14.00%)
  4 (4.00%) high mild
  10 (10.00%) high severe

Benchmarking list-branches[many local branches [tracked & packed]]/no filter: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 6.8s, enable flat sampling, or reduce sample count to 60.
Benchmarking list-branches[many local branches [tracked & packed]]/no filter: Collecting 100 samples in estimated 6.8154 s (5050 iterlist-branches[many local branches [tracked & packed]]/no filter
                        time:   [1.3169 ms 1.3252 ms 1.3351 ms]
                        thrpt:  [449.39 Kelem/s 452.76 Kelem/s 455.61 Kelem/s]
                 change:
                        time:   [-99.511% -99.505% -99.499%] (p = 0.00 < 0.05)
                        thrpt:  [+19848% +20119% +20335%]
                        Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
  2 (2.00%) high mild
  5 (5.00%) high severe
Benchmarking list-branches[many local branches [tracked & packed]]/name-filter rejecting all: Collecting 100 samples in estimated 8.8list-branches[many local branches [tracked & packed]]/name-filter rejecting all
                        time:   [872.31 µs 877.61 µs 882.88 µs]
                        thrpt:  [679.59 Kelem/s 683.67 Kelem/s 687.83 Kelem/s]
                 change:
                        time:   [-38.986% -38.602% -38.215%] (p = 0.00 < 0.05)
                        thrpt:  [+61.852% +62.873% +63.897%]
                        Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
  1 (1.00%) high mild
  1 (1.00%) high severe

Finally, a run going from gix to git2 but with core.precomposeUnicode enabled. gix looses quite a bit of performance then.

list-branches[many local branches]/no filter
                        time:   [4.3731 ms 4.6555 ms 5.0195 ms]
                        thrpt:  [59.767 Kelem/s 64.440 Kelem/s 68.601 Kelem/s]
                 change:
                        time:   [-5.5547% +0.6503% +8.8473%] (p = 0.88 > 0.05)
                        thrpt:  [-8.1282% -0.6461% +5.8814%]
                        No change in performance detected.
Found 3 outliers among 100 measurements (3.00%)
  3 (3.00%) high severe
Benchmarking list-branches[many local branches]/name-filter rejecting all: Collecting 100 samples in estimated 5.0987 s (1400 iteratilist-branches[many local branches]/name-filter rejecting all
                        time:   [3.6591 ms 3.6841 ms 3.7148 ms]
                        thrpt:  [80.759 Kelem/s 81.431 Kelem/s 81.987 Kelem/s]
                 change:
                        time:   [-15.107% -14.055% -12.957%] (p = 0.00 < 0.05)
                        thrpt:  [+14.886% +16.353% +17.796%]
                        Performance has improved.
Found 16 outliers among 100 measurements (16.00%)
  10 (10.00%) high mild
  6 (6.00%) high severe

list-branches[tiny repo]/no filter
                        time:   [809.92 µs 811.54 µs 813.35 µs]
                        thrpt:  [3.6884 Kelem/s 3.6967 Kelem/s 3.7041 Kelem/s]
                 change:
                        time:   [+48.732% +49.885% +50.971%] (p = 0.00 < 0.05)
                        thrpt:  [-33.762% -33.282% -32.765%]
                        Performance has regressed.
Found 7 outliers among 100 measurements (7.00%)
  5 (5.00%) high mild
  2 (2.00%) high severe
list-branches[tiny repo]/name-filter rejecting all
                        time:   [514.74 µs 516.26 µs 518.13 µs]
                        thrpt:  [5.7901 Kelem/s 5.8110 Kelem/s 5.8282 Kelem/s]
                 change:
                        time:   [-3.9443% -3.2164% -2.5378%] (p = 0.00 < 0.05)
                        thrpt:  [+2.6039% +3.3233% +4.1062%]
                        Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
  5 (5.00%) high mild

list-branches[many local branches [packed]]/no filter
                        time:   [741.43 µs 744.00 µs 746.54 µs]
                        thrpt:  [401.85 Kelem/s 403.23 Kelem/s 404.62 Kelem/s]
                 change:
                        time:   [-8.8907% -8.1606% -7.4633%] (p = 0.00 < 0.05)
                        thrpt:  [+8.0652% +8.8857% +9.7583%]
                        Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
  4 (4.00%) high mild
  3 (3.00%) high severe
Benchmarking list-branches[many local branches [packed]]/name-filter rejecting all: Collecting 100 samples in estimated 5.5949 s (10klist-branches[many local branches [packed]]/name-filter rejecting all
                        time:   [550.22 µs 551.25 µs 552.40 µs]
                        thrpt:  [543.08 Kelem/s 544.22 Kelem/s 545.23 Kelem/s]
                 change:
                        time:   [+11.743% +12.698% +13.644%] (p = 0.00 < 0.05)
                        thrpt:  [-12.006% -11.267% -10.509%]
                        Performance has regressed.
Found 8 outliers among 100 measurements (8.00%)
  7 (7.00%) high mild
  1 (1.00%) high severe

Benchmarking list-branches[many local branches [tracked]]/no filter: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 28.0s, or reduce sample count to 10.
list-branches[many local branches [tracked]]/no filter
                        time:   [276.61 ms 276.90 ms 277.20 ms]
                        thrpt:  [2.1645 Kelem/s 2.1669 Kelem/s 2.1691 Kelem/s]
                 change:
                        time:   [+2749.9% +2777.0% +2803.2%] (p = 0.00 < 0.05)
                        thrpt:  [-96.555% -96.524% -96.491%]
                        Performance has regressed.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild
Benchmarking list-branches[many local branches [tracked]]/name-filter rejecting all: Collecting 100 samples in estimated 5.7763 s (70list-branches[many local branches [tracked]]/name-filter rejecting all
                        time:   [8.1693 ms 8.2156 ms 8.2679 ms]
                        thrpt:  [72.570 Kelem/s 73.032 Kelem/s 73.446 Kelem/s]
                 change:
                        time:   [-10.791% -9.7886% -8.7425%] (p = 0.00 < 0.05)
                        thrpt:  [+9.5800% +10.851% +12.096%]
                        Performance has improved.
Found 12 outliers among 100 measurements (12.00%)
  7 (7.00%) high mild
  5 (5.00%) high severe

Benchmarking list-branches[many local branches [tracked & packed]]/no filter: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 27.0s, or reduce sample count to 10.
Benchmarking list-branches[many local branches [tracked & packed]]/no filter: Collecting 100 samples in estimated 27.003 s (100 iteralist-branches[many local branches [tracked & packed]]/no filter
                        time:   [269.66 ms 270.02 ms 270.43 ms]
                        thrpt:  [2.2187 Kelem/s 2.2221 Kelem/s 2.2250 Kelem/s]
                 change:
                        time:   [+19464% +19600% +19729%] (p = 0.00 < 0.05)
                        thrpt:  [-99.496% -99.492% -99.489%]
                        Performance has regressed.
Found 4 outliers among 100 measurements (4.00%)
  2 (2.00%) high mild
  2 (2.00%) high severe
Benchmarking list-branches[many local branches [tracked & packed]]/name-filter rejecting all: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 7.1s, enable flat sampling, or reduce sample count to 50.
Benchmarking list-branches[many local branches [tracked & packed]]/name-filter rejecting all: Collecting 100 samples in estimated 7.0list-branches[many local branches [tracked & packed]]/name-filter rejecting all
                        time:   [1.3909 ms 1.3935 ms 1.3966 ms]
                        thrpt:  [429.61 Kelem/s 430.56 Kelem/s 431.38 Kelem/s]
                 change:
                        time:   [+56.323% +57.769% +59.095%] (p = 0.00 < 0.05)
                        thrpt:  [-37.144% -36.616% -36.030%]
                        Performance has regressed.
Found 12 outliers among 100 measurements (12.00%)
  4 (4.00%) high mild
  8 (8.00%) high severe

[vercel]

@Byron is attempting to deploy a commit to the GitButler Team on Vercel.

A member of the Team first needs to authorize it.

[krlvi]

awesome, love seeing these performance improvements!