perf(git): speed up CLI backend startup

[idriss-mortadi]

Closes #560.

Context

backend = "cli" already exists. It selects native Git for all Git operations. This PR improves startup when that setting is enabled. The default libgit2 backend is unchanged.

Problem

The CLI backend runs separate probes for staged, tracked unstaged, and untracked files. Untracked discovery is slow in large repositories.

Measured in a large monorepo:

• Default libgit2 startup: 22.8s
• Existing CLI backend startup: 9.42s

Change

Standard Git repositories now use one machine-readable probe:

git status --porcelain=v1 -z --untracked-files=normal

The parser folds Git's two status columns into staged and unstaged booleans. It also handles the extra pathname emitted for renames and copies.

--untracked-files=normal is explicit. Without it, status.showUntrackedFiles=no can hide an untracked-only worktree. tuicr would then hide the Unstaged changes row. An integration test covers this repository configuration.

Sparse checkout and sparse index repositories keep the existing scoped probes. Plain git status can report untracked files outside the sparse cone.

Git already applies .gitignore during status. tuicr only enumerates paths when .tuicrignore adds application-specific rules. This condition lives in generic app code. Today, only GitCliBackend returns a successful cheap status result, so only that backend takes the new path. The libgit2, Jujutsu, and Mercurial behavior remains unchanged.

Result

With backend = "cli":

• Before: 9.42s
• After: 1.19s

Recording

The recording uses a synthetic repository with 1,000,001 tracked files. It contains no private source or repository metadata. Both runs use a warm filesystem cache and backend = "cli".

The recorded synthetic benchmark measured 5.96s before and 339ms after.

reproduce the synthetic benchmark

this script checks required tools, commit availability, free disk space, and platform support. it builds both revisions and creates a temporary synthetic repository. it removes all temporary files when it exits.

#!/usr/bin/env bash
set -euo pipefail

base=8323f135018ab0c1d07e228a423bd504c3503593
head=589eaad6755805a7f363e5a65e48002173f7a4e6

for command in git cargo python3 script; do
  command -v "$command" >/dev/null || {
    echo "missing required command: $command" >&2
    exit 1
  }
done

[[ $(uname -s) == Darwin ]] || {
  echo "this recording script currently expects macOS script(1)" >&2
  exit 1
}

root=$(git rev-parse --show-toplevel 2>/dev/null) || {
  echo "run this from a tuicr checkout" >&2
  exit 1
}

grep -q '^name = "tuicr"$' "$root/Cargo.toml" || {
  echo "run this from a tuicr checkout" >&2
  exit 1
}

available_kb=$(df -Pk /tmp | awk 'NR == 2 {print $4}')
(( available_kb >= 8 * 1024 * 1024 )) || {
  echo "at least 8 GiB free under /tmp is required" >&2
  exit 1
}

if ! git -C "$root" cat-file -e "$base^{commit}" 2>/dev/null ||
   ! git -C "$root" cat-file -e "$head^{commit}" 2>/dev/null; then
  git -C "$root" fetch https://github.com/agavra/tuicr.git \
    main refs/pull/568/head
fi

for commit in "$base" "$head"; do
  git -C "$root" cat-file -e "$commit^{commit}" 2>/dev/null || {
    echo "could not fetch required commit: $commit" >&2
    exit 1
  }
done

work=$(mktemp -d /tmp/tuicr-pr-568-benchmark.XXXXXX)
cleanup() {
  git -C "$root" worktree remove --force "$work/original" >/dev/null 2>&1 || true
  git -C "$root" worktree remove --force "$work/improved" >/dev/null 2>&1 || true
  rm -rf "$work"
}
trap cleanup EXIT

git -C "$root" worktree add --detach "$work/original" "$base"
git -C "$root" worktree add --detach "$work/improved" "$head"
CARGO_TARGET_DIR="$work/original-target" cargo build --release --manifest-path "$work/original/Cargo.toml"
CARGO_TARGET_DIR="$work/improved-target" cargo build --release --manifest-path "$work/improved/Cargo.toml"

repo="$work/repo"
config="$work/config"
data="$work/data"
mkdir -p "$repo" "$config/tuicr" "$data"
printf 'backend = "cli"\nno_update_check = true\n' > "$config/tuicr/config.toml"

git -C "$repo" init -q
git -C "$repo" config user.name 'tuicr benchmark'
git -C "$repo" config user.email benchmark@example.com
git -C "$repo" config core.untrackedCache true

python3 - "$repo" <<'PY'
from pathlib import Path
import sys

root = Path(sys.argv[1])
for directory in range(5_000):
    folder = root / "src" / f"package-{directory:04d}"
    folder.mkdir(parents=True)
    for filename in range(200):
        (folder / f"file-{filename:03d}.txt").write_text("benchmark\n")
(root / ".gitignore").write_text("build/\n")
(root / "untracked.txt").write_text("visible untracked file\n")
PY

git -C "$repo" add .gitignore src
git -C "$repo" commit -qm 'synthetic million-file repository'
git -C "$repo" status --porcelain=v1 >/dev/null

original="$work/original-target/release/tuicr"
improved="$work/improved-target/release/tuicr"

prewarm() {
  local binary=$1
  (
    cd "$repo"
    printf q | script -q /dev/null env \
      XDG_CONFIG_HOME="$config" \
      XDG_DATA_HOME="$data" \
      "$binary" >/dev/null 2>&1
  )
}

run_case() {
  local label=$1
  local binary=$2
  local log=$3

  clear
  printf '\n  tuicr cli startup benchmark\n\n'
  printf '  synthetic repository: 1,000,001 tracked files\n'
  printf '  warm filesystem cache\n'
  printf '  version: %s\n\n' "$label"
  printf '  opening tuicr...\n'
  sleep 1

  (
    cd "$repo"
    XDG_CONFIG_HOME="$config" \
    XDG_DATA_HOME="$data" \
    TUICR_PROFILE="$log" \
    "$binary"
  )

  clear
  elapsed=$(grep 'operation="startup.app_init"}: close' "$log" | tail -1 | sed -E 's/.*time.busy=([^ ]+).*/\1/')
  printf '\n  %s startup: %s\n\n' "$label" "$elapsed"
}

prewarm "$original"
prewarm "$improved"
run_case 'before this pr' "$original" "$work/before.log"
printf '  press enter to run the improved version'
read -r
run_case 'after this pr' "$improved" "$work/after.log"
printf '  press enter to finish'
read -r

Validation

• cargo test: 1300 passed, 2 ignored
• cargo check --all-targets: passed
• Sparse checkout status tests: passed
• status.showUntrackedFiles=no integration test: passed

cargo clippy --all-targets -- -D warnings remains blocked by the existing items_after_test_module finding in src/handler.rs:577.

[agavra]

LGTM! One question inline

[agavra]

any reason why we have to force v1? seems like Claude thinks this hsould work with git v2 clients as well

[idriss-mortadi]

v1 is the output format. it is not the git client version. git 2 clients support this format.

we only need the two status columns from v1. the parser reads those first two bytes. v2 has a different and richer record format that we do not need here.

--porcelain defaults to v1 today. writing =v1 makes the parser contract explicit.

[idriss-mortadi]

Also, git status 's porcelain flag defaults to v1. See https://git-scm.com/docs/git-status#Documentation/git-status.txt---porcelainversion