Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 47 additions & 0 deletions home-manager/services/dolt/backup-dolt-main.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
#!/usr/bin/env bash
# @git@, @mirrorDir@, @remoteUrl@, @userEmail@ are substituted by pkgs.replaceVars.
# `bd` is resolved via PATH (user-installed in ~/.local/bin).
#
# Triggered by launchd/systemd when the dolt manifest changes. Exports the
# full beads_global database to JSONL and pushes it to refs/heads/main of
# the beads GitHub repo so the data renders in the GitHub UI. Dolt's native
# git+https push only writes refs/dolt/data, which GitHub does not render.

set -euo pipefail

MIRROR="@mirrorDir@"
REMOTE="@remoteUrl@"
GIT="@git@/bin/git"
USER_EMAIL="@userEmail@"

export PATH="/etc/profiles/per-user/${USER}/bin:${HOME}/.nix-profile/bin:${HOME}/.local/bin:/run/current-system/sw/bin:/usr/bin:/bin"

if ! command -v bd >/dev/null 2>&1; then
echo "bd not on PATH; cannot export beads database" >&2
exit 1
fi

mkdir -p "$(dirname "${MIRROR}")"

if [ ! -d "${MIRROR}/.git" ]; then
"${GIT}" clone --depth 1 "${REMOTE}" "${MIRROR}"
fi

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Latent: mirror never picks up a changed remoteUrl.

The git clone is gated by [ ! -d "${MIRROR}/.git" ], and nothing later calls git remote set-url. Because the mirror lives in ~/.cache/beads-jsonl-mirror (persisted across home-manager rebuilds), any future change to remoteUrl in default.nix (rename, fork, switch to SSH) will be silently ignored — fetch/push keep hitting whatever URL is baked into the existing .git/config.

Trigger condition: changing remoteUrl in home-manager/services/dolt/default.nix while the cache directory already exists.
Consequence: push silently goes to the wrong repo or fails with a 404/permission error until the user manually rm -rf ~/.cache/beads-jsonl-mirror.

Fix: make it idempotent by always reconciling the URL after the clone gate, e.g.:

"${GIT}" -C "${MIRROR}" remote set-url origin "${REMOTE}"
Suggested change
fi
if [ ! -d "${MIRROR}/.git" ]; then
"${GIT}" clone --depth 1 "${REMOTE}" "${MIRROR}"
fi
"${GIT}" -C "${MIRROR}" remote set-url origin "${REMOTE}"


cd "${MIRROR}"
"${GIT}" fetch --depth 1 origin main
"${GIT}" reset --hard origin/main

bd --global export --all >issues.jsonl.tmp
mv issues.jsonl.tmp issues.jsonl

if "${GIT}" diff --quiet -- issues.jsonl; then
echo "no JSONL changes; skipping commit"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First-run bug: bootstrap commit is silently skipped.

At this point issues.jsonl is still untracked in the mirror — we've only mv'd it into the working tree. git diff --quiet -- issues.jsonl only reports differences for tracked files, so on an untracked file it returns 0 (no diff). The script logs no JSONL changes; skipping commit and exits 0 without ever running git add / git commit / git push, so the mirror never gets seeded.

Reproducer:

cd $(mktemp -d) && git init -q && git commit --allow-empty -m init -q
echo hi > issues.jsonl
git diff --quiet -- issues.jsonl; echo $?   # prints 0

Since the script's first effective run on any fresh machine starts with a mirror that doesn't yet contain issues.jsonl (we just reset --hard origin/main), this is the case that always hits. Subsequent runs are also stuck because git add never runs.

Fix: stage first, then check the index:

"${GIT}" add issues.jsonl
if "${GIT}" diff --cached --quiet -- issues.jsonl; then
  echo "no JSONL changes; skipping commit"
  exit 0
fi

Verified: after git add on the untracked file, git diff --cached --quiet -- issues.jsonl exits 1, so the commit/push path runs.

Suggested change
echo "no JSONL changes; skipping commit"
"${GIT}" add issues.jsonl
if "${GIT}" diff --cached --quiet -- issues.jsonl; then
echo "no JSONL changes; skipping commit"
exit 0
fi
"${GIT}" \

exit 0
fi

"${GIT}" add issues.jsonl
"${GIT}" \
-c "user.name=beads-backup" \
-c "user.email=${USER_EMAIL}" \
commit -m "chore(beads): refresh issues.jsonl snapshot"
"${GIT}" push origin main
62 changes: 62 additions & 0 deletions home-manager/services/dolt/default.nix
Original file line number Diff line number Diff line change
Expand Up @@ -10,13 +10,30 @@ let
homeDir = config.home.homeDirectory;
repoDir = "${homeDir}/dotfiles";
beadsDir = "${repoDir}/.beads";
doltManifest = "${beadsDir}/beads_global/.dolt/noms/manifest";
mirrorDir = "${homeDir}/.cache/beads-jsonl-mirror";
remoteUrl = "https://github.com/shunkakinoki/beads";
userEmail = "shunkakinoki@gmail.com";
enabled = isKyber || isGalactica;
# 1.86+ adds the git+https:// remote scheme used by the beads_global GitHub backup.
doltMinVersion = "1.86";
startScript = pkgs.replaceVars ./start.sh {
inherit beadsDir;
inherit (pkgs) dolt;
};
backupScript = pkgs.replaceVars ./backup-dolt-main.sh {
inherit mirrorDir remoteUrl userEmail;
inherit (pkgs) git;
};
in
lib.mkIf enabled {
assertions = [
{
assertion = lib.versionAtLeast pkgs.dolt.version doltMinVersion;
message = "pkgs.dolt is ${pkgs.dolt.version}; needs >= ${doltMinVersion} for git+https push (beads_global GitHub backup). Run 'nix flake update nixpkgs-unstable'.";
Comment thread
indent-zero[bot] marked this conversation as resolved.
Resolved
}
];

home.sessionVariables = {
BEADS_DOLT_SHARED_SERVER = "1";
BEADS_DOLT_SERVER_PORT = "3307";
Expand All @@ -37,6 +54,27 @@ lib.mkIf enabled {
};
};

# Mirror the live beads_global DB to refs/heads/main as JSONL so the data
# is visible in the GitHub UI (Dolt's native push only writes refs/dolt/data).
# Triggered by manifest changes inside dolt's noms store; throttled to avoid
# hammering on rapid writes.
launchd.agents.dolt-backup-main = lib.mkIf pkgs.stdenv.isDarwin {
enable = true;
config = {
ProgramArguments = [
"${pkgs.bash}/bin/bash"
"${backupScript}"
];
WatchPaths = [ doltManifest ];
ThrottleInterval = 60;
RunAtLoad = false;
KeepAlive = false;
WorkingDirectory = repoDir;
StandardOutPath = "/tmp/dolt-backup-main.log";
StandardErrorPath = "/tmp/dolt-backup-main.error.log";
};
};

systemd.user.services.dolt = lib.mkIf pkgs.stdenv.isLinux {
Unit = {
Description = "Dolt SQL server for dotfiles beads";
Expand All @@ -53,4 +91,28 @@ lib.mkIf enabled {
WantedBy = [ "default.target" ];
};
};

systemd.user.services.dolt-backup-main = lib.mkIf pkgs.stdenv.isLinux {
Unit = {
Description = "Push beads_global JSONL snapshot to GitHub main";
};
Service = {
Type = "oneshot";
ExecStart = "${pkgs.bash}/bin/bash ${backupScript}";
WorkingDirectory = repoDir;
};
};

systemd.user.paths.dolt-backup-main = lib.mkIf pkgs.stdenv.isLinux {
Unit = {
Description = "Watch dolt manifest and trigger JSONL backup";
};
Path = {
PathChanged = doltManifest;
Unit = "dolt-backup-main.service";
};
Install = {
WantedBy = [ "default.target" ];
};
};
}
102 changes: 102 additions & 0 deletions spec/backup_dolt_main_spec.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
#!/usr/bin/env bash
# shellcheck disable=SC2329

Describe 'home-manager/services/dolt/backup-dolt-main.sh'
SCRIPT="$PWD/home-manager/services/dolt/backup-dolt-main.sh"

Describe 'script properties'
It 'uses bash shebang'
When run bash -c "head -1 '$SCRIPT'"
The output should include '#!/usr/bin/env bash'
End

It 'uses strict mode'
When run bash -c "grep 'set -euo pipefail' '$SCRIPT'"
The output should include 'set -euo pipefail'
End

It 'passes bash syntax check after stripping placeholders'
When run bash -c "sed 's|@[A-Za-z_][A-Za-z0-9_]*@|/usr|g' '$SCRIPT' | bash -n"
The status should be success
End
End

Describe 'placeholder substitutions'
It 'references @git@'
When run bash -c "grep '@git@' '$SCRIPT'"
The output should include '@git@'
End

It 'references @mirrorDir@'
When run bash -c "grep '@mirrorDir@' '$SCRIPT'"
The output should include '@mirrorDir@'
End

It 'references @remoteUrl@'
When run bash -c "grep '@remoteUrl@' '$SCRIPT'"
The output should include '@remoteUrl@'
End

It 'references @userEmail@'
When run bash -c "grep '@userEmail@' '$SCRIPT'"
The output should include '@userEmail@'
End
End

Describe 'preflight checks'
It 'requires bd on PATH'
When run bash -c "grep 'command -v bd' '$SCRIPT'"
The output should include 'command -v bd'
End

It 'extends PATH for Nix profiles and user-local bins'
When run bash -c "grep 'export PATH=' '$SCRIPT'"
The output should include '.nix-profile/bin'
End
End

Describe 'mirror checkout management'
It 'creates the mirror parent directory'
When run bash -c "grep 'mkdir -p' '$SCRIPT'"
The output should include 'mkdir -p'
End

It 'clones the remote when the mirror is missing'
When run bash -c "grep 'clone --depth 1' '$SCRIPT'"
The output should include 'clone --depth 1'
End

It 'resets the mirror to origin/main before exporting'
When run bash -c "grep 'reset --hard origin/main' '$SCRIPT'"
The output should include 'reset --hard origin/main'
End
End

Describe 'export and push'
It 'exports the full beads dataset'
When run bash -c "grep -- 'bd --global export --all' '$SCRIPT'"
The output should include 'bd --global export --all'
End

It 'writes to a temporary file before renaming'
When run bash -c "grep 'issues.jsonl.tmp' '$SCRIPT'"
The output should include 'issues.jsonl.tmp'
End

It 'skips commit when JSONL is unchanged'
When run bash -c "grep -- 'diff --quiet' '$SCRIPT'"
The output should include 'diff --quiet'
End

It 'commits with a chore prefix'
When run bash -c "grep -- 'chore(beads)' '$SCRIPT'"
The output should include 'chore(beads)'
End

It 'pushes to origin main'
When run bash -c "grep 'push origin main' '$SCRIPT'"
The output should include 'push origin main'
End
End

End
5 changes: 5 additions & 0 deletions spec/coverage_spec.sh
Original file line number Diff line number Diff line change
Expand Up @@ -201,6 +201,10 @@ It 'has spec file for home-manager/services/dolt/start.sh'
The path "spec/dolt_start_spec.sh" should be exist
End

It 'has spec file for home-manager/services/dolt/backup-dolt-main.sh'
The path "spec/backup_dolt_main_spec.sh" should be exist
End

It 'has spec file for home-manager/services/k3s/activate.sh'
The path "spec/k3s_service_activate_spec.sh" should be exist
End
Expand Down Expand Up @@ -438,6 +442,7 @@ home-manager/services/docker-postgres/start-postgres.sh
home-manager/services/docker/docker-setup.sh
home-manager/services/docker/setup-docker.sh
home-manager/services/dolt/start.sh
home-manager/services/dolt/backup-dolt-main.sh
home-manager/services/dotfiles-updater/update.sh
home-manager/services/gas-town/start.sh
home-manager/services/k3s/activate.sh
Expand Down
Loading