Skip to content

feat: team-lead liveness checks, inbox watcher daemon, CI coverage gate - #645

Merged
namastex888 merged 2 commits into
devfrom
feat/genie-hardening
Mar 17, 2026
Merged

namastex888 merged 2 commits into
devfrom
feat/genie-hardening

Conversation

@namastex888

Copy link
Copy Markdown
Contributor

Summary

Fixes #531, #526, partial #574 (Phase 1-2).

  • Team-lead liveness check: isTeamActive() now verifies process liveness via isPaneAlive(), not just tmux window existence. 30s grace period prevents false negatives during startup.
  • Agent registry tracking: Team-lead pane ID stored in agent-registry for tracking and auto-respawn. New saveTeamLeadEntry()/getTeamLeadEntry() helpers.
  • Inbox watcher daemon: New inbox-watcher.ts polls ~/.claude/teams/*/inboxes/team-lead.json every 30s. Auto-spawns offline team-leads on unread messages. 3-attempt backoff prevents crash loops.
  • CI coverage gate: Enforces 68% minimum line coverage (currently at 70.57%). Graceful degradation if coverage output is unparseable.

Files Changed (8 files, +772/-40)

  • src/lib/team-auto-spawn.ts — Liveness check + pane ID storage
  • src/lib/agent-registry.ts — Team-lead entry helpers
  • src/lib/team-auto-spawn.test.ts — 5 new liveness/registry tests
  • src/lib/inbox-watcher.ts — NEW: inbox polling daemon
  • src/lib/inbox-watcher.test.ts — NEW: 5 watcher unit tests
  • src/lib/claude-native-teams.ts — listTeamsWithUnreadInbox() helper
  • .github/workflows/ci.yml — Coverage threshold enforcement
  • knip.json — Updated ignore list

Test plan

  • 773 tests pass, 0 failures
  • Typecheck clean
  • Lint clean (1 pre-existing warning in team-manager.ts)
  • Dead code check clean
  • CI passes on PR

Test added 2 commits March 17, 2026 13:08
- Wire isPaneAlive() into isTeamActive() so dead team-lead processes are detected
- Store team-lead pane ID in agent-registry for tracking and auto-respawn
- Add 30s grace period to prevent false negatives during slow startup
- New inbox-watcher daemon polls native inboxes every 30s, spawns offline team-leads
- Backoff after 3 failed spawn attempts to prevent crash loops
- CI coverage gate enforces 68% minimum line coverage threshold
- Add saveTeamLeadEntry/getTeamLeadEntry to agent-registry
- Add listTeamsWithUnreadInbox to claude-native-teams
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly improves the robustness and automation of team-lead management by introducing liveness checks, persistent tracking of team-lead processes, and an inbox-driven auto-respawn mechanism. It also strengthens code quality by enforcing a minimum test coverage in the continuous integration pipeline.

Highlights

  • Team-lead Liveness Checks: Enhanced isTeamActive() to verify process liveness using isPaneAlive() with a 30-second grace period to prevent false negatives during startup.
  • Agent Registry Tracking: Implemented storage of team-lead pane IDs in the agent registry, enabling better tracking and auto-respawn capabilities through new saveTeamLeadEntry() and getTeamLeadEntry() helpers.
  • Inbox Watcher Daemon: Introduced a new inbox-watcher.ts daemon that periodically polls team-lead inboxes for unread messages and automatically respawns offline team-leads, incorporating a 3-attempt backoff mechanism to prevent crash loops.
  • CI Coverage Gate: Established a minimum line coverage threshold of 68% in CI, with graceful degradation for unparseable coverage outputs, to maintain code quality.
Changelog
  • .github/workflows/ci.yml
    • Enforced a minimum code coverage threshold.
  • knip.json
    • Updated the ignore list to include inbox-watcher.ts.
  • src/lib/agent-registry.ts
    • Added saveTeamLeadEntry and getTeamLeadEntry functions for managing team-lead entries.
  • src/lib/claude-native-teams.ts
    • Introduced listTeamsWithUnreadInbox to scan for teams with unread messages.
  • src/lib/inbox-watcher.test.ts
    • Added new unit tests for the inbox watcher daemon.
  • src/lib/inbox-watcher.ts
    • Implemented the new inbox watcher daemon for polling and auto-spawning team-leads.
  • src/lib/team-auto-spawn.test.ts
    • Expanded tests for isTeamActive and ensureTeamLead, including liveness and registry interactions.
  • src/lib/team-auto-spawn.ts
    • Refactored isTeamActive to include pane liveness and a grace period, and updated ensureTeamLead to use the agent registry and handle stale windows.
  • src/term-commands/ship.test.ts
    • Modified git initialization command to specify the main branch.
Ignored Files
  • Ignored by pattern: .github/workflows/** (1)
    • .github/workflows/ci.yml
Activity
  • All 773 tests passed without failures.
  • Typechecking completed cleanly.
  • Linting passed, with one pre-existing warning noted in team-manager.ts.
  • Dead code checks completed cleanly.
  • CI pipeline is expected to pass on this pull request.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces several significant features, including a liveness check for team leads, an inbox watcher daemon for auto-spawning, and agent registry tracking. The implementation is generally robust, leveraging dependency injection for testability which is a great practice. I've identified a couple of areas for performance improvement by parallelizing sequential operations. More critically, there's a bug in team-auto-spawn.ts where incorrect sanitization logic is used to locate tmux windows, which could cause liveness checks and stale window cleanup to fail. My review includes suggestions to fix this and to improve performance.

Comment on lines 119 to +120
const sanitized = sanitizeTeamName(teamName);
return windows.some((w) => w.name === sanitized || w.name === teamName);
const matchingWindow = windows.find((w) => w.name === sanitized || w.name === teamName);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

There's a potential bug in how the tmux window is located. The code uses sanitizeTeamName to find the window, but windows are created using sanitizeWindowName. These two functions have different logic (e.g., sanitizeTeamName lowercases, sanitizeWindowName does not), which can lead to isTeamActive failing to find an existing window for team names with uppercase letters or periods. You should consistently use sanitizeWindowName for finding tmux windows.

Suggested change
const sanitized = sanitizeTeamName(teamName);
return windows.some((w) => w.name === sanitized || w.name === teamName);
const matchingWindow = windows.find((w) => w.name === sanitized || w.name === teamName);
const windowName = sanitizeWindowName(teamName);
const matchingWindow = windows.find((w) => w.name === windowName);

Comment on lines +169 to +170
const sanitized = sanitizeTeamName(teamName);
const staleWindow = windows.find((w) => w.name === sanitized || w.name === teamName || w.name === windowName);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Similar to the issue in isTeamActive, the logic to find a stale window is overly complex and uses sanitizeTeamName, which can be incorrect. It should be simplified to only use sanitizeWindowName, which is how window names are created.

Suggested change
const sanitized = sanitizeTeamName(teamName);
const staleWindow = windows.find((w) => w.name === sanitized || w.name === teamName || w.name === windowName);
const staleWindow = windows.find((w) => w.name === windowName);

Comment on lines +344 to +378
const results: Array<{ teamName: string; unreadCount: number; workingDir: string | null }> = [];

for (const name of teamDirs) {
// Read inbox messages
const inboxFile = join(base, name, 'inboxes', 'team-lead.json');
let messages: NativeInboxMessage[];
try {
const content = await readFile(inboxFile, 'utf-8');
messages = JSON.parse(content);
} catch {
continue; // No inbox or invalid JSON
}

if (!Array.isArray(messages)) continue;

const unreadCount = messages.filter((m) => m.read === false).length;
if (unreadCount === 0) continue;

// Get workingDir from config.json → members → team-lead → cwd
let workingDir: string | null = null;
try {
const cfgContent = await readFile(join(base, name, 'config.json'), 'utf-8');
const config: NativeTeamConfig = JSON.parse(cfgContent);
const leadMember = config.members.find((m) => m.name === 'team-lead' || m.agentId.startsWith('team-lead@'));
if (leadMember?.cwd) {
workingDir = leadMember.cwd;
}
} catch {
// Config missing or malformed — workingDir stays null
}

results.push({ teamName: name, unreadCount, workingDir });
}

return results;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The for...of loop processes each team directory sequentially due to the await calls inside for reading files. This could be slow if there are many teams. You can improve performance by processing teams in parallel using Promise.all.

  const results = await Promise.all(
    teamDirs.map(async (name) => {
      // Read inbox messages
      const inboxFile = join(base, name, 'inboxes', 'team-lead.json');
      let messages: NativeInboxMessage[];
      try {
        const content = await readFile(inboxFile, 'utf-8');
        messages = JSON.parse(content);
      } catch {
        return null; // No inbox or invalid JSON
      }

      if (!Array.isArray(messages)) return null;

      const unreadCount = messages.filter((m) => m.read === false).length;
      if (unreadCount === 0) return null;

      // Get workingDir from config.json → members → team-lead → cwd
      let workingDir: string | null = null;
      try {
        const cfgContent = await readFile(join(base, name, 'config.json'), 'utf-8');
        const config: NativeTeamConfig = JSON.parse(cfgContent);
        const leadMember = config.members.find((m) => m.name === 'team-lead' || m.agentId.startsWith('team-lead@'));
        if (leadMember?.cwd) {
          workingDir = leadMember.cwd;
        }
      } catch {
        // Config missing or malformed — workingDir stays null
      }

      return { teamName: name, unreadCount, workingDir };
    }),
  );

  return results.filter((r): r is { teamName: string; unreadCount: number; workingDir: string | null } => r !== null);

Comment thread src/lib/inbox-watcher.ts
Comment on lines +83 to +118
const spawned: string[] = [];

for (const { teamName, workingDir } of teamsWithUnread) {
// Skip teams that have exceeded max spawn failures
const failures = spawnFailures.get(teamName) ?? 0;
if (failures >= MAX_SPAWN_FAILURES) {
deps.warn(`[inbox-watcher] Skipping team "${teamName}" — ${failures} consecutive spawn failures`);
continue;
}

// Skip teams that already have an active team-lead
const active = await deps.isTeamActive(teamName);
if (active) continue;

// No working dir means we can't spawn
if (!workingDir) {
deps.warn(`[inbox-watcher] Cannot spawn team-lead for "${teamName}" — no workingDir in config`);
continue;
}

// Attempt to spawn team-lead
try {
await deps.ensureTeamLead(teamName, workingDir);
spawnFailures.set(teamName, 0); // Reset on success
spawned.push(teamName);
} catch (err) {
const newCount = failures + 1;
spawnFailures.set(teamName, newCount);
const message = err instanceof Error ? err.message : String(err);
deps.warn(
`[inbox-watcher] Failed to spawn team-lead for "${teamName}" (attempt ${newCount}/${MAX_SPAWN_FAILURES}): ${message}`,
);
}
}

return spawned;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The for...of loop processes each team with unread messages sequentially. Since each team check is independent, you can parallelize these operations using Promise.all to improve performance, especially when multiple teams need spawning.

  const spawnPromises = teamsWithUnread.map(async ({ teamName, workingDir }) => {
    // Skip teams that have exceeded max spawn failures
    const failures = spawnFailures.get(teamName) ?? 0;
    if (failures >= MAX_SPAWN_FAILURES) {
      deps.warn(`[inbox-watcher] Skipping team "${teamName}" — ${failures} consecutive spawn failures`);
      return null;
    }

    // Skip teams that already have an active team-lead
    const active = await deps.isTeamActive(teamName);
    if (active) return null;

    // No working dir means we can't spawn
    if (!workingDir) {
      deps.warn(`[inbox-watcher] Cannot spawn team-lead for "${teamName}" — no workingDir in config`);
      return null;
    }

    // Attempt to spawn team-lead
    try {
      await deps.ensureTeamLead(teamName, workingDir);
      spawnFailures.set(teamName, 0); // Reset on success
      return teamName;
    } catch (err) {
      const newCount = failures + 1;
      spawnFailures.set(teamName, newCount);
      const message = err instanceof Error ? err.message : String(err);
      deps.warn(
        `[inbox-watcher] Failed to spawn team-lead for "${teamName}" (attempt ${newCount}/${MAX_SPAWN_FAILURES}): ${message}`,
      );
      return null;
    }
  });

  const spawned = (await Promise.all(spawnPromises)).filter((t): t is string => t !== null);

  return spawned;

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ced87a7640

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

try {
const cfgContent = await readFile(join(base, name, 'config.json'), 'utf-8');
const config: NativeTeamConfig = JSON.parse(cfgContent);
const leadMember = config.members.find((m) => m.name === 'team-lead' || m.agentId.startsWith('team-lead@'));

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Derive lead cwd without hardcoding the member name

listTeamsWithUnreadInbox() assumes the lead entry is named team-lead, but teams created through the normal session bootstrap path register the leader as basename(cwd) (see ensureNativeTeamForLeader() in src/genie-commands/session.ts), so this lookup often returns no match and leaves workingDir null. In that case checkInboxes() skips spawning entirely, which means unread inboxes for those existing teams will never trigger auto-respawn.

Useful? React with 👍 / 👎.

Comment thread src/lib/inbox-watcher.ts
const message = err instanceof Error ? err.message : String(err);
deps.warn(`[inbox-watcher] Poll error: ${message}`);
});
}, getInboxPollIntervalMs());

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Disable the daemon loop when poll interval is zero

The module documents GENIE_INBOX_POLL_MS=0 as disabled, and checkInboxes() returns early for that case, but startInboxWatcher() still creates setInterval(..., 0). That schedules a tight loop that repeatedly wakes up and immediately returns, causing unnecessary CPU churn in the exact configuration intended to disable polling.

Useful? React with 👍 / 👎.

Comment thread .github/workflows/ci.yml
run: |
COVERAGE_OUTPUT=$(bun test --coverage 2>&1)
echo "$COVERAGE_OUTPUT"
LINE_COV=$(echo "$COVERAGE_OUTPUT" | grep "All files" | awk -F'|' '{print $2}' | tr -d ' ')

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Parse the % Lines column for the coverage gate

The coverage row format is All files | % Funcs | % Lines | ..., but the script extracts field 2 (awk -F'|' '{print $2}'), which is function coverage, then labels it as line coverage. This makes the gate enforce the wrong metric and can fail or pass builds contrary to the intended 68% line-coverage policy.

Useful? React with 👍 / 👎.

Comment thread .github/workflows/ci.yml
run: |
COVERAGE_OUTPUT=$(bun test --coverage 2>&1)
echo "$COVERAGE_OUTPUT"
LINE_COV=$(echo "$COVERAGE_OUTPUT" | grep "All files" | awk -F'|' '{print $2}' | tr -d ' ')

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Make unparseable coverage output truly non-fatal

The fallback branch for unparseable coverage is unreachable when grep "All files" finds no match, because this step runs with -e -o pipefail and the command substitution fails before if [ -z "$LINE_COV" ] executes. As a result, any Bun output format change hard-fails CI instead of taking the intended graceful-degradation path.

Useful? React with 👍 / 👎.

@coderabbitai

coderabbitai Bot commented Mar 17, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 48524a9d-46d2-4d02-9348-15ef811a78a2

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/genie-hardening
📝 Coding Plan
  • Generate coding plan for human review comments

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant