Skip to content

fix: run cargo test/build --workspace in CI, isolate ponytail's flaky config test - #120

Merged
getappz merged 2 commits into
masterfrom
fix/ci-workspace-tests
Jul 9, 2026
Merged

fix: run cargo test/build --workspace in CI, isolate ponytail's flaky config test#120
getappz merged 2 commits into
masterfrom
fix/ci-workspace-tests

Conversation

@getappz

@getappz getappz commented Jul 9, 2026

Copy link
Copy Markdown
Owner

Summary

  • Add PONYTAIL_CONFIG_DIR_OVERRIDE to crates/ponytail/src/config.rs::config_dir(), mirroring the root crate's paths::home()/AGENTFLARE_HOME_OVERRIDE pattern. Without it, config_dir() reads the real OS profile directory directly (dirs::config_dir() ignores env-var overrides), so defaults_to_full and roundtrip_default_mode raced each other over the real ~/.config/agentflare/ponytail/config.json under cargo's parallel test runner. All three tests that touch persisted config state are now isolated via a temp dir.
  • Add --workspace to both cargo build/cargo test steps in .github/workflows/ci.yml. This repo's root Cargo.toml defines both a [workspace] and a root [package] with no default-members set — per Cargo's default-member resolution, plain cargo test/cargo build only built/tested the root package. None of crates/ponytail, crates/caveman, crates/agent-registry, crates/skill-registry, or crates/gateway-registry's own unit tests ever ran in CI, despite PRs (including Caveman L1 integration: port markdown compressor to Rust #117) showing all-green.

These two changes are bundled together because landing --workspace alone, without the ponytail test-isolation fix, would have turned CI red for an unrelated reason (the pre-existing flake) on every future PR.

Test plan

  • cargo test -p ponytail — 36 tests, ran clean
  • cargo test --workspace x4 in a row locally — no flakes (previously reproduced the race within a handful of runs)
  • Confirmed via cargo test --verbose (no --workspace, matching the old CI command) that it only runs 3 test binaries (root package + its 2 integration tests) — none of the workspace member crates' own unit tests

Fixes #119
Fixes #118

Summary by CodeRabbit

  • Tests
    • Improved isolation for configuration-related tests by using a temporary, environment-specific config directory and automatic cleanup.
  • CI / Build
    • Expanded CI verification to run cargo build and cargo test across the full Rust workspace with more verbose output.

…ld --workspace in CI

crates/ponytail/src/config.rs: config_dir() ignored env-var overrides
(dirs::config_dir() reads the real OS profile dir directly), so
defaults_to_full and roundtrip_default_mode raced each other over the
real ~/.config/agentflare/ponytail/config.json under cargo's parallel
test runner. Add PONYTAIL_CONFIG_DIR_OVERRIDE (same pattern as
paths::home()'s AGENTFLARE_HOME_OVERRIDE) and isolate the three tests
that read/write persisted config state.

.github/workflows/ci.yml: cargo build/test had no --workspace flag, so
with a root package present and no default-members set, only the root
package was ever built/tested — none of crates/ponytail, crates/caveman,
crates/agent-registry, crates/skill-registry, or crates/gateway-registry's
own unit tests ran in CI despite PRs showing all-green. Verified locally:
--workspace correctly picks up all workspace member tests. This depends
on the config.rs fix above — landing --workspace alone would have turned
CI red on the pre-existing ponytail flake.

Fixes #119
Fixes #118
@coderabbitai

coderabbitai Bot commented Jul 9, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: ef8c9380-d217-4996-80cc-f0ed48176941

📥 Commits

Reviewing files that changed from the base of the PR and between 3b7728f and 570aabc.

📒 Files selected for processing (1)
  • crates/ponytail/src/config.rs
🚧 Files skipped from review as they are similar to previous changes (1)
  • crates/ponytail/src/config.rs

📝 Walkthrough

Walkthrough

CI now runs build and test across the full Rust workspace. Ponytail config resolution accepts a test-only config directory override, and the config tests use a temporary isolated directory for env and persisted-state cases.

Changes

CI workspace coverage and ponytail config test isolation

Layer / File(s) Summary
Config dir override and temp-dir test helper
crates/ponytail/src/config.rs
config_dir() returns PONYTAIL_CONFIG_DIR_OVERRIDE when set, otherwise it falls back to the OS config path; a test-only helper creates a temporary config directory, installs the override for a closure, and clears it on drop.
CI workspace build/test flags
.github/workflows/ci.yml
cargo build and cargo test now run with --workspace --verbose.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

  • getappz/agentflare#96: Both PRs modify crates/ponytail/src/config.rs to add env-based test isolation around config directory handling.
  • getappz/agentflare#111: Both PRs touch .github/workflows/ci.yml's cargo build and cargo test steps in the same job.
  • getappz/agentflare#94: Both PRs change crates/ponytail/src/config.rs around default-mode and persisted config handling.
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes both requested changes: workspace CI commands and ponytail config test isolation.
Linked Issues check ✅ Passed The changes satisfy #118 by adding --workspace to build/test and satisfy #119 by isolating ponytail config tests with an override.
Out of Scope Changes check ✅ Passed All code changes map to the linked goals; the RAII temp-config helper is part of the ponytail test isolation fix.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/ci-workspace-tests

Warning

Tools execution failed with the following error:

Failed to run tools: 13 INTERNAL: Received RST_STREAM with code 2 (Internal server error)


Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
crates/ponytail/src/config.rs (1)

91-107: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Stale temp dir not cleaned up after test run.

The temp directory created at line 100-102 is only removed pre-emptively at the start of the next run, never after the current one finishes. Not critical since it's OS temp space and gets wiped/reused, but worth a Drop-based or trailing cleanup for hygiene.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/ponytail/src/config.rs` around lines 91 - 107, The test-only isolation
helper with_temp_config_dir currently removes the temp directory only before the
next run, leaving stale state behind after the current test finishes. Update
with_temp_config_dir in config.rs to perform cleanup after the closure returns
by ensuring the temp dir is deleted on exit, preferably via a Drop-based guard
or equivalent trailing cleanup, while preserving the
PONYTAIL_CONFIG_DIR_OVERRIDE setup/teardown behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@crates/ponytail/src/config.rs`:
- Around line 91-107: The test-only override in with_temp_config_dir leaks
PONYTAIL_CONFIG_DIR_OVERRIDE if the closure panics or returns early because
cleanup after f() is not panic-safe. Update with_temp_config_dir to use an RAII
guard around the set_var/remove_var lifecycle so the environment variable is
always restored, even when defaults_to_full, reads_env_var, or
roundtrip_default_mode fail. Keep the fix localized to with_temp_config_dir and
the temp config dir handling in config.rs.

---

Nitpick comments:
In `@crates/ponytail/src/config.rs`:
- Around line 91-107: The test-only isolation helper with_temp_config_dir
currently removes the temp directory only before the next run, leaving stale
state behind after the current test finishes. Update with_temp_config_dir in
config.rs to perform cleanup after the closure returns by ensuring the temp dir
is deleted on exit, preferably via a Drop-based guard or equivalent trailing
cleanup, while preserving the PONYTAIL_CONFIG_DIR_OVERRIDE setup/teardown
behavior.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: e1ef6f84-83bf-4dea-9b09-225ae5751fa1

📥 Commits

Reviewing files that changed from the base of the PR and between e8f5ac5 and 3b7728f.

📒 Files selected for processing (2)
  • .github/workflows/ci.yml
  • crates/ponytail/src/config.rs

Comment thread crates/ponytail/src/config.rs
If the test closure panicked (e.g. a failing assert_eq! in
defaults_to_full/reads_env_var/roundtrip_default_mode), the bare
remove_var() after f() was skipped, leaking PONYTAIL_CONFIG_DIR_OVERRIDE
process-wide for the rest of the test binary. A Drop guard restores it
unconditionally, dropped while ENV_TEST_LOCK is still held.

(CodeRabbit review on #120)
@getappz
getappz merged commit 1637b87 into master Jul 9, 2026
11 checks passed
@getappz
getappz deleted the fix/ci-workspace-tests branch July 9, 2026 13:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant