Skip to content

fix(test): Stabilize test coverage by preventing jiti double instrumentation#928

Merged
yamadashy merged 4 commits intomainfrom
chore/coverage
Oct 30, 2025
Merged

fix(test): Stabilize test coverage by preventing jiti double instrumentation#928
yamadashy merged 4 commits intomainfrom
chore/coverage

Conversation

@yamadashy
Copy link
Owner

Summary

This PR resolves test coverage instability that occurred after implementing JS/TS config support. Coverage was fluctuating between 68-71% on each test run.

Root Cause

Test fixture files imported defineConfig from src/index.js, which caused jiti to dynamically transform the entire src/ directory. This created a double instrumentation problem:

  • Vitest instrumented src/ files for coverage tracking
  • jiti transformed the same src/ files when loading test fixtures
  • Same files existed in two versions, causing v8 coverage tracker to produce non-deterministic results

Solution

  1. Added DI pattern to loadFileConfig with defaultJitiImport factory for better testability
  2. Removed defineConfig imports from all test fixtures - now use plain objects
  3. Mock jiti in dynamic config tests to prevent src/ file transformation

Results

  • ✅ Coverage now stable at 89.34% (was fluctuating 68-71%)
  • ✅ Coverage improved by ~19% due to accurate instrumentation
  • ✅ Integration test runtime improved from 3.8s to 0.1s (38x faster)
  • ✅ All tests pass with zero lint errors

Changes

Modified Files:

  • src/config/configLoad.ts - Added DI pattern and defaultJitiImport factory
  • tests/config/configLoad.integration.test.ts - Mock jiti for dynamic config tests
  • 7 test fixture files - Removed defineConfig imports

Reverted:

  • Temporary Vitest config workarounds (sequence.shuffle, isolate settings)

Checklist

  • Run npm run test
  • Run npm run lint

Copilot AI review requested due to automatic review settings October 30, 2025 15:56
@coderabbitai
Copy link
Contributor

coderabbitai bot commented Oct 30, 2025

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Note

Other AI code review bot(s) detected

CodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review.

Walkthrough

This PR introduces dependency injection for config file loading via a pluggable jitiImport function, refactors test fixtures to export plain objects instead of defineConfig-wrapped configs, and adds comprehensive test coverage for CLI actions, spinner behavior, permission checking, and output generation features.

Changes

Cohort / File(s) Change Summary
Config Loading Refactoring
src/config/configLoad.ts
Introduces defaultJitiImport helper and extends loadFileConfig and loadAndValidateConfig signatures to accept optional deps parameter with jitiImport implementation, enabling testable/configurable config file loading via dependency injection instead of inline Jiti instantiation.
Test Fixtures - JS Configs
tests/fixtures/config-js/repomix.config.js, repomix.config.mjs, repomix-dynamic.config.js
Removes defineConfig import and wrapper calls; exports plain object literals instead. Replaces dynamic timestamp generation with fixed constants to ensure deterministic test behavior.
Test Fixtures - TS Configs
tests/fixtures/config-ts/repomix.config.ts, repomix.config.cts, repomix.config.mts, repomix-dynamic.config.ts
Removes defineConfig import and wrapper calls; exports plain object literals. Replaces dynamic timestamp generation with fixed string constants for consistent test execution.
Config Loading Tests
tests/config/configLoad.test.ts, configLoad.integration.test.ts
Adds tests for unsupported config formats, invalid extensions, and error handling. Updates integration tests to mock jitiImport hook and assert exact file paths instead of regex patterns.
CLI Action Tests
tests/cli/actions/initAction.test.ts
Adds comprehensive test coverage for createConfigFile and createIgnoreFile scenarios including cancellation handling (process.exit calls), overwrite prompts, and successful file creation returns.
CLI Report Tests
tests/cli/cliReport.test.ts
Adds tests for printing summaries with Git diffs/logs included and excluded, and detailed security issue reporting with suspicious file/diff/log handling.
CLI Spinner Tests
tests/cli/cliSpinner.test.ts
New test suite for cliSpinner covering spinner lifecycle (start, frame updates, stop), message updates, success/fail paths, frame cycling, quiet mode behavior, and interval management.
File Permission Tests
tests/core/file/permissionCheck.test.ts
Adds tests for macOS-specific error handling (EPERM, EISDIR with platform-specific messages), unknown error codes, undefined error codes, and partial/complete permission check failures.
Output Generation Tests
tests/core/output/outputGenerate.test.ts
Adds tests for Git diffs/logs inclusion, JSON style output, file/directory structure section toggles, and validates existing style outputs (plain, xml, markdown) remain functional.

Sequence Diagram(s)

sequenceDiagram
    participant Caller as loadFileConfig Caller
    participant LFC as loadFileConfig
    participant LAVC as loadAndValidateConfig
    participant DepsImporter as deps.jitiImport

    Caller->>LFC: loadFileConfig(rootDir, argConfigPath, deps?)
    alt deps provided
        LFC->>LAVC: loadAndValidateConfig(filePath, deps)
    else deps not provided
        LFC->>LFC: create defaultJitiImport
        LFC->>LAVC: loadAndValidateConfig(filePath, {jitiImport: defaultJitiImport})
    end
    
    alt TS or JS config file
        LAVC->>DepsImporter: deps.jitiImport(filePath)
        DepsImporter->>LAVC: return loaded config
    else JSON or json5 config
        LAVC->>LAVC: parse JSON/json5 directly
    end
    
    LAVC->>LFC: return validated config
    LFC->>Caller: return config
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

  • src/config/configLoad.ts: The dependency injection pattern is straightforward but introduces new function signatures that need careful review to ensure backward compatibility and correct default behavior.
  • Test fixtures: Repetitive refactoring across multiple files (removal of defineConfig wrapper), which is straightforward but numerous and should be verified consistently applied.
  • Integration tests: Updated mocking of jitiImport requires verification that mock expectations align with actual usage patterns.
  • New test suites: While test additions are generally low-risk, the permissionCheck and cliSpinner test suites introduce edge-case scenarios that should be reviewed for correctness and completeness.

Possibly related PRs

Suggested labels

enhancement

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
✅ Passed checks (2 passed)
Check name Status Explanation
Title Check ✅ Passed The pull request title "fix(test): Stabilize test coverage by preventing jiti double instrumentation" directly and specifically addresses the main objective of the changeset. It accurately describes the core problem being solved—test coverage instability caused by jiti double instrumentation—and the primary solution approach. The title is concise, uses clear language without vague terms, and a teammate reviewing the commit history would immediately understand this PR concerns test coverage stabilization and the jiti instrumentation issue. The use of conventional commit format ("fix(test):") further clarifies the scope.
Description Check ✅ Passed The pull request description comprehensively covers all required template elements and substantially exceeds the basic requirements. It includes a clear summary section explaining the context and objectives, provides root cause analysis of the double instrumentation problem, describes the three-part solution (DI pattern, removing defineConfig imports, and mocking jiti), quantifies concrete results (89.34% coverage, ~19% improvement, 38x faster integration tests), lists all modified files, and includes a completed checklist confirming both npm run test and npm run lint were executed. The description is well-structured and provides excellent context for reviewers without being overly verbose.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@gemini-code-assist
Copy link
Contributor

Summary of Changes

Hello @yamadashy, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly improves the stability and performance of the test suite by addressing a critical issue related to jiti's dynamic module loading. Previously, test coverage was inconsistent due to jiti inadvertently re-instrumenting source files already handled by Vitest. The solution involves introducing a dependency injection pattern for jiti imports, simplifying test fixtures, and explicitly mocking jiti in integration tests, resulting in stable and higher coverage, along with a substantial reduction in test execution time.

Highlights

  • Test Coverage Stability: Resolved fluctuating test coverage (68-71%) by preventing jiti's double instrumentation of source files, leading to a stable and improved coverage of 89.34%.
  • Performance Boost: Achieved a 38x speedup in integration test runtime, reducing it from 3.8s to 0.1s, by eliminating unnecessary dynamic transformations during testing.
  • Enhanced Testability: Implemented a Dependency Injection pattern for jiti imports within loadFileConfig, making the configuration loading process more testable and controllable.
  • Simplified Test Fixtures: Refactored test fixture files to use plain JavaScript/TypeScript objects instead of importing defineConfig, which was a root cause of the double instrumentation issue.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@codecov
Copy link

codecov bot commented Oct 30, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 89.50%. Comparing base (deb3c7f) to head (3811fdb).
⚠️ Report is 5 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff             @@
##             main     #928       +/-   ##
===========================================
+ Coverage   75.32%   89.50%   +14.18%     
===========================================
  Files         111      111               
  Lines        7793     7807       +14     
  Branches     1454     1498       +44     
===========================================
+ Hits         5870     6988     +1118     
+ Misses       1923      819     -1104     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@cloudflare-workers-and-pages
Copy link

cloudflare-workers-and-pages bot commented Oct 30, 2025

Deploying repomix with  Cloudflare Pages  Cloudflare Pages

Latest commit: 3811fdb
Status: ✅  Deploy successful!
Preview URL: https://81e4a9b0.repomix.pages.dev
Branch Preview URL: https://chore-coverage.repomix.pages.dev

View logs

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request addresses a critical issue of test coverage instability by preventing double instrumentation of files during testing. The changes include adding a DI pattern to loadFileConfig, removing defineConfig imports from test fixtures, and mocking jiti in dynamic config tests. These modifications stabilize test coverage, improve test runtime, and enhance the accuracy of instrumentation. The code changes are well-structured and effectively resolve the identified root cause.

@claude
Copy link
Contributor

claude bot commented Oct 30, 2025

PR Review: Stabilize Test Coverage by Preventing Jiti Double Instrumentation

Summary

Excellent work on identifying and fixing this coverage instability issue! The root cause analysis is thorough, and the solution follows best practices with dependency injection. The PR successfully resolves the double instrumentation problem and dramatically improves both coverage stability and test performance.

🎯 Strengths

  1. Root Cause Analysis: The PR clearly identifies the double instrumentation problem caused by jiti transforming src/ files while Vitest instruments them for coverage
  2. Clean DI Pattern: The dependency injection approach with defaultJitiImport factory is well-implemented and testable
  3. Significant Improvements:
    • Coverage stabilized at 89.34% (previously fluctuating 68-71%)
    • Integration test runtime improved 38x (3.8s → 0.1s)
  4. Comprehensive Test Coverage: Added 724 lines of new tests covering edge cases across multiple modules
  5. Documentation: Clear comments in fixture files explaining why defineConfig was removed

💡 Code Quality Assessment

src/config/configLoad.ts

Positive:

  • Clean separation of concerns with defaultJitiImport factory function
  • Consistent deps parameter pattern following project guidelines
  • Good comments explaining the purpose of jiti configuration options

Minor Observations:

  • Lines 60-67: The defaultJitiImport factory is well-structured and follows the project's DI pattern
  • Lines 120-122: Properly passing deps through to loadAndValidateConfig
Details: Implementation Pattern

The DI pattern implementation is exemplary:

export const loadFileConfig = async (
  rootDir: string,
  argConfigPath: string | null,
  deps = {
    jitiImport: defaultJitiImport,
  },
): Promise<RepomixConfigFile> => {

This allows tests to inject mock implementations while production code uses the real jiti import, avoiding coverage contamination.

Test Changes

Positive:

  • Integration tests properly mock jitiImport to prevent coverage instability
  • New test coverage fills important gaps (spinner, report, permission checks)
  • Tests verify the file URL being loaded before returning mock data

Observations:

  • All 7 fixture files correctly removed defineConfig imports with explanatory comments
  • Dynamic config tests (lines 53-78 in configLoad.integration.test.ts) maintain behavior verification while avoiding actual jiti transformation

🔒 Security Considerations

No security concerns identified

  • Changes are test-focused and don't affect production security boundaries
  • No handling of user input or sensitive data in modified code
  • DI pattern doesn't introduce injection vulnerabilities (factory is internal)

⚡ Performance Considerations

Significant performance improvements

  • Integration tests now run 38x faster (3.8s → 0.1s)
  • Removed double transformation overhead
  • No negative performance impacts on production code

🧪 Test Coverage

Excellent test coverage improvements

New tests added for:

  • initAction.test.ts: 80 new lines covering edge cases (cancel flows, success paths)
  • cliReport.test.ts: 168 new lines covering git diffs/logs reporting
  • cliSpinner.test.ts: 203 new lines covering all spinner states and quiet mode
  • permissionCheck.test.ts: 39 new lines for macOS-specific error handling
  • outputGenerate.test.ts: 108 new lines for git integration and JSON output

The coverage increase from ~68-71% to 89.34% represents genuine improvement, not just stabilization.

🔍 Premortem Analysis: Potential Failure Scenarios

1. Edge Case: User-Provided Config Files in Production

Risk: Low
Scenario: Users with custom config loaders or tooling that depends on defineConfig type checking
Mitigation:

  • defineConfig is still exported from the public API (only removed from test fixtures)
  • User-facing documentation should still recommend defineConfig for type safety
  • Consider adding a note in documentation that plain objects are also supported

2. Integration Issue: Third-Party Test Runners

Risk: Low
Scenario: Projects using Jest or other test runners might not benefit from the same fix
Mitigation:

  • The fix is Vitest-specific but doesn't break other environments
  • The DI pattern is universal and improves testability regardless of test runner

3. Deployment Risk: Coverage Reporting in CI

Risk: Very Low
Scenario: CI coverage thresholds might fail due to the 19% increase if thresholds are set exactly
Mitigation:

  • Coverage increased (positive change), threshold failures would only occur if thresholds were set between 68-89%
  • Easy to adjust thresholds upward
  • Recommendation: Update coverage thresholds in CI config to reflect the new 89.34% baseline

4. Regression Risk: Mock Drift

Risk: Low
Scenario: Mocked jiti behavior in tests might diverge from real jiti behavior
Mitigation:

  • Static config tests still use real jiti (only dynamic tests are mocked)
  • The mocks in dynamic tests return realistic config objects
  • Recommendation: Consider adding an E2E test that verifies actual dynamic config loading works in isolation

5. Breaking Change: Internal API

Risk: Very Low
Scenario: Internal consumers calling loadFileConfig without the deps parameter
Mitigation:

  • The deps parameter has a default value, maintaining backward compatibility
  • Internal codebase search shows no other files call loadFileConfig with explicit deps
  • ✅ This is a non-breaking change

📋 Suggested Improvements (Optional)

1. Type Safety Enhancement

Consider adding a type definition for the deps object to improve maintainability:

// Type for dependency injection
type ConfigLoadDeps = {
  jitiImport: (fileUrl: string) => Promise<unknown>;
};

export const loadFileConfig = async (
  rootDir: string,
  argConfigPath: string | null,
  deps: ConfigLoadDeps = {
    jitiImport: defaultJitiImport,
  },
): Promise<RepomixConfigFile> => {

This makes the contract more explicit and helps with IDE autocomplete.

2. Test Documentation Enhancement

The dynamic config tests have excellent comments, but consider adding a comment in the main test file explaining why only dynamic tests are mocked:

// Static config tests use real jiti to verify actual loading behavior
// Dynamic config tests mock jiti to prevent coverage instability
3. Consider E2E Test for Dynamic Configs

Add a separate E2E test file that verifies dynamic config loading works correctly without affecting coverage:

// tests/e2e/dynamic-config.e2e.test.ts
// Runs in isolation to verify real dynamic config behavior

This would provide confidence that mocked tests accurately reflect real behavior.

✅ Checklist Review

  • ✅ Tests pass (npm run test)
  • ✅ Linting passes (npm run lint)
  • ✅ Follows DI pattern from CLAUDE.md
  • ✅ Comments in English
  • ✅ Files under 250 lines (all modified files comply)
  • ✅ Commit message follows Conventional Commits format

🎉 Recommendation

APPROVE - This PR is ready to merge!

The implementation is clean, well-tested, and solves a real problem that was causing coverage instability. The 38x performance improvement on integration tests is a significant bonus. The premortem analysis identified only low-risk scenarios with appropriate mitigations already in place.

The only suggestion is to update CI coverage thresholds to reflect the new 89.34% baseline to prevent false positives in future PRs.

Great work on the thorough investigation and elegant solution! 🚀

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR improves test stability and code coverage by addressing issues with dynamic imports and test fixtures. The changes focus on preventing jiti from transforming source files during tests and making tests more deterministic.

  • Removed defineConfig imports from test fixture files to prevent jiti from transforming src/ files during test execution
  • Made dynamic timestamp values in test fixtures deterministic to ensure stable coverage
  • Added comprehensive unit tests for output generation, permission checking, config loading, CLI spinner, CLI reporting, and init actions
  • Introduced dependency injection for jitiImport in the config loading logic to enable better testability

Reviewed Changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated no comments.

Show a summary per file
File Description
tests/fixtures/config-ts/repomix.config.ts Removed defineConfig import and wrapper to prevent jiti transformation during tests
tests/fixtures/config-ts/repomix.config.mts Removed defineConfig import and wrapper to prevent jiti transformation during tests
tests/fixtures/config-ts/repomix.config.cts Removed defineConfig import and wrapper to prevent jiti transformation during tests
tests/fixtures/config-ts/repomix-dynamic.config.ts Replaced dynamic timestamp with fixed value and removed defineConfig import
tests/fixtures/config-js/repomix.config.mjs Removed defineConfig import and wrapper to prevent jiti transformation during tests
tests/fixtures/config-js/repomix.config.js Removed defineConfig import and wrapper to prevent jiti transformation during tests
tests/fixtures/config-js/repomix-dynamic.config.js Replaced dynamic timestamp with fixed value and removed defineConfig import
tests/core/output/outputGenerate.test.ts Added new test cases for git diffs, git logs, JSON output, and conditional output sections
tests/core/file/permissionCheck.test.ts Added edge case tests for macOS-specific error handling and unknown error codes
tests/config/configLoad.test.ts Added error handling tests for unsupported formats and general errors
tests/config/configLoad.integration.test.ts Added mock jiti implementation to prevent coverage instability in dynamic config tests
tests/cli/cliSpinner.test.ts Created comprehensive test suite for the CLI spinner functionality
tests/cli/cliReport.test.ts Added tests for git diffs/logs reporting and security check details
tests/cli/actions/initAction.test.ts Added tests for user cancellation scenarios and successful creation paths
src/config/configLoad.ts Added dependency injection for jitiImport function to enable testability

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between deb3c7f and 4a86076.

📒 Files selected for processing (15)
  • src/config/configLoad.ts (5 hunks)
  • tests/cli/actions/initAction.test.ts (2 hunks)
  • tests/cli/cliReport.test.ts (2 hunks)
  • tests/cli/cliSpinner.test.ts (1 hunks)
  • tests/config/configLoad.integration.test.ts (2 hunks)
  • tests/config/configLoad.test.ts (1 hunks)
  • tests/core/file/permissionCheck.test.ts (1 hunks)
  • tests/core/output/outputGenerate.test.ts (1 hunks)
  • tests/fixtures/config-js/repomix-dynamic.config.js (1 hunks)
  • tests/fixtures/config-js/repomix.config.js (2 hunks)
  • tests/fixtures/config-js/repomix.config.mjs (1 hunks)
  • tests/fixtures/config-ts/repomix-dynamic.config.ts (1 hunks)
  • tests/fixtures/config-ts/repomix.config.cts (1 hunks)
  • tests/fixtures/config-ts/repomix.config.mts (1 hunks)
  • tests/fixtures/config-ts/repomix.config.ts (2 hunks)
🧰 Additional context used
🧬 Code graph analysis (9)
tests/cli/cliReport.test.ts (4)
tests/testing/testUtils.ts (1)
  • createMockConfig (15-45)
src/core/packager.ts (1)
  • PackResult (18-32)
src/cli/cliReport.ts (2)
  • reportSummary (49-94)
  • reportSecurityCheck (96-129)
src/core/security/securityCheck.ts (1)
  • SuspiciousFileResult (10-14)
tests/cli/actions/initAction.test.ts (1)
src/cli/actions/initAction.ts (2)
  • createConfigFile (41-126)
  • createIgnoreFile (128-178)
tests/config/configLoad.test.ts (1)
src/config/configLoad.ts (1)
  • loadFileConfig (69-111)
tests/core/output/outputGenerate.test.ts (2)
tests/testing/testUtils.ts (1)
  • createMockConfig (15-45)
src/core/output/outputGenerate.ts (1)
  • generateOutput (221-262)
src/config/configLoad.ts (2)
src/index.ts (2)
  • loadFileConfig (31-31)
  • RepomixConfigFile (32-32)
src/config/configSchema.ts (1)
  • RepomixConfigFile (153-153)
tests/fixtures/config-ts/repomix-dynamic.config.ts (1)
tests/fixtures/config-js/repomix-dynamic.config.js (1)
  • timestamp (5-5)
tests/cli/cliSpinner.test.ts (1)
src/cli/cliSpinner.ts (1)
  • Spinner (9-70)
tests/config/configLoad.integration.test.ts (1)
src/config/configLoad.ts (1)
  • loadFileConfig (69-111)
tests/core/file/permissionCheck.test.ts (1)
src/core/file/permissionCheck.ts (2)
  • checkDirectoryPermissions (27-90)
  • PermissionError (16-25)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (9)
  • GitHub Check: Agent
  • GitHub Check: CodeQL analysis (javascript-typescript)
  • GitHub Check: Test (macos-latest, 22.x)
  • GitHub Check: Test with Bun (windows-latest, latest)
  • GitHub Check: Test (macos-latest, 20.x)
  • GitHub Check: Test (windows-latest, 24.x)
  • GitHub Check: Build and run (windows-latest, 24.x)
  • GitHub Check: Build and run with Bun (windows-latest, latest)
  • GitHub Check: claude-review
🔇 Additional comments (18)
src/config/configLoad.ts (3)

60-67: LGTM! Clean dependency injection implementation.

The defaultJitiImport helper provides a sensible default with moduleCache: false for fresh config loads and interopDefault: true for automatic default export handling. This enables testability while maintaining production behavior.


69-111: LGTM! Well-designed dependency injection pattern.

The optional deps parameter with a default value maintains backward compatibility while enabling test mocking. All code paths (argConfigPath, local config, global config) consistently thread the dependency through to loadAndValidateConfig.


118-165: LGTM! Consistent DI implementation in config loading.

The loadAndValidateConfig function now uses deps.jitiImport instead of directly instantiating jiti, enabling testability. The JSON/JSON5 paths remain unchanged, ensuring no behavioral changes for non-JS/TS configs.

tests/config/configLoad.test.ts (1)

210-234: LGTM! Comprehensive error handling test coverage.

The new test cases thoroughly validate error scenarios including unsupported file formats, permission errors, and non-Error objects. This ensures robust error handling across different failure modes.

tests/cli/cliReport.test.ts (2)

57-165: LGTM! Thorough test coverage for Git-related reporting.

The new test cases comprehensively validate Git diffs and logs reporting in both scenarios: when data is present (with token counts) and when absent. This ensures the reporting logic handles all cases correctly.


239-295: LGTM! Excellent coverage for security issue reporting edge cases.

The new tests validate correct pluralization ("security issue" vs "security issues") and comprehensive Git diff/log security reporting. This ensures accurate user-facing messages across different scenarios.

tests/fixtures/config-ts/repomix-dynamic.config.ts (1)

1-16: LGTM! Excellent refactoring to prevent double instrumentation.

The comments clearly explain the rationale, and the fixed timestamp ensures deterministic test coverage. This change directly addresses the root cause of coverage instability mentioned in the PR objectives.

tests/fixtures/config-ts/repomix.config.mts (1)

1-11: LGTM! Consistent refactoring with clear documentation.

The plain object export avoids jiti transformation of src/ files during tests, preventing the double instrumentation issue. The comments ensure future maintainers understand the rationale.

tests/fixtures/config-ts/repomix.config.cts (1)

1-11: LGTM! Fixture refactoring aligns with PR objectives.

The transition from defineConfig wrapper to plain object export, along with explanatory comments, successfully prevents src/ file transformation during test runs.

tests/fixtures/config-js/repomix-dynamic.config.js (1)

1-15: LGTM! Clean refactoring with fixed timestamp for determinism.

The fixed timestamp constant and plain object export ensure stable, deterministic test coverage. The comments effectively communicate the intent to prevent double instrumentation.

tests/fixtures/config-ts/repomix.config.ts (1)

1-12: LGTM! Consistent pattern applied across all fixtures.

The plain object export and explanatory comments align with the overall PR strategy to stabilize test coverage by preventing jiti from transforming src/ files during test fixture loading.

tests/core/output/outputGenerate.test.ts (5)

201-220: LGTM! Git diffs integration test is well-structured.

The test correctly validates that git diffs are included in the output when config.output.git.includeDiffs is enabled, checking for both work tree and staged diff content.


222-254: LGTM! Git logs integration test covers the key scenarios.

The test appropriately validates that git logs are included when config.output.git.includeLogs is enabled, checking for the logs section header and commit details.


256-276: LGTM! JSON output test validates the format correctly.

The test appropriately validates the JSON output style, ensuring the files are correctly mapped as a JSON object with file paths as keys and content as values.


278-292: LGTM! File exclusion test validates the negative case correctly.

The test appropriately verifies that when config.output.files is set to false, file entries are excluded from the generated output.


294-307: LGTM! Directory structure exclusion test validates the feature correctly.

The test appropriately verifies that when config.output.directoryStructure is set to false, the directory structure section is excluded from the generated output.

tests/config/configLoad.integration.test.ts (2)

53-78: LGTM! This is the key fix for coverage instability.

The jitiImport mock prevents jiti from transforming src/ files during test execution, which was causing double instrumentation. The test still validates config structure and the loading mechanism while ensuring deterministic results.

Note: The trade-off is that this test no longer validates the actual fixture file content, but this is intentional and documented in the comments.


125-150: LGTM! JavaScript config test mirrors the TypeScript approach.

The jitiImport mock consistently prevents coverage instability across both JavaScript and TypeScript config loading scenarios. The implementation follows the same pattern as the TypeScript test while using an appropriate mock fixture for JavaScript files.

Added comprehensive test coverage for critical CLI and core functionality:

- Created new test file for cliSpinner with 15 tests covering:
  * Spinner start/stop/update operations
  * Quiet/verbose/stdout mode handling
  * Success/fail message display
  * Interval management

- Enhanced initAction tests (11→17 tests):
  * Added isCancel handling for user cancellation
  * Added return value validation tests
  * Covered config and ignore file creation flows

- Enhanced cliReport tests (8→15 tests):
  * Added git diffs/logs reporting tests
  * Added security check reporting for git content
  * Added single vs multiple issue message handling

- Enhanced permissionCheck tests (13→16 tests):
  * Added macOS-specific error message tests
  * Added platform-specific error handling tests
  * Added unknown error code handling

- Enhanced outputGenerate tests (7→12 tests):
  * Added git diffs/logs inclusion tests
  * Added JSON format output tests
  * Added file/directory structure exclusion tests

Overall improvements:
- Test count: 804 → 840 (+36 tests)
- Code coverage: 70.63% → 71.00% (+0.37%)
- Branch coverage: 77.64% → 78.55% (+0.91%)
- Significant improvement in CLI modules (cliSpinner: 25% → 59.61%)
Added comprehensive error handling tests for configLoad.ts to improve coverage:

- Added test for unsupported config file format (.yaml, .toml)
- Added test for general error handling (permission denied)
- Added test for non-Error object error handling
- Improved branch coverage for error cases in loadAndValidateConfig

This addresses the coverage drop that occurred in commit 156897d when
TypeScript/JavaScript config file support was added.

Improvements:
- Test count: 840 → 844 (+4 tests)
- Config branch coverage: 80.88% → 86.11% (+5.23%)
- Overall branch coverage: 78.55% → 78.76% (+0.21%)
…ntation

This commit resolves test coverage instability that occurred after
implementing JS/TS config support. Coverage was fluctuating between
68-71% on each test run, caused by jiti dynamically transforming src/
files that were already instrumented by Vitest for coverage tracking.

Root Cause:
- Test fixture files imported `defineConfig` from src/index.js
- jiti transformed the entire src/ directory when loading fixtures
- Same files existed in two versions (Vitest-instrumented vs jiti-transformed)
- v8 coverage tracker produced non-deterministic results

Solution:
1. Added DI pattern to loadFileConfig with defaultJitiImport factory
2. Removed defineConfig imports from all test fixtures (use plain objects)
3. Mock jiti in dynamic config tests to prevent src/ transformation

Results:
- Coverage now stable at 89.34% (previously 68-71%)
- Coverage improved by ~19% due to accurate instrumentation
- Integration test runtime improved from 3.8s to 0.1s
- All tests pass with zero lint errors

Related Changes:
- Reverted temporary Vitest config workarounds (sequence.shuffle, isolate)
- Extracted defaultJitiImport as reusable factory function
Implemented mandatory and recommended fixes from code review:

1. Fix timer cleanup in cliSpinner.test.ts
   - Added vi.useRealTimers() to afterEach hook
   - Prevents fake timers from bleeding into other test files

2. Fix isCancel mock in initAction.test.ts (3 locations)
   - Changed from mockReturnValue(true) to mockImplementation
   - Now properly tests only the actual cancel symbol
   - More accurate test behavior for cancellation scenarios

3. Add DI pattern explanation comment in configLoad.ts
   - Clarifies why dependency injection is used for jitiImport
   - Documents the double instrumentation issue
   - Helps future maintainers understand the design decision
@claude
Copy link
Contributor

claude bot commented Oct 30, 2025

Pull Request Review

Excellent work on resolving the test coverage instability issue! This PR demonstrates a deep understanding of the problem and implements a clean solution. The coverage improvement from ~68-71% (unstable) to 89.34% (stable) is impressive.

Summary

Approved - This PR successfully resolves the double instrumentation issue through dependency injection while maintaining code quality and improving test coverage.

Strengths

  1. Root Cause Analysis: The problem identification is spot-on - jiti's dynamic transformation of already-instrumented src/ files was causing non-deterministic coverage results
  2. Clean Implementation: The dependency injection pattern is well-implemented and follows the project's existing patterns
  3. Comprehensive Testing: Added extensive test coverage including edge cases (spinner, report, init action, etc.)
  4. Performance Improvement: 38x faster integration tests (3.8s → 0.1s) is a significant win
  5. Documentation: Excellent inline comments explaining the rationale for the DI pattern

Code Quality Assessment

Architecture (Excellent)

  • src/config/configLoad.ts:60-67: The defaultJitiImport factory is well-structured
  • src/config/configLoad.ts:69-75: Dependency injection follows the project's established pattern
  • src/config/configLoad.ts:118-120: Clear comment explaining the DI rationale

Testing (Excellent)

  • Integration tests now mock jiti properly to prevent double instrumentation
  • Added 200+ lines of new test coverage across multiple modules
  • Edge cases are well covered (cancel scenarios, error handling, etc.)

Potential Issues & Suggestions

Details

1. Type Safety Consideration (Minor)

Location: src/config/configLoad.ts:61

The defaultJitiImport returns Promise<unknown>, which is correct for flexibility. However, consider if stricter typing would be beneficial:

// Current
const defaultJitiImport = async (fileUrl: string): Promise<unknown> => {
  // ...
};

// Suggestion (if beneficial)
type JitiImportFn = (fileUrl: string) => Promise<unknown>;

This would make the dependency injection interface more explicit, though the current approach is perfectly acceptable.

2. Test Mock Verification

Location: tests/config/configLoad.integration.test.ts:58-60

The mock implementation verifies the file URL contains the expected filename:

expect(fileUrl).toContain('repomix-dynamic.config.ts');

Consider using a more precise check to ensure the full path is correct:

expect(fileUrl).toMatch(/repomix-dynamic\.config\.ts$/);

3. Error Handling Edge Case

Location: src/config/configLoad.ts:158-167

The error handling is comprehensive, but consider logging the original error for debugging:

} catch (error) {
  rethrowValidationErrorIfZodError(error, 'Invalid config schema');
  if (error instanceof SyntaxError) {
    logger.trace('Syntax error details:', error); // Add trace logging
    throw new RepomixError(`Invalid syntax in config file ${filePath}: ${error.message}`);
  }
  // ...
}

Premortem Analysis

Potential Failure Scenarios & Mitigations

Scenario 1: Real-world Dynamic Configs Fail

Risk: Users with complex dynamic configs might encounter issues if the mock doesn't capture all use cases
Likelihood: Low
Impact: Medium
Mitigation:

  • The integration tests still run real jiti for non-dynamic configs
  • Consider adding an E2E test that loads a real dynamic config in a separate process

Scenario 2: Jiti API Changes

Risk: Future jiti updates might change the import behavior
Likelihood: Low
Impact: Low
Mitigation:

  • The DI pattern makes it easy to adapt to API changes
  • Consider pinning jiti version or adding version compatibility tests

Scenario 3: Coverage Regression

Risk: Future changes might reintroduce double instrumentation
Likelihood: Low
Impact: Medium
Mitigation:

  • Add a test that verifies coverage stability across multiple runs
  • Document the double instrumentation issue in CONTRIBUTING.md

Scenario 4: Performance Regression

Risk: The mock might hide performance issues in the real jiti import path
Likelihood: Very Low
Impact: Low
Mitigation:

  • Keep some integration tests that use real jiti (already done)
  • Monitor integration test performance in CI

Scenario 5: Testing Environment Differences

Risk: Different Node.js versions might handle module loading differently
Likelihood: Low
Impact: Low
Mitigation:

  • CI already tests multiple Node.js versions
  • The DI pattern isolates environment-specific behavior

Security Considerations

✅ No security concerns identified. The changes are purely test-related and don't affect runtime behavior for end users.

Performance Considerations

Significant improvement: Integration tests are 38x faster (3.8s → 0.1s)
✅ No runtime performance impact for users

Recommendations

  1. Consider adding a smoke test that verifies a real dynamic config loads correctly in a separate test environment
  2. Document the double instrumentation issue in CONTRIBUTING.md to prevent future regressions
  3. Consider adding a coverage stability check in CI that runs tests multiple times and verifies consistent results

Checklist Review

  • npm run test - Passed
  • npm run lint - Passed
  • ✅ Follows Conventional Commits format
  • ✅ Includes proper scope in commit message
  • ✅ Tests are comprehensive
  • ✅ Code follows project guidelines

Conclusion

This is excellent work! The PR successfully resolves the coverage instability issue with a clean, maintainable solution. The dependency injection pattern is well-implemented, thoroughly tested, and follows the project's existing conventions. The performance improvements are a bonus.

Recommendation: Approve and merge after considering the minor suggestions above (all optional).

Great job on the comprehensive testing and clear documentation of the solution! 🎉

@claude
Copy link
Contributor

claude bot commented Oct 30, 2025

PR Review: Test Coverage Stability Fix

Summary

This PR successfully addresses test coverage instability by implementing dependency injection to prevent jiti double instrumentation. The approach is sound and well-executed. Approval recommended with minor suggestions below.

✅ Strengths

  1. Excellent Problem Diagnosis: The root cause analysis is spot-on. The double instrumentation issue (Vitest + jiti both transforming src/ files) was correctly identified and elegantly solved.

  2. Clean DI Implementation: The dependency injection pattern in configLoad.ts follows best practices with default factory function that keeps production code unchanged.

  3. Impressive Coverage Improvement: Stabilized at 89.34% (was fluctuating 68-71%), approximately 19% coverage improvement, and 38x faster integration tests (3.8s to 0.1s).

  4. Comprehensive Test Additions: 728 lines of new test coverage with good edge case coverage.

📋 Code Quality

src/config/configLoad.ts

  • ✅ Clean abstraction with defaultJitiImport factory
  • ✅ Good documentation at line 118-120 explaining rationale
  • ✅ Backward compatible with default parameters
  • ✅ Consistent DI pattern applied uniformly

Test Files

  • ✅ cliSpinner.test.ts: Comprehensive spinner testing with proper timer mocking
  • ✅ initAction.test.ts: Good coverage of cancel scenarios and edge cases
  • ✅ cliReport.test.ts: Thorough testing of git diff/log reporting
  • ✅ permissionCheck.test.ts: Platform-specific error handling
  • ✅ outputGenerate.test.ts: Additional git integration test coverage

Fixture Files

  • ✅ Smart removal of defineConfig imports prevents jiti transformation
  • ✅ Clear comments explain why defineConfig was removed
  • ✅ Type safety maintained through schema validation

🔍 Premortem Analysis

Integration Test False Positives (Risk: Medium)

  • Dynamic config tests now use mocks instead of real file loading
  • Real-world dynamic config issues might not be caught
  • Suggestion: Add ONE real dynamic config integration test alongside mocked tests

Coverage Regression (Risk: Low)

  • Future changes to jiti configuration could reintroduce instability
  • Suggestion: Add comment in vitest config explaining why jiti mocking matters

Deployment Risk (Risk: Very Low)

  • Production behavior should remain unchanged
  • Suggestion: Manual smoke test with real config files before release

🐛 Potential Issues

Minor Issues Found:

  1. Dynamic config tests mock file loading but don't verify actual dynamic behavior (Severity: Minor)
  2. Error message at configLoad.ts:154 could include extension for better debugging (Severity: Very Minor)
  3. Tests use empty object type assertions instead of mock helper (Severity: Very Minor)

🔒 Security

No security issues identified

  • DI pattern doesn't introduce attack vectors
  • Mock injection only available in test context

⚡ Performance

Excellent improvements

  • Integration tests: 38x faster
  • No production performance impact
  • Coverage computation more stable

🧪 Test Coverage

Overall: Excellent

New coverage includes 625+ lines across cliSpinner, initAction, cliReport, permissionCheck, outputGenerate, and configLoad tests. Test quality is high with clear BDD-style naming, proper setup/teardown, and clean mock management.

📝 Suggestions

  1. Add JSDoc to DI parameter (Optional, Low Priority)
  2. Consider one real dynamic config test (Optional, Medium Priority)
  3. Add integration test documentation (Optional, Low Priority)
  4. Verify no regression (Recommended, High Priority): Manual test with real dynamic config before merge

✅ Checklist

  • ✅ npm run test - Confirmed
  • ✅ npm run lint - Confirmed
  • ✅ Commit message follows conventions
  • ✅ Clear PR summary with context

🎯 Final Recommendation

APPROVE

Excellent work solving a real problem with minimal risk. The DI pattern is clean, test coverage is comprehensive, and performance improvements are substantial. Minor suggestions above are optional refinements, not blockers.

Confidence: High | Risk: Low | Merge Readiness: Ready after optional smoke test

Great job on the 38x test speedup! 🚀

@yamadashy yamadashy merged commit 6d3485a into main Oct 30, 2025
48 checks passed
@yamadashy yamadashy deleted the chore/coverage branch October 30, 2025 16:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants