Conversation
…itory output When copying output files fails due to permission errors (EPERM/EACCES), provide clear and actionable error messages that explain the issue and offer specific solutions. This commonly occurs when users run repomix from protected directories like C:\Windows\System32 on Windows. Fixes #868
Disable special token validation in tiktoken encoding by passing empty arrays for allowed_special and disallowed_special parameters. This prevents "special token not allowed" errors when processing files that contain special token sequences like <|endoftext|>. This approach treats special tokens as ordinary text, which is appropriate for general code/text analysis where we're not sending content to LLM APIs. Common in tokenizer configuration files (vocab.json) which frequently contain special token definitions. Fixes #868
|
Important Review skippedAuto incremental reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the Note Other AI code review bot(s) detectedCodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review. WalkthroughThis PR addresses two issues: (1) improves error handling for permission-related failures when copying output files to the current directory by throwing descriptive Changes
Sequence DiagramsequenceDiagram
participant User
participant RemoteAction
participant FileSystem
participant ErrorHandler
User->>RemoteAction: copyOutputToCurrentDirectory()
RemoteAction->>FileSystem: copyFile(source, dest)
alt Success
FileSystem-->>RemoteAction: file copied
RemoteAction-->>User: ✓ success
else Permission Error (EPERM/EACCES)
FileSystem-->>RemoteAction: error
RemoteAction->>ErrorHandler: detect permission error
ErrorHandler->>ErrorHandler: compose friendly message
ErrorHandler-->>RemoteAction: RepomixError with guidance
RemoteAction-->>User: ✗ descriptive error
else Other Error
FileSystem-->>RemoteAction: error
RemoteAction-->>User: ✗ original error
end
Estimated code review effort🎯 2 (Simple) | ⏱️ ~15 minutes The changes follow consistent patterns: one is a straightforward error handling enhancement with corresponding tests, while the other is a targeted parameter modification to function calls across test files. Both modifications are localized with clear intent and existing logic paths remain unchanged. Possibly related PRs
Pre-merge checks and finishing touches✅ Passed checks (5 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Summary of ChangesHello @yamadashy, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly improves the robustness and user experience of the application by addressing two critical issues: providing more informative and helpful error messages for file permission failures, and ensuring accurate token counting by correctly handling special tokens in various file types. These changes aim to prevent common user frustrations and ensure consistent functionality across different operating systems and content types. Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
There was a problem hiding this comment.
Pull Request Overview
Purpose: Improve handling of permission errors during output file copy operations and adjust token counting to ignore special token validation (fixing Issue #868).
- Adds explicit EPERM/EACCES handling with actionable guidance in copyOutputToCurrentDirectory.
- Modifies TokenCounter to pass empty arrays to encode to treat special tokens as ordinary text and updates related tests accordingly.
Reviewed Changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| src/cli/actions/remoteAction.ts | Adds specific permission error handling with a detailed user-facing message. |
| src/core/metrics/TokenCounter.ts | Adjusts token encoding to disable special token validation and adds explanatory comments. |
| tests/cli/actions/remoteAction.test.ts | Adds tests asserting improved permission error messaging for EPERM and EACCES. |
| tests/core/metrics/TokenCounter.test.ts | Updates expectations to reflect new encode invocation signature with empty arrays. |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request effectively addresses two separate issues: improving error messages for permission errors on Windows and fixing token counting for files with special tokens. The changes are well-implemented. The error handling for file copy operations is now much more user-friendly, providing clear instructions on how to resolve the issue. The fix for token counting correctly handles special tokens by treating them as regular text, which is appropriate for this tool's purpose. The accompanying tests are thorough and validate the new behavior. I have one suggestion to make the error handling code in remoteAction.ts even more robust and readable.
Deploying repomix with
|
| Latest commit: |
4a87223
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://a47cd64c.repomix.pages.dev |
| Branch Preview URL: | https://fix-issue-868-permission-and.repomix.pages.dev |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
tests/core/metrics/TokenCounter.test.ts (1)
50-136: Test expectations correctly updated to match new encode signature.All tests now expect
encode(text, [], [])calls, which aligns with the implementation change in TokenCounter.ts.Consider adding a specific test case that validates token counting for content containing special token sequences (e.g.,
<|endoftext|>). This would explicitly verify the fix for Issue #868:test('should handle content with special token sequences', () => { const text = 'tokenizer_config = {"eos_token": "<|endoftext|>"}'; const mockTokens = [1, 2, 3, 4, 5]; mockEncoder.encode.mockReturnValue(mockTokens); const count = tokenCounter.countTokens(text); expect(count).toBe(5); expect(mockEncoder.encode).toHaveBeenCalledWith(text, [], []); });
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (4)
src/cli/actions/remoteAction.ts(1 hunks)src/core/metrics/TokenCounter.ts(1 hunks)tests/cli/actions/remoteAction.test.ts(1 hunks)tests/core/metrics/TokenCounter.test.ts(8 hunks)
🧰 Additional context used
🧬 Code graph analysis (2)
tests/cli/actions/remoteAction.test.ts (1)
src/cli/actions/remoteAction.ts (1)
copyOutputToCurrentDirectory(186-225)
src/cli/actions/remoteAction.ts (1)
src/shared/errorHandle.ts (1)
RepomixError(6-11)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (13)
- GitHub Check: Build and run (macos-latest, 24.x)
- GitHub Check: Test with Bun (macos-latest, latest)
- GitHub Check: Lint TypeScript
- GitHub Check: Build and run (windows-latest, 22.x)
- GitHub Check: Build and run (windows-latest, 24.x)
- GitHub Check: Build and run (windows-latest, 20.x)
- GitHub Check: Build and run (ubuntu-latest, 24.x)
- GitHub Check: Build and run (ubuntu-latest, 20.x)
- GitHub Check: Build and run (ubuntu-latest, 22.x)
- GitHub Check: Lint GitHub Actions
- GitHub Check: Memory Test
- GitHub Check: claude-review
- GitHub Check: Cloudflare Pages
🔇 Additional comments (4)
tests/cli/actions/remoteAction.test.ts (2)
260-272: LGTM! Good coverage for Windows permission scenario.The test correctly simulates an EPERM error when copying to a protected Windows directory (C:\Windows\System32) and validates that the error message includes actionable guidance.
274-286: LGTM! Good coverage for Unix/Linux permission scenario.The test correctly simulates an EACCES error when copying to a protected directory and validates that the same helpful error message is provided.
src/cli/actions/remoteAction.ts (1)
209-221: Excellent enhancement for permission error handling.The implementation properly detects EPERM and EACCES errors and provides clear, actionable guidance with three remediation options. This directly addresses the Windows System32 permission error from Issue #868.
src/core/metrics/TokenCounter.ts (1)
21-26: Good solution for special token handling.Passing empty arrays to
encode()disables special token validation, treating special tokens as ordinary text. This is appropriate for general code analysis where the content may contain token sequences like<|endoftext|>in tokenizer configuration files.The comment clearly explains the rationale and use case.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #907 +/- ##
==========================================
+ Coverage 74.36% 74.60% +0.23%
==========================================
Files 111 111
Lines 7776 7782 +6
Branches 1448 1450 +2
==========================================
+ Hits 5783 5806 +23
+ Misses 1993 1976 -17 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Code Review - PR #907This PR effectively addresses both issues reported in #868. The implementation is solid with good test coverage. Strengths
Key Suggestions
Security & Performance
SummaryRecommendation: Approve and merge This is a well-implemented PR solving real user problems with clear error messages, proper error handling, and good test coverage. The suggestions above are minor improvements that could be addressed in follow-up PRs. Current implementation is production-ready. Review conducted per CLAUDE.md guidelines |
Use a single template literal instead of string concatenation for better readability and modern JavaScript best practices. Addresses feedback from AI code review (Gemini Code Assist).
Use 'latest' instead of specific version number to avoid schema version mismatch warnings.
PR Review: Fix Permission Errors and Special Token HandlingSummaryThis PR effectively addresses two critical issues from #868:
Strengths
Code Quality & Best Practices✅ Positive Observations
💡 SuggestionsMinor: Consider extracting the error message to a constant // In remoteAction.ts, consider:
const PERMISSION_ERROR_MESSAGE = `Failed to copy output file to {path}: Permission denied.
The current directory may be protected or require elevated permissions.
Please try one of the following:
• Run from a different directory (e.g., your home directory or Documents folder)
• Use the --output flag to specify a writable location: --output ~/repomix-output.xml
• Use --stdout to print output directly to the console`;This would make testing easier and improve maintainability. However, this is very minor and can be addressed in future refactoring. Potential Bugs & Edge Cases
|
PR Review: Fix permission errors and special token handlingOverall Assessment: Approved with Minor Suggestions This PR effectively addresses both issues reported in #868 with clean, well-tested solutions. The code quality is excellent and aligns with the project's standards. Strengths
Code Quality and Best PracticesPermission Error Handling (remoteAction.ts)
Token Counter Changes (TokenCounter.ts)
Test Coverage
Potential Issues and Edge CasesMinor Observations:
Performance Considerations
Security Concerns
Test Coverage AssessmentCurrent Coverage: GoodWhat is Tested:
Potential Additions (Optional):
Test Quality:
Premortem Analysis: Potential Failure ScenariosIdentified Risks and Mitigations:1. tiktoken API Breaking ChangesRisk: Future tiktoken updates might change the encode() signature or behavior
2. Permission Error on Other PlatformsRisk: Different error codes on other OS (macOS, Linux variants, BSDs)
3. Token Count Accuracy ChangeRisk: Disabling special token validation might slightly change token counts
4. Error Message UX IssuesRisk: Multi-line error message formatting issues in different terminals
5. Integration IssuesRisk: Copy operation might fail for other reasons (disk full, network drives)
6. Backward CompatibilityRisk: Breaking changes for users relying on special token error behavior
Suggestions for Improvement
Checklist Verification
ConclusionThis is a high-quality PR that solves real user problems with minimal code changes. The fixes are well-targeted, properly tested, and include excellent user-facing improvements. The premortem analysis reveals only minor risks, all of which are already mitigated. Recommendation: Approve and merge Great work addressing both issues in #868! The improved error messages will significantly enhance user experience, especially for Windows users running from protected directories. |
Summary
Fixes #868
This PR addresses two distinct issues reported in #868:
Changes
1. Permission Error Improvements
File:
src/cli/actions/remoteAction.tsWhen file copy operations fail due to permission errors (
EPERM/EACCES), the error message now:--outputflag to specify writable location--stdoutto print directly to consoleThis commonly occurs when users run repomix from protected directories like
C:\Windows\System32on Windows.2. Special Token Handling
File:
src/core/metrics/TokenCounter.tsFixed token counting errors when processing files containing special tokens (e.g.,
<|endoftext|>in tokenizer vocab.json files):encode()methodBefore:
After:
Test Plan
npm run test- All 800 tests passingnpm run lint- No errorsChecklist
npm run testnpm run lint