Codex/add ignorecontent configuration and cli support br7p2k#822
Codex/add ignorecontent configuration and cli support br7p2k#822Ruddy35 wants to merge 3 commits intoyamadashy:mainfrom
Conversation
fix: include dotfiles in ignoreContent patterns
|
Caution Review failedThe pull request is closed. WalkthroughAdds a new ignore-content mechanism across CLI, config, core file collection/processing, and docs. Introduces --ignore-content and ignoreContent config. File collection tags tasks with skipContent to omit reading contents; downstream processing and security paths filter out files without content. Tests and schemas updated; package.json adds prepare script. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
actor User
participant CLI as CLI
participant Config as ConfigLoad/Merge
participant Collect as FileCollector
participant Worker as FileCollectWorker
participant Pack as Packager
participant Proc as FileProcessor
participant Sec as SecurityCheck
User->>CLI: repomix --ignore-content "<patterns>"
CLI->>Config: buildCliConfig(ignoreContent)
Config-->>CLI: merged config (include, ignore, ignoreContent)
CLI->>Collect: collectFiles(config)
Collect->>Collect: shouldSkipContent(filePath, ignoreContent)
alt skipContent = true
Collect->>Worker: { path, skipContent: true }
Worker-->>Collect: RawFile { path } (no content)
else skipContent = false
Collect->>Worker: { path, skipContent: false }
Worker-->>Collect: RawFile { path, content }
end
Collect-->>Pack: rawFiles (some with undefined content)
Pack->>Pack: filter files with content
Pack->>Proc: process(files with content)
Proc-->>Pack: processed files
Pack->>Sec: runSecurityCheck(files with content)
Sec-->>Pack: suspicious results
Pack-->>User: packaged output
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related PRs
Suggested reviewers
📜 Recent review detailsConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro 📒 Files selected for processing (44)
✨ Finishing Touches
🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Summary of Changes
Hello @Ruddy35, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!
This pull request significantly enhances the file processing capabilities by introducing a mechanism to selectively ignore the content of files based on glob patterns. This allows for more precise control over what data is processed, which can be crucial for managing sensitive information or optimizing performance by skipping large, irrelevant file contents. The feature is integrated into both the command-line interface and the configuration system, complete with robust testing and updated documentation.
Highlights
- New
ignoreContentFeature: Introduced a new configuration option and CLI flag (--ignore-content) to allow users to specify patterns for files whose content should be ignored during processing, while still including their paths. - Flexible Pattern Matching: The
ignoreContentpatterns support glob syntax, including negation (!) to selectively keep content for specific paths within a broader ignored set. - Refined File Processing Flow: The internal file collection and processing logic has been updated to handle files with optional content, ensuring that content-intensive operations like security checks only run on files that actually have their content loaded.
- Comprehensive Documentation: The new feature is fully documented in the
README.mdand across all localized guide pages for both command-line options and configuration files.
Using Gemini Code Assist
The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.
Invoking Gemini
You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.
| Feature | Command | Description |
|---|---|---|
| Code Review | /gemini review |
Performs a code review for the current pull request in its current state. |
| Pull Request Summary | /gemini summary |
Provides a summary of the current pull request in its current state. |
| Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in issue comments and review comments. |
| Help | /gemini help |
Displays a list of available commands. |
Customization
To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.
Limitations & Feedback
Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.
You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.
Footnotes
-
Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩
There was a problem hiding this comment.
Code Review
This pull request introduces a valuable --ignore-content feature, allowing files to be included in the project structure without their content. The implementation is thorough, with updates to the CLI, configuration, core logic, tests, and documentation across multiple languages. My review focuses on improving the robustness of the pattern matching logic, removing a small piece of redundant code, and ensuring documentation consistency. I've also noted that the new feature's documentation needs to be translated in the non-English guides.
| const shouldSkipContent = (filePath: string, patterns: string[]): boolean => { | ||
| let skip = false; | ||
| for (const pattern of patterns) { | ||
| const normalizedPattern = normalizeGlobPattern(pattern.startsWith('!') ? pattern.slice(1) : pattern); | ||
| if (pattern.startsWith('!')) { | ||
| if (minimatch(filePath, normalizedPattern, { dot: true })) { | ||
| skip = false; | ||
| } | ||
| } else if (minimatch(filePath, normalizedPattern, { dot: true })) { | ||
| skip = true; | ||
| } | ||
| } | ||
| return skip; | ||
| }; |
There was a problem hiding this comment.
The current implementation of shouldSkipContent is dependent on the order of patterns in the configuration. For example, with ['!foo', 'foo'], a file named foo will be skipped, but with ['foo', '!foo'], it will not. This can be confusing and lead to unexpected behavior. A more robust approach is to give negated patterns (!) precedence regardless of their order.
const shouldSkipContent = (filePath: string, patterns: string[]): boolean => {
// Negated patterns ("!...") should always take precedence to ensure content is included.
const isExplicitlyIncluded = patterns
.filter((pattern) => pattern.startsWith('!'))
.some((pattern) => minimatch(filePath, normalizeGlobPattern(pattern.slice(1)), { dot: true }));
if (isExplicitlyIncluded) {
return false; // Do not skip content.
}
// If not explicitly included, check if it matches any ignore pattern.
const isIgnored = patterns
.filter((pattern) => !pattern.startsWith('!'))
.some((pattern) => minimatch(filePath, normalizeGlobPattern(pattern), { dot: true }));
return isIgnored;
};| | `output.git.includeLogs` | Apakah akan menyertakan log git dalam output. Menampilkan riwayat commit dengan tanggal, pesan, dan jalur file | `false` | | ||
| | `output.git.includeLogsCount` | Jumlah commit log git yang akan disertakan dalam output | `50` | | ||
| | `include` | Pola file untuk disertakan menggunakan [pola glob](https://github.com/mrmlnc/fast-glob?tab=readme-ov-file#pattern-syntax) | `[]` | | ||
| | `ignoreContent` | Patterns of files whose content should be ignored (using [glob patterns](https://github.com/mrmlnc/fast-glob?tab=readme-ov-file#pattern-syntax)); prefix with `!` to keep specific paths | `[]` | |
There was a problem hiding this comment.
The description for ignoreContent is in English. It should be translated into Indonesian for consistency.
| | `ignoreContent` | Patterns of files whose content should be ignored (using [glob patterns](https://github.com/mrmlnc/fast-glob?tab=readme-ov-file#pattern-syntax)); prefix with `!` to keep specific paths | `[]` | | |
| | `ignoreContent` | Pola file yang kontennya harus diabaikan (menggunakan [pola glob](https://github.com/mrmlnc/fast-glob?tab=readme-ov-file#pattern-syntax)); awali dengan `!` untuk menyimpan path tertentu | `[]` | |
| ## Options de sélection de fichiers | ||
| - `--include <patterns>`: Liste des motifs d'inclusion (séparés par des virgules) | ||
| - `-i, --ignore <patterns>`: Motifs d'ignorance supplémentaires (séparés par des virgules) | ||
| - `--ignore-content <patterns>`: Skip file content for matched patterns (comma-separated; prefix with `!` to keep specific paths) |
There was a problem hiding this comment.
The description for --ignore-content is in English. It should be translated into French for consistency.
| - `--ignore-content <patterns>`: Skip file content for matched patterns (comma-separated; prefix with `!` to keep specific paths) | |
| - `--ignore-content <patterns>`: Motifs pour ignorer le contenu des fichiers correspondants (séparés par des virgules ; préfixer avec `!` pour conserver des chemins spécifiques) |
| ## Dateiauswahloptionen | ||
| - `--include <patterns>`: Liste der Einschlussmuster (kommagetrennt) | ||
| - `-i, --ignore <patterns>`: Zusätzliche Ignoriermuster (kommagetrennt) | ||
| - `--ignore-content <patterns>`: Skip file content for matched patterns (comma-separated; prefix with `!` to keep specific paths) |
There was a problem hiding this comment.
The description for --ignore-content is in English. It should be translated into German for consistency with the rest of the document.
| - `--ignore-content <patterns>`: Skip file content for matched patterns (comma-separated; prefix with `!` to keep specific paths) | |
| - `--ignore-content <patterns>`: Muster zum Überspringen von Dateiinhalten für übereinstimmende Muster (kommagetrennt; mit `!` voranstellen, um bestimmte Pfade beizubehalten) |
| | `output.git.includeLogs` | Ob Git-Logs in der Ausgabe enthalten sein sollen. Zeigt Commit-Historie mit Daten, Nachrichten und Dateipfaden an | `false` | | ||
| | `output.git.includeLogsCount` | Anzahl der Git-Log-Commits, die in die Ausgabe einbezogen werden sollen | `50` | | ||
| | `include` | Zu einschließende Dateimuster (verwendet [glob-Muster](https://github.com/mrmlnc/fast-glob?tab=readme-ov-file#pattern-syntax)) | `[]` | | ||
| | `ignoreContent` | Patterns of files whose content should be ignored (using [glob patterns](https://github.com/mrmlnc/fast-glob?tab=readme-ov-file#pattern-syntax)); prefix with `!` to keep specific paths | `[]` | |
There was a problem hiding this comment.
The description for ignoreContent is in English. It should be translated into German for consistency.
| | `ignoreContent` | Patterns of files whose content should be ignored (using [glob patterns](https://github.com/mrmlnc/fast-glob?tab=readme-ov-file#pattern-syntax)); prefix with `!` to keep specific paths | `[]` | | |
| | `ignoreContent` | Muster von Dateien, deren Inhalt ignoriert werden soll (verwendet [glob-Muster](https://github.com/mrmlnc/fast-glob?tab=readme-ov-file#pattern-syntax)); mit `!` voranstellen, um bestimmte Pfade beizubehalten | `[]` | |
| | `output.git.includeLogs` | Có nên bao gồm nhật ký git trong đầu ra hay không. Hiển thị lịch sử commit với ngày tháng, thông điệp và đường dẫn tệp | `false` | | ||
| | `output.git.includeLogsCount` | Số lượng commit git logs để bao gồm trong đầu ra | `50` | | ||
| | `include` | Các mẫu file để bao gồm sử dụng [mẫu glob](https://github.com/mrmlnc/fast-glob?tab=readme-ov-file#pattern-syntax) | `[]` | | ||
| | `ignoreContent` | Patterns of files whose content should be ignored (using [glob patterns](https://github.com/mrmlnc/fast-glob?tab=readme-ov-file#pattern-syntax)); prefix with `!` to keep specific paths | `[]` | |
There was a problem hiding this comment.
The description for ignoreContent is in English. It should be translated into Vietnamese for consistency.
| | `ignoreContent` | Patterns of files whose content should be ignored (using [glob patterns](https://github.com/mrmlnc/fast-glob?tab=readme-ov-file#pattern-syntax)); prefix with `!` to keep specific paths | `[]` | | |
| | `ignoreContent` | Các mẫu tệp có nội dung cần bỏ qua (sử dụng [mẫu glob](https://github.com/mrmlnc/fast-glob?tab=readme-ov-file#pattern-syntax)); tiền tố `!` để giữ lại các đường dẫn cụ thể | `[]` | |
| ## 文件选择选项 | ||
| - `--include <patterns>`: 包含模式列表（逗号分隔） | ||
| - `-i, --ignore <patterns>`: 附加忽略模式（逗号分隔） | ||
| - `--ignore-content <patterns>`: Skip file content for matched patterns (comma-separated; prefix with `!` to keep specific paths) |
There was a problem hiding this comment.
The description for --ignore-content is in English. It should be translated into Simplified Chinese for consistency.
| - `--ignore-content <patterns>`: Skip file content for matched patterns (comma-separated; prefix with `!` to keep specific paths) | |
| - `--ignore-content <patterns>`: 跳过匹配模式的文件内容（逗号分隔；以`!`为前缀保留特定路径） |
| | `output.git.includeLogs` | 是否在输出中包含Git日志。显示提交历史的日期、消息和文件路径 | `false` | | ||
| | `output.git.includeLogsCount` | 要包含的Git日志提交数量。限制历史深度以了解开发模式 | `50` | | ||
| | `include` | 要包含的文件模式（使用[glob模式](https://github.com/mrmlnc/fast-glob?tab=readme-ov-file#pattern-syntax)） | `[]` | | ||
| | `ignoreContent` | Patterns of files whose content should be ignored (using [glob patterns](https://github.com/mrmlnc/fast-glob?tab=readme-ov-file#pattern-syntax)); prefix with `!` to keep specific paths | `[]` | |
There was a problem hiding this comment.
The description for ignoreContent is in English. It should be translated into Simplified Chinese for consistency.
| | `ignoreContent` | Patterns of files whose content should be ignored (using [glob patterns](https://github.com/mrmlnc/fast-glob?tab=readme-ov-file#pattern-syntax)); prefix with `!` to keep specific paths | `[]` | | |
| | `ignoreContent` | 其内容应被忽略的文件的模式（使用[glob模式](https://github.com/mrmlnc/fast-glob?tab=readme-ov-file#pattern-syntax)）；以`!`为前缀保留特定路径 | `[]` | |
| ## 檔案選擇選項 | ||
| - `--include <patterns>`: 包含模式清單（逗號分隔） | ||
| - `-i, --ignore <patterns>`: 附加忽略模式（逗號分隔） | ||
| - `--ignore-content <patterns>`: Skip file content for matched patterns (comma-separated; prefix with `!` to keep specific paths) |
There was a problem hiding this comment.
The description for --ignore-content is in English. It should be translated into Traditional Chinese for consistency.
| - `--ignore-content <patterns>`: Skip file content for matched patterns (comma-separated; prefix with `!` to keep specific paths) | |
| - `--ignore-content <patterns>`: 跳過符合模式的檔案內容（逗號分隔；以`!`為前綴以保留特定路徑） |
| | `output.git.includeLogs` | 是否在輸出中包含Git記錄。顯示提交歷史包括日期、訊息和檔案路徑 | `false` | | ||
| | `output.git.includeLogsCount` | 在輸出中包含的git記錄提交數量 | `50` | | ||
| | `include` | 要包含的檔案模式（使用[glob模式](https://github.com/mrmlnc/fast-glob?tab=readme-ov-file#pattern-syntax)） | `[]` | | ||
| | `ignoreContent` | Patterns of files whose content should be ignored (using [glob patterns](https://github.com/mrmlnc/fast-glob?tab=readme-ov-file#pattern-syntax)); prefix with `!` to keep specific paths | `[]` | |
There was a problem hiding this comment.
The description for ignoreContent is in English. It should be translated into Traditional Chinese for consistency.
| | `ignoreContent` | Patterns of files whose content should be ignored (using [glob patterns](https://github.com/mrmlnc/fast-glob?tab=readme-ov-file#pattern-syntax)); prefix with `!` to keep specific paths | `[]` | | |
| | `ignoreContent` | 其內容應被忽略的檔案的模式（使用[glob模式](https://github.com/mrmlnc/fast-glob?tab=readme-ov-file#pattern-syntax)）；以`!`為前綴以保留特定路徑 | `[]` | |
Ruddy35
deleted the
codex/add-ignorecontent-configuration-and-cli-support-br7p2k
branch
1 participant
Checklist
npm run testnpm run lint