|
| 1 | +# CLAUDE.md |
| 2 | + |
| 3 | +This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. |
| 4 | + |
| 5 | +## Overview |
| 6 | + |
| 7 | +The `nf-core/tools` repository contains a comprehensive Python package that provides command-line tools for the nf-core community. This package helps users run, create, and develop nf-core Nextflow pipelines with integrated support for modules, subworkflows, and schema management. |
| 8 | + |
| 9 | +## Development Commands |
| 10 | + |
| 11 | +### Testing |
| 12 | + |
| 13 | +```bash |
| 14 | +# Run all tests with pytest |
| 15 | +python -m pytest |
| 16 | + |
| 17 | +# Run specific test file |
| 18 | +python -m pytest tests/test_<module>.py |
| 19 | + |
| 20 | +# Run tests with coverage |
| 21 | +python -m pytest --cov --cov-config=.coveragerc |
| 22 | + |
| 23 | +# Run with verbose output |
| 24 | +python -m pytest -v |
| 25 | + |
| 26 | +# Run tests with specific markers |
| 27 | +python -m pytest -m "datafiles" |
| 28 | +``` |
| 29 | + |
| 30 | +### Code Quality and Linting |
| 31 | + |
| 32 | +```bash |
| 33 | +# Run pre-commit hooks (includes ruff formatting and linting) |
| 34 | +pre-commit run --all-files |
| 35 | + |
| 36 | +# Run ruff linter |
| 37 | +ruff check . |
| 38 | + |
| 39 | +# Run ruff formatter |
| 40 | +ruff format . |
| 41 | + |
| 42 | +# Run mypy type checking |
| 43 | +mypy nf_core/ |
| 44 | +``` |
| 45 | + |
| 46 | +### Development Setup |
| 47 | + |
| 48 | +```bash |
| 49 | +# Install in development mode with dev dependencies |
| 50 | +pip install --upgrade -r requirements-dev.txt -e . |
| 51 | + |
| 52 | +# Install pre-commit hooks |
| 53 | +pre-commit install |
| 54 | + |
| 55 | +# Build package |
| 56 | +python -m build |
| 57 | + |
| 58 | +# Install package locally |
| 59 | +pip install -e . |
| 60 | +``` |
| 61 | + |
| 62 | +## Architecture Overview |
| 63 | + |
| 64 | +### Core Structure |
| 65 | + |
| 66 | +The codebase is organized into several key domains: |
| 67 | + |
| 68 | +**Command Structure**: The main CLI interface is defined in `nf_core/__main__.py` using Click with rich-click for enhanced formatting. Commands are organized hierarchically: |
| 69 | + |
| 70 | +- `pipelines/` - Pipeline management (create, lint, download, sync, etc.) |
| 71 | +- `modules/` - Module management (install, update, create, test, etc.) |
| 72 | +- `subworkflows/` - Subworkflow management (similar to modules) |
| 73 | +- `test_datasets/` - Test dataset management |
| 74 | + |
| 75 | +**Component Architecture**: |
| 76 | + |
| 77 | +- `nf_core/components/` - Shared functionality for modules and subworkflows |
| 78 | +- `nf_core/pipelines/` - Pipeline-specific operations including lint tests |
| 79 | +- `nf_core/modules/` - Module-specific operations and lint tests |
| 80 | +- `nf_core/subworkflows/` - Subworkflow-specific operations |
| 81 | + |
| 82 | +### Key Design Patterns |
| 83 | + |
| 84 | +**Lint System**: Both pipelines and modules use a comprehensive lint system: |
| 85 | + |
| 86 | +- `nf_core/pipelines/lint/` - Contains individual lint test files |
| 87 | +- `nf_core/modules/lint/` - Module-specific lint tests |
| 88 | +- Each lint test is a separate Python module with standardized interface |
| 89 | + |
| 90 | +**Template System**: Templates are stored in dedicated directories: |
| 91 | + |
| 92 | +- `nf_core/pipeline-template/` - Complete pipeline template with Jinja2 templating |
| 93 | +- `nf_core/module-template/` - Module template for creating new modules |
| 94 | +- `nf_core/subworkflow-template/` - Subworkflow template |
| 95 | + |
| 96 | +**Component Management**: Modules and subworkflows share common patterns: |
| 97 | + |
| 98 | +- JSON tracking files (`modules.json`) for version management |
| 99 | +- Git-based remote repository integration |
| 100 | +- Local vs remote component distinction |
| 101 | +- Patch system for local modifications |
| 102 | + |
| 103 | +### Data Flow |
| 104 | + |
| 105 | +1. **Command Parsing**: Click commands in `__main__.py` parse arguments and delegate to command modules |
| 106 | +2. **Context Management**: Click context objects carry configuration and state between commands |
| 107 | +3. **Git Operations**: Remote repository operations for fetching modules/subworkflows |
| 108 | +4. **Template Processing**: Jinja2 templating for generating new pipelines/modules |
| 109 | +5. **Lint Execution**: Modular lint system with individual test execution and reporting |
| 110 | + |
| 111 | +## Testing Strategy |
| 112 | + |
| 113 | +### Test Organization |
| 114 | + |
| 115 | +- Tests mirror the source structure under `tests/` |
| 116 | +- Integration tests use real pipeline/module examples |
| 117 | +- Snapshot testing for CLI output with pytest-textual-snapshot |
| 118 | +- Workflow testing with pytest-workflow for pipeline execution |
| 119 | + |
| 120 | +### Test Data |
| 121 | + |
| 122 | +- `tests/data/` - Mock configurations and test data |
| 123 | +- `tests/fixtures/` - Reusable test fixtures |
| 124 | +- Pipeline templates and module examples for integration testing |
| 125 | + |
| 126 | +### Key Testing Patterns |
| 127 | + |
| 128 | +- Use `@pytest.mark.datafiles` for tests requiring file fixtures |
| 129 | +- Snapshot tests for CLI output verification |
| 130 | +- Mock external dependencies (GitHub API, Docker registry) |
| 131 | +- Parameterized tests for multiple scenarios |
| 132 | + |
| 133 | +## Important Implementation Details |
| 134 | + |
| 135 | +### Configuration Management |
| 136 | + |
| 137 | +- Global configuration in `~/.nfcore/` directory |
| 138 | +- Rich console output with color support |
| 139 | +- Environment variable support with `NFCORE_` prefix |
| 140 | +- Logging configuration with file and console handlers |
| 141 | + |
| 142 | +### Error Handling |
| 143 | + |
| 144 | +- Custom exception hierarchy for different error types |
| 145 | +- Selective traceback display for user-friendly error messages |
| 146 | +- Graceful degradation for network failures |
| 147 | + |
| 148 | +### Performance Considerations |
| 149 | + |
| 150 | +- Parallel processing for download operations |
| 151 | +- Caching for remote repository operations |
| 152 | +- Progress bars for long-running operations |
| 153 | +- Lazy loading of large data structures |
| 154 | + |
| 155 | +## Development Notes |
| 156 | + |
| 157 | +### Adding New Commands |
| 158 | + |
| 159 | +1. Add command function to appropriate `commands_*.py` file |
| 160 | +2. Add Click decorator and options in `__main__.py` |
| 161 | +3. Update command groups in `COMMAND_GROUPS` dictionary |
| 162 | +4. Add comprehensive tests in `tests/` directory |
| 163 | +5. Update documentation if needed |
| 164 | + |
| 165 | +### Lint Test Development |
| 166 | + |
| 167 | +1. Create new lint test module in appropriate `lint/` directory |
| 168 | +2. Implement test class with `run()` method |
| 169 | +3. Add test to main lint runner |
| 170 | +4. Create comprehensive test cases |
| 171 | +5. Update documentation with test description |
| 172 | + |
| 173 | +### Template Updates |
| 174 | + |
| 175 | +1. Modify template files in respective `-template/` directories |
| 176 | +2. Test template generation with various options |
| 177 | +3. Update template tests and snapshots |
| 178 | +4. Consider backward compatibility for existing pipelines |
| 179 | + |
| 180 | +This codebase emphasizes modularity, comprehensive testing, and user experience through rich CLI interfaces and detailed error reporting. |
0 commit comments