From 6a49900fb60ecb2937b42e60130a0a6cfc31f1c3 Mon Sep 17 00:00:00 2001 From: Jakub Jares Date: Tue, 5 May 2026 16:11:47 +0200 Subject: [PATCH 1/5] Add Python and TypeScript extension files for code-testing pipeline First-draft language extension files following the dotnet.md pattern. These guide the polyglot test agent on build/test/lint/fix for each language. Python: pytest-focused, environment/runner detection (Poetry/PDM/uv/Hatch), public-API testing philosophy, common errors, mocking guidelines. TypeScript: package-manager detection (npm/pnpm/yarn/bun), Jest+Vitest+Mocha support, ESM/CJS guidance, TS-specific considerations, framework detection. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../skills/code-testing-extensions/SKILL.md | 2 + .../extensions/python.md | 223 ++++++++++++++ .../extensions/typescript.md | 288 ++++++++++++++++++ 3 files changed, 513 insertions(+) create mode 100644 plugins/dotnet-test/skills/code-testing-extensions/extensions/python.md create mode 100644 plugins/dotnet-test/skills/code-testing-extensions/extensions/typescript.md diff --git a/plugins/dotnet-test/skills/code-testing-extensions/SKILL.md b/plugins/dotnet-test/skills/code-testing-extensions/SKILL.md index cd74bfee2d..80c690d0c1 100644 --- a/plugins/dotnet-test/skills/code-testing-extensions/SKILL.md +++ b/plugins/dotnet-test/skills/code-testing-extensions/SKILL.md @@ -18,6 +18,8 @@ This skill provides access to language-specific guidance files used by the code- | File | Language | Contents | |------|----------|----------| | [extensions/dotnet.md](extensions/dotnet.md) | .NET (C#/F#/VB) | Build commands, test commands, project reference validation, common CS error codes, MSTest template | +| [extensions/python.md](extensions/python.md) | Python | Build/test commands (pytest), project layout detection, mocking guidelines, common errors, pytest template | +| [extensions/typescript.md](extensions/typescript.md) | TypeScript/JavaScript | Build/test commands (Jest/Vitest/Mocha), framework detection, mocking, TS-specific considerations | | [extensions/cpp.md](extensions/cpp.md) | C++ | Testing internals with friend declarations | | [extensions/dotnet-examples.md](extensions/dotnet-examples.md) | .NET (C#/F#/VB) | Concrete pipeline examples: sample research output, plan, generated tests, fix cycles, final report | diff --git a/plugins/dotnet-test/skills/code-testing-extensions/extensions/python.md b/plugins/dotnet-test/skills/code-testing-extensions/extensions/python.md new file mode 100644 index 0000000000..9209f064c4 --- /dev/null +++ b/plugins/dotnet-test/skills/code-testing-extensions/extensions/python.md @@ -0,0 +1,223 @@ +# Python Extension + +Language-specific guidance for Python test generation. + +## Environment and Runner Detection + +Before running any commands, detect the project's package manager/runner from lockfiles and config: + +| Indicator | Runner | Command prefix | +|-----------|--------|---------------| +| `poetry.lock` or `[tool.poetry]` in `pyproject.toml` | Poetry | `poetry run` | +| `pdm.lock` or `[tool.pdm]` in `pyproject.toml` | PDM | `pdm run` | +| `uv.lock` or `[tool.uv]` in `pyproject.toml` | uv | `uv run` | +| `Pipfile.lock` | Pipenv | `pipenv run` | +| `hatch.toml` or `[tool.hatch]` in `pyproject.toml` | Hatch | `hatch run` | +| None of the above | pip/venv | Use `python -m` prefix | + +- **Always prefer `python -m pytest` over bare `pytest`** — this ensures the correct interpreter and avoids PATH issues +- **Always prefer `python -m pip` over bare `pip`** for the same reason +- If a `Makefile`, `tox.ini`, or `nox` config exists, prefer the project's existing build/test commands +- **Do not add a new test framework if one already exists** — follow the repo's established choices + +## Build Commands + +Python is interpreted — there is no separate build step. Validate syntax and imports by running the tests or using a type checker if configured. + +| Scope | Command | +|-------|---------| +| Syntax check | `python -m py_compile path/to/file.py` | +| Type check (if configured) | `mypy path/to/file.py` or `pyright path/to/file.py` | + +- Check for `pyproject.toml`, `setup.py`, `setup.cfg`, or `requirements.txt` to understand the project layout +- If the project uses editable installs, run `python -m pip install -e .` or `python -m pip install -e ".[dev]"` before testing + +## Test Commands + +| Scope | Command | +|-------|---------| +| All tests | `pytest` | +| Specific file | `pytest tests/test_module.py` | +| Specific test | `pytest tests/test_module.py::TestClass::test_method` | +| Keyword filter | `pytest -k "keyword"` | +| Verbose output | `pytest -v` | +| Stop on first failure | `pytest -x` | + +- Prefer `pytest` over `unittest` — most Python projects use pytest even when test classes inherit `unittest.TestCase` +- **Prefer the project's existing test script** (`make test`, `tox`, `nox`) over raw `pytest` commands +- If `pytest` is not installed, check `pyproject.toml` `[project.optional-dependencies]` or `requirements-dev.txt` +- Use `pytest --tb=short` to reduce traceback noise during fix cycles +- If the project uses `unittest` exclusively (no pytest in dependencies), use `python -m unittest discover` + +## Lint Command + +Prefer the project's existing lint script (e.g., `make lint`, `tox -e lint`) over running tools directly. If no script exists, detect which tools the project uses from `pyproject.toml`, `.flake8`, `setup.cfg`, or `ruff.toml`: + +```bash +# Prefer ruff when available (fast, covers linting + formatting + import sorting) +ruff check --fix path/to/test_file.py +ruff format path/to/test_file.py + +# Fallback alternatives +black path/to/test_file.py # formatting +flake8 path/to/test_file.py # linting +isort path/to/test_file.py # import sorting +``` + +## Dependency Validation + +Before writing test code, verify the test dependencies are available: + +1. **pytest**: Check `pyproject.toml` or `requirements*.txt` for `pytest` +2. **Source package**: If the source code is in a `src/` layout, verify the package is importable (editable install) +3. **Mocking**: `unittest.mock` is in the stdlib — no extra dependency needed +4. **pytest plugins**: Check for `pytest-asyncio` (async tests), `pytest-mock` (mocker fixture) + +If imports fail with `ModuleNotFoundError`: + +```bash +python -m pip install -e . # Install source package in editable mode +python -m pip install -e ".[dev]" # Install with dev extras +python -m pip install pytest pytest-asyncio # Install test dependencies directly +``` + +## Common Errors + +| Error | Meaning | Fix | +|-------|---------|-----| +| `ModuleNotFoundError` | Package not installed or wrong import path | `pip install -e .` or fix the import statement | +| `ImportError` | Symbol not found in module | Verify the function/class name matches the source exactly | +| `AttributeError` | Wrong attribute on object | Check spelling and that the attribute exists in the source | +| `TypeError: __init__() missing required argument` | Constructor needs more args | Read the `__init__` signature and pass all required parameters | +| `TypeError: takes N positional arguments but M were given` | Wrong number of args | Match the function signature exactly | +| `fixture 'X' not found` | pytest fixture not defined or not imported | Define the fixture or add the correct import/conftest.py | +| `SyntaxError` | Invalid Python syntax in test file | Fix the syntax error at the indicated line | + +## Project Layout Detection + +Python projects use varied layouts. Detect the correct one: + +| Layout | Structure | Import Style | +|--------|-----------|-------------| +| `src/` layout | `src/package/module.py` | `from package.module import X` | +| Flat layout | `package/module.py` at repo root | `from package.module import X` | +| Single module | `module.py` at repo root | `from module import X` | + +- Check `pyproject.toml` `[tool.setuptools.packages.find]` or `[tool.setuptools.package-dir]` for layout hints +- If `conftest.py` exists at the repo root, pytest typically handles path resolution +- **Follow the existing convention first** — check where existing tests live before placing new ones +- If no convention exists, default to `tests/` mirroring the source structure: `src/billing/service.py` → `tests/billing/test_service.py` + +## Test Discovery + +Python tests are discovery-based — no registration step is needed (unlike .NET solutions). Pytest finds tests automatically if naming conventions are followed. + +## Testing Philosophy + +- **Test behavior through the public API** — do not test private functions (prefixed with `_`) directly unless they contain complex algorithms that cannot be adequately exercised through the public surface +- **Prefer output/state assertions over interaction assertions** — verify return values and observable state changes, not internal call counts +- **Do not mock the system under test** — only mock external dependencies (databases, HTTP, filesystem, time) +- **Do not patch private helpers** — if behavior is only reachable through a private function, test it via the public method that calls it +- Fewer focused tests that thoroughly exercise behavior are better than many shallow tests that only check surface behavior + +## Test File Naming + +- Test files must be named `test_*.py` or `*_test.py` (pytest default discovery) +- Test functions must start with `test_` +- Test classes must start with `Test` (no `__init__` method) + +## pytest Template + +```python +import pytest +from package.module import ClassName + + +class TestClassName: + """Tests for ClassName behavior.""" + + def test_method_name_returns_expected_result(self): + # Given + sut = ClassName() + + # When + result = sut.method_name(input_value) + + # Then + assert result == expected_value + + @pytest.mark.parametrize("a,b,expected", [ + (2, 3, 5), + (-1, 1, 0), + (0, 0, 0), + ]) + def test_add_valid_inputs_returns_sum(self, a, b, expected): + # Given + sut = ClassName() + + # When + result = sut.add(a, b) + + # Then + assert result == expected + + def test_method_raises_on_invalid_input(self): + sut = ClassName() + with pytest.raises(ValueError, match="must be positive"): + sut.method_name(-1) +``` + +## Mocking Guidelines + +Use `unittest.mock` (stdlib) or the `pytest-mock` plugin's `mocker` fixture: + +```python +from unittest.mock import Mock, patch, AsyncMock + +# Dependency injection — preferred +def test_service_calls_repository(self): + # Given + repo = Mock() + repo.find.return_value = {"id": 1, "name": "test"} + sut = Service(repository=repo) + + # When + result = sut.get_by_id(1) + + # Then + assert result["name"] == "test" + repo.find.assert_called_once_with(1) + +# Patching module-level dependencies — use sparingly +@patch("package.module.datetime") +def test_uses_current_time(self, mock_dt): + mock_dt.utcnow.return_value = datetime(2024, 1, 1) + # ... +``` + +- Prefer dependency injection over `@patch` — it produces clearer, less brittle tests +- When patching, patch where the name is **looked up**, not where it is defined +- Use `Mock(spec=RealClass)` to catch attribute errors early +- Use `AsyncMock` for async functions +- If a test needs more than 3 mocks, flag it as a design smell + +## Async Tests + +If the source code uses `async def`, tests need async support: + +```python +import pytest + +@pytest.mark.asyncio +async def test_async_method_returns_result(): + sut = AsyncService() + result = await sut.fetch_data(42) + assert result is not None +``` + +- Requires `pytest-asyncio` package +- Check for `asyncio_mode = "auto"` in `pyproject.toml` — if set, the `@pytest.mark.asyncio` decorator is not needed + +## Skip Coverage Tools + +Do not configure or run code coverage measurement tools (coverage.py, pytest-cov). Coverage is measured separately by the evaluation harness. diff --git a/plugins/dotnet-test/skills/code-testing-extensions/extensions/typescript.md b/plugins/dotnet-test/skills/code-testing-extensions/extensions/typescript.md new file mode 100644 index 0000000000..9b3666c74e --- /dev/null +++ b/plugins/dotnet-test/skills/code-testing-extensions/extensions/typescript.md @@ -0,0 +1,288 @@ +# TypeScript Extension + +Language-specific guidance for TypeScript (and JavaScript) test generation. + +## Package Manager Detection + +Before running any commands, detect the package manager from lockfiles: + +| Indicator | Manager | Run command | Exec command | +|-----------|---------|-------------|-------------| +| `pnpm-lock.yaml` | pnpm | `pnpm test` | `pnpm exec vitest` | +| `yarn.lock` | Yarn | `yarn test` | `yarn vitest` | +| `bun.lockb` or `bun.lock` | Bun | `bun test` | `bunx vitest` | +| `package-lock.json` or none | npm | `npm test` | `npx vitest` | + +- **Always prefer the project's existing scripts** (`npm test`, `pnpm test`, etc.) over raw tool CLIs +- **Do not add a new test framework if one already exists** — follow the repo's established choices +- For monorepos (workspaces), run commands from the package directory, not the root + +## Build Commands + +| Scope | Command | +|-------|---------| +| Type check (project-level) | `npx tsc --noEmit` or `npm run typecheck` | +| Build (if configured) | `npm run build` | + +- Check `package.json` `scripts` for the project's preferred build/typecheck command +- Many projects don't need an explicit build step — the test runner handles transpilation +- If `tsconfig.json` exists, TypeScript is in use; check `strict` mode and `target` settings + +## Test Commands + +Detect the test runner from `package.json` `devDependencies` and `scripts.test`: + +| Runner | All tests | Filtered | Watch mode | +|--------|-----------|----------|------------| +| **Jest** | `npx jest` | `npx jest --testPathPattern="module"` | `npx jest --watch` | +| **Vitest** | `npx vitest run` | `npx vitest run path/to/file` | `npx vitest` | +| **Mocha** | `npx mocha` | `npx mocha --grep "pattern"` | `npx mocha --watch` | + +- Prefer `npm test` or `npm run test` (or equivalent for detected package manager) if it's configured in `package.json` +- Use `npx vitest run` (not `npx vitest`) to run once without watch mode +- For Jest: use `--verbose` for detailed output, `--bail` to stop on first failure +- For Jest: filter by test name with `npx jest path/to/file.test.ts -t "test name"` +- For Vitest: use `--reporter=verbose` for detailed output +- For Vitest: filter by test name with `npx vitest run path/to/file.test.ts -t "test name"` +- Mocha should almost always be invoked via the project's existing script/config — direct CLI only if existing tests already do that + +## Lint Command + +```bash +# ESLint (most common) +npx eslint path/to/test_file.ts --fix + +# Prettier (formatting) +npx prettier --write path/to/test_file.ts + +# Biome (all-in-one) +npx biome check --write path/to/test_file.ts +``` + +- Detect which tools the project uses from `package.json` `devDependencies` and config files (`.eslintrc.*`, `prettier.config.*`, `biome.json`) +- Run `npm run lint -- --fix` if the project has a lint script configured + +## Dependency Validation + +Before writing test code, verify test infrastructure is present: + +1. **Test runner**: Check `package.json` `devDependencies` for `jest`, `vitest`, `mocha`, etc. +2. **Type definitions**: For Jest, ensure `@types/jest` is installed; Vitest includes its own types +3. **TypeScript support**: Jest needs `ts-jest` or `@swc/jest`; Vitest handles TS natively +4. **Assertion library**: Jest/Vitest have built-in `expect`; Mocha typically uses `chai` + +If imports fail or tests won't run: + +```bash +# Jest setup +npm install --save-dev jest ts-jest @types/jest + +# Vitest setup +npm install --save-dev vitest + +# Mocha + Chai setup +npm install --save-dev mocha chai @types/mocha @types/chai ts-node +``` + +## Common Errors + +| Error | Meaning | Fix | +|-------|---------|-----| +| `Cannot find module 'X'` | Import path wrong or package not installed | Fix the import path or `npm install` the package | +| `TS2305: Module has no exported member` | Named export doesn't exist | Check the source file's actual exports | +| `TS2307: Cannot find module` | Missing module or type declarations | Install `@types/package` or check `tsconfig.json` paths | +| `TS2345: Argument type not assignable` | Type mismatch in function call | Match the expected type or use type assertion | +| `TS2339: Property does not exist on type` | Wrong property name or type | Verify property name against the source interface/class | +| `TS7006: Parameter implicitly has 'any' type` | Missing type annotation (strict mode) | Add explicit type annotations | +| `SyntaxError: Unexpected token` | Test runner can't parse TypeScript | Configure `ts-jest`, `@swc/jest`, or use Vitest which handles TS natively | +| `ReferenceError: describe is not defined` | Test globals not available | For Vitest: import from `vitest` or set `globals: true` in config; for Jest: ensure tests run under Jest (not `node`); for Mocha: check test bootstrap | +| `ERR_REQUIRE_ESM` / `Cannot use import statement outside a module` | ESM/CJS mismatch | Set `"type": "module"` in `package.json`, or configure the test runner's transform/loader — see ESM section below | +| `ReferenceError: document is not defined` | Code uses browser APIs | Configure test environment: `testEnvironment: 'jsdom'` (Jest) or `environment: 'jsdom'` (Vitest) | + +## Project Layout Detection + +| Layout | Test Location | Import Style | +|--------|--------------|-------------| +| Colocated | `src/module.test.ts` next to `src/module.ts` | `import { X } from './module'` | +| Separate `__tests__` | `src/__tests__/module.test.ts` | `import { X } from '../module'` | +| Top-level `tests/` | `tests/module.test.ts` | `import { X } from '../src/module'` | + +- Check existing test files to match the project's convention +- If `tsconfig.json` has `paths` aliases (e.g., `@/`), use them in test imports +- For monorepos, import from the package name, not relative paths across packages + +## Test File Naming + +- Jest default: `*.test.ts`, `*.test.tsx`, `*.spec.ts`, `*.spec.tsx`, or files inside `__tests__/` +- Vitest default: same as Jest +- Match the existing project convention — check for `.test.` vs `.spec.` usage +- Place test files to mirror source structure + +## Jest Template + +```typescript +import { ClassName } from '../module'; + +describe('ClassName', () => { + let sut: ClassName; + + beforeEach(() => { + sut = new ClassName(); + }); + + describe('methodName', () => { + it('returns expected result for valid input', () => { + // Arrange + const input = 'test'; + + // Act + const result = sut.methodName(input); + + // Assert + expect(result).toBe(expected); + }); + + it.each([ + { a: 2, b: 3, expected: 5 }, + { a: -1, b: 1, expected: 0 }, + { a: 0, b: 0, expected: 0 }, + ])('add($a, $b) returns $expected', ({ a, b, expected }) => { + expect(sut.add(a, b)).toBe(expected); + }); + + it('throws on invalid input', () => { + expect(() => sut.methodName(null!)).toThrow('must not be null'); + }); + }); +}); +``` + +## Vitest Template + +```typescript +import { describe, it, expect, beforeEach } from 'vitest'; +import { ClassName } from '../module'; + +describe('ClassName', () => { + let sut: ClassName; + + beforeEach(() => { + sut = new ClassName(); + }); + + describe('methodName', () => { + it('returns expected result for valid input', () => { + const result = sut.methodName('test'); + expect(result).toBe(expected); + }); + + it.each([ + { a: 2, b: 3, expected: 5 }, + { a: -1, b: 1, expected: 0 }, + ])('add($a, $b) returns $expected', ({ a, b, expected }) => { + expect(sut.add(a, b)).toBe(expected); + }); + }); +}); +``` + +## Mocking Guidelines + +### Jest + +```typescript +// Manual mock +const mockRepo = { + find: jest.fn().mockResolvedValue({ id: 1, name: 'test' }), + save: jest.fn(), +}; +const sut = new Service(mockRepo as unknown as Repository); + +// Module mock +jest.mock('../repository', () => ({ + Repository: jest.fn().mockImplementation(() => ({ + find: jest.fn().mockResolvedValue({ id: 1 }), + })), +})); + +// Spy on existing method +jest.spyOn(sut, 'methodName').mockReturnValue('mocked'); +``` + +### Vitest + +```typescript +import { vi } from 'vitest'; + +const mockRepo = { + find: vi.fn().mockResolvedValue({ id: 1, name: 'test' }), + save: vi.fn(), +}; +const sut = new Service(mockRepo as unknown as Repository); + +// Module mock +vi.mock('../repository', () => ({ + Repository: vi.fn().mockImplementation(() => ({ + find: vi.fn().mockResolvedValue({ id: 1 }), + })), +})); +``` + +- Prefer dependency injection over module mocking — cleaner and less brittle +- Prefer typed mock helpers (`jest.Mocked`, `vi.mocked`) or `Pick` over `as unknown as Type` +- Use `as unknown as Type` only as a last resort for partial mocks +- For complex interfaces, consider a factory helper to reduce mock boilerplate +- If a test needs more than 3–4 mocks, flag it as a design smell +- Mock reset: if config enables `clearMocks`/`mockReset`, rely on it; otherwise reset explicitly in `beforeEach` + +## Async Tests + +```typescript +// Jest / Vitest — both support async/await natively +it('fetches data successfully', async () => { + const result = await sut.fetchData(42); + expect(result).toBeDefined(); +}); + +// Testing rejected promises +it('throws on not found', async () => { + await expect(sut.fetchData(-1)).rejects.toThrow('not found'); +}); +``` + +## TypeScript-Specific Considerations + +- **Access modifiers**: TypeScript `private` and `protected` are compile-time only — they don't exist at runtime. Tests can technically access them but **should not** — test through the public API +- **Interfaces**: When the source defines interfaces, mock against the interface type, not the concrete class +- **Enums**: Import and use enum values directly in test assertions — don't hardcode the underlying numbers +- **Generics**: Provide explicit type arguments when instantiating generic classes in tests for clarity +- **Type assertions in tests**: Use `as Type` sparingly and only for test setup (mock objects), never to silence legitimate type errors + +## ESM vs CommonJS + +Many TypeScript projects are transitioning to ESM. Watch for these signals: + +- `"type": "module"` in `package.json` → ESM project +- `"module": "ESNext"` or `"NodeNext"` in `tsconfig.json` → ESM output +- `.mjs`/`.mts` file extensions → ESM files + +If the test runner fails with ESM errors: + +- **Jest**: May need `--experimental-vm-modules` flag and ESM-compatible transform (`ts-jest` with `useESM: true`, or `@swc/jest`) +- **Vitest**: Handles ESM natively — prefer Vitest for ESM projects if no runner is established +- **Mocha**: Needs `--loader ts-node/esm` or similar loader configuration + +Check the project's existing test configuration before changing module settings. + +## Framework Detection Priority + +When the project has multiple test runners configured, prefer in this order: + +1. Whatever `npm test` / `scripts.test` runs +2. Vitest (faster, better TS support) +3. Jest (most widely used) +4. Mocha + Chai (older projects) + +## Skip Coverage Tools + +Do not configure or run code coverage measurement tools (istanbul, c8, vitest --coverage). Coverage is measured separately by the evaluation harness. From c2a68e41846626f46ecc6ff5c2aa609b75a5a35d Mon Sep 17 00:00:00 2001 From: Jakub Jares Date: Wed, 6 May 2026 13:07:39 +0200 Subject: [PATCH 2/5] Revise python.md and typescript.md based on 10-agent review Key changes: - Add 'Rule #1: Investigate the repo first' as top section - Remove generic templates (anchor to wrong conventions) - Remove testing philosophy (unactionable for agent) - Fix command inconsistency (parameterize with /) - Move dependency install to 'last resort' section - Add error-driven fixer playbook with concrete fixes - Fix Mocha --grep (test name, not file filter) - Fix ESM guidance (don't change package.json type field) - Add monorepo/workspace guidance (Nx, Turborepo) - Add framework notes (React, Express, NestJS) - Add Jest mock hoisting warning - Cut from 224+289 lines to 117+136 lines (~57% reduction) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../extensions/python.md | 262 +++++--------- .../extensions/typescript.md | 326 +++++------------- 2 files changed, 165 insertions(+), 423 deletions(-) diff --git a/plugins/dotnet-test/skills/code-testing-extensions/extensions/python.md b/plugins/dotnet-test/skills/code-testing-extensions/extensions/python.md index 9209f064c4..dcaf9d008a 100644 --- a/plugins/dotnet-test/skills/code-testing-extensions/extensions/python.md +++ b/plugins/dotnet-test/skills/code-testing-extensions/extensions/python.md @@ -2,222 +2,116 @@ Language-specific guidance for Python test generation. -## Environment and Runner Detection +## Rule #1: Investigate the Repo First -Before running any commands, detect the project's package manager/runner from lockfiles and config: +Before writing any test or running any command, read: -| Indicator | Runner | Command prefix | -|-----------|--------|---------------| -| `poetry.lock` or `[tool.poetry]` in `pyproject.toml` | Poetry | `poetry run` | -| `pdm.lock` or `[tool.pdm]` in `pyproject.toml` | PDM | `pdm run` | -| `uv.lock` or `[tool.uv]` in `pyproject.toml` | uv | `uv run` | -| `Pipfile.lock` | Pipenv | `pipenv run` | -| `hatch.toml` or `[tool.hatch]` in `pyproject.toml` | Hatch | `hatch run` | -| None of the above | pip/venv | Use `python -m` prefix | +1. **Existing tests** — find `test_*.py` / `*_test.py` files and copy their style (imports, fixtures, class vs function, assertion patterns) +2. **Config files** — `pyproject.toml`, `pytest.ini`, `setup.cfg`, `tox.ini`, `conftest.py` +3. **Package layout** — determine import paths from existing code, not guesswork -- **Always prefer `python -m pytest` over bare `pytest`** — this ensures the correct interpreter and avoids PATH issues -- **Always prefer `python -m pip` over bare `pip`** for the same reason -- If a `Makefile`, `tox.ini`, or `nox` config exists, prefer the project's existing build/test commands -- **Do not add a new test framework if one already exists** — follow the repo's established choices +Do **not** add or change test infrastructure (frameworks, plugins, configs) unless the repo already uses it. -## Build Commands +## Environment Detection -Python is interpreted — there is no separate build step. Validate syntax and imports by running the tests or using a type checker if configured. +Detect the runner from lockfiles/config and prefix all commands accordingly: -| Scope | Command | -|-------|---------| -| Syntax check | `python -m py_compile path/to/file.py` | -| Type check (if configured) | `mypy path/to/file.py` or `pyright path/to/file.py` | +| Indicator | Prefix | +|-----------|--------| +| `poetry.lock` / `[tool.poetry]` in `pyproject.toml` | `poetry run` | +| `pdm.lock` / `[tool.pdm]` in `pyproject.toml` | `pdm run` | +| `uv.lock` / `[tool.uv]` in `pyproject.toml` | `uv run` | +| `Pipfile.lock` | `pipenv run` | +| `hatch.toml` / `[tool.hatch]` in `pyproject.toml` | `hatch run` | +| None of the above | `python -m` | -- Check for `pyproject.toml`, `setup.py`, `setup.cfg`, or `requirements.txt` to understand the project layout -- If the project uses editable installs, run `python -m pip install -e .` or `python -m pip install -e ".[dev]"` before testing +If `Makefile`, `tox.ini`, or `nox` config exists, prefer those scripts over raw commands. -## Test Commands +## Build Commands + +Python has no separate build step. Validate with the type checker if one is configured: | Scope | Command | |-------|---------| -| All tests | `pytest` | -| Specific file | `pytest tests/test_module.py` | -| Specific test | `pytest tests/test_module.py::TestClass::test_method` | -| Keyword filter | `pytest -k "keyword"` | -| Verbose output | `pytest -v` | -| Stop on first failure | `pytest -x` | - -- Prefer `pytest` over `unittest` — most Python projects use pytest even when test classes inherit `unittest.TestCase` -- **Prefer the project's existing test script** (`make test`, `tox`, `nox`) over raw `pytest` commands -- If `pytest` is not installed, check `pyproject.toml` `[project.optional-dependencies]` or `requirements-dev.txt` -- Use `pytest --tb=short` to reduce traceback noise during fix cycles -- If the project uses `unittest` exclusively (no pytest in dependencies), use `python -m unittest discover` - -## Lint Command - -Prefer the project's existing lint script (e.g., `make lint`, `tox -e lint`) over running tools directly. If no script exists, detect which tools the project uses from `pyproject.toml`, `.flake8`, `setup.cfg`, or `ruff.toml`: - -```bash -# Prefer ruff when available (fast, covers linting + formatting + import sorting) -ruff check --fix path/to/test_file.py -ruff format path/to/test_file.py +| Syntax check | ` py_compile path/to/file.py` | +| Type check | ` mypy path/to/file.py` or ` pyright path/to/file.py` | -# Fallback alternatives -black path/to/test_file.py # formatting -flake8 path/to/test_file.py # linting -isort path/to/test_file.py # import sorting -``` - -## Dependency Validation - -Before writing test code, verify the test dependencies are available: - -1. **pytest**: Check `pyproject.toml` or `requirements*.txt` for `pytest` -2. **Source package**: If the source code is in a `src/` layout, verify the package is importable (editable install) -3. **Mocking**: `unittest.mock` is in the stdlib — no extra dependency needed -4. **pytest plugins**: Check for `pytest-asyncio` (async tests), `pytest-mock` (mocker fixture) - -If imports fail with `ModuleNotFoundError`: - -```bash -python -m pip install -e . # Install source package in editable mode -python -m pip install -e ".[dev]" # Install with dev extras -python -m pip install pytest pytest-asyncio # Install test dependencies directly -``` - -## Common Errors +## Test Commands -| Error | Meaning | Fix | -|-------|---------|-----| -| `ModuleNotFoundError` | Package not installed or wrong import path | `pip install -e .` or fix the import statement | -| `ImportError` | Symbol not found in module | Verify the function/class name matches the source exactly | -| `AttributeError` | Wrong attribute on object | Check spelling and that the attribute exists in the source | -| `TypeError: __init__() missing required argument` | Constructor needs more args | Read the `__init__` signature and pass all required parameters | -| `TypeError: takes N positional arguments but M were given` | Wrong number of args | Match the function signature exactly | -| `fixture 'X' not found` | pytest fixture not defined or not imported | Define the fixture or add the correct import/conftest.py | -| `SyntaxError` | Invalid Python syntax in test file | Fix the syntax error at the indicated line | +Always use the detected ``. Prefer `python -m pytest` over bare `pytest` to ensure the correct interpreter. -## Project Layout Detection +| Scope | Command | +|-------|---------| +| All tests | ` pytest` | +| Specific file | ` pytest tests/test_module.py` | +| Specific test | ` pytest tests/test_module.py::TestClass::test_method` | +| Keyword filter | ` pytest -k "keyword"` | +| Stop on first failure | ` pytest -x --tb=short` | -Python projects use varied layouts. Detect the correct one: +- If `scripts.test` exists in `Makefile`/`tox`/`nox`, prefer that +- If the project uses `unittest` only (no pytest in deps), use `python -m unittest discover` -| Layout | Structure | Import Style | -|--------|-----------|-------------| -| `src/` layout | `src/package/module.py` | `from package.module import X` | -| Flat layout | `package/module.py` at repo root | `from package.module import X` | -| Single module | `module.py` at repo root | `from module import X` | +## Lint Command -- Check `pyproject.toml` `[tool.setuptools.packages.find]` or `[tool.setuptools.package-dir]` for layout hints -- If `conftest.py` exists at the repo root, pytest typically handles path resolution -- **Follow the existing convention first** — check where existing tests live before placing new ones -- If no convention exists, default to `tests/` mirroring the source structure: `src/billing/service.py` → `tests/billing/test_service.py` +Use the repo's existing lint script first (`make lint`, `tox -e lint`). Otherwise detect tools from config: -## Test Discovery +- `ruff.toml` or `[tool.ruff]` → ` ruff check --fix && ruff format` +- `[tool.black]` → ` black` +- `.flake8` → ` flake8` -Python tests are discovery-based — no registration step is needed (unlike .NET solutions). Pytest finds tests automatically if naming conventions are followed. +## Project Layout and Imports -## Testing Philosophy +| Layout | Import Style | +|--------|-------------| +| `src/package/module.py` | `from package.module import X` | +| `package/module.py` at root | `from package.module import X` | +| `module.py` at root | `from module import X` | -- **Test behavior through the public API** — do not test private functions (prefixed with `_`) directly unless they contain complex algorithms that cannot be adequately exercised through the public surface -- **Prefer output/state assertions over interaction assertions** — verify return values and observable state changes, not internal call counts -- **Do not mock the system under test** — only mock external dependencies (databases, HTTP, filesystem, time) -- **Do not patch private helpers** — if behavior is only reachable through a private function, test it via the public method that calls it -- Fewer focused tests that thoroughly exercise behavior are better than many shallow tests that only check surface behavior +- **Match existing test imports exactly** — do not invent `src.` prefixes unless existing tests use them +- Check `pyproject.toml` `[tool.setuptools.package-dir]` for layout hints +- Default test placement: `tests/` mirroring source structure (`src/billing/service.py` → `tests/billing/test_service.py`) ## Test File Naming -- Test files must be named `test_*.py` or `*_test.py` (pytest default discovery) -- Test functions must start with `test_` -- Test classes must start with `Test` (no `__init__` method) - -## pytest Template - -```python -import pytest -from package.module import ClassName - - -class TestClassName: - """Tests for ClassName behavior.""" - - def test_method_name_returns_expected_result(self): - # Given - sut = ClassName() - - # When - result = sut.method_name(input_value) - - # Then - assert result == expected_value - - @pytest.mark.parametrize("a,b,expected", [ - (2, 3, 5), - (-1, 1, 0), - (0, 0, 0), - ]) - def test_add_valid_inputs_returns_sum(self, a, b, expected): - # Given - sut = ClassName() - - # When - result = sut.add(a, b) +- Files: `test_*.py` or `*_test.py` +- Functions: `test_` prefix +- Classes: `Test` prefix, no `__init__` +- No registration step needed — pytest discovers automatically - # Then - assert result == expected - - def test_method_raises_on_invalid_input(self): - sut = ClassName() - with pytest.raises(ValueError, match="must be positive"): - sut.method_name(-1) -``` - -## Mocking Guidelines - -Use `unittest.mock` (stdlib) or the `pytest-mock` plugin's `mocker` fixture: - -```python -from unittest.mock import Mock, patch, AsyncMock - -# Dependency injection — preferred -def test_service_calls_repository(self): - # Given - repo = Mock() - repo.find.return_value = {"id": 1, "name": "test"} - sut = Service(repository=repo) - - # When - result = sut.get_by_id(1) - - # Then - assert result["name"] == "test" - repo.find.assert_called_once_with(1) - -# Patching module-level dependencies — use sparingly -@patch("package.module.datetime") -def test_uses_current_time(self, mock_dt): - mock_dt.utcnow.return_value = datetime(2024, 1, 1) - # ... -``` +## Common Errors -- Prefer dependency injection over `@patch` — it produces clearer, less brittle tests -- When patching, patch where the name is **looked up**, not where it is defined -- Use `Mock(spec=RealClass)` to catch attribute errors early +| Error | Fix | +|-------|-----| +| `ModuleNotFoundError: No module named 'src'` | Import from the package name used by the repo, not from `src` | +| `ModuleNotFoundError: No module named 'X'` | Check existing imports for the correct package name; if editable install needed: ` pip install -e .` | +| `ImportError: attempted relative import` | Convert to absolute imports matching existing test patterns | +| `fixture 'X' not found` | Check `conftest.py` for existing fixtures; reuse them instead of creating new ones | +| `TypeError: missing required argument` | Read the full `__init__`/function signature; pass all required parameters | +| `async def functions are not natively supported` | Use `@pytest.mark.asyncio` only if `pytest-asyncio` is already in deps; check for `asyncio_mode = "auto"` in config | +| `SyntaxError` | Fix syntax at the indicated line | + +## Mocking Rules + +- Use `unittest.mock` (stdlib) — no extra dependency needed +- **Patch where the name is looked up**, not where it is defined: `@patch("mypackage.module.datetime")` not `@patch("datetime.datetime")` +- Use `Mock(spec=RealClass)` to catch attribute errors - Use `AsyncMock` for async functions +- Prefer dependency injection over `@patch` - If a test needs more than 3 mocks, flag it as a design smell -## Async Tests - -If the source code uses `async def`, tests need async support: +## Dependency Installation (Last Resort) -```python -import pytest +Only install packages after investigation confirms they are missing. Use the detected prefix: -@pytest.mark.asyncio -async def test_async_method_returns_result(): - sut = AsyncService() - result = await sut.fetch_data(42) - assert result is not None -``` +| Manager | Install command | +|---------|----------------| +| Poetry | `poetry add --group dev pytest` | +| PDM | `pdm add -dG test pytest` | +| uv | `uv add --dev pytest` | +| pip | `python -m pip install -e ".[dev]"` | -- Requires `pytest-asyncio` package -- Check for `asyncio_mode = "auto"` in `pyproject.toml` — if set, the `@pytest.mark.asyncio` decorator is not needed +Never run bare `pip install` in a Poetry/PDM/uv project — it bypasses the lockfile. ## Skip Coverage Tools -Do not configure or run code coverage measurement tools (coverage.py, pytest-cov). Coverage is measured separately by the evaluation harness. +Do not configure or run coverage tools (coverage.py, pytest-cov). Coverage is measured separately by the evaluation harness. diff --git a/plugins/dotnet-test/skills/code-testing-extensions/extensions/typescript.md b/plugins/dotnet-test/skills/code-testing-extensions/extensions/typescript.md index 9b3666c74e..961b37726e 100644 --- a/plugins/dotnet-test/skills/code-testing-extensions/extensions/typescript.md +++ b/plugins/dotnet-test/skills/code-testing-extensions/extensions/typescript.md @@ -2,287 +2,135 @@ Language-specific guidance for TypeScript (and JavaScript) test generation. +## Rule #1: Investigate the Repo First + +Before writing any test or running any command, read: + +1. **Existing tests** — find `*.test.ts` / `*.spec.ts` files and copy their style (imports, describe/it vs test, assertion patterns, mock approach) +2. **`package.json`** — `scripts.test`, `devDependencies`, `type` field +3. **Config files** — `tsconfig.json`, `jest.config.*`, `vitest.config.*`, `eslint.config.*` + +Do **not** add or change test infrastructure (frameworks, configs, transforms) unless the repo already uses it. Use the runner the repo already has — do not switch runners. + ## Package Manager Detection -Before running any commands, detect the package manager from lockfiles: +Detect the package manager from lockfiles and use it consistently for **all** commands: -| Indicator | Manager | Run command | Exec command | -|-----------|---------|-------------|-------------| -| `pnpm-lock.yaml` | pnpm | `pnpm test` | `pnpm exec vitest` | -| `yarn.lock` | Yarn | `yarn test` | `yarn vitest` | -| `bun.lockb` or `bun.lock` | Bun | `bun test` | `bunx vitest` | -| `package-lock.json` or none | npm | `npm test` | `npx vitest` | +| Indicator | Manager | Run script | Execute binary | +|-----------|---------|------------|----------------| +| `pnpm-lock.yaml` | pnpm | `pnpm test` | `pnpm exec ` | +| `yarn.lock` | Yarn | `yarn test` | `yarn ` | +| `bun.lockb` / `bun.lock` | Bun | `bun test` | `bunx ` | +| `package-lock.json` or none | npm | `npm test` | `npx ` | -- **Always prefer the project's existing scripts** (`npm test`, `pnpm test`, etc.) over raw tool CLIs -- **Do not add a new test framework if one already exists** — follow the repo's established choices -- For monorepos (workspaces), run commands from the package directory, not the root +Use `` below as shorthand for the detected exec command. ## Build Commands | Scope | Command | |-------|---------| -| Type check (project-level) | `npx tsc --noEmit` or `npm run typecheck` | -| Build (if configured) | `npm run build` | +| Type check | ` tsc --noEmit` or the repo's `typecheck` script | +| Build (if configured) | The repo's `build` script | -- Check `package.json` `scripts` for the project's preferred build/typecheck command -- Many projects don't need an explicit build step — the test runner handles transpilation -- If `tsconfig.json` exists, TypeScript is in use; check `strict` mode and `target` settings +Many projects don't need an explicit build step — the test runner handles transpilation. ## Test Commands -Detect the test runner from `package.json` `devDependencies` and `scripts.test`: +Detect the runner from `devDependencies` and `scripts.test`. Always prefer the repo's test script first. -| Runner | All tests | Filtered | Watch mode | -|--------|-----------|----------|------------| -| **Jest** | `npx jest` | `npx jest --testPathPattern="module"` | `npx jest --watch` | -| **Vitest** | `npx vitest run` | `npx vitest run path/to/file` | `npx vitest` | -| **Mocha** | `npx mocha` | `npx mocha --grep "pattern"` | `npx mocha --watch` | +| Runner | Run once | Filter by file | Filter by name | +|--------|----------|----------------|----------------| +| **Jest** | ` jest` | ` jest path/to/file` | ` jest -t "name"` | +| **Vitest** | ` vitest run` | ` vitest run path/to/file` | ` vitest run -t "name"` | +| **Mocha** | ` mocha` | (use config or positional args) | ` mocha --grep "name"` | -- Prefer `npm test` or `npm run test` (or equivalent for detected package manager) if it's configured in `package.json` -- Use `npx vitest run` (not `npx vitest`) to run once without watch mode -- For Jest: use `--verbose` for detailed output, `--bail` to stop on first failure -- For Jest: filter by test name with `npx jest path/to/file.test.ts -t "test name"` -- For Vitest: use `--reporter=verbose` for detailed output -- For Vitest: filter by test name with `npx vitest run path/to/file.test.ts -t "test name"` -- Mocha should almost always be invoked via the project's existing script/config — direct CLI only if existing tests already do that +- **Always use `vitest run`** (not bare `vitest`) — bare `vitest` starts watch mode +- **Never use `--watch`** — the agent must not start interactive/watch mode +- For Jest: `--bail` to stop on first failure, `--verbose` for detail +- Mocha `--grep` filters by **test name**, not file path ## Lint Command -```bash -# ESLint (most common) -npx eslint path/to/test_file.ts --fix - -# Prettier (formatting) -npx prettier --write path/to/test_file.ts +Use the repo's lint script first. Otherwise detect from `devDependencies` and config: -# Biome (all-in-one) -npx biome check --write path/to/test_file.ts -``` +- `eslint.config.*` or `.eslintrc.*` → ` eslint --fix path/to/file.ts` +- `prettier` → ` prettier --write path/to/file.ts` +- `biome.json` → ` biome check --write path/to/file.ts` -- Detect which tools the project uses from `package.json` `devDependencies` and config files (`.eslintrc.*`, `prettier.config.*`, `biome.json`) -- Run `npm run lint -- --fix` if the project has a lint script configured +## Project Layout and Imports -## Dependency Validation +| Layout | Import Style | +|--------|-------------| +| Colocated (`src/module.test.ts`) | `import { X } from './module'` | +| `__tests__/` dir | `import { X } from '../module'` | +| Top-level `tests/` | `import { X } from '../src/module'` | -Before writing test code, verify test infrastructure is present: - -1. **Test runner**: Check `package.json` `devDependencies` for `jest`, `vitest`, `mocha`, etc. -2. **Type definitions**: For Jest, ensure `@types/jest` is installed; Vitest includes its own types -3. **TypeScript support**: Jest needs `ts-jest` or `@swc/jest`; Vitest handles TS natively -4. **Assertion library**: Jest/Vitest have built-in `expect`; Mocha typically uses `chai` - -If imports fail or tests won't run: - -```bash -# Jest setup -npm install --save-dev jest ts-jest @types/jest - -# Vitest setup -npm install --save-dev vitest - -# Mocha + Chai setup -npm install --save-dev mocha chai @types/mocha @types/chai ts-node -``` - -## Common Errors - -| Error | Meaning | Fix | -|-------|---------|-----| -| `Cannot find module 'X'` | Import path wrong or package not installed | Fix the import path or `npm install` the package | -| `TS2305: Module has no exported member` | Named export doesn't exist | Check the source file's actual exports | -| `TS2307: Cannot find module` | Missing module or type declarations | Install `@types/package` or check `tsconfig.json` paths | -| `TS2345: Argument type not assignable` | Type mismatch in function call | Match the expected type or use type assertion | -| `TS2339: Property does not exist on type` | Wrong property name or type | Verify property name against the source interface/class | -| `TS7006: Parameter implicitly has 'any' type` | Missing type annotation (strict mode) | Add explicit type annotations | -| `SyntaxError: Unexpected token` | Test runner can't parse TypeScript | Configure `ts-jest`, `@swc/jest`, or use Vitest which handles TS natively | -| `ReferenceError: describe is not defined` | Test globals not available | For Vitest: import from `vitest` or set `globals: true` in config; for Jest: ensure tests run under Jest (not `node`); for Mocha: check test bootstrap | -| `ERR_REQUIRE_ESM` / `Cannot use import statement outside a module` | ESM/CJS mismatch | Set `"type": "module"` in `package.json`, or configure the test runner's transform/loader — see ESM section below | -| `ReferenceError: document is not defined` | Code uses browser APIs | Configure test environment: `testEnvironment: 'jsdom'` (Jest) or `environment: 'jsdom'` (Vitest) | - -## Project Layout Detection - -| Layout | Test Location | Import Style | -|--------|--------------|-------------| -| Colocated | `src/module.test.ts` next to `src/module.ts` | `import { X } from './module'` | -| Separate `__tests__` | `src/__tests__/module.test.ts` | `import { X } from '../module'` | -| Top-level `tests/` | `tests/module.test.ts` | `import { X } from '../src/module'` | - -- Check existing test files to match the project's convention -- If `tsconfig.json` has `paths` aliases (e.g., `@/`), use them in test imports -- For monorepos, import from the package name, not relative paths across packages +- **Match existing test imports** — copy path style from neighboring tests +- If `tsconfig.json` has `paths` aliases (e.g., `@/`), use them in tests too +- For monorepos: import from the package name, not relative cross-package paths +- For monorepo workspaces (Nx, Turborepo, Lerna): run tests via the workspace tool (`nx test `, `turbo test`), not from a random package directory ## Test File Naming -- Jest default: `*.test.ts`, `*.test.tsx`, `*.spec.ts`, `*.spec.tsx`, or files inside `__tests__/` -- Vitest default: same as Jest -- Match the existing project convention — check for `.test.` vs `.spec.` usage -- Place test files to mirror source structure - -## Jest Template +- Match existing convention — check for `.test.ts` vs `.spec.ts` +- Jest/Vitest default: `*.test.ts`, `*.spec.ts`, or files inside `__tests__/` +- Place test files to mirror the existing project pattern -```typescript -import { ClassName } from '../module'; - -describe('ClassName', () => { - let sut: ClassName; - - beforeEach(() => { - sut = new ClassName(); - }); - - describe('methodName', () => { - it('returns expected result for valid input', () => { - // Arrange - const input = 'test'; - - // Act - const result = sut.methodName(input); +## Common Errors - // Assert - expect(result).toBe(expected); - }); +| Error | Fix | +|-------|-----| +| `Cannot find module 'X'` | Check existing imports for correct paths; verify `tsconfig.json` `paths`; check `moduleNameMapper` (Jest) or `resolve.alias` (Vitest) | +| `TS2305: has no exported member` | Verify the exact export name from the source file | +| `TS2345: type not assignable` | Match the expected type; use type assertion only for mock objects | +| `SyntaxError: Unexpected token` / `Jest encountered an unexpected token` | Verify TS transform config (`ts-jest`, `@swc/jest`, or Vitest handles natively) | +| `ReferenceError: describe is not defined` | Vitest: import from `vitest` or set `globals: true` in config; Jest: ensure tests run under Jest not bare `node` | +| `Cannot use import statement outside a module` / `ERR_REQUIRE_ESM` | ESM/CJS mismatch — align runner config with the project's module system (see ESM section); do **not** blindly set `"type": "module"` | +| `ReferenceError: document is not defined` | Set test environment: `testEnvironment: 'jsdom'` (Jest) or `environment: 'jsdom'` (Vitest) | +| `jest.mock() ... out-of-scope variables` | Keep `jest.mock()` at top level; don't reference variables declared after the mock call (Jest hoists mocks) | +| `Cannot find module '@/...'` | Mirror the project's alias config in the test runner's module resolution | +| `Warning: not wrapped in act(...)` | Await async UI updates using the repo's existing pattern (`waitFor`, `act`) | - it.each([ - { a: 2, b: 3, expected: 5 }, - { a: -1, b: 1, expected: 0 }, - { a: 0, b: 0, expected: 0 }, - ])('add($a, $b) returns $expected', ({ a, b, expected }) => { - expect(sut.add(a, b)).toBe(expected); - }); +## ESM vs CommonJS - it('throws on invalid input', () => { - expect(() => sut.methodName(null!)).toThrow('must not be null'); - }); - }); -}); -``` +Check these signals to determine the project's module system: -## Vitest Template - -```typescript -import { describe, it, expect, beforeEach } from 'vitest'; -import { ClassName } from '../module'; - -describe('ClassName', () => { - let sut: ClassName; - - beforeEach(() => { - sut = new ClassName(); - }); - - describe('methodName', () => { - it('returns expected result for valid input', () => { - const result = sut.methodName('test'); - expect(result).toBe(expected); - }); - - it.each([ - { a: 2, b: 3, expected: 5 }, - { a: -1, b: 1, expected: 0 }, - ])('add($a, $b) returns $expected', ({ a, b, expected }) => { - expect(sut.add(a, b)).toBe(expected); - }); - }); -}); -``` +- `"type": "module"` in `package.json` → ESM +- `"module": "ESNext"` or `"NodeNext"` in `tsconfig.json` → ESM output (but not sufficient alone) +- `.mjs`/`.mts` extensions → ESM files -## Mocking Guidelines +If the test runner fails with ESM errors, align the runner's config with the project's module system. **Do not change `package.json` `type` field** — align the test runner to match whatever the project uses: -### Jest +- **Jest**: `--experimental-vm-modules` + `ts-jest` with `useESM: true`, or `@swc/jest` +- **Vitest**: handles ESM natively +- **Mocha**: `--loader ts-node/esm` -```typescript -// Manual mock -const mockRepo = { - find: jest.fn().mockResolvedValue({ id: 1, name: 'test' }), - save: jest.fn(), -}; -const sut = new Service(mockRepo as unknown as Repository); +## Mocking Rules -// Module mock -jest.mock('../repository', () => ({ - Repository: jest.fn().mockImplementation(() => ({ - find: jest.fn().mockResolvedValue({ id: 1 }), - })), -})); +- Prefer dependency injection over module mocking +- Use typed mocks: `jest.Mocked`, `vi.mocked(obj)`, or `Partial` with `as T` +- Jest: `jest.mock()` is hoisted — keep at top level, don't close over local variables +- Vitest: `vi.mock()` follows the same hoisting rules +- If a test needs more than 3–4 mocks, flag it as a design smell +- Mock reset: rely on `clearMocks`/`restoreMocks` config if present; otherwise reset in `beforeEach` -// Spy on existing method -jest.spyOn(sut, 'methodName').mockReturnValue('mocked'); -``` +## Framework-Specific Notes -### Vitest +- **React/Preact**: use `@testing-library/react`, wrap with necessary providers (router, query client, theme) matching existing test setup +- **Express/Koa**: use `supertest` for HTTP testing if the repo already uses it +- **NestJS**: build testing module with `Test.createTestingModule` — don't instantiate controllers directly -```typescript -import { vi } from 'vitest'; +## Dependency Installation (Last Resort) -const mockRepo = { - find: vi.fn().mockResolvedValue({ id: 1, name: 'test' }), - save: vi.fn(), -}; -const sut = new Service(mockRepo as unknown as Repository); +Only install packages after investigation confirms they are missing. Use the detected package manager: -// Module mock -vi.mock('../repository', () => ({ - Repository: vi.fn().mockImplementation(() => ({ - find: vi.fn().mockResolvedValue({ id: 1 }), - })), -})); ``` - -- Prefer dependency injection over module mocking — cleaner and less brittle -- Prefer typed mock helpers (`jest.Mocked`, `vi.mocked`) or `Pick` over `as unknown as Type` -- Use `as unknown as Type` only as a last resort for partial mocks -- For complex interfaces, consider a factory helper to reduce mock boilerplate -- If a test needs more than 3–4 mocks, flag it as a design smell -- Mock reset: if config enables `clearMocks`/`mockReset`, rely on it; otherwise reset explicitly in `beforeEach` - -## Async Tests - -```typescript -// Jest / Vitest — both support async/await natively -it('fetches data successfully', async () => { - const result = await sut.fetchData(42); - expect(result).toBeDefined(); -}); - -// Testing rejected promises -it('throws on not found', async () => { - await expect(sut.fetchData(-1)).rejects.toThrow('not found'); -}); + add --save-dev jest ts-jest @types/jest + add --save-dev vitest ``` -## TypeScript-Specific Considerations - -- **Access modifiers**: TypeScript `private` and `protected` are compile-time only — they don't exist at runtime. Tests can technically access them but **should not** — test through the public API -- **Interfaces**: When the source defines interfaces, mock against the interface type, not the concrete class -- **Enums**: Import and use enum values directly in test assertions — don't hardcode the underlying numbers -- **Generics**: Provide explicit type arguments when instantiating generic classes in tests for clarity -- **Type assertions in tests**: Use `as Type` sparingly and only for test setup (mock objects), never to silence legitimate type errors - -## ESM vs CommonJS - -Many TypeScript projects are transitioning to ESM. Watch for these signals: - -- `"type": "module"` in `package.json` → ESM project -- `"module": "ESNext"` or `"NodeNext"` in `tsconfig.json` → ESM output -- `.mjs`/`.mts` file extensions → ESM files - -If the test runner fails with ESM errors: - -- **Jest**: May need `--experimental-vm-modules` flag and ESM-compatible transform (`ts-jest` with `useESM: true`, or `@swc/jest`) -- **Vitest**: Handles ESM natively — prefer Vitest for ESM projects if no runner is established -- **Mocha**: Needs `--loader ts-node/esm` or similar loader configuration - -Check the project's existing test configuration before changing module settings. - -## Framework Detection Priority - -When the project has multiple test runners configured, prefer in this order: - -1. Whatever `npm test` / `scripts.test` runs -2. Vitest (faster, better TS support) -3. Jest (most widely used) -4. Mocha + Chai (older projects) +Never install test infrastructure that conflicts with what the repo already uses. ## Skip Coverage Tools -Do not configure or run code coverage measurement tools (istanbul, c8, vitest --coverage). Coverage is measured separately by the evaluation harness. +Do not configure or run coverage tools (istanbul, c8, `vitest --coverage`). Coverage is measured separately by the evaluation harness. From dbfb9cdaae947b1ccbf7431286d29dde4e3ea621 Mon Sep 17 00:00:00 2001 From: Jakub Jares Date: Wed, 6 May 2026 13:28:14 +0200 Subject: [PATCH 3/5] Add powershell.md extension for Pester v5 test generation Key sections: - Repo investigation first (shell target detection) - Pester v5 discovery vs run phase rules - Import patterns for modules, library scripts, and executable scripts - Cross-platform guidance (pwsh vs powershell.exe, Join-Path, casing) - Non-terminating error handling with Should -Throw - Mock scoping, -ModuleName, PesterBoundParameters - TestDrive: for file-based tests - Non-obvious assertion gotchas (Contain vs Be, Throw needs scriptblock) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../skills/code-testing-extensions/SKILL.md | 1 + .../extensions/powershell.md | 110 ++++++++++++++++++ 2 files changed, 111 insertions(+) create mode 100644 plugins/dotnet-test/skills/code-testing-extensions/extensions/powershell.md diff --git a/plugins/dotnet-test/skills/code-testing-extensions/SKILL.md b/plugins/dotnet-test/skills/code-testing-extensions/SKILL.md index 80c690d0c1..e404673cf6 100644 --- a/plugins/dotnet-test/skills/code-testing-extensions/SKILL.md +++ b/plugins/dotnet-test/skills/code-testing-extensions/SKILL.md @@ -20,6 +20,7 @@ This skill provides access to language-specific guidance files used by the code- | [extensions/dotnet.md](extensions/dotnet.md) | .NET (C#/F#/VB) | Build commands, test commands, project reference validation, common CS error codes, MSTest template | | [extensions/python.md](extensions/python.md) | Python | Build/test commands (pytest), project layout detection, mocking guidelines, common errors, pytest template | | [extensions/typescript.md](extensions/typescript.md) | TypeScript/JavaScript | Build/test commands (Jest/Vitest/Mocha), framework detection, mocking, TS-specific considerations | +| [extensions/powershell.md](extensions/powershell.md) | PowerShell | Test commands (Pester v5), module import patterns, discovery/run pitfalls, mocking, common errors | | [extensions/cpp.md](extensions/cpp.md) | C++ | Testing internals with friend declarations | | [extensions/dotnet-examples.md](extensions/dotnet-examples.md) | .NET (C#/F#/VB) | Concrete pipeline examples: sample research output, plan, generated tests, fix cycles, final report | diff --git a/plugins/dotnet-test/skills/code-testing-extensions/extensions/powershell.md b/plugins/dotnet-test/skills/code-testing-extensions/extensions/powershell.md new file mode 100644 index 0000000000..389410465b --- /dev/null +++ b/plugins/dotnet-test/skills/code-testing-extensions/extensions/powershell.md @@ -0,0 +1,110 @@ +# PowerShell Extension + +Language-specific guidance for PowerShell test generation using Pester v5. + +## Rule #1: Investigate the Repo First + +Before writing any test or running any command, read: + +1. **Existing tests** — find `*.Tests.ps1` files and copy their style (structure, assertions, mock approach, import method) +2. **Module structure** — look for `.psd1` (manifest), `.psm1` (root module), `Public/`/`Private/` organization +3. **Build/test scripts** — check for `build.ps1`, `Invoke-Build` (`*.build.ps1`), `psake`, or CI scripts +4. **Shell target** — check `.psd1` for `PowerShellVersion`/`CompatiblePSEditions`, CI matrix for `pwsh` vs `powershell.exe` + +Do **not** add Pester or change test infrastructure unless the repo already uses it. + +## Build Commands + +PowerShell is interpreted — no build step. If the repo has a build script, use it. Otherwise validate with: + +- **Module loads**: `Import-Module ./MyModule.psd1 -Force -ErrorAction Stop` +- **Script analyzer**: `Invoke-ScriptAnalyzer -Path ./src -Recurse` (if PSScriptAnalyzer is available) +- **Lint**: `Invoke-ScriptAnalyzer -Path path/to/file.ps1 -Fix` + +## Test Commands + +| Scope | Command | +|-------|---------| +| All tests | `Invoke-Pester` | +| Specific file | `Invoke-Pester -Path ./Tests/Get-Widget.Tests.ps1` | +| Filter by name | `Invoke-Pester -FullNameFilter '*Get-Widget*'` | +| Filter by tag | `Invoke-Pester -TagFilter 'Unit'` | +| Non-interactive (CI) | `Invoke-Pester -CI` | +| Detailed output | `Invoke-Pester -Output Detailed` | + +- Prefer the repo's build/test script over raw `Invoke-Pester` +- Use `-Output Detailed` during fix cycles, `-Output Minimal` for final validation + +## Project Layout and Imports + +| Layout | Import in `BeforeAll` | +|--------|-----------------------| +| Module (`.psd1`) | `Import-Module "$PSScriptRoot/../MyModule.psd1" -Force` | +| Library script (defines functions) | `. $PSScriptRoot/Get-Widget.ps1` | +| Co-located test | `. $PSCommandPath.Replace('.Tests.ps1', '.ps1')` | +| Executable script (has `param()`) | Do **not** dot-source — invoke with `& $PSScriptRoot/script.ps1 -Param value` and assert on output/errors | + +- **All imports go in `BeforeAll`** — never at script top level +- **Use `$PSScriptRoot` or `$PSCommandPath`** — never `$MyInvocation.MyCommand.Path` (returns empty in `BeforeAll`) +- Use `-Force` on `Import-Module` to pick up changes between runs + +## Test File Naming + +- Files: `*.Tests.ps1` — match existing convention (co-located vs `Tests/` directory) + +## Pester v5 Discovery vs Run (Critical) + +Pester v5 runs in **two phases**: Discovery (collects test metadata) then Run (executes tests). This is the #1 source of agent errors. + +**Rules:** +- All setup code goes in `BeforeAll` or `BeforeEach` — never at script top level or loose inside `Describe`/`Context` +- Code directly inside `Describe`/`Context` (but outside `It`/`Before*`/`After*`) runs during **Discovery** — do not put setup, imports, or variable assignments there +- Data for `-ForEach` / `-TestCases` must be set in `BeforeDiscovery`, not `BeforeAll` (BeforeAll runs after discovery) +- `-Skip:$condition` evaluates at Discovery time — conditions from `BeforeAll` will be `$null` +- Use `foreach` loops for dynamic test generation only with `BeforeDiscovery` data +- Use `TestDrive:` for file-based tests instead of touching repo files — Pester cleans it up automatically + +## Common Errors + +| Error | Fix | +|-------|-----| +| Variable is `$null` in `It` block | Move assignment into `BeforeAll` — variables set there are visible to child `It` blocks without `$script:` | +| `-ForEach` data is empty | Move data setup from `BeforeAll` to `BeforeDiscovery` | +| `CommandNotFoundException` for Mock target | The function must exist before mocking — import the module in `BeforeAll` first | +| `$MyInvocation.MyCommand.Path` returns empty | Use `$PSCommandPath` or `$PSScriptRoot` instead | +| `Should Be` (no dash) fails | Use v5 syntax: `Should -Be` (with dash prefix) | +| `Assert-MockCalled` not recognized | Use v5 syntax: `Should -Invoke` | +| Mock has no effect | Check scope — mocks in `It` only apply to that `It`; use `BeforeAll`/`BeforeEach` for broader scope | +| `Should -Throw` doesn't catch cmdlet errors | Most cmdlet errors are non-terminating — wrap with `{ cmd -ErrorAction Stop }` or set `$ErrorActionPreference = 'Stop'` in `BeforeEach` | +| Tests pass on Windows but fail on Linux | Use `Join-Path` not string concatenation; match exact file casing; avoid Windows-only cmdlets (Registry, EventLog) | + +## Mocking Rules + +- Place mocks in `BeforeAll` (shared) or `BeforeEach` (reset per test) +- Mock where the command is **called from** — use `-ModuleName` to mock inside a module's scope +- Use `-ParameterFilter` for selective mocking (no `param()` block needed in v5) +- Verify calls with `Should -Invoke` — default scope inside `It` counts only that test's calls +- Use `InModuleScope` sparingly and as narrowly as possible — prefer `Mock -ModuleName` for testing via public API +- Inside mock bodies, use `$PesterBoundParameters` not `$PSBoundParameters` +- If a test needs more than 3 mocks, flag it as a design smell + +## Non-Obvious Assertions + +Most `Should` operators are self-explanatory. These are the ones agents get wrong: + +- `Should -Throw` requires a **scriptblock**: `{ risky-op } | Should -Throw` — not a direct call +- `Should -Contain` is for **collections** — use `Should -Be` for scalar equality +- `Should -HaveParameter` validates cmdlet signatures: `Get-Command X | Should -HaveParameter 'Name' -Mandatory` +- `Should -Invoke` verifies mock calls: `Should -Invoke Get-Item -Times 1 -Exactly` + +## Cross-Platform + +- Prefer `pwsh` (PowerShell 7+) unless the repo explicitly targets Windows PowerShell 5.1 +- Use `Join-Path` for paths — never string concatenation with `\` +- Linux/macOS file systems are **case-sensitive** — match exact casing in imports and paths +- Windows ships Pester 3.4.0 — if v5 is needed: `Install-Module Pester -Force -SkipPublisherCheck` +- Check `$PSVersionTable.PSEdition` to detect Core vs Desktop + +## Skip Coverage Tools + +Do not configure or run coverage tools (Pester CodeCoverage, JaCoCo export). Coverage is measured separately by the evaluation harness. From c2d1fadddb874fcef320dc34ff071847fb5bfdf0 Mon Sep 17 00:00:00 2001 From: Jakub Jares Date: Wed, 6 May 2026 14:19:06 +0200 Subject: [PATCH 4/5] Reword test infra guidance; add framework detection to dotnet.md - Reword 'do not add' to 'use existing, only introduce if none exist' - Add test framework detection table to dotnet.md (MSTest/xUnit/NUnit) - Tailor wording per language (Python defaults to pytest, TS follows scripts.test, PS defaults to Pester) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../code-testing-extensions/extensions/dotnet.md | 12 ++++++++++++ .../code-testing-extensions/extensions/powershell.md | 2 +- .../code-testing-extensions/extensions/python.md | 2 +- .../code-testing-extensions/extensions/typescript.md | 2 +- 4 files changed, 15 insertions(+), 3 deletions(-) diff --git a/plugins/dotnet-test/skills/code-testing-extensions/extensions/dotnet.md b/plugins/dotnet-test/skills/code-testing-extensions/extensions/dotnet.md index 7c30a78174..019c76b179 100644 --- a/plugins/dotnet-test/skills/code-testing-extensions/extensions/dotnet.md +++ b/plugins/dotnet-test/skills/code-testing-extensions/extensions/dotnet.md @@ -71,6 +71,18 @@ If a new test project was created, register it with the solution so `dotnet test 4. Skip this if the project is already included in the solution or solution filter used for testing. 5. Prefer the researched test command. If you need to run the solution directly, use `dotnet test --solution ` only for repos on .NET SDK 10+ with MTP-style syntax; otherwise use the standard positional form `dotnet test `. +## Test Framework Detection + +Detect the framework from the test project's `.csproj` package references and match its conventions: + +| Package Reference | Framework | Attributes | Assertion Style | +|-------------------|-----------|------------|-----------------| +| `MSTest.Sdk` or `MSTest.TestFramework` | MSTest | `[TestClass]`, `[TestMethod]`, `[DataRow]` | `Assert.AreEqual(expected, actual)` | +| `xunit` | xUnit | `[Fact]`, `[Theory]`, `[InlineData]` | `Assert.Equal(expected, actual)` | +| `NUnit` | NUnit | `[TestFixture]`, `[Test]`, `[TestCase]` | `Assert.That(actual, Is.EqualTo(expected))` | + +Use the repo's existing framework — do not introduce a different one. + ## MSTest Template ```csharp diff --git a/plugins/dotnet-test/skills/code-testing-extensions/extensions/powershell.md b/plugins/dotnet-test/skills/code-testing-extensions/extensions/powershell.md index 389410465b..e0b1201d8d 100644 --- a/plugins/dotnet-test/skills/code-testing-extensions/extensions/powershell.md +++ b/plugins/dotnet-test/skills/code-testing-extensions/extensions/powershell.md @@ -11,7 +11,7 @@ Before writing any test or running any command, read: 3. **Build/test scripts** — check for `build.ps1`, `Invoke-Build` (`*.build.ps1`), `psake`, or CI scripts 4. **Shell target** — check `.psd1` for `PowerShellVersion`/`CompatiblePSEditions`, CI matrix for `pwsh` vs `powershell.exe` -Do **not** add Pester or change test infrastructure unless the repo already uses it. +Use the repo's existing test conventions. Only add Pester if the repo has no tests at all. ## Build Commands diff --git a/plugins/dotnet-test/skills/code-testing-extensions/extensions/python.md b/plugins/dotnet-test/skills/code-testing-extensions/extensions/python.md index dcaf9d008a..1e8f80394c 100644 --- a/plugins/dotnet-test/skills/code-testing-extensions/extensions/python.md +++ b/plugins/dotnet-test/skills/code-testing-extensions/extensions/python.md @@ -10,7 +10,7 @@ Before writing any test or running any command, read: 2. **Config files** — `pyproject.toml`, `pytest.ini`, `setup.cfg`, `tox.ini`, `conftest.py` 3. **Package layout** — determine import paths from existing code, not guesswork -Do **not** add or change test infrastructure (frameworks, plugins, configs) unless the repo already uses it. +Use the repo's existing test framework and conventions. If multiple frameworks are present, follow whichever existing tests use. Only introduce a framework (default to pytest) if the repo has no tests at all. ## Environment Detection diff --git a/plugins/dotnet-test/skills/code-testing-extensions/extensions/typescript.md b/plugins/dotnet-test/skills/code-testing-extensions/extensions/typescript.md index 961b37726e..de26a70ab9 100644 --- a/plugins/dotnet-test/skills/code-testing-extensions/extensions/typescript.md +++ b/plugins/dotnet-test/skills/code-testing-extensions/extensions/typescript.md @@ -10,7 +10,7 @@ Before writing any test or running any command, read: 2. **`package.json`** — `scripts.test`, `devDependencies`, `type` field 3. **Config files** — `tsconfig.json`, `jest.config.*`, `vitest.config.*`, `eslint.config.*` -Do **not** add or change test infrastructure (frameworks, configs, transforms) unless the repo already uses it. Use the runner the repo already has — do not switch runners. +Use the repo's existing test runner and conventions — do not switch frameworks. If multiple runners are configured, follow whichever `scripts.test` invokes. Only introduce a framework if the repo has no tests at all. ## Package Manager Detection From 22035263e28d97a0c539b8adb71d623045d4cdcd Mon Sep 17 00:00:00 2001 From: Jakub Jares Date: Tue, 12 May 2026 13:23:13 +0200 Subject: [PATCH 5/5] Make Python extension framework-adaptive instead of pytest-centric Rule #1 now searches for ALL test file formats (not just test_*.py), explicitly calls out custom frameworks like UTscapy, and emphasizes adopting repo conventions fully rather than layering pytest on top. Test Commands section now has a custom framework block before the pytest block, and Test File Naming defers to repo conventions. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../skills/code-testing-extensions/SKILL.md | 2 +- .../extensions/python.md | 37 +++++++++++++------ 2 files changed, 27 insertions(+), 12 deletions(-) diff --git a/plugins/dotnet-test/skills/code-testing-extensions/SKILL.md b/plugins/dotnet-test/skills/code-testing-extensions/SKILL.md index e404673cf6..5d7510917a 100644 --- a/plugins/dotnet-test/skills/code-testing-extensions/SKILL.md +++ b/plugins/dotnet-test/skills/code-testing-extensions/SKILL.md @@ -18,7 +18,7 @@ This skill provides access to language-specific guidance files used by the code- | File | Language | Contents | |------|----------|----------| | [extensions/dotnet.md](extensions/dotnet.md) | .NET (C#/F#/VB) | Build commands, test commands, project reference validation, common CS error codes, MSTest template | -| [extensions/python.md](extensions/python.md) | Python | Build/test commands (pytest), project layout detection, mocking guidelines, common errors, pytest template | +| [extensions/python.md](extensions/python.md) | Python | Framework-adaptive test commands (pytest, custom runners), project layout detection, mocking guidelines, common errors | | [extensions/typescript.md](extensions/typescript.md) | TypeScript/JavaScript | Build/test commands (Jest/Vitest/Mocha), framework detection, mocking, TS-specific considerations | | [extensions/powershell.md](extensions/powershell.md) | PowerShell | Test commands (Pester v5), module import patterns, discovery/run pitfalls, mocking, common errors | | [extensions/cpp.md](extensions/cpp.md) | C++ | Testing internals with friend declarations | diff --git a/plugins/dotnet-test/skills/code-testing-extensions/extensions/python.md b/plugins/dotnet-test/skills/code-testing-extensions/extensions/python.md index 1e8f80394c..6584e520a8 100644 --- a/plugins/dotnet-test/skills/code-testing-extensions/extensions/python.md +++ b/plugins/dotnet-test/skills/code-testing-extensions/extensions/python.md @@ -4,13 +4,18 @@ Language-specific guidance for Python test generation. ## Rule #1: Investigate the Repo First -Before writing any test or running any command, read: +Before writing any test or running any command, discover what the repo already does: -1. **Existing tests** — find `test_*.py` / `*_test.py` files and copy their style (imports, fixtures, class vs function, assertion patterns) -2. **Config files** — `pyproject.toml`, `pytest.ini`, `setup.cfg`, `tox.ini`, `conftest.py` -3. **Package layout** — determine import paths from existing code, not guesswork +1. **Find ALL existing test files** — search broadly: `test_*.py`, `*_test.py`, `*.uts`, `test/*.sh`, or any other test format. Do not assume pytest. +2. **Identify the test framework** — look for: + - Custom test runners (e.g. `UTscapy` for scapy, project-specific harnesses) + - Standard frameworks (`pytest`, `unittest`, `nose2`) + - Test runner scripts in `Makefile`, `tox.ini`, `nox`, `scripts/` + - Config entries in `pyproject.toml`, `setup.cfg`, `pytest.ini`, `conftest.py` +3. **Read existing tests thoroughly** — copy their exact style: file format, imports, fixtures, assertion patterns, helper utilities, setup/teardown conventions +4. **Package layout** — determine import paths from existing code, not guesswork -Use the repo's existing test framework and conventions. If multiple frameworks are present, follow whichever existing tests use. Only introduce a framework (default to pytest) if the repo has no tests at all. +**Use whatever framework and conventions the repo already uses.** If the repo uses a custom test framework (custom file formats, custom runners, domain-specific test utilities), adopt it fully — do not layer pytest on top. Only introduce pytest if the repo has no tests at all. ## Environment Detection @@ -38,7 +43,15 @@ Python has no separate build step. Validate with the type checker if one is conf ## Test Commands -Always use the detected ``. Prefer `python -m pytest` over bare `pytest` to ensure the correct interpreter. +If the repo uses a **custom test framework** (custom file formats, custom runner), use its native commands — do not wrap them in pytest. Examples: + +| Framework | Command | +|-----------|---------| +| UTscapy (`.uts` files) | ` scapy.tools.UTscapy -f test/test_file.uts` | +| Custom runner script | `make test`, `./run_tests.sh`, `tox` | +| Repo-defined script | Whatever `scripts.test` in Makefile/tox/nox specifies | + +For **pytest** projects (the most common case), use the detected ``: | Scope | Command | |-------|---------| @@ -48,7 +61,7 @@ Always use the detected ``. Prefer `python -m pytest` over bare `pytest` | Keyword filter | ` pytest -k "keyword"` | | Stop on first failure | ` pytest -x --tb=short` | -- If `scripts.test` exists in `Makefile`/`tox`/`nox`, prefer that +- Prefer `python -m pytest` over bare `pytest` to ensure the correct interpreter - If the project uses `unittest` only (no pytest in deps), use `python -m unittest discover` ## Lint Command @@ -73,10 +86,12 @@ Use the repo's existing lint script first (`make lint`, `tox -e lint`). Otherwis ## Test File Naming -- Files: `test_*.py` or `*_test.py` -- Functions: `test_` prefix -- Classes: `Test` prefix, no `__init__` -- No registration step needed — pytest discovers automatically +Match the repo's existing conventions. Common patterns: + +- **pytest**: Files `test_*.py` or `*_test.py`, functions `test_` prefix, classes `Test` prefix +- **Custom frameworks**: Use whatever format existing tests use (e.g. `.uts` for UTscapy, custom extensions) + +If writing new tests in a repo with no tests, default to pytest conventions. ## Common Errors