Improve dotnet-test eval coverage and efficiency

plugins/dotnet-test/README.md

@@ -80,7 +80,6 @@ These are the entry-point agents you invoke directly:
 
 | Agent | Purpose |
 |---|---|
-| **code-testing-generator** | Orchestrates the full test generation pipeline (research → plan → implement → build → test → fix → lint) |
 | **test-quality-auditor** | Runs multi-skill audit pipelines for comprehensive test suite assessment |
 | **testability-migration** | End-to-end testability improvement: detect → generate wrappers → migrate call sites |
 
@@ -92,6 +91,7 @@ These are pipeline stages invoked automatically by the agents above (`user-invoc
 
 | Agent | Called by | Purpose |
 |---|---|---|
+| **code-testing-generator** | code-testing-agent skill | Orchestrates the full test generation pipeline (research → plan → implement → build → test → fix → lint) |
 | **code-testing-researcher** | code-testing-generator | Analyzes codebase structure, testing patterns, and testability |
 | **code-testing-planner** | code-testing-generator | Creates phased test implementation plans from research findings |
 | **code-testing-implementer** | code-testing-generator | Implements one phase from the plan, runs build-test-fix cycles |

plugins/dotnet-test/agents/code-testing-generator.agent.md

@@ -1,11 +1,10 @@
 ---
 description: >-
-  Orchestrates comprehensive test generation using
-  Research-Plan-Implement pipeline. Use when asked to generate tests, write unit
-  tests, improve test coverage, or add tests. DO NOT USE FOR: diagnosing
-  coverage plateaus or project-wide coverage/CRAP analysis without writing tests
-  (use coverage-analysis); targeted method/class CRAP scores (use crap-score).
+  Internal implementation agent for the code-testing-agent skill. Orchestrates
+  the Research-Plan-Implement pipeline after that public entry-point skill
+  delegates a test-generation request. Do not route user prompts here directly.
 name: code-testing-generator
+user-invocable: false
 tools: ["agent", "skill", "read", "search", "edit", "execute", "Task", "Skill", "Read", "Glob", "Grep", "Edit", "Write", "Bash", "read_file", "replace", "write_file", "glob", "grep_search", "run_shell_command"]
 agents:
   - code-testing-researcher
@@ -38,6 +37,14 @@ Understand what the user wants: scope (project, files, classes), priority areas,
 
 Before writing code, read the language-specific base extension. Reuse it for the whole run; sub-agents must not independently reload the same reference unless they need a section that was not captured in `.testagent/research.md`.
 
+Create a **requirement checklist** from the request before choosing a strategy.
+Preserve each explicit behavior, layer, collaborator seam, boundary case,
+integration, coverage threshold, and required artifact as a separate item. For
+example, "mock the repository in service tests", "exercise SQLite in memory",
+and "cover pagination boundaries" are three independently verifiable
+requirements. Direct strategy keeps this checklist in context; delegated
+strategies record it in `.testagent/research.md`.
+
 ### Step 2: Choose Execution Strategy
 
 Based on the request scope, pick exactly one strategy and follow it:
@@ -134,15 +141,21 @@ Additional self-review heuristics (still required, even when running the skills)
 
 After the previous phases complete, use the target inventory already recorded in `.testagent/research.md` and the files reported by implementers. Do not rescan or reread the workspace.
 
-1. Compare the bounded target inventory with implemented test files.
-2. If the user requested a measurable coverage target, collect coverage once and prioritize only the reported gaps.
-3. Otherwise, add tests only for requested targets that remain unaddressed.
-4. Stop when the requested scope is covered or the stated target is met; do not recursively expand into unrelated files.
-5. If this step added or modified any tests, re-run the full Step 7 pre-completion gate (`test-gap-analysis` + `assertion-quality` + prompt-scenario coverage) on the new or changed tests before reporting completion.
+1. Compare the requirement checklist and bounded target inventory with the implemented tests.
+2. Inspect the generated test bodies for evidence of every checklist item. A covered line does not prove that a requested collaborator was mocked, a concrete result was asserted, or a boundary/property combination was exercised.
+3. If the user requested a measurable coverage target, collect coverage once and prioritize only gaps inside the requested scope.
+4. Add tests for any unaddressed checklist item before adding optional cases merely to raise test count.
+5. Stop only when every feasible checklist item is covered and the stated target is met; do not recursively expand into unrelated files.
+6. If this step added or modified tests, re-run the full Step 7 pre-completion gate (`test-gap-analysis` + `assertion-quality` + prompt-scenario coverage) on those tests before reporting completion.
 
 ### Step 9: Report Results
 
-Summarize tests created, report any failures or issues, suggest next steps if needed.
+Summarize tests created, report any failures or issues, and include a compact
+**Requirement coverage** section that maps each explicit request to the test
+file or test group that satisfies it. Name concrete evidence such as the mock
+or fake used, fixed inputs and expected values, boundary combinations,
+in-memory integration fixture, and generated coverage artifact. Do not report
+a requirement as covered based only on aggregate coverage.
 
 **Example final report:**
 

plugins/dotnet-test/agents/code-testing-researcher.agent.md

@@ -56,12 +56,16 @@ Based on files found:
 ### 3. Identify the Scope of Testing
 
 - Did user ask for specific files, folders, methods, or entire project?
-- If specific scope is mentioned, focus research on that area. If not, analyze entire codebase.
+- If specific scope is mentioned, focus research on that area.
+- If scope is omitted, bound research to the nearest project or package rooted
+  at the working directory, as identified by its closest manifest. Do not
+  inventory sibling projects. If no project boundary can be inferred, record
+  the ambiguity for the generator instead of expanding to the entire workspace.
 
 ### 4. Use the cheapest discovery path
 
 - Prefer project manifests, language-server references, and deterministic pairing tools over whole-tree text searches.
-- For C#/.NET multi-file scopes, invoke `find-untested-sources` once and consume its JSON instead of manually walking source and test trees.
+- For multi-file scopes in C#, Python, TypeScript/JavaScript, Go, Java, Rust, or Ruby, invoke `find-untested-sources` once and consume its JSON instead of manually walking source and test trees.
 - Do not spawn sub-agents for discovery that can be completed with one bounded search.
 - Use parallel sub-agents only when the requested scope contains independent projects or languages that need separate context.
 
@@ -108,7 +112,7 @@ Locate tests paired to the bounded target inventory:
   - Whether tests cover only happy paths or also edge cases and error paths
 - Do not invent numeric coverage percentages without a coverage report.
 
-**For C# / .NET repos**, before manually pairing source ↔ test files, invoke the `find-untested-sources` skill (when available in the workspace). It parses every `.cs` file with Roslyn — no build, no `Compilation`, no `MetadataReferences` — and returns a deterministic JSON map: `source_to_tests` (which test files reference which source), an `untested` list ordered by API surface (`decl_count`) descending, and a `suggested_test_path` derived from existing `<ProjectReference>` edges. Use its `untested` list as the prioritized worklist and `source_to_tests` for pairing. Do not then repeat the same discovery manually. Fall back to bounded manual discovery only when the skill is unavailable or the code is non-C#.
+Before manually pairing source ↔ test files in C#, Python, TypeScript/JavaScript, Go, Java, Rust, or Ruby, invoke the `find-untested-sources` skill when available. It returns a deterministic JSON pairing map, an untested list ordered by declared API surface, and suggested test paths. For .NET-only repositories, prefer its namespace-aware Roslyn engine; otherwise use its tree-sitter engine. Use the untested list as the prioritized worklist and do not repeat the same discovery manually. Fall back to bounded manual discovery only when the skill is unavailable or the language is unsupported.
 
 ### 8. Generate Research Document
 

plugins/dotnet-test/skills/code-testing-agent/SKILL.md

@@ -1,22 +1,17 @@
 ---
 name: code-testing-agent
 description: >-
-  Generates and writes new unit tests for any programming language —
-  scaffolds test projects and configures coverage tooling
-  (coverlet, pytest-cov, @vitest/coverage-v8) as part of test
-  generation. Use when asked to generate tests, generate pytest
-  tests, generate Vitest tests, write unit tests, add tests, improve
-  coverage, comprehensive tests, or scaffold a new test project or
-  suite for an app, service, library, REST API, blueprint, or
-  package — including project-wide, multi-file test generation
-  across services, repositories, routes, and modules. Supports
-  C#/.NET, Python (pytest, Flask/Django), TypeScript/JavaScript
-  (Vitest, Jest, Mocha), Go, Rust, Java (JUnit). Runs a research,
-  planning, and implementation pipeline so tests compile and pass.
-  DO NOT USE FOR: running existing tests (use run-tests); analyzing
-  existing coverage reports (use coverage-analysis or crap-score);
-  writing, fixing, or modernizing MSTest-specific tests, assertions,
-  attributes, or lifecycle (use writing-mstest-tests).
+  MANDATORY ENTRY POINT for generating or writing tests. Invoke this skill
+  before editing files whenever the user asks to generate tests, write/add unit
+  tests, scaffold a test project or pytest/Vitest/Jest suite, create
+  comprehensive tests, improve/achieve coverage, or test an app, API, service,
+  repository, route, module, library, or package. Supports C#/.NET, Python,
+  TypeScript/JavaScript, Go, Rust, Java, and Ruby. For sparse, gutted-looking,
+  synthetic, or incomplete workspaces, test only the source that remains and
+  never restore missing source.
+  DO NOT USE FOR: running existing tests (use run-tests); analyzing coverage
+  reports (use coverage-analysis or crap-score); MSTest-specific test authoring
+  or modernization (use writing-mstest-tests).
 license: MIT
 ---
 
@@ -85,21 +80,54 @@ When the user does not express strong requirements for test style, coverage goal
 Start by calling the `code-testing-generator` agent with your test generation request:
 
 ```text
-Generate unit tests for [path or description of what to test], following the [unit-test-generation.prompt.md](unit-test-generation.prompt.md) guidelines
+Generate unit tests for [path or description of what to test], following the [unit-test-generation.prompt.md](unit-test-generation.prompt.md) guidelines. Treat the current workspace as authoritative even when it is sparse, gutted-looking, synthetic, or missing tracked files; never restore or reconstruct it.
 ```
 
 The Test Generator will manage the entire pipeline automatically.
 
+If `code-testing-generator` is unavailable, do not skip the workflow. Execute the
+same Research → Plan → Implement sequence inline, create the `.testagent/`
+artifacts described below, and apply the same completion contract.
+
 ### Step 3: Execute with bounded context
 
 For multi-file requests:
 
-1. Research only the requested module or project and write a compact `.testagent/research.md`.
-2. Reuse manifests, symbol references, and deterministic pairing tools instead of reading every source and test file.
-3. For C# multi-file scopes, run `find-untested-sources` once and consume its `source_to_tests`, `untested`, and `suggested_test_path` output; do not repeat that pairing manually.
-4. Plan each target file once, then implement phases sequentially.
-5. Build and test the narrow target during fix cycles; run workspace-level validation once at the end.
-6. Read a language example from `code-testing-extensions` only when the repository has no representative tests and the base extension is insufficient.
+1. Turn every explicit user requirement into a checklist before implementation. Include requested layers, collaborators to mock, boundary cases, integrations, coverage thresholds, and report artifacts.
+2. Research only the requested module or project and write the checklist plus a compact target inventory to `.testagent/research.md`.
+3. Reuse manifests, symbol references, and deterministic pairing tools instead of reading every source and test file.
+4. For multi-file scopes in C#, Python, TypeScript/JavaScript, Go, Java, Rust, or Ruby, run `find-untested-sources` once and consume its pairing and suggested-path output; do not repeat that discovery manually.
+5. Plan each target file once, then implement phases sequentially. Map every checklist item to at least one concrete test or explain why it is blocked.
+6. Build and test the narrow target during fix cycles; run workspace-level validation once at the end.
+7. Before reporting success, re-open the generated tests and verify every checklist item against concrete test names and assertions. Coverage alone is not evidence that a requested mock seam, boundary, state transition, or property combination was tested.
+8. Read a language example from `code-testing-extensions` only when the repository has no representative tests and the base extension is insufficient.
+
+### Completion contract
+
+Do not report completion until all of these are true:
+
+1. `.testagent/research.md` records the bounded target inventory, existing test
+   conventions, and the acceptance checklist.
+2. `.testagent/plan.md` maps each checklist item to a planned test or an explicit
+   blocker.
+3. Generated tests compile and pass with the narrowest relevant test command.
+4. Every explicit user requirement is backed by a concrete test and assertion.
+   Fix missing mock seams, boundary cases, state transitions, and property
+   combinations even when coverage already passes. In the final summary, cite
+   at least one generated test name for every checklist item so completion is
+   auditable; if an item has no test to cite, keep implementing or report it as
+   blocked. For non-behavioral requirements such as scaffolding, scope limits,
+   commands, or coverage artifacts, cite the relevant file, command, or report
+   instead of forcing a test-name mapping.
+5. Review the generated tests for behavior gaps and weak assertions. Invoke
+   `test-gap-analysis` and `assertion-quality` when available; otherwise perform
+   the equivalent review inline and record the findings and fixes in
+   `.testagent/status.md`.
+
+The final response MUST include a compact `Requirement | Evidence` table.
+Behavioral rows cite exact generated test names. Non-behavioral rows cite the
+relevant project file, validation command, or coverage report. A generic list
+of tested areas is not a substitute for requirement-by-requirement evidence.
 
 ## State Management
 
@@ -111,40 +139,6 @@ All pipeline state is stored in `.testagent/` folder:
 | `.testagent/plan.md`     | Phased implementation plan   |
 | `.testagent/status.md`   | Progress tracking (optional) |
 
-## Examples
-
-### Strategy Selection
-
-The generator picks a strategy based on request scope:
-
-| User Request | Strategy | Why |
-|---|---|---|
-| "Generate tests for `src/services/UserService.ts`" | **Direct** | Single file, small scope — write tests immediately, skip sub-agents (but still run the generator's Step 7 pre-completion gate — `test-gap-analysis` + `assertion-quality` — before finishing) |
-| "Add unit tests for my billing project" | **Single pass** | Moderate scope — one Research → Plan → Implement cycle covers it |
-| "Achieve 80% coverage across the entire solution" | **Iterative** | Large scope — multiple R→P→I cycles, each narrowing remaining gaps |
-
-### Pipeline Walkthrough
-
-Given a request like *"Generate unit tests for my InvoiceService"*, the pipeline produces:
-
-1. **Research** → `.testagent/research.md` containing detected language/framework, build commands, files to test ranked by priority, and existing test inventory
-2. **Plan** → `.testagent/plan.md` containing phased approach with specific methods and test scenarios (happy path, edge cases, error cases) for each file
-3. **Implement** → Test files written, built, and verified per phase. Fix cycle runs automatically if build/test errors occur
-4. **Validate** → Full workspace build + full test run to catch cross-project issues
-5. **Report** → Summary of tests created, pass/fail counts, coverage notes, and next steps
-
-### Language-Specific Examples
-
-The `code-testing-extensions` skill provides concrete, filled-in examples for each pipeline phase showing real source code, real research output, real plans, and real generated tests. Call the `code-testing-extensions` skill to discover available extension files, then read:
-
-- **`dotnet-examples.md`** — MSTest example with InvoiceService: research output, plan output, generated test file, fix cycle walkthrough, and final report
-- **`python-examples.md`** — pytest example with the same InvoiceService scenario: research, plan, generated test file (parametrized, `unittest.mock`), fix cycles (`ModuleNotFoundError`, patch target, `Mock(spec=...)`), and final report
-- **`typescript-examples.md`** — Vitest example (also applicable to Jest) showing `it.each` parameterization, async tests, fake timers, and ESM/CJS fix cycles
-- **`go-examples.md`** — Standard `testing` package example with table-driven subtests, hand-written fake repository, injected clock, and `-run` regex fix cycle
-- **`java-examples.md`** — JUnit 5 + Mockito example on Maven showing `@ExtendWith(MockitoExtension.class)`, `@ParameterizedTest` + `@CsvSource`, `Clock.fixed(...)` for time, and Surefire fix cycles
-
-For languages without a dedicated examples file (Rust, Ruby, Swift, Kotlin, C++, PowerShell), use the base extension file (`<language>.md`) plus the example file for the closest paradigm — the pipeline shape (research → plan → generate → fix) and the categories of decisions (test layout, mocking strategy, fixed clock for time-dependent code, parameterization style) translate directly.
-
 ## Agent Reference
 
 | Agent                      | Purpose              |

plugins/dotnet-test/skills/code-testing-agent/unit-test-generation.prompt.md

@@ -49,6 +49,11 @@ When the task specifies particular test scenarios or behaviors to cover:
 2. **Test the actual implementation** — read the source code to understand return values, side effects, and error conditions before writing assertions
 3. **Fewer focused tests beat many shallow ones** — 5 tests that thoroughly exercise the function are better than 20 that only check surface behavior
 4. **Every test must pass** — run tests after writing them; fix immediately if they fail
+5. **Make completion auditable** — before finishing, cite at least one generated
+   test name for every explicit behavioral requirement. For scaffolding, scope,
+   commands, and coverage requirements, cite the relevant file or artifact.
+   Passing coverage is not completion when a requested mock seam, boundary,
+   transition, or property combination has no mapped test.
 
 ## Write Tests That Pin Down Behavior