Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions tests/dotnet-test/coverage-analysis/eval.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ scenarios:
- "Identifies CalculateGpa as the highest-risk method (high complexity, low coverage)"
- "Provides actionable recommendations for the riskiest methods"
- "Generates or displays a coverage analysis report"
timeout: 300
timeout: 480

- name: "Run coverage from scratch without existing data"
prompt: "Analyze code coverage and risk hotspots for this project. I haven't run any coverage yet."
Expand Down Expand Up @@ -88,4 +88,4 @@ scenarios:
- "Quantifies the impact — covering CalculateGpa would raise overall coverage significantly"
- "Uses CRAP scores to distinguish the risky gap (CalculateGpa) from trivial ones"
- "Provides specific test recommendations for the uncovered method"
timeout: 300
timeout: 480
4 changes: 2 additions & 2 deletions tests/dotnet-test/run-tests/eval.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -193,7 +193,7 @@ scenarios:
- "xUnit v3 on MTP requires framework-specific filter flags, not VSTest expression syntax"
- "Correctly handles the '--' separator for SDK 9"
expect_tools: ["bash"]
timeout: 120
timeout: 240

- name: "Filter xUnit v3 tests by class pattern and trait using query filter language"
prompt: "I have an xUnit v3 project with several test classes. I need to run only the Smoke tests from classes whose name contains 'Integration' — I want a single combined filter expression for dotnet test rather than separate flags."
Expand All @@ -219,7 +219,7 @@ scenarios:
- "The expression correctly targets only Smoke tests from integration test classes, not all Smoke tests or all integration tests"
- "Used the path-segment syntax with wildcard segments and a trait qualifier like [Category=Smoke]"
expect_tools: ["bash"]
timeout: 120
timeout: 240

- name: "Filter TUnit tests by class using treenode-filter"
prompt: "I want to run only the SmsNotificationTests from this TUnit project using dotnet test. What's the command?"
Expand Down
4 changes: 2 additions & 2 deletions tests/dotnet-test/test-gap-analysis/eval.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,7 @@ scenarios:
- "Acknowledged that both zero and negative quantities are tested for guard clauses"
- "Acknowledged that RemoveStock verifies both the return value and the resulting stock level"
- "If any gaps were found, they were presented as minor improvements rather than critical weaknesses"
timeout: 300
timeout: 420

# ==========================================================================
# Scenario 4: Non-activation — user wants to write new tests
Expand Down Expand Up @@ -140,7 +140,7 @@ scenarios:
rubric:
- "Wrote test methods for the ShoppingCart class"
- "Covered the AddItem and GetTotal methods"
timeout: 120
timeout: 300

# ==========================================================================
# Scenario 5: Report quality — exclude trivial/generated code, trace call
Expand Down
8 changes: 4 additions & 4 deletions tests/dotnet-test/writing-mstest-tests/eval.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ scenarios:
- "Tests both success and failure paths (e.g., null input, empty items)"
- "Uses Assert.ThrowsExactly or Assert.Throws for exception testing instead of [ExpectedException]"
- "Uses Assert.AreEqual with expected as first argument and actual as second"
timeout: 180
timeout: 360

- name: "Write data-driven tests for a calculator"
prompt: "I have a PriceCalculator class with CalculateTotal and ApplyDiscount methods. Help me write data-driven MSTest tests using DataRow and DynamicData that cover multiple input combinations following modern MSTest best practices."
Expand All @@ -44,7 +44,7 @@ scenarios:
- "Seals the test class"
- "Uses Assert.ThrowsExactly or Assert.Throws for testing invalid argument scenarios"
- "Does not use manual foreach loops to iterate test data — uses proper data-driven attributes instead"
timeout: 180
timeout: 360

- name: "Write async tests with cancellation"
prompt: |
Expand Down Expand Up @@ -348,7 +348,7 @@ scenarios:
- "Tests BuildGreeting validates the output starts with 'Dear' and ends with a comma"
- "Tests GenerateSlug validates the slug pattern contains only lowercase letters, digits, and hyphens"
- "Tests exception handling for null or invalid inputs"
timeout: 180
timeout: 360

# ============================================================================
# Goal 6: Comparison assertions
Expand Down Expand Up @@ -400,7 +400,7 @@ scenarios:
- "Tests boundary conditions at grade thresholds (e.g., exactly 90 vs 89)"
- "Tests that NormalizeScore caps the result at 100 when rawScore exceeds maxRaw"
- "Seals the test class"
timeout: 120
timeout: 240

# ============================================================================
# Goal 7: Collection, null, and reference assertions
Expand Down