Skip to content

Commit

Permalink
Separate code cov and mutation cov unittest generator (#19)
Browse files Browse the repository at this point in the history
Fixes #18
  • Loading branch information
jungs1 authored Aug 7, 2024
1 parent 021a063 commit 118b430
Show file tree
Hide file tree
Showing 8 changed files with 570 additions and 225 deletions.
9 changes: 4 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,6 @@ We'd love to hear your feedback, suggestions, and any thoughts you have on mutat

[![Run on Replit](https://replit.com/badge/github/codeintegrity-ai/mutahunter)](https://replit.com/@raghuramabilash/Mutahunterai)


## Table of Contents

- [Features](#features)
Expand Down Expand Up @@ -55,10 +54,10 @@ This tool generates unit tests to increase both line and mutation coverage, insp
## go to examples/java_maven
## remove some tests from BankAccountTest.java

mutahunter gen --test-command "mvn clean test" --code-coverage-report-path "target/site/jacoco/jacoco.xml" --test-file-path "src/test/java/BankAccountTest.java" --source-file-path "src/main/java/com/example/BankAccount.java" --coverage-type jacoco  --model "gpt-4o"
mutahunter gen-line --test-command "mvn test -Dtest=BankAccountTest" --code-coverage-report-path "target/site/jacoco/jacoco.xml" --coverage-type jacoco --test-file-path "src/test/java/BankAccountTest.java" --source-file-path "src/main/java/com/example/BankAccount.java" --model "gpt-4o" --target-line-coverage 0.9 --max-attempts 3

Line coverage increased from 47.00% to 100.00%
Mutation coverage increased from 92.86% to 92.86%
Line Coverage increased from 47.00% to 100.00%
Mutation Coverage increased from 92.86% to 92.86%
```

## Getting Started with Mutation Testing
Expand Down Expand Up @@ -178,4 +177,4 @@ jobs:
## Cash Bounty Program
Help us improve Mutahunter and get rewarded! We have a cash bounty program to incentivize contributions to the project. Check out the [bounty board](https://docs.google.com/spreadsheets/d/1cT2_O55m5txrUgZV81g1gtqE_ZDu9LlzgbpNa_HIisc/edit?gid=0#gid=0) to see the available bounties and claim one today!
Help us improve Mutahunter and get rewarded! We have a cash bounty program to incentivize contributions to the project. Check out the [bounty board](https://docs.google.com/spreadsheets/d/1cT2_O55m5txrUgZV81g1gtqE_ZDu9LlzgbpNa_HIisc/edit?gid=0#gid=0) to see the available bounties and claim one today!
2 changes: 1 addition & 1 deletion examples/java_maven/readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ mutahunter run --test-command "mvn test" --code-coverage-report-path "target/sit

```bash
# remove some tests
mutahunter gen --test-command "mvn test" --code-coverage-report-path "target/site/jacoco/jacoco.xml" --coverage-type jacoco --test-file-path "src/test/java/BankAccountTest.java" --source-file-path "src/main/java/com/example/BankAccount.java" --model "gpt-4o"
mutahunter gen-line --test-command "mvn test -Dtest=BankAccountTest" --code-coverage-report-path "target/site/jacoco/jacoco.xml" --coverage-type jacoco --test-file-path "src/test/java/BankAccountTest.java" --source-file-path "src/main/java/com/example/BankAccount.java" --model "gpt-4o" --target-line-coverage 0.9 --max-attempts 3
```

Check `logs/_latest/html` for mutation report.
17 changes: 11 additions & 6 deletions src/mutahunter/core/controller.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,17 +13,22 @@
from mutahunter.core.db import MutationDatabase
from mutahunter.core.entities.config import MutationTestControllerConfig
from mutahunter.core.error_parser import extract_error_message
from mutahunter.core.exceptions import (CoverageAnalysisError,
MutantKilledError, MutantSurvivedError,
MutationTestingError,
ReportGenerationError,
UnexpectedTestResultError)
from mutahunter.core.exceptions import (
CoverageAnalysisError,
MutantKilledError,
MutantSurvivedError,
MutationTestingError,
ReportGenerationError,
UnexpectedTestResultError,
)
from mutahunter.core.git_handler import GitHandler
from mutahunter.core.io import FileOperationHandler
from mutahunter.core.llm_mutation_engine import LLMMutationEngine
from mutahunter.core.logger import logger
from mutahunter.core.prompts.mutant_generator import (
SYSTEM_PROMPT_MUTANT_ANALYSUS, USER_PROMPT_MUTANT_ANALYSIS)
SYSTEM_PROMPT_MUTANT_ANALYSUS,
USER_PROMPT_MUTANT_ANALYSIS,
)
from mutahunter.core.report import MutantReport
from mutahunter.core.router import LLMRouter
from mutahunter.core.runner import MutantTestRunner
Expand Down
14 changes: 13 additions & 1 deletion src/mutahunter/core/entities/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ class MutationTestControllerConfig:


@dataclass
class UnittestGeneratorConfig:
class UnittestGeneratorLineConfig:
model: str
api_base: str
test_file_path: str
Expand All @@ -24,5 +24,17 @@ class UnittestGeneratorConfig:
code_coverage_report_path: Optional[str]
coverage_type: str
target_line_coverage_rate: float
max_attempts: int


@dataclass
class UnittestGeneratorMutationConfig:
model: str
api_base: str
test_file_path: str
source_file_path: str
test_command: str
code_coverage_report_path: Optional[str]
coverage_type: str
target_mutation_coverage_rate: float
max_attempts: int
2 changes: 1 addition & 1 deletion src/mutahunter/core/prompts/unittest_generator.py
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ class NewTest(BaseModel):
test_behavior: str = Field(..., description="Short description of the behavior the test covers.")
lines_to_cover: str = Field(..., description="List of line numbers, currently uncovered, that this specific new test aims to cover.")
test_name: str = Field(..., description="A short unique test name reflecting the test objective.")
test_code: str = Field(..., description="A single test function testing the behavioral description.")
test_code: str = Field(..., description="A single test function testing the behavioral description. Do not include boilerplate code. Just the test function.")
new_imports_code: str = Field(..., description="New imports required for the new test function, or an empty string if none.")
test_tags: List[str] = Field(...,
description="Tags for the test such as 'happy path', 'edge case', 'other'.")
Expand Down
Loading

0 comments on commit 118b430

Please sign in to comment.