Normalize NL/T JUnit names and robust summary

[dsarno]

Summary

• Normalize merged NL/T JUnit fragments by deriving testcase IDs from filenames or system output and prefixing them to testcase names.
• Build markdown summary tolerant of missing IDs by detecting IDs in testcase names or system output and reporting unmapped cases.
• Tighten prompt instructions to emit testcases with {TESTID} — {Title} names and avoid creating directories during reporting.

Testing

• pytest

---

https://chatgpt.com/codex/tasks/task_e_68bba3b53ec88327988af4a6db486fef

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

✨ Finishing Touches

🧪 Generate unit tests

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch codex/fix-testcase-naming-mismatch-in-reports

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[greptile-apps]

Greptile Summary

This PR standardizes JUnit XML testcase naming and improves test report robustness in the Unity MCP project's CI/CD pipeline. The changes focus on two main areas: updating prompt instructions for consistent test case naming and enhancing the GitHub Actions workflow to handle test result processing more reliably.

The primary change updates the Claude prompt template to use a standardized {TESTID} — {Title} format for test case names instead of the previous <TESTID>: <description> format. This new format uses em-dash separators which provide better XML parsing compatibility and make test IDs more easily extractable by automated tools. The prompt also now explicitly instructs not to create directories during reporting, assuming the reports directory already exists.

The GitHub Actions workflow receives significant enhancements to the JUnit fragment processing logic. The new implementation derives test IDs from multiple sources (filenames, system output) and normalizes them into canonical NL-X/T-X formats. This addresses inconsistencies where test cases might not have proper ID prefixes, ensuring all test cases are consistently named and trackable across CI runs. The markdown summary generation is also made more robust by implementing fallback detection mechanisms that can identify test IDs in various locations within the test output, preventing test results from being lost when they don't match expected patterns.

These changes integrate with the existing Unity MCP testing infrastructure, which includes comprehensive test suites in the /tests directory and the TestProjects structure. The standardization will improve the reliability of test result parsing in the CI pipeline and make test reports more machine-readable for automated analysis.

Important Files Changed

Changed Files

Filename	Score	Overview
.claude/prompts/nl-unity-suite-full-additive.md	5/5	Updated XML testcase naming format from colon to em-dash separator and added directory creation instructions
.github/workflows/claude-nl-suite.yml	4/5	Major refactor of JUnit processing with enhanced test ID normalization, robust summary generation, and improved error handling

Confidence score: 4/5

• This PR is safe to merge with minimal risk as it primarily improves test reporting consistency without affecting core functionality
• Score reflects well-structured changes that enhance robustness, though the workflow logic is complex and could benefit from additional testing
• Pay close attention to .github/workflows/claude-nl-suite.yml due to the significant refactoring of test processing logic

Sequence Diagram

sequenceDiagram
    participant User
    participant GitHub as "GitHub Actions"
    participant Unity as "Unity Container"
    participant MCP as "MCP Server"
    participant Claude as "Claude Agent"
    participant Reports as "Report System"

    User->>GitHub: "Trigger workflow_dispatch"
    GitHub->>GitHub: "Check secrets (Unity, Anthropic)"
    GitHub->>Unity: "Setup Unity license & container"
    Unity-->>GitHub: "License activation complete"
    
    GitHub->>Unity: "Start headless Unity with MCP bridge"
    Unity->>MCP: "Initialize MCP bridge on TCP port"
    MCP-->>GitHub: "Bridge ready signal"
    
    GitHub->>GitHub: "Create MCP config (.claude/mcp.json)"
    GitHub->>GitHub: "Prepare report directories and skeletons"
    GitHub->>GitHub: "Snapshot baseline test file"
    
    GitHub->>Claude: "Execute NL/T test suite with additive design"
    Claude->>MCP: "Read target file (LongUnityScriptClaudeTest.cs)"
    MCP->>Unity: "Request file content"
    Unity-->>MCP: "Return file content"
    MCP-->>Claude: "File content response"
    
    loop "For each test NL-0 to NL-4, T-A to T-J"
        Claude->>MCP: "Apply text edits (additive)"
        MCP->>Unity: "Execute edit operation"
        Unity-->>MCP: "Edit result"
        MCP-->>Claude: "Edit confirmation"
        
        Claude->>MCP: "Validate script"
        MCP->>Unity: "Validate compilation"
        Unity-->>MCP: "Validation result"
        MCP-->>Claude: "Validation response"
        
        Claude->>Reports: "Write testcase XML fragment"
    end
    
    GitHub->>Reports: "Canonicalize testcase names (NL/T prefixes)"
    GitHub->>Reports: "Backfill missing test placeholders"
    GitHub->>Reports: "Merge fragments into single JUnit XML"
    GitHub->>Reports: "Build markdown summary from JUnit"
    
    GitHub->>GitHub: "Publish JUnit report with test results"
    GitHub->>GitHub: "Upload artifacts (reports, fragments, transcript)"
    
    GitHub->>Unity: "Stop Unity container"
    Unity-->>GitHub: "Container stopped"

Loading

_{2 files reviewed, 1 comment}

_{Edit Code Review Bot Settings | Greptile}

[greptile-apps]

style: The em-dash (—) character may cause encoding issues in some XML parsers or CI systems. Consider using a standard hyphen (-) for better compatibility.