Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 38 additions & 16 deletions .github/workflows/test-finder.yml
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,7 @@ jobs:
5. Focus on the goose crate first, then goose-cli, then others

Process:
0. Immediately write these requirements in your TODO list tool and periodically check against them
1. Find a suitable untested function (use your `analyze` tool and ripgrep)
2. Write a comprehensive unit test for it
3. Apply your changes to the codebase
Expand All @@ -68,10 +69,8 @@ jobs:
- Apply the fix and run the test again
- Repeat up to 3 times until the test passes
6. Once the test passes (or after 3 attempts), save the final changes as a git diff to /tmp/test_addition.patch
7. Report the outcome:
- If successful: write "SUCCESS" to /tmp/test_result.txt
- If no suitable function found: write "NO_FUNCTION_FOUND" to /tmp/test_result.txt
- If test couldn't be fixed: write "TEST_FAILED" to /tmp/test_result.txt
7. If successful, write the name of the function you tested to /tmp/function_tested.txt (just the function name, e.g., "check_tool_call" or "MyStruct::my_method")
8. Only create the patch file if the test actually passes

Important:
- Only add ONE test for ONE function
Expand All @@ -82,23 +81,45 @@ jobs:

goose run -i /tmp/create_working_test.txt --with-builtin developer

# Check the result
if [ -f /tmp/test_result.txt ]; then
RESULT=$(cat /tmp/test_result.txt)
echo "Test creation result: $RESULT"
# Debug: Check what files were created
echo "Checking for patch file..."
if [ -f /tmp/test_addition.patch ]; then
echo "Patch file exists"
echo "Patch size: $(wc -c < /tmp/test_addition.patch) bytes"
echo "First few lines of patch:"
head -5 /tmp/test_addition.patch || true
else
echo "No patch file found at /tmp/test_addition.patch"
fi

if [ "$RESULT" = "SUCCESS" ] && [ -f /tmp/test_addition.patch ]; then
echo "patch_created=true" >> $GITHUB_OUTPUT
# Extract function name from patch for PR title
FUNC_NAME=$(grep -E "fn test_|#\[test\]" /tmp/test_addition.patch | head -1 | sed 's/.*test_//' | sed 's/(.*//' || echo "function")
echo "function_name=${FUNC_NAME}" >> $GITHUB_OUTPUT
# Check for new commits that Goose might have made
COMMITS_AHEAD=$(git rev-list HEAD --not --remotes=origin --count 2>/dev/null || echo "0")
echo "Commits ahead of origin: $COMMITS_AHEAD"

# Check if we have changes to create a PR from
# Either: 1) A patch file exists, OR 2) There are new commits
if [ -f /tmp/test_addition.patch ] && [ -s /tmp/test_addition.patch ] || [ "$COMMITS_AHEAD" -gt 0 ]; then
echo "Changes detected (patch file or new commits)"
echo "Attempting to apply patch..."
# Apply the patch for the PR
git apply /tmp/test_addition.patch 2>/dev/null || echo "Patch already applied or changes already present"
echo "patch_created=true" >> $GITHUB_OUTPUT
echo "Test creation successful - patch file created"

# Try to get the function name from Goose's output file
if [ -f /tmp/function_tested.txt ]; then
FUNC_NAME=$(cat /tmp/function_tested.txt | head -1)
else
echo "patch_created=false" >> $GITHUB_OUTPUT
echo "Reason: $RESULT"
# Fallback: Extract from test name in the actual changes (staged or committed)
# Try staged changes first, then last commit, then patch file
FUNC_NAME=$(git diff --cached | grep "^+.*fn test_" | head -1 | sed 's/.*fn test_//' | sed 's/(.*//' || git diff HEAD~1 | grep "^+.*fn test_" | head -1 | sed 's/.*fn test_//' | sed 's/(.*//' || grep "fn test_" /tmp/test_addition.patch 2>/dev/null | head -1 | sed 's/.*fn test_//' | sed 's/(.*//' || echo "function")
fi
# Clean up the function name (remove any trailing whitespace or special chars)
FUNC_NAME=$(echo "$FUNC_NAME" | tr -d '\n\r' | sed 's/[[:space:]]*$//')
echo "function_name=${FUNC_NAME}" >> $GITHUB_OUTPUT
else
echo "patch_created=false" >> $GITHUB_OUTPUT
echo "No result file created"
echo "No patch file created - either no suitable function found or test failed"
fi

- name: Extract token metrics
Expand Down Expand Up @@ -136,6 +157,7 @@ jobs:
token: ${{ secrets.GITHUB_TOKEN }}
commit-message: "test: add test for ${{ steps.find_untested.outputs.function_name }}"
title: "test: add test coverage for ${{ steps.find_untested.outputs.function_name }}"
draft: true
body: |
## 🤖 Automated Test Addition

Expand Down
Loading