feat(ai-test): handle no-changes-needed case in contracts test system#18484
Merged
feat(ai-test): handle no-changes-needed case in contracts test system#18484
Conversation
aliersh
force-pushed
the
ari/ai-test-no-changes-needed
branch
from
abf42f4 to
5638758
Compare
- add structured_output schema to devin prompt instructing completion status - add check for structured_output in devin_client.py blocked status handler - add finished_no_changes status when analysis complete without changes - create slack-notifier component with prepare_notification.sh script - add prepare-slack-notification command to justfile - add slack notification for finished_no_changes status - replace inline slack logic in circleci config with just command
aliersh
force-pushed
the
ari/ai-test-no-changes-needed
branch
from
5638758 to
dcd5398
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## develop #18484 +/- ##
===========================================
- Coverage 75.62% 75.32% -0.30%
===========================================
Files 187 187
Lines 11199 11199
===========================================
- Hits 8469 8436 -33
- Misses 2584 2619 +35
+ Partials 146 144 -2
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
smartcontracts
approved these changes
Dec 3, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds support for handling test files that require no changes in the AI Contracts Test Maintenance System.
Reason
Previously, when Devin analyzed a test file and determined no changes were needed, the system would fail because it expected either a PR or an error state. This caused legitimate successful analyses to be treated as failures, preventing proper logging and notifications. The system now correctly handles this third outcome by using Devin's structured output feature to distinguish between "no changes needed" and actual failures.
Changes
analysis_completeandchanges_neededfields when handling blocked sessionsfinished_no_changesstatus for sessions that complete analysis without requiring modificationsprepare-slack-notificationcommand to justfilefinished_no_changesstatusrunbook.mdto reflect new status and structured output handling