Skip to content

Accept QA7 routine wording variants - #5423

Merged
serrrfirat merged 3 commits into
mainfrom
codex/fix-qa7-routine-wording
Jun 29, 2026
Merged

serrrfirat merged 3 commits into
mainfrom
codex/fix-qa7-routine-wording

Conversation

@serrrfirat

Copy link
Copy Markdown
Collaborator

Summary

  • allow Reborn WebUI v2 live QA text gates to express explicit wording alternatives with |
  • update QA 7C to accept routine/automation/cron/schedule/fires/watches wording while still requiring bug
  • keep the trigger-record creation check unchanged, so the case still fails if no routine is persisted

Why

QA 7C can legitimately report the created automation as "fires every 5 minutes" or "watches Slack" instead of literally saying "trigger". The latest /canary all failure showed the routine was described successfully but rejected by an overly literal text gate.

Verification

  • python3 -m pytest scripts/reborn_webui_v2_live_qa/test_run_live_qa.py -q

@railway-app
railway-app Bot temporarily deployed to ironclaw-ci-preview / ironclaw-pr-5423 June 29, 2026 21:00 Destroyed
@github-actions github-actions Bot added size: S 10-49 changed lines risk: low Changes to docs, tests, or low-risk modules contributor: core 20+ merged PRs labels Jun 29, 2026
@coderabbitai

coderabbitai Bot commented Jun 29, 2026 •

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 7bf23acd-b767-438b-9c43-88c2a47f2dd8

📥 Commits

Reviewing files that changed from the base of the PR and between 615bfeb and 646e141.

📒 Files selected for processing (2)
  • scripts/reborn_webui_v2_live_qa/run_live_qa.py
  • scripts/reborn_webui_v2_live_qa/test_run_live_qa.py

📝 Walkthrough

Summary by CodeRabbit

  • Bug Fixes

    • Improved Live QA reply validation to evaluate required keywords using grouped, case-insensitive alternatives (AND across items; OR within each item), with smarter whole-word detection for stricter terms.
    • Updated the acceptance criteria for one QA routine so routine/scheduling-related keywords can satisfy the validation.
  • Tests

    • Added unit coverage to verify explicit alternative patterns are accepted and updated the expected validation inputs for the affected QA routine.

Walkthrough

Adds _required_text_matches for |-separated alternatives in required_text, routes _wait_for_assistant_reply through it, updates qa_7c_slack_bug_logger_routine to use alternation keywords, and adds tests for the matcher and updated case.

Changes

OR-matching helper and qa_7c update

Layer / File(s) Summary
_required_text_matches helper and call-site
scripts/reborn_webui_v2_live_qa/run_live_qa.py
Adds _required_text_matches(normalized_text, required_text) -> bool plus _required_option_matches(); _wait_for_assistant_reply now delegates required-text checks, and qa_7c uses `["trigger
Tests
scripts/reborn_webui_v2_live_qa/test_run_live_qa.py
Adds test_required_text_accepts_explicit_alternatives and updates the qa_7c sheet-prep expectation to the alternation string.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~4 minutes

Poem

A pipe now splits the words apart,
one match will do to play its part.
The bug finds friends in kindred text,
and tests confirm what comes next.
Small gate, neat path, no drama chart.

🚥 Pre-merge checks | ✅ 3 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Description check ⚠️ Warning Summary/Why/Verification are present, but the template is mostly incomplete and omits change type, linked issue, impact, rollback, and checklist sections. Add the missing template sections: Change Type, Linked Issue, Security/Database/Blast Radius/Rollback, Trust-Boundary Checklist, and Review Follow-Through.
✅ Passed checks (3 passed)
Check name Status Explanation
Title check ✅ Passed The title matches the main change: relaxing QA7 wording gates to accept routine-variant phrasing.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a helper function _required_text_matches to support pipe-separated alternative options in the required_text matching logic for live QA routines, updating the qa_7c_slack_bug_logger_routine case and adding corresponding unit tests. The review feedback points out a potential issue where empty options (e.g., due to trailing or consecutive pipes) could vacuously succeed, and suggests filtering out empty strings and handling text normalization directly within the helper function.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread scripts/reborn_webui_v2_live_qa/run_live_qa.py Outdated
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
@railway-app
railway-app Bot temporarily deployed to ironclaw-ci-preview / ironclaw-pr-5423 June 29, 2026 21:06 Destroyed

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@scripts/reborn_webui_v2_live_qa/run_live_qa.py`:
- Around line 1031-1033: The word-check in the required-text matching logic is
currently using raw substring containment, which lets `bug` match inside
unrelated words like `debug` or `bugfix`. Update the matching in
`run_live_qa.py` within the `all(...)`/`any(...)` block to perform word-aware
matching instead of `option.strip().lower() in normalized_text`, so the `bug`
requirement is only satisfied by a standalone word while preserving the existing
handling of `piece.split('|')` alternatives.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 3d68b416-9eae-43f6-94af-9fa41ae9b889

📥 Commits

Reviewing files that changed from the base of the PR and between 6990ba4 and 615bfeb.

📒 Files selected for processing (1)
  • scripts/reborn_webui_v2_live_qa/run_live_qa.py

Comment thread scripts/reborn_webui_v2_live_qa/run_live_qa.py
@railway-app
railway-app Bot temporarily deployed to ironclaw-ci-preview / ironclaw-pr-5423 June 29, 2026 21:20 Destroyed
@github-actions github-actions Bot added size: M 50-199 changed lines and removed size: S 10-49 changed lines labels Jun 29, 2026
@serrrfirat
serrrfirat merged commit 988c281 into main Jun 29, 2026
32 checks passed
@serrrfirat
serrrfirat deleted the codex/fix-qa7-routine-wording branch June 29, 2026 21:26

This branch was successfully deployed

No deployments
ironclaw-ci-preview / ironclaw-pr-5423 — 646e1410 Deployed Jun 29, 2026 by railway-app[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

contributor: core 20+ merged PRs risk: low Changes to docs, tests, or low-risk modules size: M 50-199 changed lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant