feat: add template sync validation to prevent consumer repo sync failures#857
Merged
feat: add template sync validation to prevent consumer repo sync failures#857
Conversation
…d slashes The task decomposer was incorrectly splitting compound words like 'additions/removals' or file paths like 'src/utils' into separate tasks because it matched any '/' character. This caused the agents:apply-suggestions workflow to corrupt issue bodies by creating spurious subtasks like: - Color-coded additions (verify: confirm completion in repo) - removals (verify: confirm completion in repo) Fix: - task_decomposer._split_task_parts: Only split on spaced slashes ' / ' (indicating alternatives) not unspaced slashes (compound words/paths) - issue_optimizer._is_large_task: Only flag tasks with spaced slashes as 'large' requiring decomposition Closes: Issue with corrupted tasks in PRs #4355 and #4353
The Automated Status Summary in PR bodies was extracting checkbox patterns (- [ ]) from inside fenced code blocks and treating them as tasks. For example, code examples like: ```python tasks = ['- [ ] example task'] ``` Were being parsed as actual tasks, corrupting the PR body. Fix: - Add stripCodeBlocks() function that removes fenced code blocks (both ``` and ~~~ style) before parsing - Apply code block stripping in collectSections() before extracting tasks - Add regression tests for code block handling This is the actual fix for the corrupted tasks issue - the previous commit (task decomposer slash splitting) addressed a different but related problem.
- Add isCodeFenceLine() helper to detect ``` and ~~~ fences - Update normaliseChecklist() to skip checkbox normalization inside code blocks - Update extractListBlocks() to continue through code blocks - Update extractListBlocksWithOffsets() similarly for offset tracking - Update collectSections() to skip heading detection inside code blocks This fixes the issue where code examples in issue bodies containing patterns like 'tasks:' or '- example item' were being incorrectly detected as section headings or converted to task checkboxes. Code blocks are now: 1. Preserved in the output (not stripped) 2. Excluded from heading detection 3. Excluded from checkbox normalization Closes #TBD
…k.py Per Copilot reviewer feedback: capability_check.py contains a similar function _is_multi_action_task that also checks for '/' in task. This had the same issue - incorrectly flagging compound words like 'additions/removals' and paths like 'src/utils' as multi-action tasks. Changes: - Updated _is_multi_action_task to check for ' / ' (spaced slash) instead - Added comprehensive tests for the fix
…ures PROBLEM: Repeatedly modifying .github/scripts/ without updating templates/consumer-repo/.github/scripts/ causes: - No sync PRs created for consumer repos - Consumer repos missing critical workflow fixes - Wasted cleanup effort SOLUTION: Automated guards that make this mistake impossible Changes: 1. **scripts/validate_template_sync.py** - Validates template files are in sync with source files by comparing SHA256 hashes 2. **scripts/sync_templates.sh** - Helper script to sync all changed files from .github/scripts/ to templates/consumer-repo/ 3. **.github/workflows/validate-template-sync.yml** - CI workflow that fails if templates are out of sync (runs on PR and push to main) 4. **docs/CONTRIBUTING.md** - Added prominent warning section with instructions on using the validation and sync scripts 5. **Synced 7 out-of-sync template files**: - issue_scope_parser.js (code block handling fix) - keepalive_worker_gate.js - keepalive_instruction_template.js - agents_pr_meta_keepalive.js - agents-guard.js - agents_pr_meta_update_body.js - verifier_ci_query.js This will now trigger consumer repo sync PRs for the code block fix.
stranske
temporarily deployed
to
agent-standard
GitHub Actions
Inactive
stranske
temporarily deployed
to
agent-standard
GitHub Actions
Inactive
Contributor
Automated Status SummaryHead SHA: 40703b4
Coverage Overview
Coverage Trend
Top Coverage Hotspots (lowest coverage)
Updated automatically; will refresh on subsequent CI/Docker completions. Keepalive checklistScopeAfter merging PR #103 (multi-agent routing infrastructure), we need to: Context for AgentDesign Decisions & Constraints
Related Issues/PRsReferencesBlockers & Dependencies
Tasks
Acceptance criteria
|
Contributor
🤖 Keepalive Loop StatusPR #857 | Agent: Codex | Iteration 0/5 Current State
🔍 Failure Classification| Error type | infrastructure | |
stranske
temporarily deployed
to
agent-high-privilege
GitHub Actions
Inactive
Contributor
|
Status | ✅ no new diagnostics |
Contributor
|
Autofix updated these files:
|
agents-workflows-bot
bot
temporarily deployed
to
agent-high-privilege
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 96ef587f22
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
- SYNC_WORKFLOW.md: Added step 0 to verify template sync before any sync - SETUP_CHECKLIST.md: Added explanation of template sync process - Clarifies how consumer repos receive updates - Shows validation/sync commands for contributors - Warns consumer repo users not to manually edit workflow files
stranske
temporarily deployed
to
agent-high-privilege
GitHub Actions
Inactive
Contributor
There was a problem hiding this comment.
Pull request overview
This PR introduces automated validation to ensure workflow scripts in .github/scripts/ stay synchronized with their template counterparts in templates/consumer-repo/.github/scripts/. The PR addresses a recurring problem where source file updates weren't being propagated to the template directory, preventing sync PRs to consumer repos.
Changes:
- Added Python validation script to detect hash mismatches between source and template files
- Added bash sync helper script to automate copying changed files to the template directory
- Added CI workflow that fails builds when templates are out of sync with source files
- Updated 7 out-of-sync template files (including the code block handling fix from PR #855)
- Enhanced slash-handling logic in Python modules to distinguish between spaced slashes (alternatives) and compound words with unspaced slashes
- Updated CONTRIBUTING.md with template sync requirements and instructions
Reviewed changes
Copilot reviewed 19 out of 21 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| scripts/validate_template_sync.py | New validation script that compares SHA256 hashes of source vs template files |
| scripts/sync_templates.sh | New helper script to automatically copy changed files to template directory |
| .github/workflows/validate-template-sync.yml | New CI workflow that enforces template synchronization on PRs |
| docs/CONTRIBUTING.md | Added critical template sync guard section with usage instructions |
| scripts/langchain/task_decomposer.py | Updated to only split on spaced slashes, preserving compound words and file paths |
| scripts/langchain/issue_optimizer.py | Updated slash detection for large task identification |
| scripts/langchain/capability_check.py | Updated slash detection for multi-action task identification |
| tests/scripts/test_*.py | Added comprehensive tests for spaced vs unspaced slash handling |
| templates/consumer-repo/.github/scripts/*.js | Synced 7 files with source (code block handling, retry logic, indentation, unused variable cleanup) |
| .github/scripts/issue_scope_parser.js | Added code fence tracking to prevent checkbox addition inside code blocks |
| .github/scripts/tests/issue_scope_parser.test.js | Added tests for code block preservation |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
templates/consumer-repo/.github/scripts/keepalive_instruction_template.js
Show resolved
Hide resolved
…ew files Codex Review P2 Issues Fixed: 1. Validator now FAILS when template counterpart is missing (not just warns) - Prevents CI from passing when new .github/scripts/*.js files lack templates 2. Sync script now CREATES missing template files (not just updates existing) - Makes sync effective for new files, not just changed ones Also: - Renamed workflow to health-72-template-sync.yml (follows naming convention) - Fixed shellcheck SC2181: check exit code directly instead of using $?
- Added to EXPECTED_NAMES in test_workflow_naming.py - Added to docs/ci/WORKFLOWS.md workflow list - Added to docs/ci/WORKFLOW_SYSTEM.md workflow table This fixes the 4 failing pytest tests that check workflow naming conventions and documentation completeness.
stranske
temporarily deployed
to
agent-high-privilege
GitHub Actions
Inactive
stranske
temporarily deployed
to
agent-high-privilege
GitHub Actions
Inactive
Addresses Copilot feedback requesting test coverage. Tests verify: - Detection of hash mismatches - Detection of missing template files (exit 1) - Handling of missing template directories - Exclusion of non-.js files - Proper error messaging and sync suggestions - Multiple simultaneous mismatches Follows patterns from test_validate_version_pins.py and test_sync_dev_dependencies.py.
stranske
temporarily deployed
to
agent-high-privilege
GitHub Actions
Inactive
agents-workflows-bot
bot
temporarily deployed
to
agent-high-privilege
…s time) The validator detected 13 source files without template counterparts. These are needed by consumer repos (e.g., keepalive_instruction_template.js requires keepalive_prompt_routing.js). Created template copies: - conflict_detector.js - keepalive_prompt_routing.js - keepalive_state.js - keepalive_loop.js - gate-docs-only.js - detect-changes.js - coverage-normalize.js - failure_comment_formatter.js - prompt_injection_guard.js - checkout_source.js - github-api-with-retry.js - maint-post-ci.js - error_classifier.js Previous attempt synced in wrong directory and didn't actually commit the files. Addresses Copilot feedback: keepalive_instruction_template.js now has required dependencies.
stranske
temporarily deployed
to
agent-high-privilege
GitHub Actions
Inactive
The previous commit updated tests and docs but forgot to rename the actual workflow file. This fixes all remaining references: - Renamed .github/workflows/validate-template-sync.yml - Updated docs/SYNC_WORKFLOW.md - Updated docs/keepalive/SETUP_CHECKLIST.md Fixes test failures: - test_workflow_slugs_follow_wfv1_prefixes - test_inventory_docs_list_all_workflows - test_workflow_names_match_filename_convention - test_canonical_workflow_names_match_expected_mapping
stranske
temporarily deployed
to
agent-high-privilege
GitHub Actions
Inactive
Changed 'Health 71 Template Sync' to 'Health 72 Template Sync' to match the filename health-72-template-sync.yml. Fixes test failures: - test_canonical_workflow_names_match_expected_mapping - test_workflow_names_match_filename_convention
stranske
temporarily deployed
to
agent-high-privilege
GitHub Actions
Inactive
Tests were failing because they ran the script from tmp_path without copying it there. Fixed by copying the script to tmp_path/scripts/ before running tests. All tests now pass locally.
stranske
temporarily deployed
to
agent-high-privilege
GitHub Actions
Inactive
agents-workflows-bot
bot
temporarily deployed
to
agent-high-privilege
When template directory doesn't exist, validator prints 'Template directory not found' instead of listing individual files. Updated test to accept either message.
stranske
temporarily deployed
to
agent-high-privilege
GitHub Actions
Inactive
CodeQL failed due to API rate limits (transient error). Retriggering all checks.
stranske
temporarily deployed
to
agent-high-privilege
GitHub Actions
Inactive
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Automated Status Summary
Scope
After merging PR #103 (multi-agent routing infrastructure), we need to:
Context for Agent
Design Decisions & Constraints
<!-- keepalive-loop-summary -->| github-actions[bot] | NEW: CLI agent iteration tracking | ✅ Keep for CLI agents |<!-- keepalive-state:v1 -->| agents-workflows-bot[bot] | State tracking |<!-- keepalive-round: N -->| stranske | OLD: Instruction comment | ❌ CLI agents dont need this |agent:*label), we should have exactly one updating comment (<!-- keepalive-loop-summary -->) instead of accumulating 10+ comments per PR.Related Issues/PRs
References
Blockers & Dependencies
Tasks
GITHUB_STEP_SUMMARYoutput so iteration results are visible in the Actions UIAcceptance criteria
Head SHA: 77b9331
Latest Runs: ✅ success — Gate
Required: gate: ✅ success