fix: Prevent short tokens from matching keywords via prefix by stranske · Pull Request #735 · stranske/Workflows

stranske · 2026-01-10T04:11:39Z

Source: Issue #123

Automated Status Summary

Scope

After merging PR #103 (multi-agent routing infrastructure), we need to:

Validate the CLI agent pipeline works end-to-end with the new task-focused prompts
Add GITHUB_STEP_SUMMARY output so iteration results are visible in the Actions UI
Streamline the Automated Status Summary to reduce clutter when using CLI agents
Clean up comment patterns to avoid a mix of old UI-agent and new CLI-agent comments

Context for Agent

Design Decisions & Constraints

1. Clean up comment patterns to avoid a mix of old UI-agent and new CLI-agent comments
The keepalive loop now:
|  | github-actions[bot] | NEW: CLI agent iteration tracking | ✅ Keep for CLI agents |
|  | agents-workflows-bot[bot] | State tracking | ⚠️ Multiple copies accumulate |
|  | stranske | OLD: Instruction comment | ❌ CLI agents dont need this |
The goal: For CLI agents (agent:* label), we should have exactly one updating comment () instead of accumulating 10+ comments per PR.
Requires PR #103 to be merged first
This round you MUST:
Review the Scope/Tasks/Acceptance below, identify the next incomplete task that requires code, implement it, then post a reply comment with the completed items using their exact original text.

Related Issues/PRs

References

https://github.com/stranske/Workflows/compare/main...codex/issue-123?expand=1

Blockers & Dependencies

After merging PR #103 (multi-agent routing infrastructure), we need to:
1. Mark a task checkbox complete ONLY after verifying the implementation works.

Tasks

Pipeline Validation

After PR chore(codex): bootstrap PR for issue #101 #103 merges, create a test PR with agent:codex label
Verify task appendix appears in Codex prompt (check workflow logs)
Verify Codex works on actual tasks (not random infrastructure work)
Verify keepalive comment updates with iteration progress

GITHUB_STEP_SUMMARY

Add step summary output to agents-keepalive-loop.yml after agent run
Include: iteration number, tasks completed, files changed, outcome
Ensure summary is visible in workflow run UI

Conditional Status Summary

Modify buildStatusBlock() in agents_pr_meta_update_body.js to accept agentType parameter
When agentType is set (CLI agent): hide workflow table, hide head SHA/required checks
Keep Scope/Tasks/Acceptance checkboxes for all cases
Pass agent type from workflow to the update_body job

Comment Pattern Cleanup

Acceptance criteria

CLI agent receives explicit tasks in prompt and works on them
Iteration results visible in Actions workflow run summary
PR body shows checkboxes but not workflow clutter when using CLI agents
UI Codex path (no agent label) continues to show full status summary
CLI agent PRs have ≤3 bot comments total (summary, one per iteration update) instead of 10+
State tracking is consolidated in the summary comment, not scattered

Dependencies

- Requires PR chore(codex): bootstrap PR for issue #101 #103 to be merged first

Head SHA: 9be18ab
Latest Runs: ❔ in progress — Gate
Required: gate: ❔ in progress

Workflow / Job	Result	Logs
Agents PR meta manager	❔ in progress	View run
CI Autofix Loop	✅ success	View run
Gate	❔ in progress	View run
Health 40 Sweep	✅ success	View run
Health 44 Gate Branch Protection	✅ success	View run
Health 45 Agents Guard	✅ success	View run
Health 50 Security Scan	✅ success	View run
Maint 52 Validate Workflows	✅ success	View run
PR 11 - Minimal invariant CI	✅ success	View run
Selftest CI	✅ success	View run

The _token_matches_keyword function was allowing single-character tokens to match keywords via prefix matching. For example: - 'd' (from 'Describe') matched 'defect' - 'a' could match 'add' This caused feature requests to get 'bug' label (0.91) because typical issue text contains short tokens that prefix-match bug keywords. Fix: Require token to be >= 4 chars before allowing prefix matching in either direction. Before: token='d', keyword='defect' → True (defect.startswith('d')) After: token='d', keyword='defect' → False (len('d') < 4)

github-actions · 2026-01-10T04:13:03Z

github-actions · 2026-01-10T04:13:15Z

Automated Status Summary

Head SHA: b413caf
Latest Runs: ⏳ pending — Gate
Required contexts: Gate / gate, Health 45 Agents Guard / Enforce agents workflow protections
Required: core tests (3.11): ⏳ pending, core tests (3.12): ⏳ pending, docker smoke: ⏳ pending, gate: ⏳ pending

Workflow / Job	Result	Logs
(no jobs reported)	⏳ pending	—

Coverage Overview

Coverage history entries: 1

Coverage Trend

Metric	Value
Current	92.21%
Baseline	85.00%
Delta	+7.21%
Minimum	70.00%
Status	✅ Pass

Top Coverage Hotspots (lowest coverage)

File	Coverage	Missing
`scripts/workflow_health_check.py`	62.6%	28
`scripts/classify_test_failures.py`	62.9%	37
`scripts/ledger_validate.py`	65.3%	63
`scripts/mypy_return_autofix.py`	82.6%	11
`scripts/ledger_migrate_base.py`	85.5%	13
`scripts/fix_cosmetic_aggregate.py`	92.3%	1
`scripts/coverage_history_append.py`	92.8%	2
`scripts/workflow_validator.py`	93.3%	4
`scripts/update_autofix_expectations.py`	93.9%	1
`scripts/pr_metrics_tracker.py`	95.7%	3
`scripts/generate_residual_trend.py`	96.6%	1
`scripts/build_autofix_pr_comment.py`	97.0%	2
`scripts/aggregate_agent_metrics.py`	97.2%	0
`scripts/fix_numpy_asserts.py`	98.1%	0
`scripts/sync_test_dependencies.py`	98.3%	1

Updated automatically; will refresh on subsequent CI/Docker completions.

Keepalive checklist

Scope

After merging PR #103 (multi-agent routing infrastructure), we need to:

Validate the CLI agent pipeline works end-to-end with the new task-focused prompts
Add GITHUB_STEP_SUMMARY output so iteration results are visible in the Actions UI
Streamline the Automated Status Summary to reduce clutter when using CLI agents
Clean up comment patterns to avoid a mix of old UI-agent and new CLI-agent comments

Context for Agent

Design Decisions & Constraints

1. Clean up comment patterns to avoid a mix of old UI-agent and new CLI-agent comments
The keepalive loop now:
|  | github-actions[bot] | NEW: CLI agent iteration tracking | ✅ Keep for CLI agents |
|  | agents-workflows-bot[bot] | State tracking | ⚠️ Multiple copies accumulate |
|  | stranske | OLD: Instruction comment | ❌ CLI agents dont need this |
The goal: For CLI agents (agent:* label), we should have exactly one updating comment () instead of accumulating 10+ comments per PR.
Requires PR #103 to be merged first
This round you MUST:
Review the Scope/Tasks/Acceptance below, identify the next incomplete task that requires code, implement it, then post a reply comment with the completed items using their exact original text.

Related Issues/PRs

References

https://github.com/stranske/Workflows/compare/main...codex/issue-123?expand=1

Blockers & Dependencies

After merging PR #103 (multi-agent routing infrastructure), we need to:
1. Mark a task checkbox complete ONLY after verifying the implementation works.

Tasks

Pipeline Validation

After PR chore(codex): bootstrap PR for issue #101 #103 merges, create a test PR with agent:codex label
Verify task appendix appears in Codex prompt (check workflow logs)
Verify Codex works on actual tasks (not random infrastructure work)
Verify keepalive comment updates with iteration progress

GITHUB_STEP_SUMMARY

Add step summary output to agents-keepalive-loop.yml after agent run
Include: iteration number, tasks completed, files changed, outcome
Ensure summary is visible in workflow run UI

Conditional Status Summary

Modify buildStatusBlock() in agents_pr_meta_update_body.js to accept agentType parameter
When agentType is set (CLI agent): hide workflow table, hide head SHA/required checks
Keep Scope/Tasks/Acceptance checkboxes for all cases
Pass agent type from workflow to the update_body job

Comment Pattern Cleanup

Acceptance criteria

CLI agent receives explicit tasks in prompt and works on them
Iteration results visible in Actions workflow run summary
PR body shows checkboxes but not workflow clutter when using CLI agents
UI Codex path (no agent label) continues to show full status summary
CLI agent PRs have ≤3 bot comments total (summary, one per iteration update) instead of 10+
State tracking is consolidated in the summary comment, not scattered

Dependencies

- Requires PR chore(codex): bootstrap PR for issue #101 #103 to be merged first

github-actions · 2026-01-10T04:13:37Z

🤖 Keepalive Loop Status

PR #735 | Agent: Codex | Iteration 0/5

Current State

Metric	Value
Iteration progress	[----------] 0/5
Action	wait (missing-agent-label)
Disposition	skipped (transient)
Gate	success
Tasks	0/28 complete
Keepalive	❌ disabled
Autofix	❌ disabled

🔍 Failure Classification

Copilot

Pull request overview

This PR fixes a bug where short tokens (< 4 characters) from issue text were incorrectly matching keywords via prefix matching, causing issues to receive incorrect labels. The fix adds a length requirement (>= 4 chars) to prevent short tokens like "d" (from "Describe") from matching keywords like "defect".

Changes:

Modified _token_matches_keyword to require tokens be at least 4 characters long before allowing prefix matching
Added inline comments explaining the rationale for the 4-character minimum

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-01-10T04:14:29Z

scripts/langchain/label_matcher.py

 def _token_matches_keyword(token: str, keyword: str) -> bool:
    if token == keyword:
        return True
+    # Only allow prefix matching for tokens >= 4 chars to avoid false positives
+    # from short tokens like "d" matching "defect" or "a" matching "add"
    if len(token) >= 4 and token.startswith(keyword):
        return True
-    return bool(len(keyword) >= 4 and keyword.startswith(token))
+    # Check if keyword starts with token (both must be >= 4 chars)
+    return len(token) >= 4 and len(keyword) >= 4 and keyword.startswith(token)


The fix correctly prevents short tokens from matching keywords via prefix. However, there's no test coverage for this specific scenario. Consider adding a test case that verifies tokens shorter than 4 characters (e.g., "d" from "Describe") do not match keywords like "defect", which was the root cause of the bug described in the PR.

…results - Document PRs #733, #735 (deep label matcher fixes) - Record validation test results (issues #265-267) - Mark auto-label validation as complete - Key win: 2FA feature request now gets only 'enhancement' (was 3 labels)

* docs: Update SHORT_TERM_PLAN with label matcher fixes and validation results - Document PRs #733, #735 (deep label matcher fixes) - Record validation test results (issues #265-267) - Mark auto-label validation as complete - Key win: 2FA feature request now gets only 'enhancement' (was 3 labels) * docs: Add LONG_TERM_PLAN for Phases 4-5 - Phase 4: Auto-pilot workflow, user guide, conflict resolution - Phase 5: Learning from feedback, multi-model arbitration - Infrastructure: Performance, monitoring, cost optimization - Risk assessment and success metrics - Prioritized 8-week roadmap

* docs: Update SHORT_TERM_PLAN with label matcher fixes and validation results - Document PRs #733, #735 (deep label matcher fixes) - Record validation test results (issues #265-267) - Mark auto-label validation as complete - Key win: 2FA feature request now gets only 'enhancement' (was 3 labels) * docs: Add LONG_TERM_PLAN for Phases 4-5 - Phase 4: Auto-pilot workflow, user guide, conflict resolution - Phase 5: Learning from feedback, multi-model arbitration - Infrastructure: Performance, monitoring, cost optimization - Risk assessment and success metrics - Prioritized 8-week roadmap * Expand cleanup_labels.py classifications - Add autofix:*, integration-*, agents:keepalive-nudge to functional - Add common component labels (app, engine, ui, backend, cli) - Add tech labels (javascript, python, github:actions) - Add domain labels (metrics, modeling, schema, etc.) - Reduces idiosyncratic labels from 150+ to 24 - Remaining 24 are legitimate project-specific labels

Copilot AI review requested due to automatic review settings January 10, 2026 04:11

stranske temporarily deployed to agent-standard January 10, 2026 04:11 — with GitHub Actions Inactive

github-actions bot added the autofix Opt-in automated formatting & lint remediation label Jan 10, 2026

Copilot started reviewing on behalf of stranske January 10, 2026 04:12 View session

Copilot AI reviewed Jan 10, 2026

View reviewed changes

stranske enabled auto-merge (squash) January 10, 2026 04:15

Merge branch 'main' into fix/keyword-matcher-short-token-prefix

9be18ab

stranske temporarily deployed to agent-standard January 10, 2026 04:18 — with GitHub Actions Inactive

stranske merged commit aa61bd1 into main Jan 10, 2026
99 checks passed

stranske deleted the fix/keyword-matcher-short-token-prefix branch January 10, 2026 04:20

stranske mentioned this pull request Jan 10, 2026

docs: Update SHORT_TERM_PLAN with label matcher progress #737

Merged

43 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: Prevent short tokens from matching keywords via prefix#735

fix: Prevent short tokens from matching keywords via prefix#735
stranske merged 2 commits intomainfrom
fix/keyword-matcher-short-token-prefix

stranske commented Jan 10, 2026 •

edited by agents-workflows-bot bot

Loading

Uh oh!

github-actions bot commented Jan 10, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Jan 10, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Jan 10, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Jan 10, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

stranske commented Jan 10, 2026 • edited by agents-workflows-bot bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Automated Status Summary

Scope

Context for Agent

Design Decisions & Constraints

Related Issues/PRs

References

Blockers & Dependencies

Tasks

Pipeline Validation

GITHUB_STEP_SUMMARY

Conditional Status Summary

Comment Pattern Cleanup

Acceptance criteria

Dependencies

Uh oh!

github-actions bot commented Jan 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jan 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Automated Status Summary

Coverage Overview

Coverage Trend

Top Coverage Hotspots (lowest coverage)

Keepalive checklist

Scope

Context for Agent

Design Decisions & Constraints

Related Issues/PRs

References

Blockers & Dependencies

Tasks

Pipeline Validation

GITHUB_STEP_SUMMARY

Conditional Status Summary

Comment Pattern Cleanup

Acceptance criteria

Dependencies

Uh oh!

github-actions bot commented Jan 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🤖 Keepalive Loop Status

Current State

🔍 Failure Classification

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Jan 10, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

stranske commented Jan 10, 2026 •

edited by agents-workflows-bot bot

Loading

github-actions bot commented Jan 10, 2026 •

edited

Loading

github-actions bot commented Jan 10, 2026 •

edited

Loading

github-actions bot commented Jan 10, 2026 •

edited

Loading