Skip to content

chore: sync workflow templates#272

Closed
stranske wants to merge 1 commit intomainfrom
sync/workflows-63e90e8d6749
Closed

chore: sync workflow templates#272
stranske wants to merge 1 commit intomainfrom
sync/workflows-63e90e8d6749

Conversation

@stranske
Copy link
Copy Markdown
Owner

Sync Summary

Files Updated

  • agents-verifier.yml: Verifier - validates agent work meets acceptance criteria
  • agents-auto-label.yml: Auto-label - suggests/applies labels based on semantic matching (Phase 5A)
  • agents-dedup.yml: Duplicate detection - flags similar open issues (Phase 3C)
  • agents-auto-pilot.yml: Auto-pilot - end-to-end automation orchestrator (format → optimize → agent → verify)
  • parse_chatgpt_topics.py: Parses ChatGPT conversation into issue topics
  • error_classifier.js: Classifies CI errors for agent handling
  • conflict_detector.js: Conflict detector - identifies merge conflicts for targeted resolution
  • issue_formatter.py: Issue formatter - converts raw text to AGENT_ISSUE_TEMPLATE format
  • llm_provider.py: LLM provider configuration - GitHub Models and OpenAI client setup

Files Skipped

  • pr-00-gate.yml: File exists and sync_mode is create_only
  • ci.yml: File exists and sync_mode is create_only
  • dependabot.yml: File exists and sync_mode is create_only

Review Checklist

  • CI passes with updated workflows
  • No repo-specific customizations were overwritten

Source: stranske/Workflows
Manifest: .github/sync-manifest.yml

Automated sync from stranske/Workflows
Template hash: 63e90e8d6749

Changes synced from sync-manifest.yml
Copilot AI review requested due to automatic review settings January 10, 2026 18:25
@stranske stranske added sync Automated sync from Workflows automated Automated sync from Workflows labels Jan 10, 2026
@github-actions
Copy link
Copy Markdown
Contributor

⚠️ Action Required: Unable to determine source issue for PR #272. The PR title, branch name, or body must contain the issue number (e.g. #123, branch: issue-123, or the hidden marker ).

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Jan 10, 2026

🤖 Keepalive Loop Status

PR #272 | Agent: Codex | Iteration 0/5

Current State

Metric Value
Iteration progress [----------] 0/5
Action wait (missing-agent-label)
Disposition skipped (transient)
Gate success
Tasks 0/12 complete
Keepalive ❌ disabled
Autofix ❌ disabled

🔍 Failure Classification

| Error type | infrastructure |
| Error category | resource |
| Suggested recovery | Confirm the referenced resource exists (repo, PR, branch, workflow, or file). |

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR syncs workflow templates and supporting scripts from the stranske/Workflows repository, updating several agent automation workflows and related tooling. The changes introduce a new comprehensive auto-pilot workflow and update existing workflows for duplicate detection, auto-labeling, and verification.

Changes:

  • Upgraded default LLM model from gpt-4o-mini to gpt-4o for better evaluation quality
  • Added new agents-auto-pilot.yml workflow for end-to-end issue automation
  • Refactored multiple workflows to checkout locally instead of external Workflows repo
  • Enhanced duplicate detection with stricter similarity thresholds and title-based filtering
  • Updated auto-label logic to apply only the best matching label to prevent over-labeling

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 11 comments.

Show a summary per file
File Description
tools/llm_provider.py Changed default model to gpt-4o with explanation
scripts/langchain/issue_formatter.py Added task decomposition functionality with new helper functions
.github/workflows/agents-verifier.yml Updated default models and descriptions for evaluation
.github/workflows/agents-dedup.yml Changed to local checkout, raised similarity threshold, added title-based filtering
.github/workflows/agents-auto-pilot.yml New comprehensive workflow for end-to-end issue automation
.github/workflows/agents-auto-label.yml Changed to local checkout, refined auto-apply logic
.github/scripts/parse_chatgpt_topics.py Added admin/dependency keywords to implementation_notes section
.github/scripts/error_classifier.js Added debug logging for error classification
.github/scripts/conflict_detector.js Added filtering for automated comments in conflict detection

Comment on lines +140 to +143
# Require at least 40% of words to overlap for a duplicate flag
max_words = max(len(title_words_new), len(title_words_match), 1)
overlap_ratio = len(shared_words) / max_words
if m.score >= 0.95 or overlap_ratio >= 0.4:
Copy link

Copilot AI Jan 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The condition on line 143 checks for very high similarity (score >= 0.95) OR title overlap >= 0.4. However, this creates an inconsistency: matches with score >= 0.95 bypass the title overlap requirement entirely. This means issues with 95%+ content similarity but completely different titles (overlap_ratio = 0) would still be flagged as duplicates. Consider if this is the intended behavior or if high-similarity matches should also require some minimum title overlap.

Suggested change
# Require at least 40% of words to overlap for a duplicate flag
max_words = max(len(title_words_new), len(title_words_match), 1)
overlap_ratio = len(shared_words) / max_words
if m.score >= 0.95 or overlap_ratio >= 0.4:
# Require at least 40% of words to overlap for a duplicate flag,
# and ensure even very high-similarity matches have some title overlap.
max_words = max(len(title_words_new), len(title_words_match), 1)
overlap_ratio = len(shared_words) / max_words
if (m.score >= 0.95 and overlap_ratio > 0.0) or overlap_ratio >= 0.4:

Copilot uses AI. Check for mistakes.

- name: Set up Python
uses: actions/setup-python@v5
with:
Copy link

Copilot AI Jan 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Python version was downgraded from 3.12 to 3.11. If this change is intentional for compatibility reasons, it should be documented why. If the codebase or dependencies require Python 3.12 features, this could cause runtime errors.

Suggested change
with:
with:
# Use Python 3.11 intentionally: some dependencies in this workflow (including optional extras)
# are not yet fully compatible with Python 3.12 in our CI environment.

Copilot uses AI. Check for mistakes.
repository: stranske/Workflows
path: workflows-repo
- name: Checkout repository
uses: actions/checkout@v4
Copy link

Copilot AI Jan 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The checkout action was downgraded from v6 to v4. This is an unusual change as typically actions are upgraded to newer versions. If this is intentional for compatibility, it should be documented. Otherwise, consider keeping v6 which may have security or performance improvements.

Suggested change
uses: actions/checkout@v4
uses: actions/checkout@v6

Copilot uses AI. Check for mistakes.
if not tasks:
return formatted

from scripts.langchain import task_decomposer
Copy link

Copilot AI Jan 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The function attempts to import 'task_decomposer' from 'scripts.langchain', but this module doesn't exist in the repository. This will cause an ImportError at runtime when _apply_task_decomposition is called, breaking the issue formatting functionality.

Suggested change
from scripts.langchain import task_decomposer
try:
from scripts.langchain import task_decomposer
except ImportError:
# Task decomposition is optional; if the module is not available,
# fall back to returning the original formatted text.
return formatted

Copilot uses AI. Check for mistakes.
auto_apply = [best_match]
# Move other high-confidence matches to suggestions
for m in matches[1:]:
if m.score >= auto_threshold and m not in suggestions:
Copy link

Copilot AI Jan 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The condition 'm not in suggestions' on line 141 is redundant. The 'suggestions' list (line 132) only contains matches where score < auto_threshold, so any match with score >= auto_threshold will never be in 'suggestions'. The check 'm not in suggestions' will always evaluate to True when 'm.score >= auto_threshold' is True, making the second condition unnecessary.

Suggested change
if m.score >= auto_threshold and m not in suggestions:
if m.score >= auto_threshold:

Copilot uses AI. Check for mistakes.
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
PYTHONPATH: ${{ github.workspace }}/workflows-repo
PYTHONPATH: ${{ github.workspace }}
Copy link

Copilot AI Jan 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PYTHONPATH environment variable is set but unused. In line 38 and 83, 'cd workflows-repo' commands were removed but the PYTHONPATH was not removed from line 79. Since the checkout is now local (line 29), this environment variable is unnecessary.

Suggested change
PYTHONPATH: ${{ github.workspace }}

Copilot uses AI. Check for mistakes.

- name: Set up Python
uses: actions/setup-python@v5
with:
Copy link

Copilot AI Jan 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Python version was downgraded from 3.12 to 3.11. If this change is intentional for compatibility reasons, it should be documented why. If the codebase or dependencies require Python 3.12 features, this could cause runtime errors.

Suggested change
with:
with:
# Intentionally pinned to Python 3.11 for current dependency/runtime compatibility.
# TODO: Re-evaluate and upgrade to 3.12+ once all dependencies fully support it.

Copilot uses AI. Check for mistakes.
repository: stranske/Workflows
path: workflows-repo
- name: Checkout repository
uses: actions/checkout@v4
Copy link

Copilot AI Jan 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The checkout action was downgraded from v6 to v4. This is an unusual change as typically actions are upgraded to newer versions. If this is intentional for compatibility, it should be documented. Otherwise, consider keeping v6 which may have security or performance improvements.

Suggested change
uses: actions/checkout@v4
uses: actions/checkout@v6

Copilot uses AI. Check for mistakes.
if not suggestions:
return formatted

from scripts.langchain import issue_optimizer
Copy link

Copilot AI Jan 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The function attempts to import 'issue_optimizer' from 'scripts.langchain' and call '_apply_task_decomposition', but this module doesn't exist in the repository. This will cause an ImportError at runtime, breaking the issue formatting functionality.

Copilot uses AI. Check for mistakes.
Comment on lines 157 to 165
if (mode === 'compare') {
core.setOutput('model', 'gpt-4o'); // GitHub Models - current flagship
core.setOutput('model2', 'gpt-5.2'); // OpenAI - GPT-5.2
// gpt-5.2 (OpenAI) + Mistral-large (GitHub Models) for cross-provider comparison
core.setOutput('model', 'gpt-5.2');
core.setOutput('model2', 'Mistral-large-2407');
} else {
core.setOutput('model', ''); // Use default
core.setOutput('model', ''); // Use default (gpt-4o)
core.setOutput('model2', '');
}
core.setOutput('provider', ''); // Use default
Copy link

Copilot AI Jan 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When compare mode is triggered via label, the model is set to 'gpt-5.2' (an OpenAI model) on line 159, but the provider is set to empty string on line 165, which will use the default 'github-models' provider (line 62). This mismatch will cause the workflow to fail as gpt-5.2 is not available on GitHub Models. The provider should be set to 'openai' when using gpt-5.2, or the logic should be updated to handle cross-provider comparison correctly.

Copilot uses AI. Check for mistakes.
@stranske stranske closed this Jan 11, 2026
@stranske stranske deleted the sync/workflows-63e90e8d6749 branch January 11, 2026 04:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

automated Automated sync from Workflows sync Automated sync from Workflows

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants