feat: Enable LLM for agents:apply-suggestions by default by stranske · Pull Request #639 · stranske/Workflows

stranske · 2026-01-07T12:25:21Z

Source: Issue #184

Automated Status Summary

Scope

Address unmet acceptance criteria from PR #183.

Original scope:

Scope section missing from source issue.

Context for Agent

Related Issues/PRs

References

Tasks

Tasks section missing from source issue.

Acceptance criteria

Acceptance criteria section missing from source issue.

Head SHA: 3bf71f7
Latest Runs: ✅ success — Gate
Required: gate: ✅ success

Workflow / Job	Result	Logs
Agents PR meta manager	❔ in progress	View run
Auto-label Dependabot PRs	⏭️ skipped	View run
CI Autofix Loop	✅ success	View run
Copilot code review	✅ success	View run
Gate	✅ success	View run
Health 40 Sweep	✅ success	View run
Health 44 Gate Branch Protection	✅ success	View run
Health 45 Agents Guard	✅ success	View run
Health 50 Security Scan	✅ success	View run
Maint 52 Validate Workflows	✅ success	View run
PR 11 - Minimal invariant CI	✅ success	View run
Selftest CI	✅ success	View run
Validate Sync Manifest	✅ success	View run

When a user adds agents:apply-suggestions label, they explicitly want the LLM suggestions applied. The previous use_llm=False only did basic reformatting without populating sections with analyzed content. Changes: - apply_suggestions now uses use_llm=True by default - Will populate Tasks with suggested splits from analysis - Will extract Why/Scope/Non-Goals from context - Will add objective acceptance criteria from suggestions Expected impact: Quality score 6/10 → 8.5/10 Rationale: Label addition is explicit user consent to apply LLM-generated suggestions. No reason to use non-LLM mode when user requests application.

github-actions · 2026-01-07T12:27:04Z

Automated Status Summary

Head SHA: d002e2c
Latest Runs: ⏳ pending — Gate
Required contexts: Gate / gate, Health 45 Agents Guard / Enforce agents workflow protections
Required: core tests (3.11): ⏳ pending, core tests (3.12): ⏳ pending, docker smoke: ⏳ pending, gate: ⏳ pending

Workflow / Job	Result	Logs
(no jobs reported)	⏳ pending	—

Coverage Overview

Coverage history entries: 1

Coverage Trend

Metric	Value
Current	92.21%
Baseline	85.00%
Delta	+7.21%
Minimum	70.00%
Status	✅ Pass

Top Coverage Hotspots (lowest coverage)

File	Coverage	Missing
`scripts/workflow_health_check.py`	62.6%	28
`scripts/classify_test_failures.py`	62.9%	37
`scripts/ledger_validate.py`	65.3%	63
`scripts/mypy_return_autofix.py`	82.6%	11
`scripts/ledger_migrate_base.py`	85.5%	13
`scripts/fix_cosmetic_aggregate.py`	92.3%	1
`scripts/coverage_history_append.py`	92.8%	2
`scripts/workflow_validator.py`	93.3%	4
`scripts/update_autofix_expectations.py`	93.9%	1
`scripts/pr_metrics_tracker.py`	95.7%	3
`scripts/generate_residual_trend.py`	96.6%	1
`scripts/build_autofix_pr_comment.py`	97.0%	2
`scripts/aggregate_agent_metrics.py`	97.2%	0
`scripts/fix_numpy_asserts.py`	98.1%	0
`scripts/sync_test_dependencies.py`	98.3%	1

Updated automatically; will refresh on subsequent CI/Docker completions.

Keepalive checklist

Scope

No scope information available

Tasks

No tasks defined

Acceptance criteria

No acceptance criteria defined

github-actions · 2026-01-07T12:27:30Z

🤖 Keepalive Loop Status

PR #639 | Agent: Codex | Iteration 0/5

Current State

Metric	Value
Iteration progress	[----------] 0/5
Action	wait (missing-agent-label)
Disposition	skipped (transient)
Gate	success
Tasks	0/2 complete
Keepalive	❌ disabled
Autofix	❌ disabled

🔍 Failure Classification

Copilot

Pull request overview

This PR enables LLM-powered content population for the agents:apply-suggestions workflow phase by removing an explicit override that was preventing the apply_suggestions function from using its default LLM behavior. The change aims to improve the quality of applied suggestions from 6/10 to ~8.5/10 by intelligently populating issue template sections rather than just creating empty structure.

Key Changes:

Removed use_llm=False override in the workflow, allowing apply_suggestions to use its default use_llm=True behavior
Updated documentation to reflect testing results, expected quality improvements, and deployment status
Added detailed test results documentation showing substantive quality assessments

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File	Description
templates/consumer-repo/.github/workflows/agents-issue-optimizer.yml	Removed explicit `use_llm=False` override to enable intelligent section population with LLM
docs/plans/langchain-post-code-rollout.md	Documented testing results, quality improvements, and deployment status for the LLM-enabled apply-suggestions workflow

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-01-07T12:28:47Z

+   - **Gap:** Doesn't populate missing sections; could add task priority ordering
+   - **Workflow:** `agents-issue-optimizer.yml` Phase 1 (analyze)
+
+2. **issue_formatter.py** (apply_suggestions) - Intelligent content extraction


The documentation incorrectly attributes the apply_suggestions function to issue_formatter.py. The apply_suggestions function is actually defined in issue_optimizer.py (line 677). This section should reference issue_optimizer.py instead of issue_formatter.py.

Suggested change

2. **issue_formatter.py** (apply_suggestions) - Intelligent content extraction

2. **issue_optimizer.py** (`apply_suggestions`) - Intelligent content extraction

Copilot · 2026-01-07T12:28:48Z

+   - **Test:** Manager-Database #184 (same issue)
+   - **Quality Score: 6/10 (with use_llm=False)** → **Expected 8.5/10 (with use_llm=True)**
+   - **Strengths:** Consistent structure; preserves original content
+   - **Change:** Now uses `use_llm=True` by default - will populate sections with analyzed content


The statement "Now uses use_llm=True by default" is misleading. The apply_suggestions function in issue_optimizer.py already has use_llm=True as its default parameter value (line 678). This change is actually removing the explicit override of use_llm=False in the workflow, allowing it to use the function's default behavior. Consider rephrasing to "Now uses function default (use_llm=True)" or "Removed explicit override, now uses default LLM behavior".

Suggested change

- **Change:** Now uses `use_llm=True` by default - will populate sections with analyzed content

- **Change:** Removed explicit `use_llm=False` override; now uses function default (`use_llm=True`) - will populate sections with analyzed content

Copilot · 2026-01-07T12:28:48Z

+4. ~~**Live test `agents:apply-suggestions`**~~ ✅ Tested on Manager-Database #184 - Quality: 6/10
+
+### High Priority Enhancements
+1. ~~**Enable LLM for apply_suggestions**~~ ✅ Changed `use_llm=False` to `use_llm=True` in workflow


The statement "Now uses use_llm=True by default" is misleading in this context. The apply_suggestions function already defaults to use_llm=True in its signature. This change is removing an explicit override (use_llm=False) that was preventing the function from using its default LLM behavior. Consider rephrasing for clarity.

Suggested change

1. ~~**Enable LLM for apply_suggestions**~~ ✅ Changed `use_llm=False` to `use_llm=True` in workflow

1. ~~**Allow apply_suggestions to use default LLM behavior**~~ ✅ Removed explicit `use_llm=False` override so workflow uses `apply_suggestions`'s default `use_llm=True`

Copilot AI review requested due to automatic review settings January 7, 2026 12:25

stranske temporarily deployed to agent-standard January 7, 2026 12:25 — with GitHub Actions Inactive

Copilot started reviewing on behalf of stranske January 7, 2026 12:25 View session

Copilot AI reviewed Jan 7, 2026

View reviewed changes

stranske merged commit 8fb6785 into main Jan 7, 2026
261 checks passed

stranske deleted the feat/enable-llm-apply-suggestions branch January 7, 2026 12:32

stranske mentioned this pull request Jan 7, 2026

fix: Add API keys to Phase 2 (apply-suggestions) step #640

Merged

21 tasks

github-actions Bot mentioned this pull request Apr 12, 2026

deps(deps-dev): update wheel requirement from >=0.43.0 to >=0.46.3 #1743

Merged

21 tasks

github-actions Bot mentioned this pull request Apr 26, 2026

deps(deps-dev): update wheel requirement from >=0.46.3 to >=0.47.0 #1890

Closed

21 tasks

stranske mentioned this pull request Jun 23, 2026

[design-system] Fleet adoption tracker — wire the synced kit into each app (theme + helpers) #2521

Closed

8 tasks

Navigation Menu

feat: Enable LLM for agents:apply-suggestions by default - #639

feat: Enable LLM for agents:apply-suggestions by default#639
stranske merged 1 commit into
mainfrom
feat/enable-llm-apply-suggestions

stranske commented Jan 7, 2026 •

edited by agents-workflows-bot Bot

Loading

Uh oh!

github-actions Bot commented Jan 7, 2026

Uh oh!

github-actions Bot commented Jan 7, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Jan 7, 2026

Uh oh!

Copilot AI Jan 7, 2026

Uh oh!

Copilot AI Jan 7, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	2. issue_formatter.py (apply_suggestions) - Intelligent content extraction
	2. issue_optimizer.py (`apply_suggestions`) - Intelligent content extraction

	- Change: Now uses `use_llm=True` by default - will populate sections with analyzed content
	- Change: Removed explicit `use_llm=False` override; now uses function default (`use_llm=True`) - will populate sections with analyzed content

	1. ~~Enable LLM for apply_suggestions~~ ✅ Changed `use_llm=False` to `use_llm=True` in workflow
	1. ~~Allow apply_suggestions to use default LLM behavior~~ ✅ Removed explicit `use_llm=False` override so workflow uses `apply_suggestions`'s default `use_llm=True`

Conversation

stranske commented Jan 7, 2026 • edited by agents-workflows-bot Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Automated Status Summary

Scope

Context for Agent

Related Issues/PRs

References

Tasks

Acceptance criteria

Uh oh!

github-actions Bot commented Jan 7, 2026

Automated Status Summary

Coverage Overview

Coverage Trend

Top Coverage Hotspots (lowest coverage)

Keepalive checklist

Scope

Tasks

Acceptance criteria

Uh oh!

github-actions Bot commented Jan 7, 2026

🤖 Keepalive Loop Status

Current State

🔍 Failure Classification

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Jan 7, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 7, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 7, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

stranske commented Jan 7, 2026 •

edited by agents-workflows-bot Bot

Loading