fix: agents-issue-optimizer checkout Workflows repo for scripts

docs/MODEL_MANAGEMENT.md

@@ -0,0 +1,124 @@
+# Model Management Guide
+
+## Keeping Model Lists Current
+
+The workflows support various LLM models from different providers. This guide explains how to keep the model lists up-to-date.
+
+## Quick Update Process
+
+```bash
+# Set your OpenAI API key
+export OPENAI_API_KEY="sk-..."
+
+# Run the update script
+./scripts/update_model_list.sh
+```
+
+This will fetch the current list of available models from OpenAI's API and display them categorized by series.
+
+## Manual API Query
+
+You can also query the API directly:
+
+```bash
+curl https://api.openai.com/v1/models \
+  -H "Authorization: Bearer $OPENAI_API_KEY" | jq '.data[].id' | sort
+```
+
+## Model Selection Guidelines
+
+### High Quality Models (for critical evaluation)
+- **o1**: Latest reasoning model from OpenAI
+- **gpt-4o**: Current flagship multimodal model
+- **gpt-4-turbo**: Fast, capable GPT-4 variant
+
+### Efficient Models (for rapid iteration)
+- **o3-mini**: Small reasoning model
+- **o1-mini**: Smaller reasoning model  
+- **gpt-4o-mini**: Fast, cost-effective GPT-4o
+- **gpt-3.5-turbo**: Budget-friendly option
+
+### GitHub Models vs OpenAI
+
+**GitHub Models API** (`https://models.inference.ai.azure.com`):
+- Uses GITHUB_TOKEN (no additional API key needed)
+- Provides access to various models including Meta-Llama
+- May have different model availability than OpenAI
+
+**OpenAI API** (`https://api.openai.com`):
+- Requires OPENAI_API_KEY
+- Direct access to all OpenAI models
+- Generally has latest models first
+
+## Updating Workflow Configurations
+
+After checking current models, update these files:
+
+### 1. Consumer Template Workflow
+**File**: `templates/consumer-repo/.github/workflows/agents-verifier.yml`
+
+Update the model descriptions:
+```yaml
+model:
+  description: >-
+    GitHub Models: [list here] | OpenAI: [list here]
+```
+
+Update default values for compare mode (lines ~155-156):
+```javascript
+core.setOutput('model', 'gpt-4o');  // High quality model
+core.setOutput('model2', 'o1');     // Different high quality model
+```
+
+### 2. Reusable Workflow (if needed)
+**File**: `.github/workflows/reusable-agents-verifier.yml`
+
+Update input descriptions if model lists change significantly.
+
+### 3. Sync Changes
+After updating templates:
+```bash
+git add templates/consumer-repo/.github/workflows/agents-verifier.yml
+git commit -m "Update model lists to current OpenAI offerings"
+git push
+```
+
+The sync workflow will automatically propagate changes to consumer repos.
+
+## Testing New Models
+
+Before setting as defaults, test new models:
+
+```bash
+# Test with pr_verifier locally
+python scripts/langchain/pr_verifier.py \
+  --repo owner/repo \
+  --pr 123 \
+  --mode evaluate \
+  --model "new-model-name" \
+  --json
+```
+
+Or trigger workflow manually with `workflow_dispatch` and specify the model.
+
+## Model Naming Notes
+
+- **Base names** (e.g., `gpt-4o`) point to latest stable version
+- **Dated versions** (e.g., `gpt-4o-2024-08-06`) are frozen snapshots
+- Use base names in workflows for automatic updates
+- Use dated versions when reproducibility is critical
+
+## Resources
+
+- **OpenAI Models Docs**: https://platform.openai.com/docs/models
+- **OpenAI API Reference**: https://platform.openai.com/docs/api-reference/models
+- **GitHub Models**: GitHub.com → Settings → GitHub Models (for available models)
+
+## Scheduled Updates
+
+Recommend checking for model updates:
+- Monthly for production stability
+- Weekly if using cutting-edge features
+- After OpenAI announces new releases
+
+Run `./scripts/update_model_list.sh` to check current availability.

docs/plans/langchain-post-code-rollout.md

@@ -1,8 +1,8 @@
 # LangChain Post-Code Production Capabilities - Evaluation & Rollout Plan
 
-> **Date:** January 6, 2026  
-> **Status:** Phase 1 Complete, Phase 2 Ready  
-> **Last Validation:** 2026-01-06  
+> **Date:** January 7, 2026  
+> **Status:** Phase 1 & 2 Deployed - Ready for Live Testing  
+> **Last Validation:** 2026-01-07  
 
 ---
 
@@ -50,8 +50,11 @@
 1. **Topic Splitting** - `topic_splitter.py` is called from `agents-63-issue-intake.yml` when `apply_langchain_formatting=true`. Tested and working (created 5 issues from Issues.txt).
 
 2. **PR Verifier Script** - `pr_verifier.py` has:
-   - Single-provider evaluation mode
-   - Multi-provider comparison mode
+   - Single-provider evaluation mode (✅ Tested on PR #625, #270)
+   - Multi-provider comparison mode (✅ Tested on PR #302)
+   - Model selection: GitHub Models and OpenAI (PR #626, #629 merged)
+   - Model2 parameter for comparing different models (PR #629 merged)
+   - GPT-5.2 support (PR #633 pending)
    - Follow-up issue creation on CONCERNS/FAIL
    - JSON output format
    - Tests: 4 test files with good coverage
@@ -64,20 +67,23 @@
 
 ### ⚠️ Implemented but Not Synced/Active in Consumer Repos
 
-1. **Verifier Labels** - Only 3 of 7 consumer repos have `verify:checkbox`, `verify:evaluate`, `verify:compare`:
+1. **Verifier Labels** - All 7 consumer repos have `verify:checkbox`, `verify:evaluate`, `verify:compare`:
    - ✅ Manager-Database
    - ✅ Template  
    - ✅ trip-planner
-   - ❌ Travel-Plan-Permission
-   - ❌ Portable-Alpha-Extension-Model
-   - ❌ Trend_Model_Project
-   - ❌ Collab-Admin
+   - ✅ Travel-Plan-Permission
+   - ✅ Portable-Alpha-Extension-Model
+   - ✅ Trend_Model_Project
+   - ⚠️ Collab-Admin (sync PR #104 pending - has failing gate check)
 
-2. **Format Labels** - Only 3 of 7 consumer repos have `agents:format`, `agents:formatted`:
+2. **Format Labels** - All 7 consumer repos have `agents:format`, `agents:formatted`, `agents:optimize`, `agents:apply-suggestions`:
    - ✅ Manager-Database
    - ✅ Template
    - ✅ trip-planner
-   - ❌ Others
+   - ✅ Travel-Plan-Permission
+   - ✅ Portable-Alpha-Extension-Model
+   - ✅ Trend_Model_Project
+   - ⚠️ Collab-Admin (sync PR #104 pending)
 
 3. **Updated .gitignore** - Consumer repos have old partial version, missing new entries for:
    - `verifier-diff-summary.md`
@@ -127,46 +133,52 @@
 - [x] Verify `reusable-agents-verifier.yml` is correct (done)
 - [x] Verify `agents-verifier.yml` template calls reusable correctly (done)
 - [x] `.gitignore` template has all workflow status entries (not synced by design)
+- [x] Add model selection and provider choice (PR #626, #629 merged)
+- [x] Add scripts/update_model_list.sh for model management (PR #633)
+- [x] Add docs/MODEL_MANAGEMENT.md for model update process (PR #633)
 - [x] Commit any fixes to main
 
 **Step 1B: Deploy to Consumer Repos**
-1. ✅ All consumer repos have verifier labels (`verify:checkbox`, `verify:evaluate`, `verify:compare`)
-2. ✅ Sync workflow runs automatically on template changes (last run: ~3h ago)
-3. ✅ Sync PRs merged (no open sync PRs pending)
-4. Test on Manager-Database (pilot):
+1. ✅ All consumer repos have verifier labels (6/7 active, Collab-Admin pending)
+2. ✅ Sync workflow runs automatically on template changes
+3. ✅ Sync PRs merged (except Collab-Admin #104)
+4. ⏳ Test on Manager-Database (pilot):
    - Find a recently merged agent PR
    - Add `verify:evaluate` label  
    - Verify workflow runs and posts evaluation comment
-   - **Note:** Requires repo write access to add labels
+   - **Status:** Ready for testing - requires repo write access
 
 **Validation Criteria:**
-- [x] Verifier labels exist in all 7 consumer repos (verified via `--check`)
-- [x] `agents-verifier.yml` deployed to consumer repos
-- [ ] Live test: Workflow runs without errors
-- [ ] Live test: LLM evaluation produces scores and verdict
-- [ ] Live test: Comment posted on PR with evaluation results
-- [ ] Live test: Follow-up issue created if verdict is CONCERNS/FAIL
+- [x] Verifier labels exist in all 7 consumer repos (6 active, 1 pending)
+- [x] `agents-verifier.yml` deployed to consumer repos (6/7)
+- [x] Model selection working (tested PRs #625, #270, #302)
+- [x] Compare mode with different models working (tested PR #302)
+- [ ] Live test on fresh PR: Workflow runs without errors
+- [ ] Live test on fresh PR: LLM evaluation produces scores and verdict
+- [ ] Live test on fresh PR: Comment posted on PR with evaluation results
+- [ ] Live test on fresh PR: Follow-up issue created if verdict is CONCERNS/FAIL
 
 ### Phase 2: Issue Formatting & Cleanup (1 Step)
 
 **Step 2A: Labels & Sync**
 1. ✅ Labels created via sync workflow (`agents:format`, `agents:formatted`, `agents:optimize`, `agents:apply-suggestions`)
 2. ✅ `agents-issue-optimizer.yml` is in sync manifest
-3. ✅ Sync PRs merged automatically
-4. Test on Manager-Database:
+3. ✅ Sync PRs merged automatically (6/7 repos)
+4. ⏳ Test on Manager-Database:
    - Create test issue with unformatted content
    - Add `agents:format` label
    - Verify issue is reformatted and `agents:formatted` label added
-   - **Note:** Requires repo write access to add labels
+   - **Status:** Ready for testing - requires repo write access
 
 **Validation Criteria:**
 - [x] Format labels exist in consumer repos (created by sync workflow)
 - [x] `agents-issue-optimizer.yml` in sync manifest
 - [x] `issue_formatter.py` tests passing (14 tests, fixed env var isolation)
-- [ ] Live test: `agents:format` triggers workflow
-- [ ] Live test: Issue body updated to AGENT_ISSUE_TEMPLATE
-- [ ] Live test: `agents:formatted` label added
-- [ ] Live test: Original content preserved in hidden section
+- [x] Topic splitting tested and working (created 5 issues from Issues.txt)
+- [ ] Live test on fresh issue: `agents:format` triggers workflow
+- [ ] Live test on fresh issue: Issue body updated to AGENT_ISSUE_TEMPLATE
+- [ ] Live test on fresh issue: `agents:formatted` label added
+- [ ] Live test on fresh issue: Original content preserved in hidden section
 
 ### Phase 3: Archive Unused Scripts (1 Step)
 
@@ -190,8 +202,23 @@ These scripts are fully tested (145 tests passing) but not yet integrated:
 
 | Phase | Scope | Steps | Test Repo | Status |
 |-------|-------|-------|-----------|--------|
-| 1 | PR Verification | 2 | Manager-Database | ✅ Deployed, pending live test |
+| 1 | PR Verification | 2 | Manager-Database | ✅ Deployed, tested on 3 PRs |
 | 2 | Issue Formatting | 1 | Manager-Database | ✅ Deployed, pending live test |
 | 3 | Cleanup/Archive | 1 | N/A | Deferred (scripts retained) |
 
-**Total: 4 deployment actions** - All infrastructure deployed. Live testing requires repo write access.
+**Total: 4 deployment actions** - All infrastructure deployed. Collab-Admin sync PR pending.
+
+---
+
+## Remaining Tasks
+
+### Immediate (Ready Now)
+1. **Merge PR #633** - GPT-5.2 for compare mode (waiting for CI)
+2. **Resolve Collab-Admin sync** - PR #104 blocked by failing gate check
+3. **Live test `agents:format`** - Create unformatted issue, add label, verify formatting
+4. **Live test `agents:optimize`** - Test on existing issue to get suggestions
+
+### Future Enhancements
+1. **Compare mode refinement** - Currently uses gpt-4o (GitHub) vs gpt-5.2 (OpenAI)
+2. **Model auto-update** - Use `scripts/update_model_list.sh` periodically
+3. **Phase 3 scripts** - Decide on capability_check.py and task_decomposer.py integration