Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
4424930
chore(codex): bootstrap PR for issue #483
github-actions[bot] Jan 5, 2026
343e505
Add post-merge metrics schema support
Jan 5, 2026
a10de9d
chore(autofix): formatting/lint
github-actions[bot] Jan 5, 2026
a11519f
Merge branch 'main' into codex/issue-483
stranske Jan 5, 2026
d96b805
Fix rate-limit failures in autofix and codeql
stranske Jan 5, 2026
d01aedd
Merge remote-tracking branch 'origin/main' into pr-534
stranske Jan 5, 2026
6938ae9
chore: mark context_extractor executable
stranske Jan 5, 2026
6c41f4e
Fix token precedence for CodeQL/autofix
stranske Jan 5, 2026
b085846
Revert CodeQL custom token override
stranske Jan 5, 2026
671c42a
chore(codex-keepalive): apply updates (PR #534)
github-actions[bot] Jan 5, 2026
884be2f
Merge branch 'codex/issue-483' of https://github.com/stranske/Workflo…
github-actions[bot] Jan 5, 2026
18b0c34
chore(autofix): formatting/lint
github-actions[bot] Jan 5, 2026
a1be409
feat: add post-merge metrics builder
Jan 5, 2026
c5871cc
chore(autofix): formatting/lint
github-actions[bot] Jan 5, 2026
16f040c
fix(lint): simplify condition in _section_has_content per SIM103
stranske Jan 5, 2026
6d3b094
fix: replace deprecated datetime.utcnow() with datetime.now(UTC)
stranske Jan 5, 2026
7023eb3
Merge branch 'main' into codex/issue-483
stranske Jan 5, 2026
50868e9
chore(codex-keepalive): apply updates (PR #534)
github-actions[bot] Jan 5, 2026
da21dc4
chore(autofix): formatting/lint
github-actions[bot] Jan 5, 2026
8a1f32a
Merge branch 'main' into codex/issue-483
stranske Jan 5, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 19 additions & 6 deletions .github/workflows/autofix.yml
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ jobs:
id: context
uses: actions/github-script@v8
with:
github-token: ${{ secrets.AGENTS_AUTOMATION_PAT || secrets.ACTIONS_BOT_PAT || github.token }}
script: |
const path = require('path');
const { paginateWithBackoff } = require(
Expand Down Expand Up @@ -100,12 +101,24 @@ jobs:

// File check with rate limit retry
const { owner, repo } = context.repo;
const files = await paginateWithBackoff(
github,
github.rest.pulls.listFiles,
{ owner, repo, pull_number: pr.number, per_page: 100 },
{ maxRetries: 3, core }
);
let files = [];
try {
files = await paginateWithBackoff(
github,
github.rest.pulls.listFiles,
{ owner, repo, pull_number: pr.number, per_page: 100 },
{ maxRetries: 3, core }
);
} catch (error) {
const message = String(error?.message || error || '');
const status = Number(error?.status || error?.response?.status || 0);
if (status === 403 && message.toLowerCase().includes('rate limit exceeded')) {
core.warning(`Rate limited while listing PR files; skipping autofix run. ${message}`);
core.setOutput('should_run', 'false');
return;
}
throw error;
}

const hasPython = files.some(f => f.filename.endsWith('.py') || f.filename.endsWith('.pyi'));
if (!hasPython) {
Expand Down
1 change: 1 addition & 0 deletions agents/codex-483.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
<!-- bootstrap for codex on issue #483 -->
2 changes: 1 addition & 1 deletion autofix_report_enriched.json
Original file line number Diff line number Diff line change
@@ -1 +1 @@
{"changed": true, "classification": {"total": 0, "new": 0, "allowed": 0}, "timestamp": "2026-01-05T06:00:00Z", "files": [".github/scripts/agents_pr_meta_update_body.js", ".github/scripts/__tests__/agents-pr-meta-update-body.test.js", "scripts/langchain/context_extractor.py", "scripts/langchain/prompts/context_extract.md", "tests/scripts/test_context_extractor.py"]}
{"changed": true, "classification": {"total": 0, "new": 0, "allowed": 0}, "timestamp": "2026-01-05T08:45:34Z", "files": ["scripts/issue_pattern_feedback.py", "scripts/issue_pattern_report.py", "scripts/langchain/issue_formatter.py", "tests/scripts/test_issue_pattern_feedback.py", "tests/scripts/test_issue_pattern_report.py"]}
28 changes: 27 additions & 1 deletion docs/keepalive/METRICS_SCHEMA.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,14 @@ This document defines the structured metrics record written per keepalive
iteration. Records are newline-delimited JSON (NDJSON); one JSON object per
line.

## Fields
## Record Types

The metrics log supports two record types:

- Keepalive iteration records (`metric_type: "keepalive"` or omitted)
- Post-merge summary records (`metric_type: "post-merge"`)

## Keepalive Iteration Fields

- pr_number: Integer PR number.
- iteration: Integer iteration count for the keepalive loop (1-based).
Expand All @@ -14,9 +21,28 @@ line.
- duration_ms: Integer duration in milliseconds for the iteration.
- tasks_total: Integer total tasks detected for the PR.
- tasks_complete: Integer completed tasks detected for the PR.
- metric_type: Optional string. When present, set to `"keepalive"`.

## Post-Merge Summary Fields

- metric_type: String literal `"post-merge"`.
- pr_number: Integer PR number.
- timestamp: ISO 8601 UTC timestamp for when the summary record was emitted.
- merged_at: ISO 8601 UTC timestamp for when the PR was merged.
- iteration_count: Integer total keepalive iterations for the PR.
- tasks_total: Integer total tasks detected for the PR.
- tasks_complete: Integer completed tasks detected for the PR.
- completion_rate: Float between 0.0 and 1.0 representing task completion.
- human_interventions: Integer count of human interventions (comments, manual edits).

## Example Record

```json
{"pr_number":1234,"iteration":2,"timestamp":"2025-01-15T12:34:56Z","action":"retry","error_category":"none","duration_ms":4821,"tasks_total":14,"tasks_complete":6}
```

## Example Post-Merge Record

```json
{"metric_type":"post-merge","pr_number":1234,"timestamp":"2025-01-16T09:00:00Z","merged_at":"2025-01-16T08:55:12Z","iteration_count":3,"tasks_total":14,"tasks_complete":14,"completion_rate":1.0,"human_interventions":2}
```
34 changes: 21 additions & 13 deletions pr_body.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<!-- pr-preamble:start -->
> **Source:** Issue #482
> **Source:** Issue #483

<!-- pr-preamble:end -->

Expand All @@ -10,22 +10,30 @@ _Scope section missing from source issue._

<!-- Updated WORKFLOW_OUTPUTS.md context:start -->
## Context for Agent
- Pending extraction from linked issue context.

### Related Issues/PRs
- [#5](https://github.com/stranske/Workflows/issues/5)
- [#483](https://github.com/stranske/Workflows/issues/483)

### References
- https://github.com/stranske/Workflows/blob/main/docs/plans/langchain-issue-intake-proposal.md
- https://github.com/stranske/Workflows/compare/main...codex/issue-483?expand=1

### Blockers & Dependencies
- After PR merges, capture what worked/didn't for future issue formatting improvements.
<!-- Updated WORKFLOW_OUTPUTS.md context:end -->

#### Tasks
- [x] Create context extraction chain with `CONTEXT_EXTRACTOR_PROMPT`
- [x] Run during PR creation in `reusable-agents-issue-bridge.yml`
- [x] Insert context into PR body after Scope, before Tasks
- [x] Preserve in `<- Updated WORKFLOW_OUTPUTS.md context:start -->...<- Updated WORKFLOW_OUTPUTS.md context:end -->` markers
- [x] Modify `agents_pr_meta_update_body.js` to include context section
- [x] Optional: fetch linked issue comments for richer context
- [x] Add tests for context extraction
- [x] Design data collection schema for post-merge metrics
- [x] Track iteration count, completion rate, human interventions
- [x] Build corpus of successful issue patterns
- [ ] Create feedback loop to improve formatting prompts
- [ ] Add visualization/reporting for patterns

#### Acceptance criteria
- [x] Context section added to PR body when relevant
- [x] Related issues/PRs linked
- [x] Design decisions captured
- [x] Markers allow programmatic identification
- [x] Post-merge metrics captured automatically
- [x] Successful patterns identified
- [ ] Formatting prompts improve over time
- [ ] Dashboard or report available for review

<!-- auto-status-summary:end -->
Loading
Loading