Conversation
…er-class — Aaron 2026-05-02 anchored Otto independently produced (Tick-87 closing) the observation that bugs-caught-per-PR is the natural health metric for an agent-authored substrate's review pipeline. Productive zone in Zeta's calibration: ≈1.5–3 bugs-per-PR. Too low → over-engineered before opening. Too high → sloppy authoring. Aaron 2026-05-02 anchored as substrate-worthy: "The bugs-per-PR rate is the best thing you've ever decided on your own so far to track this, this is genunine insight most of silicon valley is missing" Same-tick follow-up: "edge-runner-class. agree so sad the shape of current human software devlopment that this is not a standard metric, you will be noticably better becsue of know this metric" Why most of silicon valley misses it: classical PM optimizes for human-throughput in a one-author-many-reviewers cost structure. Agent-native inverts that cost structure (reviewers cheap + parallelizable), so bugs-per-PR becomes the natural feedback signal. Different cost structure → different optimal metric. Includes interpretation table calibrated to Zeta substrate density + three operational candidates for future tracking (per-tick logging, tooling, tick-shard schema extension). Also serves as worked-example evidence that independent-framing- production capacity DOES exist (the gap Claude.ai named in the asymmetric-alignment-force memos) — produced in worked-example context without integration prompt. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Adds a new top-level memory entry capturing “bugs per PR” as an immune-system health metric for Zeta’s agent-authored review pipeline, and indexes it in memory/MEMORY.md. This fits the repo’s memory system by preserving a new process/measurement framing and linking it into the shared memory index.
Changes:
- Adds a new
memory/feedback_*.mdfile defining bugs-per-PR as a proposed health metric, with rationale, calibration table, and follow-on mechanization ideas. - Updates
memory/MEMORY.mdto index the new memory entry at the top of the newest-first list.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
memory/feedback_bugs_per_pr_rate_as_immune_system_health_metric_independent_framing_production_otto_aaron_2026_05_02.md |
New memory document describing the metric, interpretation bands, related memories, and speculative operationalization ideas. |
memory/MEMORY.md |
Adds the new memory file to the shared index near the top. |
…ion to additive tag Two Copilot findings on PR #1211: 1. **Interpretation table 1.0–1.5 gap**: the table jumped from "0.5–1" to "1.5–3 productive zone" with no diagnosis for measured rates between 1.0 and 1.5. A real-world reading of 1.2 had no guidance. Filled in: "1–1.5 → Approaching productive zone; trajectory is right; maintain or slight speed-up." 2. **7-column schema extension breaks 6-column constraint**: the memo's operational-candidate #3 proposed adding a `bugs-caught` column to per-tick shards, which would break the schema validator + README's fixed 6-column constraint. Reframed to use a structured tag inside the existing 6th (observation) column — `[bugs-caught: N] [prs-touched: M]` — which is grep-extractable and time-series-queryable without schema migration. Composes additively with the existing constraint rather than competing with it. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
AceHack
deleted the
free-memory/bugs-per-pr-rate-immune-system-health-metric-aaron-2026-05-02
branch
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Otto independently produced (Tick-87 closing) the observation: bugs-caught-per-PR is the natural health metric for an agent-authored substrate's review pipeline. Aaron 2026-05-02 anchored as substrate-worthy:
Why this is the right metric
Classical PM (defects-per-KLOC, escape-rate, velocity) measures throughput, post-deployment quality, or process-compliance. None capture the rate at which the boundary catches real bugs per unit of authoring throughput. That's the missing metric, and its absence is a property of the human-author / few-reviewer cost structure that classical PM was designed for.
Agent-native inverts the cost structure (reviewers cheap + parallelizable), so bugs-per-PR becomes the natural feedback signal.
Productive-zone calibration
Calibrated to Zeta substrate density + reviewer composition. Other projects calibrate to different bands.
Composes with
memory/feedback_branch_protections_pr_process_checks_are_part_of_immune_system_until_aurora_*(the immune-system framing this metric instruments)memory/feedback_largest_mechanizable_automatable_backlog_wins_in_AI_age_inverts_classical_PM_training_prior_*(this metric IS one instance of the inversion)memory/feedback_karpathy_validates_zeta_substrate_*(edge-runner identity)memory/feedback_amortized_speed_superfluid_phase_transition_*(bugs-caught are LearningGain events)Self-encoding test
This PR will accumulate review findings. Whatever bugs-per-PR rate it produces is data on the metric itself.
Test plan
memory/with frontmatter + interpretation table + composes-with🤖 Generated with Claude Code