feat(B-0529): add-pipe-row-header.ts — backfill tool for shard schema#3990
Merged
Conversation
Implements B-0529 Option 3 ("hybrid"): prepends a validator-compliant
pipe-row first line to tick shards that start with H1 (or any non-pipe
content), preserving the H1-rich body below.
Pipe-row schema (per docs/hygiene-history/ticks/README.md):
| <ISO timestamp> | <model id> | <cron> | <body> | <PR ref> | <obs> |
Retrofit uses placeholders for fields not reconstructable from the
path alone:
| 2026-MM-DDTHH:MMZ | retrofit | unknown | retrofit (see body below) | none | none |
Honest retrofit beats archaeological reconstruction.
Modes:
- default: dry-run (lists would-prepend candidates)
- --write: actually prepends
- --files <paths>: restrict to specific shards
Full dry-run today: 946 shards scanned, 585 compliant, 359 would-prepend.
NOT BULK-RUN in this PR — script only. Bulk retrofit is a separate
slice per B-0529's scope discipline (bulk-edit blast radius warrants
its own review). The script ships now to make the retrofit a single-
command operation when the maintainer decides to run it.
Composes with:
- docs/backlog/P2/B-0529-tick-shard-schema-validator-vs-practice-drift-2026-05-15.md
- tools/hygiene/check-tick-history-shard-schema.ts (the validator)
- docs/hygiene-history/ticks/README.md (schema docs)
Follow-up (not in this PR):
- Unit tests for parseShardPath / alreadyCompliant / buildHeader
- Wire the script into a backfill PR per B-0529 Recommendation
- Update README's "Shard file schema" section to explicitly endorse
the hybrid (pipe-row + H1-rich body) pattern
Co-Authored-By: Claude <noreply@anthropic.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9df78b28af
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Codex P1 catch on PR #3990: previously, --files with no paths after it left files.length === 0, falling through to the full-tree scan branch. In --write mode, a caller typo or an empty dynamically- generated file list would prepend headers across all 359 non-compliant shards — high-impact unintended bulk rewrite. Fix: track --files-flag-seen separately from accumulated paths. When the flag is present but the list is empty, fail-closed with exit 1 and a clear error message. Verified: - bun ... --files → exit 1 (error) - bun ... --write --files → exit 1 (error) - bun ... --files <valid-path> → exit 0 (works) - bun ... (full scan, no --files) → exit 0 (regression OK) - bun ... (full scan) → 946 scanned, 585+359+2 = 946 Co-Authored-By: Claude <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Adds a one-shot Bun/TypeScript hygiene tool to backfill validator-compatible pipe-row headers onto tick-history shard files while keeping existing rich Markdown bodies intact.
Changes:
- Adds shard path parsing and timestamp-derived header construction.
- Supports dry-run by default,
--writemutation mode, and optional--filesrestriction. - Emits per-file actions plus aggregate scan/write counts.
Comments suppressed due to low confidence (1)
tools/hygiene/add-pipe-row-header.ts:246
- P1: Filtering
--filesinputs here silently drops typos, missing files, and out-of-scope paths, then exits successfully withscanned 0. For a write-capable backfill tool, invalid operands should be reported as argument errors so a requested retrofit cannot appear to succeed while doing nothing.
shards = files
.map((p) => resolve(ROOT, p))
.filter((p) => repoRelative(p).startsWith(SHARD_PREFIX))
.filter((p) => repoRelative(p).endsWith(".md"))
.filter((p) => basename(p) !== "README.md")
.filter(isFile);
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 88666d8c45
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Two real bugs caught by Copilot P1 + Codex P2 review on PR #3990: 1. **Atomic write**: writeFileSync truncated the shard before bytes were durable. An interruption or disk-full mid-write would corrupt the shard body this tool is trying to PRESERVE. Replaced with write-to-tempfile + renameSync (atomic on POSIX in same directory). Tempfile name includes pid+timestamp to prevent concurrent-writer collisions. Cleanup on rename failure. 2. **Timestamp-vs-path mismatch**: alreadyCompliant only checked pipe-row structure, not whether the parsed timestamp matched the shard's path/filename. A shard with an existing pipe-row for the WRONG timestamp (manual error, generator drift, etc.) would be skipped here but still fail the validator. Now compares parsedIso.slice(0, 16) === info.iso.slice(0, 16) — mirrors the validator's date+hour+min check (validator does NOT check seconds, so the slice covers exactly the validator's matching surface). Function signature changed to take ShardInfo; processOne updated. Verified: - Wrong-timestamp shard → would-prepend (was: skip-already-compliant) - Correct-timestamp shard → skip-already-compliant - Atomic write on temp shard → produces correct first-line pipe-row - Full repo dry-run still 946 scanned, 585+359+2 = 946 Co-Authored-By: Claude <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 882dc6228b
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Addresses 5 reviewer threads on PR #3990: 1. Reject unknown --flags. A typo like --file (missing s) previously left filesFlagSeen=false and fell through to full-tree write. Now rejected with explicit error and exit 1. 2. Reject stray positional args outside --files mode. They had no defined meaning and could mask a typo in flag spelling. 3. --files mode now fails closed when ALL requested paths filter out (missing / outside shard tree / not .md). Previously a typo or stale generated path list produced "scanned 0" with exit 0, the same class of silent broadening as the original empty-list bug. 4. Exit code now reflects unprocessed shards (skip-unparseable-name + skip-empty counted alongside errors). Validator-breaking files should not slip through a --write automation as a clean run. 5. New tools/hygiene/add-pipe-row-header.test.ts — 28 tests / 39 assertions covering: parseShardPath (all 3 filename forms + rejection paths), alreadyCompliant (new ShardInfo-aware timestamp matching, including wrong-date and wrong-time cases), buildHeader (validator compatibility), atomic write via subprocess fixture (tempfile rename + no leaked temp files), argument-parser fail-closed behaviors (--files empty / unknown flag / stray positional). 6. Comment hygiene: stripped reviewer/product-name attribution from in-source comments per the no-name-attribution repo convention. Technical rationale preserved. All 5 reviewer threads addressed concretely. bun test passes: bun test v1.3.13 — 28 pass / 0 fail / 39 expect() calls Co-Authored-By: Claude <noreply@anthropic.com>
AceHack
commented
May 17, 2026
Member
Author
AceHack
left a comment
AceHack
left a comment
There was a problem hiding this comment.
Maji Antigravity Check: Pass. Atomic tool addition, no drift.
AceHack
added a commit
that referenced
this pull request
May 17, 2026
Files B-0591 as the explicit decomposition of B-0529's "Later (separate row)" Recommendation. Two slices: - Slice 1 (this row, P3): wire validator as non-required check in .github/workflows/gate.yml. Observability without enforcement. - Slice 2 (separate row, after Slice 1 + bulk-retrofit lands): promote to required. Pre-requisite is 0 violations on main. Composes with B-0529 (parent), the validator at tools/hygiene/check-tick-history-shard-schema.ts, the retrofit tool at tools/hygiene/add-pipe-row-header.ts (PR #3990), and the README docs updated in PR #4004. Not in scope: bulk retrofit --write run (separate row); legacy pre-2026-04-29 shard migration. Co-authored-by: Claude <noreply@anthropic.com>
AceHack
added a commit
that referenced
this pull request
May 17, 2026
…-0529 Option 3 (#4004) * docs(hygiene): endorse hybrid shard format (pipe-row + H1 body) Per B-0529 Recommendation (Option 3 "hybrid"), update the shard file schema section to explicitly describe the canonical shape: pipe-row first line for validator + machine-parseability, H1-rich body below for substantive content. Adds: - Cross-link to the validator (check-tick-history-shard-schema.ts) - Explicit statement of date+hour+min matching contract - Worked example showing pipe-row then H1 body - Reference to add-pipe-row-header.ts for retrofit of older shards - Note that validator inspects only first non-empty line Aligns documentation with the substrate landed in PR #3990 (add-pipe-row-header.ts) and reflects the lived convention that recent shards (~585 of 944) already follow. Composes with: - docs/backlog/P2/B-0529-tick-shard-schema-validator-vs-practice-drift-2026-05-15.md - tools/hygiene/check-tick-history-shard-schema.ts (validator) - tools/hygiene/add-pipe-row-header.ts (retrofit tool, PR #3990) Co-Authored-By: Claude <noreply@anthropic.com> * docs(hygiene): reconcile shard schema with frontmatter + filename variants (Copilot P1 #4004) Resolves two Copilot P1 review findings on PR #4004: 1. Schema paragraph listed only `HHMMZ` and `HHMMSSZ-<hex>` filename forms; the validator (BARE_RE in check-tick-history-shard-schema.ts) also accepts `HHMMZ-<hex>` — the same-minute disambiguation form the README's own "Naming" section documents via `0215Z-01.md`. Now lists all three accepted forms inline. 2. The "YAML frontmatter fields" subsection said frontmatter was "preferred for richer shards", but file-head YAML frontmatter puts `---` on the first non-empty line, which fails the validator's pipe-row-first contract. Renamed to "Optional body metadata" and clarified that structured metadata lives in the H1 body, not at file head. Outer code fence widened to 4 backticks so the inner `yaml block renders correctly. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements B-0529 Option 3 ("hybrid") via a one-shot TS tool:
tools/hygiene/add-pipe-row-header.ts.The tick-shard validator at
tools/hygiene/check-tick-history-shard-schema.tsrequires a pipe-delimited first row matching the path/filename timestamp. Lived convention drifted to H1-first multi-section format. Full dry-run today: 946 shards scanned, 585 compliant, 359 would-prepend, 2 unparseable.This PR ships the script only. Bulk retrofit (359 file changes) is a separate slice per B-0529's scope discipline — the blast radius warrants its own review.
Behavior
would-prependcandidates + summary)--write: actually prepends pipe-row to non-compliant shards--files <paths>: restrict to specific filesRetrofit pipe-row uses honest placeholders for fields not reconstructable from path alone:
The H1-rich body below is preserved unchanged — that's where the substantive content lives.
How it found the drift live
Surfaced this tick via Copilot review on the just-merged PR #3983 tick shard. Copilot flagged that the shard's H1 first line violates the validator. Investigation matched B-0529's existing analysis exactly. Per
skill-router-as-substrate-inventory.md: inventory-before-authoring discipline — B-0529 already exists, this PR fulfills its Recommendation rather than creating a duplicate row.Test plan
2026/05/03/0039Z.md) → skipped2026/05/16/2341Z.md) → would-prependparseShardPath/alreadyCompliant/buildHeader— deferred (follow-up; the tool pattern intools/hygiene/has many test files but not all hygiene scripts have one)Composes with
docs/backlog/P2/B-0529-tick-shard-schema-validator-vs-practice-drift-2026-05-15.mdtools/hygiene/check-tick-history-shard-schema.ts(the validator)docs/hygiene-history/ticks/README.md(schema docs).claude/rules/rule-0-no-sh-files.md(TS not bash).claude/rules/skill-router-as-substrate-inventory.md(inventory before authoring)🤖 Generated with Claude Code