docs(corpus): standard for the confidence field - #1468
Merged
Conversation
Reviewed by an independent cross-model review-plan pass (Codex, gpt-5.6-sol, high effort): eleven findings, all applied. Three more were found afterwards, the load-bearing one being that the plan blocked its own STEP 2 on an OPEN item -- an OPEN item a step must consume is a missing input, not an open question. Refs #636 Signed-off-by: Serina Mcfall <serina.mcfall@gmail.com>
The corpus's first hand-authored node. Frontmatter validates against node.schema.json with zero errors; 11 evidence entries; no relationships, because no sibling node exists to target and an unresolved target is a hard validator error. The repository revision is recorded in the evidence ledger -- the schema calls that array the provenance ledger and permits no other field, so it is the only schema-legal place for it. STEP 3 (RUNS HERE) proved the validator actually scans the node rather than skipping it: mutating type: to a value outside the enum produced 'FAIL corpus-agents: schema violation at type' and exit 1, naming the node's id; restoring it returned exit 0. Also replaces test_real_corpus_root_currently_has_no_content_outside_schema, which asserted the corpus root is EMPTY and therefore could not survive the first authored node. Authorised by Serina as a deliberate widening of the plan's LEFT OUT scope, recorded in the plan. The replacement asserts what the original's own comment said it was for -- content present AND nothing from schema/ -- and both halves were negative-controlled before being trusted. Refs #636 Signed-off-by: Serina Mcfall <serina.mcfall@gmail.com>
Adds the lookup table of authoritative sources and the 'what a corpus node is' section: one file per node, one idea per node, where nodes go, id permanence, relationship resolution, and the authored-vs-generated boundary. Reproduces no enum member list and no field-combination matrix -- those stay canonical in node.schema.json and schema/README.md and are linked instead. The checker never reads body prose, so a copy here would stay green while drifting. Verified: no type-enum member and none of entry_class/provided_by/ confidence appear in the body. Refs #636 Signed-off-by: Serina Mcfall <serina.mcfall@gmail.com>
The part that lives nowhere else: what the checker actually does with each of CONTRACT.md's six citation shapes, and three things a passing run does not mean -- that a citation supports its claim (checking is structural, the file is never opened and compared), that UNVERIFIED is a pass, or that a line number was verified (#1459). Names the three evidence classes and what each is FOR, but not which fields the schema requires per class -- that matrix stays in node.schema.json. Verified absent from the body: entry_class, provided_by, confidence. Refs #636 Signed-off-by: Serina Mcfall <serina.mcfall@gmail.com>
This is #605's acceptance criterion for the file -- a cold-start agent creating, updating or retiring one node without oral guidance. Three numbered procedures, each ending in the same check command. Adds a 'Running the check' section giving the interpreter form (works without Hermit; the just recipe needs activation first) and documenting --root, since without it the command always validates the real corpus whatever directory you are standing in. Retire covers the three things most likely to be got wrong: inbound relationships resolved before retirement, status set rather than the file deleted, and the id never reused or renamed. Refs #636 Signed-off-by: Serina Mcfall <serina.mcfall@gmail.com>
Names each deferred area with its owning issue rather than leaving a reader unable to tell a gap from an omission: the standards and templates (#1307-#1351, none merged), generated-artifact provenance (#1316), claim-type and flagged state (#1410), the human entry point (#639), and unverified line numbers in citations (#1459). Records why this node carries no relationships -- at its recorded revision it was the only authored node, and a target no node carries is a hard error -- so the absence reads as deliberate rather than forgotten. Every repo path cited in the body verified to resolve to a real file. Refs #636 Signed-off-by: Serina Mcfall <serina.mcfall@gmail.com>
The audit found two things the earlier steps had missed. The document's own rule -- 'a FACT resting only on UNVERIFIED citations has not been checked by anything' -- condemned the node's own ledger entry 1, the provenance record, whose only citation is a commit reference. Resolved by stating the one carve-out narrowly and giving it a real check (git cat-file -e <sha>, run, exit 0) rather than by quietly exempting it. A commit citation on a claim about repository CONTENT is explicitly still not covered. Ledger entry 11 overreached: 'every change under launchpad/docs/corpus is gated in CI' where the workflow triggers on pull requests and pushes to launchpad. Narrowed to what the workflow actually does. Also records a scope judgement the DoD asks for: the citation-shape table is reference material rather than instruction and belongs in the evidence standard (#1314) when that lands. Noted in place rather than folded in silently. Verified: all 11 ledger entries resolve, every body path is a real file, merge-base equals the recorded revision, and exactly one hand-authored .md was added under the corpus root. Refs #636 Signed-off-by: Serina Mcfall <serina.mcfall@gmail.com>
… finding) An independent review-tests pass defeated the first replacement: patching discover_markdown_files to a hardcoded 'return [root / "AGENTS.md"]' -- a constant that never touches the filesystem -- satisfied both 'non-empty' and 'nothing from schema/' while proving neither discovery nor exclusion. My own negative control had broken the exclusion set and emptied the corpus, but never tried a constant. Showing that assertions CAN fail is not the same as showing they can only pass for the right reason. The expectation is now derived from an independent filesystem walk and compared for equality, which catches partial walks and both exclusion directions. It does NOT catch the constant today, and the docstring says so rather than claiming otherwise: the corpus holds exactly one node, so [root/'AGENTS.md'] IS the correct answer and no assertion against the real tree can tell a constant from a real walk. That resolves itself at the second node. test_sibling_discovered_schema_dir_excluded catches it now -- verified FAIL under that mutant -- and is the real proof of discovery behaviour. Measured, after fixing a mutation harness that had patched a different module instance than the test uses and so reported every mutant as caught: returns nothing -> FAIL (caught) exclusion disabled -> FAIL (caught) constant return -> PASS (not caught; sibling catches it) Refs #636 Signed-off-by: Serina Mcfall <serina.mcfall@gmail.com>
FINDING A -- the carve-out claimed enforcement it does not have. 'There is exactly one carve-out, and widening it defeats the rule' read as though some check held the line. Nothing does: a second, third or tenth commit-only FACT produces extra non-fatal UNVERIFIED notices and still exits 0, which the reviewer proved by adding one. Now says plainly that this is a convention a REVIEWER holds, names the signal to look for (more than one commit-only FACT in a ledger), and states that no check will raise it. FINDING B -- create-node step 3 asked the author to record four categories of evidence and gave a destination to exactly one of them. Adds the mapping: revision to the ledger, sources to the citations of the claims they support, and expected-but-unverified to the body's scope section. Plus the rule that anything reaching none of the three was either a missing claim or a stale note. Applying B to this node exposed that it did not follow its own new rule -- the same self-contradiction class STEP 8 caught. Its scope section now names the two things unverified when it was written: no agent harness was tested reading it as a resolved AGENTS.md, and relationships.schema.json's enum was never read directly. Re-audited after the change: exactly one commit-only FACT remains, the enforcement claim is gone, validator exits 0, 79 tests pass. Refs #636 Signed-off-by: Serina Mcfall <serina.mcfall@gmail.com>
Codex review-final returned REQUEST_CHANGES with seven conditions, then a
second pass confirmed five fixed and refused two. Both refusals were correct.
This commit is the whole of that work.
1. LEDGER COMPLETENESS. Seven entries added (11 -> 18) for claims the body
taught with no classified entry behind them: the evidence classes, the
"supersedes" vocabulary, the syntax-only GitHub check, the external-URL
verdict, and the delete-vs-retire distinction.
The recheck caught two of these still wrong. An entry said the schema
"permits no field outside that set" after listing the six required fields
-- it also permits "relationships", so the entry was wrong about the file
it cited. And the "supersedes" entry cited node.schema.json, which carries
only the enum; the directionality it asserts ("source replaces target;
target becomes historical") lives in relationships.schema.json:30, a file
this node had listed as read-but-unverified and cited anyway. Read it,
cited it.
2. RETIREMENT WAS BUILT ON A FALSE PREMISE. It claimed inbound relationships
"no longer resolve" after retirement. Measured: with the file retained --
which the same procedure requires -- they resolve fine; only deletion
breaks them. Rewritten so the reason matches the mechanism, the danger is
stated correctly (edges keep pointing at a node that stopped being
current, and nothing warns you), and a replacement declares "supersedes",
which the procedure never mentioned.
3. CITATION TABLE OVERSTATED VERIFICATION. A GitHub link pinned to a real SHA
naming a file that never existed returns ok -- the checker never contacts
GitHub. The table now carries a "does it prove the target exists?" column,
gains the external-URL row it was missing, and names the trap. All seven
rows independently confirmed against the classifier.
4. PROVENANCE SEMANTICS -- and the defect the recheck was right to refuse.
The update procedure said to bump the revision on every edit; this node
kept its original across seven. The first attempt defined the field as an
evidence snapshot and classified that as INFERENCE citing node.schema.json.
That schema says nothing whatever about revisions, so the citation could
not support the statement: a policy choice had been given a class that made
it look derived. It is now TEAM_KNOWLEDGE attributed to #636's own
definition of done -- "the draft is checked against the repository revision
recorded in provenance" -- which is where the meaning comes from. ADR-0029
requires exactly that: history and team knowledge stay attributed rather
than being promoted to fact.
The node also claimed it was "the only authored node in the corpus" at the
recorded revision. It did not exist at that revision and neither did any
other node: git cat-file -e on that path at that sha exits 128, and the
corpus held zero .md files outside schema/. Corrected.
The revision still does not move, and the reason is checkable rather than
asserted: every cited source is byte-identical between that revision and
now, so verifying a claim at HEAD and at the recorded revision were the
same act. git diff --name-only <sha> -- <cited paths> returns nothing.
5. AUDIENCE. Added "reviewer": the document assigns reviewers an enforcement
duty no tool performs, so they are addressed by it. The adjudicator had
declined this finding (nothing consumes the field today); the final pass
argued audience describes who is addressed, not who parses it. Went with
the latter. Whether "developer" also belongs is left undecided.
6. The test module docstring claimed absolute fixture isolation that was false
before this branch and more visibly false after it. States the rule with
its one named exception instead. Flagged independently by the adjudicator
and the final pass.
Pre-existing, fixed while in the file: the same docstring advertised
"python3 -m unittest launchpad.project_intelligence.corpus.tests.test_validate",
which has never worked -- the directory is "project-intelligence" with a
hyphen, not a legal package name. Confirmed ModuleNotFoundError, removed.
7. Full suites re-run in a writable environment: 79 validator tests OK, 27
schema tests OK, real corpus exit 0. The plan's two-node create/update/
retire charter ran with its negative control -- a deliberately invalid node
failed and named itself, proving the scratch root was the one scanned;
retirement kept beta's edge resolving; deleting alpha instead broke it
exactly as the corrected text now says.
Final state: 18 ledger entries, schema errors 0, every citation resolves, no
cited source changed since the recorded revision, exactly one commit-only
FACT, 79 + 27 tests green, real corpus exit 0.
Refs #636
Signed-off-by: Serina Mcfall <serina.mcfall@gmail.com>
Adds launchpad/docs/corpus/standards/confidence.md, the canonical policy node for the confidence key on an evidence entry. The field is a number in a closed interval, required for INFERENCE only and forbidden on FACT and TEAM_KNOWLEDGE. The schema holds that much. What it cannot hold is what the number means, and an uninterpreted number between 0 and 1 is decoration -- an author picks 0.8 because it feels about right and a reader treats it as calibrated. The standard states that no calibration exists or is collected, that values are not comparable between authors, and that no confidence however high promotes an INFERENCE to a FACT. Its load-bearing section separates reasoning from evidence from dressing up a decision: an entry whose citation supports the subject of a claim but not the choice the claim makes is a decision in disguise and belongs in TEAM_KNOWLEDGE. That move was caught by cross-model review on #636 and no check can catch it, so it is documented with both worked examples. Records a measured divergence: a NaN confidence satisfies node.schema.json and passes corpus validation while memory.py rejects the identical value, because every comparison against NaN is false. Filed as #1463; the standard carries a MUST for finite values to cover it until that lands. Declares no relationships -- every sibling standard is unmerged, and a target naming an id no loaded node carries is a hard validation error. Signed-off-by: Serina Mcfall <serina.mcfall@gmail.com>
#636) Third cross-model pass, third refusal on the same condition. It was right three times, and the third refusal finally identified why the first two fixes failed: the document was writing corpus-wide provenance policy it had no authority to write. Reclassifying it (INFERENCE, then TEAM_KNOWLEDGE) only changed the label on an unsourced decision. WHAT ACTUALLY CHANGED: the document stops making the claim. #1321 ("document corpus standard for provenance", unlanded) owns the question of whether a recorded revision may stay put across edits. This node now says so, states its own approach as working practice pending that standard rather than as a rule others must follow, and defers. The ledger entry no longer asserts the policy -- it records only what #636's definition of done actually requires, which is a fact about the issue and is attributed to it. Also fixes a self-contradiction the last fix opened, exactly the failure shape this repository keeps producing. The previous commit added a FACT citing relationships.schema.json for the supersedes directionality. The same node still said, in its unverified list, that relationships.schema.json had never been read. Both could not stand. The file WAS read; the stale disclaimer is now narrowed to what remains genuinely unchecked -- that the two enum lists agree, which rests on a test that was not run. Ledger completeness, third sweep: entries added for the generated-artifact rejection, the non-enforcement of the one-commit-only-FACT convention, and positional citations not being bounds-checked. Prose that asserted this node's own history -- node counts at the recorded revision, byte-identity of cited sources -- is removed rather than given entries: it told the reader to trust the document about a fact they can check themselves in one command, so the document now gives them the command instead. TEAM_KNOWLEDGE's description corrected. It said "a person told you", which the schema does not require (it asks who or what told the corpus) and which made the class look unavailable for an attributed issue or decision record. It now also says outright what it is not: a place to park a decision you made yourself. Attributing an extrapolation to the thing it started from does not make it something you were told. Updating step 4 restated as four explicit branches including the byte-identical case, which the earlier "bump on every edit" wording omitted while this very node relied on it -- the inconsistency the pass named. Verified: 21 ledger entries, schema errors 0, every citation resolves, exactly one commit-only FACT, no cited source changed since the recorded revision, 79 validator tests OK, real corpus exit 0. Refs #636 Signed-off-by: Serina Mcfall <serina.mcfall@gmail.com>
Two review findings, both adjudicated Medium, both defects in this document rather than in anything it describes. A standard that contradicts itself teaches the wrong lesson to every sibling that copies it, so both are fixed here rather than deferred. review-code, confirmed by review-adjudicate: the Guidance section said to pick one number per band and reuse it, but never said which number belonged to which band -- and then shipped two INFERENCE entries at two different values with no band label and no stated reason. The bands now carry explicit pegs of 0.8, 0.6 and 0.4, the criterial column is named so the illustrations stop reading as tests, and this node's own two entries are labelled to their bands as the nearest worked example. The scope-and-omissions entry now disclaims the specific values, not just the idea of banding: they are agreed pegs so two authors mean the same thing by the same numeral, and they measure nothing. review-plan, surviving into the document and caught by review-adjudicate: the header claimed the field-combination rules and enum members were deliberately not copied, while Requirement 1 restates the confidence row and names all three classes. A standard about this field cannot omit the field's own rule, so the claim was the wrong half to keep. The header now names that one restatement as a deliberate drift surface and points at the two places that must change if the schema's rule ever does. One neighbouring hole opened by the first fix and closed here: Requirement 6 forbids moving a number, and the second entry moved from 0.7 to 0.6. Snapping an unchanged assessment onto a band peg is not the move that rule prohibits, but from a diff the two are indistinguishable, so Requirement 6 now says so and requires the reason be recorded. This message is that record. Signed-off-by: Serina Mcfall <serina.mcfall@gmail.com>
review-final found nine findings, and the cluster that mattered was not about confidence at all -- it was the document's claims about its own contents. For a node whose whole subject is not asserting more than the evidence supports, and which roughly 45 siblings are meant to copy, that is the report. All nine are defects in this document, all small, all inside the issue's scope, so they are fixed here rather than deferred. High. The stated reason for declaring no relationships was false. It said there was nothing this node could legally point at, and the next sentence named AGENTS.md. AGENTS.md carries id corpus-agents, is a loaded node, and a references edge to it validates today. The convention of declaring none is settled and unchanged; the justification is now the true one -- add the edge set in one pass once the siblings land -- and names corpus-agents as the first edge to make. The same false premise was in the plan's LEFT OUT and is corrected there too. Medium, four of them. The Requirements preamble claimed the first three MUSTs were machine-enforced while Requirement 3 said the schema does not reach it; it now says two. The header named two places tracking the schema when three do, and is now a table. The peg rationale asserted cross-author shared meaning that the reader rules and this node's own 0.8 INFERENCE both deny; adopting pegs is now stated as a hope about future practice, never as something a reader may sort on. The only-INFERENCE-in-the-corpus FACT cited AGENTS.md alone, which cannot establish a quantifier -- it now also cites validate.py for the schema/ exclusion and the fixture that carries the other one, which is exactly the merely-on-topic citation this document calls the tell. Medium, and the sharpest. Requirement 6's carve-out claimed re-encoding means the judgement is identical and only its expression changed. Measured against the document's own rules that is false: 0.7 to 0.6 moved the entry a full band from its neighbour, and ranking within a class is the one thing the number legitimately does. The carve-out now says re-encoding preserves the assessment but can change the rank, requires the record to name the Means row it matches, and admits there is no tie-break for a midpoint. Low, two. A number between 0 and 1 restated bounds the header said were left to the schema. And MUST be finite was overbroad -- .inf and -.inf are both caught on range, so NaN is the single value that escapes, because every comparison against it is false and the assertions never fire. Requirement 3 and the enforcement table both say so now. Per Requirement 6 as amended: the 0.6 entry matches the Medium band's Means row, an absence-of-evidence argument over a scope that was checked, because its scope is three files rather than the repository. Not fixed here: the NaN divergence itself stays filed as #1463, since the schema sits outside this issue's scope. Signed-off-by: Serina Mcfall <serina.mcfall@gmail.com>
Author
|
Needs a human before this leaves draft: the mandatory cross-model review could not run.
That is not a theoretical worry here. The gates that did run were What I need from you: either refill Codex credits so the pass can run, or waive the gate explicitly. I am not treating "no reviewer was available" as "no findings remain." Two other things worth your eyes, both in the PR body:
Checks: 7 pass, 16 skipped, 0 fail — including the corpus This comment is not an approval, and I am the author of the branch — nothing here is an independent verdict. |
#636) Three sibling agents authored corpus nodes following this document literally, and reported what broke. These four are the defects that produced real false claims in their work, not stylistic complaints. Filed as #1471; fixed here because #1462 is still open and 45 further nodes would inherit them. 1. THE ONLY-NODE JUSTIFICATION WENT FALSE, AND TWO AGENTS COPIED IT. "A node with no sibling to point at correctly declares none" read as a general rule. It was a fact about one moment -- when this was the corpus's only node -- and stopped holding as soon as a second existed. Both the #1309 and #1307 agents reproduced the reasoning and wrote a justification that was false at the time they wrote it; their adjudicators caught it, not their own reviews. Now says to enumerate what exists and give the real reason, and records why. 2. AN ISSUE-ONLY SOURCE HAD NO HONEST CLASS. The citation table lists tool results as legitimate while "Three things a passing run does not mean" says a FACT resting only on UNVERIFIED citations "has not been checked by anything", with one named exception. A claim whose only source is a GitHub issue has no openable form and cannot be SHA-pinned, so both passages applied and disagreed. The #1307 agent hit this with nothing honest to choose. Now states it: TEAM_KNOWLEDGE with provided_by naming the issue, which is what ADR-0029 requires of GitHub history anyway. 3. A RANGE WAS READ AS A MAPPING. The not-covered table paired eight subjects with "#1307-#1351", which is 45 issues and no subject-to-issue pairing at all. The #1307 agent cited this table for nine invented mappings -- exactly the "FACT cited to a file that does not discuss the claim" failure the document warns about two sections earlier. Now says outright that it is a range, not a mapping, and gives the search command. 4. CREATE STEP 3 ROUTED SOMETHING STEP 8 DID NOT ACCEPT. Step 3 sends expected-but-unverified items to the scope section; step 8 said only "state what the node does not cover". A boundary and a confidence disclosure are different things, so working the steps in order left that category homeless. Both the #1307 and #1309 agents reported it independently. Step 8 now names both. Verified: schema errors 0, 21 ledger entries, validator exit 0, 79 tests OK. Refs #636 #1471 Signed-off-by: Serina Mcfall <serina.mcfall@gmail.com>
serina-mcfall
marked this pull request as ready for review
August 26, 2026 06:15
#636) Two more defects reported by the #1308 agent after authoring against this document. Both are mine, and the first is embarrassing in context. 1. THE CITATION TABLE MISATTRIBUTED ITSELF, IN THE SECTION ABOUT NOT DOING THAT. It said "CONTRACT.md §3 defines the six shapes" above a SEVEN-row table whose GitHub-link and external-URL rows appear nowhere in §3. Measured: grepping that section for http/url/github returns zero hits. So the sentence cited a file that does not discuss two of the rows beneath it -- the exact failure this document warns about, committed by this document. The #1308 agent took the table as an accurate summary and built a scope argument on "four of six shapes name code" before their plan review caught it. Now states what §3 actually enumerates, that the two URL forms are the validator's and not §3's, and that the table is seven rows and not a summary. 2. "ADD RELATIONSHIPS ONLY TO NODES THAT EXIST" WAS UNFOLLOWABLE, AND DANGEROUSLY SO. "Exist" was ambiguous between the branch you are working on and the branch you merge into, and the checker loads whatever is present where it runs. An agent branched off this unmerged node could therefore target `corpus-agents`, validate clean locally, and break CI on `launchpad`, where the node is absent. The #1308 agent hit exactly that and their review-code raised it as a Blocker. Step 9 now says "exists on the branch you are merging INTO", explains why the distinction is the trap, and gives the command to check the merge base rather than the local tree. This is the second round of AGENTS.md fixes sourced from agents authoring against it, and the pattern is worth naming: every defect found this way was invisible to three review passes over the document itself, and obvious within minutes to someone trying to follow it. Reviewing a procedure is not the same as executing one. Verified: validator exit 0, 79 tests OK, schema clean. Refs #636 #1471 #1478 Signed-off-by: Serina Mcfall <serina.mcfall@gmail.com>
Author
Held in draft deliberately — the cross-model gate has not runDo not mark this ready for review yet. This is not an oversight. The review pipeline for corpus work under #605 requires a cross-vendor final pass. That No alternative external provider is installed ( Same-model reviews did run and their findings were addressed — see this PR's own Decision (Serina, 2026-08-26): hold every corpus PR in draft until credits are When credits return, the sequence is: run the cross-model final on this branch's current |
serina-mcfall
marked this pull request as draft
August 26, 2026 06:21
serina-mcfall
marked this pull request as ready for review
August 26, 2026 06:27
…task/1309-corpus-standard-confidence Signed-off-by: Serina Mcfall <serina.mcfall@gmail.com>
Author
Correction: the draft hold above is liftedMy earlier comment on this PR said "do not mark this ready for review". That no longer Serina marked this PR ready for review deliberately, superseding the earlier hold. I What is still true, and is the reason the earlier comment existed: The cross-vendor final review has not run on this branch. So this PR is review-ready with a known, recorded gap in review coverage, rather than When credits return, the cross-model pass on the current head is still worth running. |
tucktuck101
approved these changes
Aug 26, 2026
tucktuck101
left a comment
tucktuck101
left a comment
Collaborator
There was a problem hiding this comment.
Approved. The confidence standard is schema-consistent (cols 0.0-1.0 INFERENCE-only, NaN gap correctly scoped), ledger classes match node.schema.json, and the contested TEAM_KNOWLEDGE entry names a genuine external source (the #636 cross-model review). Note: merge #1462 first — this PR embeds the shared AGENTS.md base also carried by #1462. [panel: fable+codex]
This was referenced Aug 26, 2026
serina-mcfall
added a commit
that referenced
this pull request
Aug 26, 2026
launchpad gained launchpad/docs/corpus/AGENTS.md while this branch was open, via PRs #1468, #1469 and #1477 merging -- each carried the shared #636 base. That made an add/add conflict against the copy this branch had merged from task/636-corpus-agents-md. Resolved in favour of launchpad's copy (e422283), NOT this branch's newer one (8c6c21a). The newer copy carries #1462's fixes for its outstanding review findings. Those fixes belong to #1462 and are still under review; landing them through this PR would merge unreviewed changes to another PR's file under cover of an atomicity-standard change. This branch's own fix is unaffected either way: the "What the corpus has NOT settled about revisions" passage that the recast ledger entry cites is present in both copies, so the claim is true against whichever version ends up on launchpad. Signed-off-by: Serina Mcfall <serina.mcfall@gmail.com>
serina-mcfall
added a commit
that referenced
this pull request
Aug 26, 2026
…ed at (#636) Rebased onto `launchpad` and reduced to one file. #1468, #1469 and #1477 merged while this PR sat on CHANGES_REQUESTED, and each carried the shared #636 base, so two of this branch's three files -- the #636 plan and test_validate.py -- are now byte-identical on `launchpad` and no longer belong in this diff. What is left is the fix itself. The consequence worth stating plainly: those merges landed the UNFIXED corpus AGENTS.md on `launchpad`, including the line this commit removes -- "Do not take that on this document's word -- the command is the check". The High finding is on the default branch now, which is why this is a defect fix rather than a pre-merge correction. HIGH -- the provenance-update shortcut could certify stale evidence. Two passages said an empty `git diff --name-only <recorded-sha> -- <ledger paths>` established that every cited source was unchanged and every claim still stood. Both are false for citation forms this document itself permits. Measured, not argued. A file that DOES differ reports as changed with a bare path and reports empty output with exit status 0 once `:127` or `:219-221` is appended -- no error, no warning. So an unnormalized `path:line` citation makes a changed file indistinguishable from an unchanged one, which is the worst available failure shape for a provenance check. Recorded as a new FACT entry with the tool-result citation that demonstrates it. The rule now states its reach: normalize positions off the path first; only bare path, file line and file range are in scope; graph edge, tool result, commit and the two URL forms are excluded explicitly, so the branch is simply unavailable when a ledger contains one. The conclusion is narrowed to what a diff can support -- those files are unchanged -- with re-verification left as the only thing that establishes a FACT still holds. MEDIUM -- the one-node rule had no supporting ledger entry. "One node is one independently maintainable idea" is a substantive instruction and the ledger's only #636 entry backed the recorded-revision requirement instead. Added as TEAM_KNOWLEDGE attributed to #636's definition of done, which states it near-verbatim, matching the existing entry's classification for the same source. Also corrected a wrong-direction cross-reference: "*Updating a node* below" in Scope and omissions, where that section is above it. The recorded revision stays at 0052f5a, re-checked against the NEW base after the rebase rather than assumed to still hold: the two non-file citations were excluded, the nine normalized file paths were passed to `git diff` against 0052f5a from `launchpad` tip 0ffc1c9, and the result was empty. The second Medium is not addressed because it is stale -- the citation table was already corrected by ebe2daf and says outright that it is seven rows and is not a summary of CONTRACT.md section 3. The remaining cross-document disagreement is tracked as #1478. Evidence is in the PR thread. Validated against the full four-node corpus now on `launchpad`, not just this node: PASS, 5 non-fatal UNVERIFIED notices, all commit or tool-result citations that name no openable file. Signed-off-by: Serina Mcfall <serina.mcfall@gmail.com>
3 tasks
serina-mcfall
added a commit
that referenced
this pull request
Aug 26, 2026
Addresses all three findings from the review panel on PR #1480. Each verdict below was measured by calling validate._classify_citation directly rather than read off the document. High -- entry 15 and section 8 asserted as FACT that AGENTS.md "introduces its citation table as CONTRACT.md section 3's six shapes and then lists seven rows". The AGENTS.md this branch carries says the opposite in as many words: the table "is not a summary of section 3", because its two URL rows are forms validate.py recognises and section 3 does not enumerate. That correction landed in ebe2daf, after this node recorded its claim against 60d4947. The entry and section 8 now describe the AGENTS.md that actually ships beside them. Section 8 also now records what happened, because the node is a standard about citation discipline and this was a live instance of the failure it teaches against: a claim recorded against one revision of a moving document is not a claim about the document that finally ships. AGENTS.md records the same incident from its own side. The open CONTRACT.md-versus-validator divergence is unchanged and still points at #1478. Medium -- the ledger said a bare repository path "is opened on disk" while a later entry said a citation resolving to a real file "is never opened". Both cannot hold. Measured: a bare path returns ok on resolution alone and the file's contents are never read. The first entry now says resolved, and says explicitly that resolution establishes only that the file is there. Medium -- the ledger scoped the UNVERIFIED channel to "non-GitHub URLs", while the node's own table records GitHub issue and pull-request URLs as unverified. Measured: #1459 -> unverified #1462 -> unverified https://example.com/spec -> unverified .../blob/<full-40-sha>/path -> ok .../blob/launchpad/path -> error (mutable ref, ADR-0003) The entry now scopes the channel to external URLs that are not pinned repository links, and names GitHub issue and pull-request URLs as included -- which matches the table and the measured behaviour. Also merges origin/launchpad, which gained the corpus tree when #1468, #1469 and #1477 merged. No conflict: this branch already carried the same AGENTS.md blob. Verification: python3 launchpad/project-intelligence/corpus/validate.py PASS corpus validation found no errors; 5 item(s) reported unverified Signed-off-by: Serina Mcfall <serina.mcfall@gmail.com>
3 tasks
serina-mcfall
added a commit
that referenced
this pull request
Aug 26, 2026
, declare the corpus-agents edge (#1314) Four sentences asserted unattributed prevalence/effectiveness ("single most common," "almost never chosen," "single highest-yield," "nearly always available and nearly never taken") with no ledger entry backing them, conflicting with this node's own MUST 1. Recast each as normative guidance without the frequency claim, since no measurement exists to cite honestly. The citation-form table carried a "provisional copy" of #1308's three shape rules, which SHOULD 6 -- link another document's rules rather than restating them -- already forbids doing. Removed the copy; the table now distinguishes the verdicts without asserting what #1308 currently says. The merge-order section claimed AGENTS.md had not merged and that nine citations would fail on launchpad -- true when authored, false now: AGENTS.md landed via #1468/#1469/#1477 before this review. Re-ran the section's own reproduction command against current origin/launchpad (PASS, 0 errors, was FAIL/9 errors) and rewrote it as history rather than a live constraint. corpus-agents is therefore loadable on the merge target, so this node now declares relationships: [depends-on: corpus-agents] instead of stating merge order as the reason for none. corpus-standard-code-references (#1308, still unmerged) stays deferred. review-final finding (fable+codex panel), verified against branch head before fixing: none of this was closed, and the merge-order staleness extended past what the review flagged. Signed-off-by: Serina Mcfall <serina.mcfall@gmail.com>
serina-mcfall
added a commit
that referenced
this pull request
Aug 26, 2026
… and declare the corpus-agents edge (#1313) D1 required the six sections "in this order and no other" while the same sentence permitted additional sections between them -- a direct contradiction. Reworded to "in this relative order and none reordered among themselves," which states what was actually meant without banning the insertions the rest of the requirement allows. The enforcement section claimed "nothing automated enforces any requirement on this page, and nothing can." The evidence backing it only shows the current checker discards the body before any check runs -- it says nothing about whether section structure is inherently unenforceable. Dropped "and nothing can" and stated the distinction directly. The relationships-absence claim assumed no corpus node existed on launchpad; that was true when authored and is false now -- corpus-agents landed via #1468/#1469/#1477. Verified by materializing the current merge target and validating this node against it (PASS, 0 errors). Declared relationships: [depends-on: corpus-agents] and rewrote the prose as history rather than a live constraint. review-final finding (fable+codex panel), verified against branch head before fixing: none of this was closed. The Blocking finding on this PR (#1486's reading decision) is unresolved and left for Serina, per the review's own scope -- not addressed in this commit. Signed-off-by: Serina Mcfall <serina.mcfall@gmail.com>
serina-mcfall
added a commit
that referenced
this pull request
Aug 26, 2026
…r siblings reconcile (#1313) Serina decided #1486 on 2026-08-27: this node's built scope (a meta-standard for what a corpus standard document must itself be and contain) is the intended reading, and the four sibling standards drafted alongside it -- #1470, #1480, #1468, #1477 -- reconcile to it before merging, including the two (#1468, #1477) already merged onto launchpad. Cites the decision in scope-and-authority, states it as settled rather than provisional in Exceptions and escalation, corrects Scope and omissions' stale "pull requests are open" framing to name which two have already merged, and records the decision as a TEAM_KNOWLEDGE ledger entry attributed to #1486. This closes the Blocking finding from the review-final pass (fable+codex panel) that #1496 raised -- the node no longer publishes a binding meta-standard ahead of its authority being decided, because the authority is now decided. Signed-off-by: Serina Mcfall <serina.mcfall@gmail.com>
This was referenced Aug 27, 2026
serina-mcfall
added a commit
that referenced
this pull request
Aug 27, 2026
…ion-standard Per Serina's decision on #1486: every sibling standard drafted alongside #1313 reconciles to it, including nodes that had already merged before the decision was recorded. This one (#1309, PR #1468) merged 2026-08-26. D10 -- H1 was "# Standard: `confidence`", code-formatting the topic. Changed to "# Standard: confidence", matching the plain-text convention every other reconciled standard uses. D1/D3 -- the normative sections were named "Requirements" and "Guidance" rather than "MUST" and "SHOULD". Renamed both, and updated the two in-body cross-references that named the old section by name. Recorded the reconciliation as a ledger entry attributed to #1486. Signed-off-by: Serina Mcfall <serina.mcfall@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
launchpad/docs/corpus/standards/confidence.md, the canonical policy node for theconfidencefield on a corpus evidence entry. The schema already enforces that it is a number in a closed interval, required for INFERENCE only and forbidden on FACT and TEAM_KNOWLEDGE; what no schema can hold is what the number means, so the standard states that nothing calibrates it, that values are not comparable across authors, and that no confidence however high promotes an INFERENCE to a FACT. Its load-bearing section separates reasoning from evidence from dressing up a decision, which is the failure a cross-model review caught on #636 and which no check can catch.Related issue
Closes #1309
Issue type
Task
Agent provenance
claude-opus-5[1m]Objective
One corpus node stating what the
confidencefield means, when it is required, how an author picks a value, and what a reader is entitled to conclude from one.Impacted components
launchpad/docs/corpus/standards/confidence.mdlaunchpad/plans/2026-08-26-issue-1309-corpus-standard-confidence.mdNo generated corpus outputs changed; no generator exists yet.
launchpad/docs/corpus/AGENTS.mdis deliberately untouched.Approach and rejected alternatives
The document commits to concrete positions rather than hedging, because it is meant to be the normative exemplar for roughly 45 sibling standards and vagueness would propagate. Enum member lists and the schema's field-combination matrix are linked rather than copied, except for the
confidencerow itself and how the value is bounded — a standard about this field cannot omit the field's own rule. Those unavoidable restatements are named in a drift-surface table in the header, so a maintainer changing the schema knows exactly which passages track it.Rejected: declaring a
referencesrelationship tocorpus-agents. It would validate today —AGENTS.mdis a loaded node — but every sibling standard (#1307–#1351) is unmerged, and arelationships[].targetnaming an unloaded id is a hard validation error. Adding the whole edge set in one pass once the siblings land is cheaper than adding one edge now and revisiting. The body states that reason explicitly and namescorpus-agentsas the first edge to make.Rejected: fixing the NaN gap in
node.schema.json.schema/sits outside this issue's scope and its Definition of Done forbids a second authored document. Filed as #1463 instead, with the standard carrying a MUST for real numbers in the interim.Rejected: a calibrated numeric scale. Nothing in the repository records whether a past inference turned out correct, so any calibration claim would be the exact overclaim this document exists to prevent. The three band pegs are declared a convention with no evidence behind them.
Verification
Command run:
Raw output:
The single
FAILline inside the corpus run is a negative-path test's own stdout, not a failing test; the suite reportsOKfor all 79. The twoUNVERIFIEDnotices are the two nodes' provenance commit citations, which no file can corroborate — one is this node's, and it is the one permitted commit-only FACT.The four technical claims the document asserts as FACT were each measured rather than read, by probing
node.schema.jsonandmemory.pydirectly:Not verified
ERROR: Your workspace is out of credits.Retried once to confirm it was not transient. No alternative external CLI is installed (gemini,agy,pi,llm,ollamaall absent).review-finalwas run instead and found nine findings, all now fixed — but it is the same model family as the author and is not a substitute. The gate this branch's plan declares mandatory is still open, and the memory notefeedback_final-review-cross-modelrecords that three same-model passes on task: treat all pull request content as untrusted data #120 missed what one Codex pass caught immediately. This PR should not leave draft until a cross-model pass runs.check-ledger.shexits 1 because this branch was built directly rather than throughbuild-change, so noprogress.mdwas written. The reviews demonstrably ran — each is named with its findings in the commit messages — but the mechanical check fails for want of a record, not for want of a review.confidencevalue, because no generator exists. How a projection renders or ranks these numbers is unknown, and the "not comparable" rule has not been tested against a consumer that might assume otherwise.memory.pyonly. Whether any other consumer of the field shares the schema's permissiveness was not established.AGENTS.mdis resolved as governing instructions; whether this file's front matter degrades it for such a harness is untested.Security implications
None. The change adds two Markdown documents and no executable code, no configuration, no dependency and no network surface. It does not alter what
validate.pyaccepts, so it cannot widen what merges. One adjacent observation: the document's honesty rules reduce a real risk ADR-0029 names — presenting inference as established fact — by requiring that a decision never be dressed as a derivation.Escalations
bug: a NaN confidence passes corpus validation but memory.py rejects it. Raised while measuring the constraints this document describes. Deferred rather than fixed becausenode.schema.jsonlives underschema/, outside this issue's scope, and the Definition of Done forbids materially editing a second document. The standard carries a MUST for real numbers and an enforcement-table row naming the gap until bug: a NaN confidence passes corpus validation but memory.py rejects it #1463 lands.audiencesisagentandreviewer, notdeveloper. The task brief called this question deliberately unsettled. An author picking a number acts as agent, a person checking it acts as reviewer, and the document addresses developers only through those roles. Stated so a reviewer can overturn it cheaply; the node itself records nothing about the question, so a sibling author will not know it was ever open.review-finalwere fixed on this branch rather than filed, against the standing policy that High and Medium findings become issues. All nine were self-contradictions inside this new document — including a false justification for declaring no relationships, a claim that three requirements were machine-enforced when two are, and an overbroad claim that the schema does not enforce finiteness when only NaN escapes. Filing issues against an unmerged draft's own internal consistency would have shipped a standard that contradicts itself seven ways. Flagging the deviation rather than assuming it was authorised.