feat: the truth gate — the DJ stops lying (T329–T335) - #609
Merged
Conversation
…as amended) Pure static checker beside SpeechText (F68.6 posture): digit-run / weekday / condition-word extraction with whole-token fact support (decimal-prefix allowance kept deliberately), and present-frame clock claims — weekday under a closed marker set, daypart under greeting/ copula, overlapping daypart windows, track-title exemption, both apostrophe forms. Vocabulary v2 (clear dropped — its non-weather sense dominates DJ prose). Violations are closed-vocabulary data safe for re-ask prompt interpolation (reviewer-verified fence-forging-free). When in doubt, PASSES: anticipation, recall, and displaced references are the bread of DJ patter and never violate. Review: 3 opus rounds + 1 confirm — round 1's probes showed bare-token matching rejecting 5/10 realistic patter lines (SPEC F138.2/.3 amended to present-frame semantics, the F135.5 precedent), the substring digit rule unfalsifiable against date-bearing facts, and clear firing on 'let me be clear'; round 3 caught U+2019 disabling the it's marker — the aired-defect shape passing clean. Both gh-#434 and gh-#438 exhibits pinned in their real forms. Wiring lands in T331/T332.
…3 as amended) Eight typed causes (MalformedResponse added at review — a 12-line mistagged script is not an 'empty completion', and the model-floor signal depends on the distinction) stamped where calls resolve, decided once at the source: CleanCopy's fit/over-length/empty split, the timeout-vs-connection classifier, crosstalk's parser rejects, and the worker's window cancellation (reusing T328's GenerationAttempted signal). LlmCallRecorder (required dep) is the one seam feeding both the F73 ring and the new (cause, model, kind) counters — hourly-bucket 24-25h band, TimeProvider-driven, nothing persists (F73.3 stands). Ring entry grows Cause + Model additively; Story196's ctor-arity proof untouched. Rides along (ruled in at review): Story317's date bomb defused (hardcoded 2026-08-20 red every run from today); the crosstalk test harness no longer leaks a temp dir per build (69k dirs had inode-choked tmpfs, silently redding 160+ unrelated facts). Review: 3 opus rounds — the malformed-shape fold rejected (SPEC F139.1 amended), the optional-counter-param pattern rejected mutation-proven (deleting every counter feed left the suite green; the recorder makes it structurally undeletable), the 24h retention letter corrected to the honest band. Surface lands in T334; TruthGateReject stamps in T331.
CopyClaims.CheckFacts wired at the one completion seam for context
segments with a non-empty fact block: a violating first reply records
TruthGateReject and triggers exactly ONE re-ask naming the claim
('Your last reply stated "6" and "sunshine" but the facts above never
said that…' — comma-free, fence-safe by T329's closed-vocabulary
guarantee), sharing the first call's timeout clock so the worst-case
feeder hold never grows; a still-violating re-ask lands on the
F107.6 floor (the ContextSegment skip) via a new closed-hierarchy
TruthGateRejected arm whose WARN names the real cause — never the
MaxCopyChars lever. Both calls get honest ring entries (own start, own
prompt, own cause). The F138.5 guard line rides EVERY patter prompt
('It is {weekday} {daypart}. Never name another day or time of day.').
Admin previews are structurally ungated (no ContextFacts by
construction — documented). The ladder is an extracted choreography
T332's clock check composes into.
Review: 3 opus rounds — r1's mutations caught the budget fact vacuous
(a fresh re-ask clock survived 729 facts), the guard line pinned on one
kind only, the exhausted-ladder WARN lying about its lever, and
unpinned ring attribution; all closed mutation-red. gh-#434's exhibit
airs clean end-to-end: poisoned reply → re-ask → honest second reply.
CheckTruthGate composes facts and clock into ONE ladder run — the facts half scoped by its own null-guard to context segments, the clock half unconditional on every LLM-authored kind including previews (the factless-context exemption was deleted at review: reachable via the preview endpoint, and a missing fact block excuses nothing about a wrong-day claim). Rejection wording is honest per violation class (unsupported claim / wrong-day claim / wrong-time-of-day claim) in both the WARN and the re-ask, keyed on one discriminator (ClaimViolation.IsClockClaim). Floors verified per kind: context/ sign-off/sign-on drop (F107.6/F92 skip), lead-in/back-announce land on the template whose fixed prose names no day. Preview failures name the truth gate, never 'empty or over-length' (the wrong-lever class, kept out this time). gh-#438's exhibit airs clean end-to-end: 'Saturday morning… Tonight' at Sunday 11:50 re-asks once and the honest reply airs. Review: 2 opus rounds + confirm — r1 caught the reachable exemption, the preview wrong-lever message, zero preview coverage, and a surviving log-wording mutant; all closed mutation-red. Tts 739/0.
…as amended) The crosstalk validator's fail-closed discard gains five mechanical truth checks (after shape validation — structure passed, content lied): frequency shapes incl. integer FM dial positions (review caught 'Radio 101 FM' airing under a decimal-only rule; '9 AM' the clock time still passes), K/W call signs, weather condition words (the shared CopyClaims matcher — no second list), date shapes, and whole-script clock claims via CheckClock at the request's own instant. All stamp TruthGateReject — the honest per-model signal for 'this model invents broadcast facts'. The narrow anti-fabrication clause plus the F138.5 clock guard line now ride the banter system prompt (review: banter was the one kind clock-checked with no ladder AND no prompt rule — the model deserved to be told). Fictional lore passes untouched: recurring characters and running gags validate clean by construction. Crosstalk:DurationTargetSeconds default 25→50 (the ratified live-convergence figure), coherent through the F123.1 cap derivation and the settings help text. Review: 2 opus rounds + confirm — integer-FM hole, the missing guard line, and a stale 'Defaults to 25' on the deployed settings page; all closed probe-proven. Tts 751/0.
…9.2/.4)
GET /api/llm-calls returns {calls, causeSummary}: per-call cause+model
plus the unscoped 24h (cause, model, kind) counters — the crosstalk
lane visible without SSH. /api/status's llm block gains the dominant
failure triple via LlmCallCauseCounters.DominantFailure(Copy) —
honestly scoped to the one lane whose verdict the tile reflects
(LlmCopyStatusHolder has exactly one writer), riding the existing
status poll (no new poller — the gh-#558 lesson). The red tile renders
'Red: 6 timeouts in the last 24h, gemma3:12b' beneath the failure
line; green tiles never explain a fault that didn't cause them. The
llm-calls feed gains a cause chip. Success is never a candidate
(pinned with Success in the outright majority), ties break
deterministically (enum order then model — direction inversion-pinned),
and a Success-only day yields the null triple on both halves.
Review: 2 opus rounds + confirm — both invariants were mutation
survivors ('Red: 200 success' was live-reachable); closed spec-side.
Shared LlmCompletionsStub extracted to Support/ for T335. gh-#365's
acceptance: no SSH, no Loki, no darts.
…F139 build) Story350_TruthLaneEndToEnd drives the production DI graph: a poisoned context render re-asks once and airs the clean reply with both causes on GET /api/llm-calls; a double-poisoned lead-in degrades to the template with /api/status naming truthgatereject as the dominant cause; the crosstalk lane's counters visible on the unscoped summary; the F138.5 guard line asserted on every captured wire prompt. Three discriminating review mutations across gate, endpoint projection, and tile scoping. LlmCompletionsStub grows call-sequenced replies + request capture, additively. Live smoke (real Kestrel + real Postgres + scripted stub LLM through the deployed preview endpoint): one poisoned reply re-asked once and the clean text returned; both-poisoned exhausted the ladder to HTTP 502 'the truth gate rejected the re-ask too (wrong-day claim: Saturday)' — the honest lever, live; causeSummary counted 3 truth-gate rejects for the stub model. Review: 1 opus round, PASS.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The gate cluster of the truth-lane epic (SPEC F138–F139, designed 2026-08-20), built under /build-loop with opus review gates — 17 review rounds across seven tasks, every finding closed mutation-red. Builds on #605 (the mechanics lane).
Closes #434. Closes #365.
What ships
F138 — the truth gate. A pure
CopyClaimschecker (present-frame collocation semantics — anticipation, recall, and displaced references never violate; both aired exhibits do, in their real curly-apostrophe forms) wired at the one LlmCopyWriter seam: a violating reply gets exactly ONE re-ask naming the claim, sharing the first call's timeout budget, then the kind's own floor (context skips per F107.6; lead-ins land on a template whose fixed prose names no day). The F138.5 guard line rides every patter prompt including banter. Crosstalk gains five mechanical verifiable checks (integer + decimal FM, AM frequencies vs clock times, call signs, weather, dates, whole-script clock) through F127.4's fail-closed discard — fictional lore passes untouched.Crosstalk:DurationTargetSecondsdefaults to the ratified 50s.F139 — red explains itself. Eight typed causes (
MalformedResponseadded at review — a mistagged 12-line script is not an "empty completion") stamped where calls resolve, fed through one requiredLlmCallRecorderinto the F73 ring + 24h (cause, model, kind) counters.GET /api/llm-callsreturns{calls, causeSummary}; the dashboard's red LLM tile names its dominant recent cause and model ("Red: 6 timeouts in the last 24h, gemma3:12b"); the llm-calls feed gains a cause chip. Success is never a candidate, ties are deterministic — both inversion-pinned.The wire proof
Story350_TruthLaneEndToEnddrives the production DI graph end to end, and the live smoke ran through the deployed preview endpoint on a real Kestrel + Postgres + scripted stub LLM: one poisoned reply re-asked once and the clean text returned; both-poisoned exhausted the ladder to an honest502 — the truth gate rejected the re-ask too (wrong-day claim: Saturday); the cause summary counted every reject against the stub model.Spec honesty along the way
Four SPEC/STORIES amendments were forced by review evidence (the F135.5 precedent): F138.2/.3 narrowed to present-frame semantics after live probes showed bare-token matching rejecting half of realistic patter; F139.1 gained its eighth value; the counters' letter corrected to the honest 24–25h band; AC6 reconciled to the F107.6 skip floor.
Verification
Final full run: 4,418 .NET tests + 1,117 jest, zero failures, zero warnings, fitness laws green. 30+ review mutations across the epic, each now red against a named fact. gh-#438 closes after Dean's bench week (T336 — reading the per-model gate counters on the 4090 and recording the model floor in HARDWARE.md).