Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
ba04aea
feat(mining): P3-15'-c-ii — the criterion (c) synteny predicate, its …
bioedcam Aug 7, 2026
4b49352
fix(mining): P3-15'-c-ii review round 1 — the tandem carve-out now re…
bioedcam Aug 7, 2026
b61fd3a
fix(mining): P3-15'-c-ii review round 2 — the 5'UTR false-pass arm co…
bioedcam Aug 7, 2026
c7beb0a
fix(mining): P3-15'-c-ii review round 3 — a filtered subset that outn…
bioedcam Aug 7, 2026
b6234ec
fix(mining): P3-15'-c-ii review round 4 — "the same windows, filtered…
bioedcam Aug 7, 2026
b9aefb4
test(mining): P3-15'-c-ii review round 5 — assert object identity, no…
bioedcam Aug 7, 2026
1f9e3ab
fix(mining): P3-15'-c-ii review round 6 — the sub-threshold test used…
bioedcam Aug 7, 2026
3991fa2
fix(mining): P3-15'-c-ii review round 7 — ProducerError was not the f…
bioedcam Aug 7, 2026
a5b1fd0
fix(mining): P3-15'-c-ii review round 8 — duplicate-key collapse on t…
bioedcam Aug 7, 2026
cd3c01d
fix(mining): P3-15'-c-ii review round 9 — the false-pass rate divided…
bioedcam Aug 7, 2026
2f9563d
fix(mining): P3-15'-c-ii review round 10 — the positive control publi…
bioedcam Aug 7, 2026
bd493ba
fix(mining): P3-15'-c-ii review round 11 — a bare gene symbol suppres…
bioedcam Aug 7, 2026
24fa382
fix(mining): P3-15'-c-ii review round 12 — one bad GFF among 339 woul…
bioedcam Aug 7, 2026
9069d95
fix(mining): P3-15'-c-ii review round 13 — round 11's guard keyed on …
bioedcam Aug 7, 2026
ee9108c
test(mining): P3-15'-c-ii review round 14 — a one-class assertion gua…
bioedcam Aug 7, 2026
65f64af
fix(mining): P3-15'-c-ii review round 15 — a required parameter that …
bioedcam Aug 7, 2026
9b7a4e9
fix(mining): P3-15'-c-ii review round 16 — round 12 guarded one of tw…
bioedcam Aug 7, 2026
14a66b7
docs(dev-log): correct the round-16 pass count — 4,683, not 4,687
bioedcam Aug 7, 2026
bb76e3a
fix(mining): P3-15'-c-ii review round 17 — strided sharding defeated …
bioedcam Aug 7, 2026
231ffde
docs(dev-log): add the round-17 stanza dropped by a short-circuited c…
bioedcam Aug 7, 2026
3cf5e7b
test(mining): P3-15'-c-ii review round 18 — three instances of "fixed…
bioedcam Aug 7, 2026
be43605
docs(dev-log): P3-15'-c-ii review round 19 — no new findings, the gat…
bioedcam Aug 7, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
101 changes: 101 additions & 0 deletions analyses/phase3_log.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -3941,3 +3941,104 @@ shared predicate, since breaking only the predicate would leave a per-clause rep
Full unit+golden **4,490 passed / 15 failed** vs the matched baseline at `2be5ff4`
(**4,240 / 15**) — failure sets identical, `comm` empty both ways. Counts across nine rounds:
**9, 2, 4, 5, 5, 5, 1, 2, 1**.

## Step P3-15′-c-ii — 2026-08-07: the criterion (c) predicate, its per-candidate producer, and D4's two symmetric diagnostics

**Inputs.** `data/processed/mining/round0_fp_manifest.json` (941 candidates / 76 hosts);
`data/interim/production_annotations/*.gff.gz` (339, md5-verified at `-c-i`);
`data/interim/production_genomes/*.fna` (the contig-id bridge);
`data/processed/mining/production_genomes_v0.parquet` (GTDB clade);
`data/processed/negatives/{decoys,mining_pool}_v0.parquet` (the §9.1 arm).
**Process.** `mining/synteny.py` implements ADR-0006 D4's frozen `criterion_c` verbatim plus
the downstream walk, the tandem carve-out and the gene-identity vocabulary D4 delegates here;
`mining/synteny_producer.py` writes `$ROUND_DIR/synteny_status.json` (`candidate_id → status`,
the (a)-producer's shape) and the two committed diagnostic reports. `SYNTENY_SUPPLY_AVAILABLE`
was added beside `MSA_SUPPLY_AVAILABLE`, `candidate_evidence` / `remine_candidate_evidence`
gained the synteny status map with the same fail-closed absent-id rule, and `--synteny-available`
was promoted from a one-way `store_true` to the two-way pair with an exit-4 unevidenced preflight.
**Outputs.** `src/tbox_finder/mining/{synteny,synteny_producer}.py`;
`reports/p3/synteny_false_pass.json`; `reports/p3/synteny_exclusion_diagnostic.json`;
`tests/unit/test_synteny{,_producer}.py` (+112 tests).
**Validation.** 941/941 statuses (541 passed / 101 failed / 299 unavailable); the false-pass
control is **powered** (margin 0.975 ≥ 0.20); full unit+golden **4,604 passed / 13 failed**
against a matched baseline worktree at `95d0149` (**4,492 / 13**) — failure sets identical,
`comm` empty both ways; **19 sabotages / 19 RED against their named test**.
**Decisions.** ADR-0006 D4 (vocabulary delegated here, §10.1-verified); ADR-0005 D14 (fail-closed
`unavailable` ⇒ spared), D15 (both-strand carry-through).
**Commit.** see below.

**The criterion discriminates, and by a wide margin.** Candidates pass at **57.5 %** against a
clade-matched random-leader background of **1.67 %** and a §9.1 5′UTR-decoy background of
**4.52 %** — a 13–34× enrichment, with a positive-context arm at **99.20 %** proving the control
is powered rather than merely quiet. The reading is not that the criterion is loose: the
round-0 pool is Stage-1 *false positives* drawn from a masked negative substrate, and a 96-nt
element scoring 0.9977 that sits 19 bp upstream of an `alanine--tRNA ligase` is most plausibly
an uncatalogued T-box. ADR-0005 D14's spare rule exists precisely so those are spared rather
than trained against, and 541 of them now are.

**⚠ THE BIGGEST DEFECT WAS FOUND BY DISBELIEVING A NUMBER, NOT BY A TEST.** The first run
returned **541 of 941 passing** and the per-CDS background density of D4's four classes is
**4.14 %** — an irreconcilable gap. The cause was that the downstream walk bounded the distance
only from above: a bacterial genome is ~88 % coding, so a window buried inside a long gene found
that gene as its "downstream" gene and inherited its identity. D4's class-II case genuinely
needs a *negative* distance (a translational T-box abuts/overlaps the start codon), so the fix
could not be "reject overlap"; it is that the CDS start must lie at or after the **element's own
5′ end**, which pins no new number. The corpus-level counts barely moved because Stage-1
candidates land in leader regions rather than inside genes — so the number that exposed the bug
was not the number the bug corrupted.

**⚠ TWO DEFECTS THE DIAGNOSTIC FOUND IN ITSELF, AND BOTH WERE SILENT ZEROS.** (1) The pseudogene
diagnostic reported **0** pseudogenized first-downstream ORFs corpus-wide, because the tandem
carve-out hops exactly the population the diagnostic exists to size; the counts now accumulate
across the whole walk (15 pseudogenized, 88 unjudgeable). (2) D4's hypothetical/pseudogene route
was **entirely unreachable**: `gff3.gene_identity_text` collects `Dbxref`/`Name` as well as
`product`, and on NCBI CDS those are accessions (`Dbxref` is present on 29,852 of 30,273
measured CDS), so every `hypothetical protein` still had non-empty "identity text", never
resolved to unjudgeable, and was scored a *decided* `failed` instead of being routed to the
fallback. Identifier-shaped values are now filtered, with a positive control asserting the
filter removes identifiers rather than removing judgement.

**⚠ THE VOCABULARY IS PRECISION-FIRST BECAUSE THE PERMISSIVE VERSION WAS MEASURED FIRST.** An
"amino-acid word + enzyme word" rule looks reasonable and swallows `HAMP domain-containing
histidine kinase` (38 CDS over 12 hosts, two-component signalling), `serine/threonine protein
kinase`, `glutamine amidotransferase` (purine/His/Trp — **not** tRNA-dependent transamidation),
`methylated-DNA--[protein]-cysteine S-methyltransferase` (DNA repair) and `peptide-methionine
(S)-S-oxide reductase` (protein repair). Every `EXCLUSIONS` entry was observed on this corpus
before it was written down, exclusions are applied **before** every include rule, and the
transamidation rules all require the literal `tRNA`. Two ordering corrections came out of the
audit: the curated product name outranks the gene symbol (`hisJ` is the histidine transporter's
binding protein in the enterics and **histidinol phosphatase** in Firmicutes), and the symbol
route fires only when the product name says nothing at all — consulting it otherwise put `PTS
galactitol transporter subunit IIB` into the transamidation class via `gatB`, `cytochrome C-552`
into transport via `cycA`, and `homocitrate synthase` into aaRS via `lysS`. 24 of 4,991 matches
came from that route and 3 of the 24 were wrong.

**⚠ THE CONTIG BRIDGE IS BY IDENTITY, AND AN INDEX JOIN WOULD HAVE BEEN WRONG ON HALF THE
CORPUS.** A candidate's contig is `:c<ci>`, an index into the genomic FASTA's record order; a
GFF's `seqid` is the NCBI contig accession. **23 of the 48** candidate-carrying annotated hosts
serve their GFF seqids in a different order than their FASTA records, so `records[ci]` ↔
`seqids[ci]` would silently read the wrong contig on ~48 % of hosts. The id join resolves
**134 of 134** needed (host, contig) pairs.

**Three gaps reported rather than papered over.** (1) The **shipped §9.1 decoy rows carry
coordinates on 0 annotated production hosts** — `decoys_v0`'s three non-structured pools have no
accession at all, its 2,999 `structured_rna` accessions are Rfam source contigs, and
`mining_pool_v0`'s 45,988 leader/trail windows sit on the TBDB source genomes — so that arm is
`available: false` **with its counts**, and the §9.1 leader class is re-instantiated inside the
annotated corpus instead (5′UTR and tRNA-adjacent windows). (2) The **false-FAIL arm has almost
no positive denominator**: only **2 TBDB records on 1 assembly** intersect the 339, because the
production substrate is by construction a negative one. (3) The **joint a ∧ b ∧ c arm is
structurally unavailable** — (b) has no backend until P3-15′-d — and is withheld, never computed
from two unavailable disjuncts.

**Per-clade exclusion is the clade-structured cap D4 commissioned, and it is large.** 31.8 %
of candidates are `unavailable` (286 host-unannotated, 11 unjudgeable, 2 contig-absent), with
five phyla at **100 %** exclusion (Chloroflexota, Desulfobacterota, Fibrobacterota, Thermotogota,
Bdellovibrionota_C), Bacillota at 38.5 %, and Atribacterota / Sysuimicrobiota / Fusobacteriota at
0 %. That is the recall cap the commissioned bakta/prokka arm exists to lift. Empirical passing
distances: p50 117 bp, p95 269 bp, p99 364 bp, max 454 bp — all inside D4's 500 bp pad, which is
the sensitivity check D4 asks for.

**⚠ `n_mined` IS STILL STRUCTURALLY 0.** Sparing is a disjunction so mining is a conjunction:
`relaxed_architecture` remains the one unavailable backend, so `max_possible_yield` still caps
the round at 0 until P3-15′-d lands. This step removes one of the two blockers, not both.
274 changes: 274 additions & 0 deletions reports/p3/synteny_exclusion_diagnostic.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,274 @@
{
"adr": "ADR-0006 D4 (criterion (c)); ADR-0005 D14 (spare rule), D15 (strand)",
"config": {
"hmm_fallback_available": false,
"max_intervening_orfs": 1,
"strand_policy": "both",
"sub_threshold_orf_nt": 150,
"window_bp": 500
},
"exclusion_rate": 0.3177470775770457,
"exclusion_reason_totals": {
"contig_absent_from_gff": 2,
"first_downstream_orf_unjudgeable": 11,
"host_unannotated": 286
},
"false_fail_probe": {
"available": false,
"reason": "the production substrate is a NEGATIVE substrate (host-order-admissible windows masked of known T-boxes), so it carries essentially no catalogued T-box loci to measure a false-FAIL RATE on; the annotation-driven failure mode is carried by the positive-context arm of the false-pass report"
},
"n_candidates": 941,
"n_unavailable": 299,
"passing_class_counts": {
"aa_biosynthesis": 220,
"aars": 332,
"transamidation": 2
},
"passing_distance_sensitivity": {
"max_bp": 1652,
"n": 554,
"note": "D4 asks for the empirical p95/p99 as a sensitivity check on the 500 bp pad; these are measured on the passing candidates of THIS corpus, not on catalogued Firmicutes T-boxes",
"p50_bp": 117,
"p95_bp": 272,
"p99_bp": 389,
"window_bp": 500
},
"per_clade": {
"Armatimonadota": {
"exclusion_rate": 0.6571428571428571,
"n": 35,
"reasons": {
"host_unannotated": 23
},
"status_counts": {
"passed": 12,
"unavailable": 23
}
},
"Atribacterota": {
"exclusion_rate": 0.047619047619047616,
"n": 126,
"reasons": {
"first_downstream_orf_unjudgeable": 6
},
"status_counts": {
"failed": 48,
"passed": 72,
"unavailable": 6
}
},
"Bacillota": {
"exclusion_rate": 0.3848396501457726,
"n": 343,
"reasons": {
"host_unannotated": 132
},
"status_counts": {
"failed": 23,
"passed": 188,
"unavailable": 132
}
},
"Bacillota_I": {
"exclusion_rate": 0.4444444444444444,
"n": 54,
"reasons": {
"first_downstream_orf_unjudgeable": 2,
"host_unannotated": 22
},
"status_counts": {
"failed": 13,
"passed": 17,
"unavailable": 24
}
},
"Bdellovibrionota_C": {
"exclusion_rate": 1.0,
"n": 2,
"reasons": {
"contig_absent_from_gff": 2
},
"status_counts": {
"unavailable": 2
}
},
"Chloroflexota": {
"exclusion_rate": 1.0,
"n": 38,
"reasons": {
"host_unannotated": 38
},
"status_counts": {
"unavailable": 38
}
},
"Deinococcota": {
"exclusion_rate": 0.2857142857142857,
"n": 42,
"reasons": {
"host_unannotated": 12
},
"status_counts": {
"failed": 3,
"passed": 27,
"unavailable": 12
}
},
"Desulfobacterota": {
"exclusion_rate": 1.0,
"n": 7,
"reasons": {
"host_unannotated": 7
},
"status_counts": {
"unavailable": 7
}
},
"Dictyoglomota": {
"exclusion_rate": 0.2549019607843137,
"n": 102,
"reasons": {
"host_unannotated": 26
},
"status_counts": {
"failed": 4,
"passed": 72,
"unavailable": 26
}
},
"Fibrobacterota": {
"exclusion_rate": 1.0,
"n": 1,
"reasons": {
"host_unannotated": 1
},
"status_counts": {
"unavailable": 1
}
},
"Fusobacteriota": {
"exclusion_rate": 0.0,
"n": 6,
"reasons": {},
"status_counts": {
"failed": 1,
"passed": 5
}
},
"Hadarchaeota": {
"exclusion_rate": 0.0,
"n": 1,
"reasons": {},
"status_counts": {
"failed": 1
}
},
"Hinthialibacterota": {
"exclusion_rate": 0.0,
"n": 4,
"reasons": {},
"status_counts": {
"passed": 4
}
},
"Nitrospirota": {
"exclusion_rate": 0.4444444444444444,
"n": 18,
"reasons": {
"host_unannotated": 8
},
"status_counts": {
"passed": 10,
"unavailable": 8
}
},
"Spirochaetota": {
"exclusion_rate": 0.39285714285714285,
"n": 28,
"reasons": {
"host_unannotated": 11
},
"status_counts": {
"passed": 17,
"unavailable": 11
}
},
"Sumerlaeota": {
"exclusion_rate": 0.3333333333333333,
"n": 6,
"reasons": {
"host_unannotated": 2
},
"status_counts": {
"passed": 4,
"unavailable": 2
}
},
"Sysuimicrobiota": {
"exclusion_rate": 0.02586206896551724,
"n": 116,
"reasons": {
"first_downstream_orf_unjudgeable": 3
},
"status_counts": {
"failed": 8,
"passed": 105,
"unavailable": 3
}
},
"Thermodesulfobiota": {
"exclusion_rate": 0.0,
"n": 8,
"reasons": {},
"status_counts": {
"passed": 8
}
},
"Thermotogota": {
"exclusion_rate": 1.0,
"n": 4,
"reasons": {
"host_unannotated": 4
},
"status_counts": {
"unavailable": 4
}
}
},
"provenance": {
"adr": "ADR-0006 D4 (criterion (c)); ADR-0005 D14 (spare rule), D15 (strand)",
"env_lock_hash": null,
"extra": {
"hmm_fallback_available": false,
"max_intervening_orfs": 1,
"step": "P3-15'-c-ii",
"strand_policy": "both",
"sub_threshold_orf_nt": 150,
"window_bp": 500
},
"generated_at_utc": "2026-08-07T02:32:38.019777+00:00",
"git_sha": "95d0149449458d793cec31018aab650c5d1ed063",
"inputs": {
"/home/bioedca/.claude/jobs/b0674bf6/tmp/round0/synteny_status.json": "fcd2ce0a841dd9d67bdc4a72d3bab1cd06773e9bcaa147380e6596eb781e1b93",
"data/processed/mining/production_genomes_v0.parquet": "e52b34b79256d16a8cf9986cdc8e1d0d5de229ee43562b5a6ddb9528e7109f22"
},
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved
"outputs": {},
"rule": "synteny_producer::diagnostics.exclusion",
"schema_version": "1.0",
"script": "src/tbox_finder/mining/synteny_producer.py",
"seed": 42
},
"pseudogene_diagnostic": {
"consequence": "D4 routes a pseudogenized or hypothetical first-downstream ORF to targeted Pfam/KO profiles. That profile database is an unmet \u00a710.2 acquisition, so these loci resolve 'unavailable' and ADR-0005 D14 SPARES them \u2014 the fail-closed direction. The count above is the size of the recall the missing fallback costs.",
"hmm_fallback_available": false,
"n_pseudogenized_orfs_encountered": 15,
"n_unjudgeable_orfs_encountered": 88
},
"schema_version": "1.0",
"step": "P3-15'-c-ii",
"tandem_carve_out": {
"max_intervening_orfs": 1,
"n_strand_evaluations_using_carve_out": 73,
"sub_threshold_orf_nt": 150
}
}
Loading
Loading