Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@ All notable changes to TEPP are documented here. The format follows Keep a Chang
- `psychometric_core` posterior-aware structural input gates: construct classification, refusal of raw-proportion Pearson/OLS, explicit ALR-versus-ILR geometry boundaries, CPU `f64` OLS recovery, posterior-draw loading point-estimate averaging without Rubin uncertainty claims, invariance-gated latent-mean comparison, and causal-heuristic refusal (ADR 0005 first production slice; no new migration).
### Added

- `tepp_simulation` cutoff eligibility: delayed-reporting documents are excluded when `available_time` exceeds the knowledge cutoff, with recovered eligible counts matching known truth.
- `topic_measurement` bounded deterministic CPU `f64` TRSL-TM reference estimator: canonical CSR/CSC inputs, cutoff-safe documents, standardized event time, weighted multiple memberships, prevalence covariates, explicit predecessor/successor regularization, multi-seed generalized EM, diagonal Laplace uncertainty, and fitted topic-lineage counts with known-truth RMSE plus exact line/branch coverage (ADR 0012; no persistence or accelerated-backend claim).
- `topic_measurement` logistic-normal additive log-ratio and sequential Egozcue isometric log-ratio coordinates: fail-closed simplex validation, max-shifted stable ALR/ILR inverses with true-parameter RMSE, pairwise CLR Aitchison distance recovered by ILR Euclidean isometry for valid composition pairs, and refusal of TF-IDF/BM25/keyword scores as inferential topic coordinates (ADR 0012 first production slice; no new migration).
- Coverage contract now excludes Rust multiline string continuation records emitted by LLVM LCOV, keeping the 100% authored-line gate focused on executable production lines.
Expand Down
5 changes: 4 additions & 1 deletion DOCUMENTATION.md

@devin-ai-integration devin-ai-integration Bot Aug 25, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 Documentation map file is duplicated end-to-end

DOCUMENTATION.md contains the whole map twice (headers at line 1 and line 83). This predates the PR, but the PR edits only the first copy, so the two now diverge. Worth consolidating.

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ TEPP's approved PRD v0.4 and implementation plan are the primary product baselin
| Hourly NIM product-development operations | [`docs/operations/HOURLY_NIM_PRODUCT_DEVELOPMENT.md`](docs/operations/HOURLY_NIM_PRODUCT_DEVELOPMENT.md) |
| Actions workflow fleet audit | [`docs/operations/ACTIONS_WORKFLOW_FLEET.md`](docs/operations/ACTIONS_WORKFLOW_FLEET.md) |
| Actions fleet research doctoring | [`docs/research/actions-workflow-fleet.md`](docs/research/actions-workflow-fleet.md) |
| Simulation cutoff-eligibility doctoring | [`docs/research/simulation-cutoff-eligibility.md`](docs/research/simulation-cutoff-eligibility.md) |
| Posterior ESEM/DSEM input-gate doctoring | [`docs/research/posterior-esem-input-gates.md`](docs/research/posterior-esem-input-gates.md) |
| Multilevel/event-time recovery doctoring | [`docs/research/multilevel-event-time-recovery.md`](docs/research/multilevel-event-time-recovery.md) |
| Rubin total-variance doctoring | [`docs/research/rubin-total-variance.md`](docs/research/rubin-total-variance.md) |
Expand All @@ -53,9 +54,11 @@ TEPP's approved PRD v0.4 and implementation plan are the primary product baselin
| Entity/project target SQL doctoring | [`docs/research/entity-project-sql.md`](docs/research/entity-project-sql.md) |
| Scientific claim-promotion gate doctoring | [`docs/research/scientific-claim-promotion-gates.md`](docs/research/scientific-claim-promotion-gates.md) |
| Retention/deletion/legal-hold doctoring | [`docs/research/retention-deletion-legal-hold.md`](docs/research/retention-deletion-legal-hold.md) |
| Stopword-deletion doctoring | [`docs/research/stopword-deletion.md`](docs/research/stopword-deletion.md) |
| Provider-payload minimization doctoring | [`docs/research/provider-payload-minimization.md`](docs/research/provider-payload-minimization.md) |
| Adaptive orchestration router doctoring | [`docs/research/adaptive-orchestration-router.md`](docs/research/adaptive-orchestration-router.md) |
| Mention-confidence Brier doctoring | [`docs/research/mention-confidence-brier.md`](docs/research/mention-confidence-brier.md) |
| Event-intelligence status-gate doctoring | [`docs/research/event-intelligence-status-gates.md`](docs/research/event-intelligence-status-gates.md) |
Comment on lines +59 to +60

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Documentation map lists two entries twice

The Mention-confidence Brier doctoring and Event-intelligence status-gate doctoring rows are re-added here, duplicating the identical rows already present earlier in the same table (DOCUMENTATION.md).

Suggested change
| Mention-confidence Brier doctoring | [`docs/research/mention-confidence-brier.md`](docs/research/mention-confidence-brier.md) |
| Event-intelligence status-gate doctoring | [`docs/research/event-intelligence-status-gates.md`](docs/research/event-intelligence-status-gates.md) |
Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

| Stopword-deletion doctoring | [`docs/research/stopword-deletion.md`](docs/research/stopword-deletion.md) |
| Topic log-ratio coordinate doctoring | [`docs/research/topic-logratio-coordinates.md`](docs/research/topic-logratio-coordinates.md) |
| Hourly NIM OpenCode doctoring | [`docs/doctoring/hourly-nim-opencode-development.md`](docs/doctoring/hourly-nim-opencode-development.md) |
| Analysis engine v1 doctoring | [`docs/doctoring/analysis-engine-v1.md`](docs/doctoring/analysis-engine-v1.md) |
Expand Down
19 changes: 18 additions & 1 deletion crates/tepp_simulation/src/document_process.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
//! Document generation with reporting delays, method effects, and memberships.

use crate::SimulationError;
use temporal_core::{AvailableTime, DocumentTime, EventTime};
use temporal_core::{AvailableTime, DocumentTime, EventTime, KnowledgeCutoff};
use uuid::Uuid;

/// Governed method-effect labels for generated document variants.
Expand Down Expand Up @@ -201,6 +201,23 @@ impl SimulatedDocument {
}
}

/// Refuse a delayed-reporting document whose availability exceeds the cutoff.
///
/// # Errors
///
/// Returns [`SimulationError::TemporalInvariantViolation`] when
/// `available_time` is after `cutoff`.
pub fn refuse_unavailable_document(
document: &SimulatedDocument,
cutoff: &KnowledgeCutoff,
) -> Result<(), SimulationError> {
if document.available_time().instant() <= cutoff.instant() {
Ok(())
} else {
Err(SimulationError::TemporalInvariantViolation)
}
Comment thread
seonghobae marked this conversation as resolved.
}

/// Hours spanned by the synthetic non-wrapping calendar (`12 * 28 * 24`).
pub const SYNTHETIC_YEAR_HOURS: u32 = 12 * 28 * 24;

Expand Down
2 changes: 2 additions & 0 deletions crates/tepp_simulation/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,8 @@ pub use document_process::SimulatedMembership;
pub use document_process::delayed_clocks;
/// Membership role vocabulary helper.
pub use document_process::membership_role_at;
/// Refuse a document that is not yet available at the cutoff.
pub use document_process::refuse_unavailable_document;
/// Fail-closed simulation errors.
pub use error::SimulationError;
/// Latent event truth row.
Expand Down
13 changes: 13 additions & 0 deletions crates/tepp_simulation/src/truth_manifest.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ use crate::latent_event::LatentEvent;
use crate::relation_process::{ObservedRelation, TrueRelation};
use sha2::{Digest, Sha256};
use std::collections::BTreeSet;
use temporal_core::KnowledgeCutoff;

/// Immutable known-truth corpus bound to an explicit seed and content digest.
#[derive(Clone, Debug, Eq, PartialEq)]
Expand Down Expand Up @@ -97,6 +98,18 @@ impl TruthManifest {
self.documents.len()
}

/// Documents whose availability time is not after `cutoff`.
#[must_use]
pub fn documents_eligible_at_cutoff(
&self,
cutoff: &KnowledgeCutoff,
) -> Vec<&SimulatedDocument> {
self.documents
.iter()
.filter(|document| document.available_time().instant() <= cutoff.instant())
.collect()
}

/// Verify scientific invariants required of every truth corpus.
///
/// # Errors
Expand Down
50 changes: 50 additions & 0 deletions crates/tepp_simulation/tests/cutoff_eligibility_contract.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
//! Delayed-reporting documents cannot enter a historical fit before they exist.

use temporal_core::KnowledgeCutoff;
use tepp_simulation::{SimulationConfig, SimulationError, generate, refuse_unavailable_document};

#[test]
fn delayed_documents_are_excluded_and_counts_match_known_truth() {
let config = SimulationConfig::new(21, 4, 1, 2, 48, 24, 0, 0, 0, 0, 0, 0).expect("cfg");
let manifest = generate(config).expect("corpus");
let cutoff = KnowledgeCutoff::parse_rfc3339("2026-01-03T00:00:00Z").expect("cutoff");

let eligible = manifest.documents_eligible_at_cutoff(&cutoff);
let truth_count = manifest
.documents()
.iter()
.filter(|document| document.available_time().instant() <= cutoff.instant())
.count();
assert_eq!(eligible.len(), truth_count);
Comment on lines +13 to +18

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 Info: Cutoff eligibility test mirrors the implementation

The test computes its expected truth_count with the same predicate available_time().instant() <= cutoff.instant() that documents_eligible_at_cutoff uses internally (truth_manifest.rs:109). Checking output against a copy of the implementation cannot catch an inverted comparison. An independent oracle would give real coverage of the count-matches-known-truth claim.

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

assert!(
truth_count < manifest.document_count(),
"cutoff must exclude at least one delayed document"
);

let boundary_document = manifest.documents().first().expect("boundary document");
let boundary_cutoff =
KnowledgeCutoff::parse_rfc3339(&boundary_document.available_time().to_rfc3339())
.expect("boundary cutoff");
let boundary_eligible = manifest.documents_eligible_at_cutoff(&boundary_cutoff);
assert!(
boundary_eligible
.iter()
.any(|document| document.document_id() == boundary_document.document_id()),
"a document available exactly at cutoff must be eligible"
);
refuse_unavailable_document(boundary_document, &boundary_cutoff)
.expect("availability equal to cutoff is valid");

for document in eligible {
refuse_unavailable_document(document, &cutoff).expect("eligible");
}
let late = manifest
.documents()
.iter()
.find(|document| document.available_time().instant() > cutoff.instant())
.expect("late document");
assert_eq!(
refuse_unavailable_document(late, &cutoff),
Err(SimulationError::TemporalInvariantViolation)
);
}
Comment thread
seonghobae marked this conversation as resolved.
2 changes: 2 additions & 0 deletions docs/TRACEABILITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,8 @@ The full APA 7th standards/literature register remains `docs/research/standards-
| recovery metrics (RMSE, bias, coverage, graph, temporal order, Monte Carlo SE gates) | PRD; Test Strategy; ADR 0007/0014 | `validation_core` on protected main (PR #19); SE-aware Monte Carlo gates included | implemented-main |
| PostgreSQL bitemporal/lineage persistence | ADR 0013; Architecture/ERD | `persistence_postgres` migration contracts, in-memory adapters, live SQL session/document SQL port, tenant RLS (`0002` + session GUC/role helpers), `DATABASE_URL` SQLx gate, optional session-affine `live-sqlx` `PgPool` driver, exact-head live PostgreSQL CI with isolation proof, append-only immutability triggers (`0004`), temporal interval ordering CHECKs (`0005`), typed membership assignment (`0006` implemented-main), event-relation/mention/instance SQL (#37–#39 implemented-main), source-artifact SQL (#40 implemented-main), audit-event SQL (#41 implemented-main), concurrent document-write stress (#43 implemented-main), backup/restore integrity revalidation (#44 implemented-main), `revision_order` later-revision system-time ordering implemented-main, entity/project target SQL on PR #131; remaining physical ERD constraints | partial |
| known-truth temporal/event simulation manifests | PRD; TRD; Test Strategy | `tepp_simulation` on protected main; recovery metrics in `validation_core` | implemented-main |
| delayed-reporting cutoff eligibility in truth corpora | ADR 0002; research | `tepp_simulation` eligible-at-cutoff filter on the active PR | active-PR |
| versioned service/API contracts and exports | PRD; API contract; ADR 0011/0013 | `tepp_api` analysis-run/export/JSON-LD/GraphML contracts on protected main (PR #21); HTTP service remaining accepted-target | partial |
| versioned service/API contracts and exports | PRD; API contract; ADR 0011/0013 | `tepp_api` analysis-run/export/JSON-LD/GraphML contracts on protected main (PR #21); LineageWeave loopback contracts and request-bound terminal result are composed on the active product branch; production TLS remaining | partial |
| executable cutoff-safe analysis runs | ADR 0012/0022; temporal research; API terminal-result contract | `analysis_engine` availability cutoff, snapshot binding, multiple-membership aggregation, digest-bound readiness artifact, and `tepp.trsl_topic_lineage.v1` execution through `topic_measurement`; synthetic recovery plus tamper/non-convergence tests and exact coverage on the active product branch | active-PR |
| immutable split/run/reproducibility manifests | ADR 0013; ERD | `tepp_api` reproducibility manifest contract on protected main; `persistence_postgres` append-only SQL insert/lookup for `reproducibility_manifest`, `corpus_split_manifest`, `model_run`, and `model_artifact` (migration `0003`); full physical ERD constraints remaining | partial |
Expand Down
8 changes: 0 additions & 8 deletions docs/adr/0009-purpose-bound-pii-governance.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,5 @@
# ADR 0009 — Purpose-bound PII governance without blanket masking

**Decision status:** Accepted
**Implementation maturity:** partial — persistence retention/deletion/legal-hold (migration `0007`) and purpose-bound provider-payload minimization are implemented-main; untrusted-intake grant presence in `intake_authorization` is on the active PR and is not implemented-main until exact-head checks, review, and protected-main integration complete; deployment/provider-region evidence remains accepted-target
**Implementation maturity:** active-PR — `encrypted_mapping` seals source identity with AES-256-GCM authenticated encryption and an operating-system-generated nonce; persistence retention/deletion/legal-hold (migration `0007`) and purpose-bound provider-payload minimization are implemented-main; persistence/KMS and remaining authorization adapters stay accepted-target; deployment/provider-region evidence remains accepted-target
**Implementation maturity:** partial — persistence retention/deletion/legal-hold (migration `0007`) and purpose-bound provider-payload minimization are implemented-main; provider-disclosure receipts with field-code-only evidence are on this active PR and are not implemented-main until exact-head checks, review, and protected-main integration complete; deployment/provider-region evidence remains accepted-target
**Implementation maturity:** active-PR — persistence retention/deletion/legal-hold (migration `0007`) and purpose-bound provider-payload minimization are implemented-main; `operational_log::try_record` and inspected `audit_event` inserts are on this active PR and are not implemented-main until exact-head checks, review, and protected-main integration complete; deployment/provider-region evidence remains accepted-target
**Date:** 2026-08-10
**Implementation maturity:** active-PR — persistence retention/deletion/legal-hold (migration `0007`) and purpose-bound provider-payload minimization are implemented-main; `derived_sensitivity` inheritance is on the active PR and is not implemented-main until exact-head checks, review, and protected-main integration complete; deployment/provider-region evidence remains accepted-target
**Implementation maturity:** partial — persistence retention/deletion/legal-hold (migration `0007`) is implemented-main; purpose-bound provider-payload minimization (expired-purpose denial, log/source separation, separately authorized re-identification) is on the active PR and is not implemented-main until exact-head checks, review, and protected-main integration complete; deployment/provider-region evidence remains accepted-target
**Decision status:** Accepted
**Implementation maturity:** partial — persistence retention/deletion/legal-hold (migration `0007`) and purpose-bound provider-payload minimization (expired-purpose denial, log/source separation, separately authorized re-identification) are implemented-main; deployment/provider-region evidence remains accepted-target

Expand Down
1 change: 0 additions & 1 deletion docs/adr/0010-adaptive-llm-orchestration.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
# ADR 0010 — Adaptive LLM orchestration and test-time compute

**Decision status:** Accepted
**Decision status:** Accepted
**Implementation maturity:** partial — `tepp_api` governed router, comparable-budget ablation record, credential-free contextual-orchestrator binding are implemented-main; evidence-bounded `interpretation_gateway` is on the active PR and is not implemented-main until exact-head checks, review, and protected-main integration complete; live NIM execution, learned conductor calibration, and production ablation evidence remain accepted-target
**Date:** 2026-08-10
Expand Down
27 changes: 27 additions & 0 deletions docs/research/simulation-cutoff-eligibility.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Simulation cutoff eligibility

## Scope

This note documents the `tepp_simulation` historical-fit filter:

1. a delayed-reporting document is eligible only when `available_time <= knowledge_cutoff`;
2. eligible counts must match the known-truth count computed from generated clocks;
3. a late document fails closed.

No database migration is allocated. Interval-aware cutoff on uncertain availability remains a separate follow-up capability; no open PR currently claims it.

## Authoritative sources

Jensen, C. S., & Snodgrass, R. T. (1999). Temporal data management. *IEEE Transactions on Knowledge and Data Engineering, 11*(1), 36–44. https://doi.org/10.1109/69.755613

Dyreson, C. E., & Snodgrass, R. T. (1998). Supporting valid-time indeterminacy. *ACM Transactions on Database Systems, 23*(1), 1–57. https://doi.org/10.1145/288086.288087

## Application

Jensen and Snodgrass (1999) distinguish valid time from transaction time. TEPP defines `available_time` as an application-specific knowledge-availability clock for historical fits. This mapping is specified by ADR 0002 and the `tepp_simulation` eligibility and fail-closed validation APIs. A document written at `document_time` but released later cannot enter a cutoff that precedes `available_time` (Jensen & Snodgrass, 1999; Dyreson & Snodgrass, 1998).

## Verification

- a delayed corpus has fewer eligible documents than total documents at an early cutoff;
- eligible count equals the known-truth count from generated clocks;
- `refuse_unavailable_document` admits a document whose availability is exactly the cutoff and denies late ones with `TemporalInvariantViolation`.
Loading
Loading