diff --git a/CHANGELOG.md b/CHANGELOG.md index 062a69412..8c1c79ce3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -38,6 +38,8 @@ All notable changes to TEPP are documented here. The format follows Keep a Chang ## [Unreleased] +- **Interpreter/verifier analysis-run profile**: cutoff-safe `interpreter_verifier_v1` binds `interpretation_gateway` evidence-span proposals, numerical-authority refusal, and unsupported-claim rates (`analysis_engine`). Hypothetical only; not a live LLM provider and not implemented-main. + - `event_core` adds bounded Allen interval-consistency classification, atomic path-consistency closure, contradiction/resource refusals, and an explicit dependency-error fallback without claiming unrestricted global satisfiability. - `psychometric_core` recovers the Driver, Oud, and Voelkle (2017, Table 2, p. 12 `MANIFESTTRAITVAR`; §7.1, p. 19; p. 16 `MANIFESTTRAITVARstd`; footnote 4; 2017-era ctsem `summary.ctsemFit.R`; JSS PDF re-opened 2026-08-27T14:20Z from https://www.jstatsoft.org/index.php/jss/article/download/v077i05/1104) scalar standardised manifest-trait variance on current main after `0ce16e8` dropped the pre-consolidation code while research notes already named the map (register items 83–84). Table 2 names `MANIFESTTRAITVAR` `Ψ_τ` the additional time-invariant variance-covariance on the measurement level and sets it `NULL` when there is no manifest trait. Equation 5 writes `Γ ~ N(τ, Ψ)` and names that covariance the manifest traits. Section 7.1 names manifest traits stable individual differences in indicator levels, distinct from process-level `TRAITVAR` `φ_ξ`. Page 16 prints standardised matrices with the suffix `std` when appropriate. The printed example on p. 16 is `discreteDRIFTstd`, not `MANIFESTTRAITVARstd`. Footnote 4 standardises using only the relevant variance, not the total. The relevant variance for that named indicator-level correlation is `MANIFESTTRAITVAR`, not process-level `TRAITVAR` and not residual `MANIFESTVAR` `θ`. The 2017-era source forms `MANIFESTTRAITVARstd` only when `MANIFESTTRAITVAR != 0`, as `solve(sqrt(diag(MANIFESTTRAITVAR) + ridging)) %&% MANIFESTTRAITVAR` when `verbose = TRUE`. OpenMx `%&%` is `t(A) %*% B %*% A`. Unlike `TRAITVARstd`, that formation adds `diag(c(ridging), n.manifest)`. The default `ridging = FALSE` adds 0, not `0.0001`; that ridge is a numerical hack and is not this exact map. The scalar correlation is `ψ / ψ = 1` after strictly positive `MANIFESTTRAITVAR`. Form strictly positive `ψ` first, then `1 / √ψ`, then `(1 / √ψ) ψ (1 / √ψ)`. Unstandardised `MANIFESTTRAITVAR` is defined for a zero trait; standardised `MANIFESTTRAITVAR` is not. Zero `MANIFESTTRAITVAR` skips forming `MANIFESTTRAITVARstd` in the 2017-era source and fails closed here. Indicator-level trait variance is an event-time structural quantity, so a non-event clock fails closed. `MANIFESTTRAITVAR` does not require stable `a < 0`. Distinct positive `ψ` recover the same 1. `trait / trait = 1` is `TRAITVARstd` and recovers the same number and remains a distinct named quantity. `θ` is `MANIFESTVAR` and is measurement error, not this correlation. Meredith (1993) remains unread (web search 2026-08-27T14:20Z: Springer/Cambridge Core paywalled; Unpaywall historically `is_oa: false`; Springer `content/pdf` is an HTML stub). Mislevy (1991, *Psychometrika, 56*, 177–196) remains unread on the same terms (DOI `10.1007/bf02294457`). Still not a Kalman filter, not a matrix `expm`, not ESEM estimation, not DSEM, and not ctsem estimation. diff --git a/Cargo.lock b/Cargo.lock index 454a7d612..2682fc3e7 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -73,6 +73,7 @@ version = "0.2.0" dependencies = [ "corpus_split", "event_core", + "interpretation_gateway", "membership_core", "relation_graph", "serde", diff --git a/crates/analysis_engine/Cargo.toml b/crates/analysis_engine/Cargo.toml index 7322212b2..5ed0e3b5f 100644 --- a/crates/analysis_engine/Cargo.toml +++ b/crates/analysis_engine/Cargo.toml @@ -15,6 +15,7 @@ publish = false [dependencies] event_core = { path = "../event_core", version = "0.2.0" } +interpretation_gateway = { path = "../interpretation_gateway", version = "0.2.0" } serde = { workspace = true } serde_json = { workspace = true } sha2 = { workspace = true } diff --git a/crates/analysis_engine/src/interpreter_verifier_artifact.rs b/crates/analysis_engine/src/interpreter_verifier_artifact.rs new file mode 100644 index 000000000..82a6f7ea2 --- /dev/null +++ b/crates/analysis_engine/src/interpreter_verifier_artifact.rs @@ -0,0 +1,399 @@ +//! Digest-bound interpreter/verifier output as an analysis-run profile. + +use interpretation_gateway::{ + ClaimSupport, EvidenceBoundInterpretation, InterpretationError, InterpretationId, + refuse_interpretation_as_estimator_result, refuse_interpretation_as_observed_fact, + unsupported_claim_rate, +}; +use serde::{Deserialize, Serialize}; +use sha2::{Digest, Sha256}; +use temporal_core::KnowledgeCutoff; +use tepp_api::{ + AnalysisResultSummary, AnalysisRunAccepted, AnalysisRunRequest, AnalysisRunTerminalResult, +}; +use uuid::Uuid; + +use crate::{AnalysisEngineError, format_digest, require_receipt_identity, valid_identifier}; + +/// Versioned schema for a completed interpreter/verifier artifact. +pub const INTERPRETER_VERIFIER_ARTIFACT_SCHEMA_VERSION: &str = "tepp.interpreter_verifier.v1"; +/// Model contract required by the interpreter/verifier execution path. +pub const INTERPRETER_VERIFIER_MODEL_CONTRACT_VERSION: &str = "interpreter_verifier_v1"; +/// Analysis-run output profile required for an interpreter/verifier artifact. +pub const INTERPRETER_VERIFIER_OUTPUT_PROFILE: &str = "interpreter_verifier_v1"; +/// Maximum canonical artifact JSON size. +pub const INTERPRETER_VERIFIER_ARTIFACT_BYTE_LIMIT: usize = 256 * 1024; +const INTERPRETER_VERIFIER_INFERENCE_STATUS: &str = + "hypothetical_interpretation_not_scientific_authority"; +const HYPOTHETICAL_STATUS: &str = "hypothetical"; + +/// Offered evidence-bounded interpretation plus known-truth claim labels. +#[derive(Clone, Debug, Eq, PartialEq)] +pub struct InterpreterVerifierInput { + interpretation_id: InterpretationId, + evidence_span_ids: Vec, + truth: Vec, + decided: Vec, +} + +impl InterpreterVerifierInput { + /// Bundle one interpretation identity, cited spans, and known-truth labels. + #[must_use] + pub fn new( + interpretation_id: InterpretationId, + evidence_span_ids: Vec, + truth: Vec, + decided: Vec, + ) -> Self { + Self { + interpretation_id, + evidence_span_ids, + truth, + decided, + } + } + + /// Return the interpretation identity. + #[must_use] + pub const fn interpretation_id(&self) -> InterpretationId { + self.interpretation_id + } + + /// Borrow the cited evidence span identities. + #[must_use] + pub fn evidence_span_ids(&self) -> &[Uuid] { + &self.evidence_span_ids + } + + /// Borrow the known-truth support labels. + #[must_use] + pub fn truth(&self) -> &[ClaimSupport] { + &self.truth + } + + /// Borrow the decided support labels. + #[must_use] + pub fn decided(&self) -> &[ClaimSupport] { + &self.decided + } +} + +/// Completed, bounded interpreter/verifier result for analysis-run clients. +#[derive(Clone, Debug, Deserialize, PartialEq, Serialize)] +#[serde(deny_unknown_fields)] +pub struct InterpreterVerifierArtifact { + /// Exact versioned schema identity. + pub schema_version: String, + /// Opaque accepted-run identity. + pub run_id: String, + /// Immutable source snapshot identity. + pub snapshot_id: String, + /// Historical evidence cutoff used by the run. + pub knowledge_cutoff: String, + /// Opaque interpretation identity. + pub interpretation_id: String, + /// Number of cited evidence spans. + pub cited_span_count: u64, + /// False-support rate over unsupported known truth. + pub unsupported_claim_rate: f64, + /// Whether estimator-result promotion was refused. + pub estimator_result_refused: bool, + /// Whether observed-fact promotion was refused. + pub observed_fact_refused: bool, + /// Interpretation remains hypothetical. + pub interpretation_status: String, + /// Fixed claim boundary for consumer copy. + pub inference_status: String, +} + +impl InterpreterVerifierArtifact { + /// Parse and fully validate a bounded artifact JSON payload. + /// + /// # Errors + /// + /// Returns [`AnalysisEngineError::InvalidInterpreterVerifierArtifact`] when + /// the schema, identifiers, counts, rate, or claim boundary fail. + pub fn from_json(payload: &str) -> Result { + if payload.len() > INTERPRETER_VERIFIER_ARTIFACT_BYTE_LIMIT { + return Err(AnalysisEngineError::LimitExceeded); + } + let artifact: Self = serde_json::from_str(payload) + .map_err(|_| AnalysisEngineError::InvalidInterpreterVerifierArtifact)?; + artifact.validate()?; + Ok(artifact) + } + + /// Serialize canonical validated artifact JSON. + /// + /// # Errors + /// + /// Returns a typed validation or serialization failure. + pub fn to_json(&self) -> Result { + self.validate()?; + serde_json::to_string(self).map_err(|_| AnalysisEngineError::SerializationFailure) + } + + /// Return the lowercase SHA-256 digest of canonical artifact JSON. + /// + /// # Errors + /// + /// Returns a typed validation or serialization failure. + pub fn sha256(&self) -> Result { + self.to_json() + .map(|json| format_digest(Sha256::digest(json.into_bytes()))) + } + + fn validate(&self) -> Result<(), AnalysisEngineError> { + if self.schema_version != INTERPRETER_VERIFIER_ARTIFACT_SCHEMA_VERSION + || !valid_identifier(&self.run_id) + || !valid_identifier(&self.snapshot_id) + || KnowledgeCutoff::parse_rfc3339(&self.knowledge_cutoff).is_err() + || Uuid::parse_str(&self.interpretation_id).is_err() + || self.cited_span_count == 0 + || !self.unsupported_claim_rate.is_finite() + || self.unsupported_claim_rate < 0.0 + || self.unsupported_claim_rate > 1.0 + || !self.estimator_result_refused + || !self.observed_fact_refused + || self.interpretation_status != HYPOTHETICAL_STATUS + || self.inference_status != INTERPRETER_VERIFIER_INFERENCE_STATUS + { + return Err(AnalysisEngineError::InvalidInterpreterVerifierArtifact); + } + Ok(()) + } +} + +/// One completed interpreter/verifier artifact and its terminal result. +#[derive(Clone, Debug, PartialEq)] +pub struct InterpreterVerifierExecution { + /// Digest-bound completed interpretation artifact. + pub artifact: InterpreterVerifierArtifact, + /// Terminal result carrying the artifact identity, digest, and schema. + pub terminal_result: AnalysisRunTerminalResult, +} + +/// Execute cutoff-safe interpreter/verifier composition as one analysis-run profile. +/// +/// The executor invokes [`EvidenceBoundInterpretation::propose`], +/// [`refuse_interpretation_as_estimator_result`], +/// [`refuse_interpretation_as_observed_fact`], and [`unsupported_claim_rate`]. +/// It does not call a live LLM provider, does not weaken numerical-authority +/// refusal, and cannot promote an interpretation to scientific truth. +/// +/// # Errors +/// +/// Returns a request/receipt/snapshot/cutoff/profile error, interpretation +/// refusal, support-rate failure, or invalid artifact error. +pub fn execute_interpreter_verifier_run( + request: &AnalysisRunRequest, + accepted: &AnalysisRunAccepted, + snapshot_id: &str, + knowledge_cutoff: KnowledgeCutoff, + input: &InterpreterVerifierInput, + completed_at: impl Into, +) -> Result { + request.to_json()?; + accepted.to_json()?; + require_receipt_identity(request, accepted)?; + if request.snapshot_id != snapshot_id { + return Err(AnalysisEngineError::SnapshotMismatch); + } + if request.knowledge_cutoff != knowledge_cutoff.to_rfc3339() + || request.model_contract_version != INTERPRETER_VERIFIER_MODEL_CONTRACT_VERSION + || request.output_profile != INTERPRETER_VERIFIER_OUTPUT_PROFILE + { + return Err(AnalysisEngineError::InvalidEvidence); + } + + let interpretation = + EvidenceBoundInterpretation::propose(input.interpretation_id(), input.evidence_span_ids())?; + let estimator_result_refused = + refuse_interpretation_as_estimator_result(interpretation.interpretation_id()) + == Err(InterpretationError::InterpretationIsNotEstimatorResult); + let observed_fact_refused = + refuse_interpretation_as_observed_fact(interpretation.interpretation_id()) + == Err(InterpretationError::InterpretationIsNotObservedFact); + let rate = unsupported_claim_rate(input.truth(), input.decided())?; + let cited_span_count = interpretation.evidence_span_ids().len() as u64; + let artifact = InterpreterVerifierArtifact { + schema_version: INTERPRETER_VERIFIER_ARTIFACT_SCHEMA_VERSION.into(), + run_id: accepted.run_id.clone(), + snapshot_id: snapshot_id.to_owned(), + knowledge_cutoff: knowledge_cutoff.to_rfc3339(), + interpretation_id: interpretation.interpretation_id().as_uuid().to_string(), + cited_span_count, + unsupported_claim_rate: rate, + estimator_result_refused, + observed_fact_refused, + interpretation_status: HYPOTHETICAL_STATUS.into(), + inference_status: INTERPRETER_VERIFIER_INFERENCE_STATUS.into(), + }; + let digest = artifact.sha256()?; + let summary = AnalysisResultSummary::new( + "interpreter_verifier", + cited_span_count, + 2, + INTERPRETER_VERIFIER_INFERENCE_STATUS, + )?; + let terminal_result = AnalysisRunTerminalResult::succeeded( + request, + accepted, + format!("interpreter_verifier_artifact_{}", &digest[..16]), + digest, + INTERPRETER_VERIFIER_ARTIFACT_SCHEMA_VERSION, + completed_at, + summary, + )?; + Ok(InterpreterVerifierExecution { + artifact, + terminal_result, + }) +} + +#[cfg(test)] +mod tests { + use super::{ + HYPOTHETICAL_STATUS, INTERPRETER_VERIFIER_ARTIFACT_BYTE_LIMIT, + INTERPRETER_VERIFIER_ARTIFACT_SCHEMA_VERSION, INTERPRETER_VERIFIER_INFERENCE_STATUS, + InterpreterVerifierArtifact, InterpreterVerifierInput, + }; + use crate::AnalysisEngineError; + use interpretation_gateway::{ClaimSupport, InterpretationId}; + use uuid::Uuid; + + fn artifact() -> InterpreterVerifierArtifact { + InterpreterVerifierArtifact { + schema_version: INTERPRETER_VERIFIER_ARTIFACT_SCHEMA_VERSION.into(), + run_id: "run-1".into(), + snapshot_id: "snapshot-1".into(), + knowledge_cutoff: "2026-08-01T00:00:00Z".into(), + interpretation_id: "00000000-0000-0000-0000-000000000002".into(), + cited_span_count: 1, + unsupported_claim_rate: 0.0, + estimator_result_refused: true, + observed_fact_refused: true, + interpretation_status: HYPOTHETICAL_STATUS.into(), + inference_status: INTERPRETER_VERIFIER_INFERENCE_STATUS.into(), + } + } + + fn assert_invalid(artifact: &InterpreterVerifierArtifact) { + assert_eq!( + artifact.to_json(), + Err(AnalysisEngineError::InvalidInterpreterVerifierArtifact) + ); + } + + #[test] + fn artifact_round_trip_and_size_bounds_fail_closed() { + let artifact = artifact(); + let payload = artifact.to_json().expect("json"); + assert_eq!( + InterpreterVerifierArtifact::from_json(&payload), + Ok(artifact.clone()) + ); + assert_eq!(artifact.sha256().expect("digest").len(), 64); + assert_eq!( + InterpreterVerifierArtifact::from_json("{}"), + Err(AnalysisEngineError::InvalidInterpreterVerifierArtifact) + ); + assert_eq!( + InterpreterVerifierArtifact::from_json( + &"x".repeat(INTERPRETER_VERIFIER_ARTIFACT_BYTE_LIMIT + 1) + ), + Err(AnalysisEngineError::LimitExceeded) + ); + } + + #[test] + fn artifact_metadata_tampering_fails_closed() { + let artifact = artifact(); + let invalid_artifacts = [ + { + let mut value = artifact.clone(); + value.schema_version.clear(); + value + }, + { + let mut value = artifact.clone(); + value.run_id.clear(); + value + }, + { + let mut value = artifact.clone(); + value.snapshot_id.clear(); + value + }, + { + let mut value = artifact.clone(); + value.knowledge_cutoff = "invalid".into(); + value + }, + { + let mut value = artifact.clone(); + value.interpretation_id.clear(); + value + }, + { + let mut value = artifact.clone(); + value.cited_span_count = 0; + value + }, + { + let mut value = artifact.clone(); + value.unsupported_claim_rate = -0.1; + value + }, + { + let mut value = artifact.clone(); + value.unsupported_claim_rate = f64::NAN; + value + }, + { + let mut value = artifact.clone(); + value.unsupported_claim_rate = 1.5; + value + }, + { + let mut value = artifact.clone(); + value.estimator_result_refused = false; + value + }, + { + let mut value = artifact.clone(); + value.observed_fact_refused = false; + value + }, + { + let mut value = artifact.clone(); + value.interpretation_status = "scientific".into(); + value + }, + { + let mut value = artifact.clone(); + value.inference_status.clear(); + value + }, + ]; + for invalid in invalid_artifacts { + assert_invalid(&invalid); + } + } + + #[test] + fn input_accessors_preserve_offered_spans_and_labels() { + let id = InterpretationId::from_uuid(Uuid::from_u128(2)); + let span = Uuid::from_u128(7); + let input = InterpreterVerifierInput::new( + id, + vec![span], + vec![ClaimSupport::Unsupported], + vec![ClaimSupport::Supported], + ); + assert_eq!(input.interpretation_id(), id); + assert_eq!(input.evidence_span_ids(), &[span]); + assert_eq!(input.truth(), &[ClaimSupport::Unsupported]); + assert_eq!(input.decided(), &[ClaimSupport::Supported]); + } +} diff --git a/crates/analysis_engine/src/lib.rs b/crates/analysis_engine/src/lib.rs index 72bd5854c..40fdeadf2 100644 --- a/crates/analysis_engine/src/lib.rs +++ b/crates/analysis_engine/src/lib.rs @@ -8,13 +8,17 @@ //! through [`tepp_api`]. It deliberately does not claim latent-variable or topic //! estimation authority; those estimators remain separate scientific crates. //! estimation authority; it invokes estimators through their scientific crate -//! contracts and preserves their artifact meaning. +//! contracts and preserves their artifact meaning. Interpreter/verifier +//! composition invokes [`interpretation_gateway`] and cannot promote an +//! interpretation to an estimator result or observed fact. mod case_deletion_refit; +mod interpreter_verifier_artifact; mod lineage_criterion; mod topic_context_posterior; mod topic_lineage_artifact; +use interpretation_gateway::InterpretationError; use serde::Serialize; use sha2::{Digest, Sha256}; use std::collections::BTreeSet; @@ -41,6 +45,17 @@ pub use case_deletion_refit::ExhaustiveCaseDeletionError; pub use case_deletion_refit::ExhaustiveCaseDeletionFits; /// Fit the full corpus and every actual one-document deletion. pub use case_deletion_refit::fit_exhaustive_case_deletion; +/// Known-truth claim support labels used by interpreter/verifier composition. +pub use interpretation_gateway::ClaimSupport; +/// Opaque interpretation identity used by interpreter/verifier composition. +pub use interpretation_gateway::InterpretationId; +/// Interpreter/verifier artifact and execution contracts from this engine. +pub use interpreter_verifier_artifact::{ + INTERPRETER_VERIFIER_ARTIFACT_BYTE_LIMIT, INTERPRETER_VERIFIER_ARTIFACT_SCHEMA_VERSION, + INTERPRETER_VERIFIER_MODEL_CONTRACT_VERSION, INTERPRETER_VERIFIER_OUTPUT_PROFILE, + InterpreterVerifierArtifact, InterpreterVerifierExecution, InterpreterVerifierInput, + execute_interpreter_verifier_run, +}; /// Rust-owned independent TDT link-criterion posterior fitting contracts. pub use lineage_criterion::{ LineageCriterionFit, LineageCriterionFitError, LineageCriterionObservation, @@ -248,6 +263,10 @@ pub enum AnalysisEngineError { TopicMeasurement(TopicMeasurementError), /// A topic-lineage artifact violated its bounded schema or count invariants. InvalidTopicLineageArtifact, + /// An interpretation-gateway gate rejected the offered proposal or labels. + Interpretation(InterpretationError), + /// An interpreter/verifier artifact violated its bounded schema or rates. + InvalidInterpreterVerifierArtifact, } impl fmt::Display for AnalysisEngineError { @@ -262,6 +281,8 @@ impl fmt::Display for AnalysisEngineError { Self::LimitExceeded => "analysis corpus exceeded its execution bound", Self::TopicMeasurement(error) => return error.fmt(formatter), Self::InvalidTopicLineageArtifact => "invalid topic lineage artifact", + Self::Interpretation(error) => return error.fmt(formatter), + Self::InvalidInterpreterVerifierArtifact => "invalid interpreter verifier artifact", }; formatter.write_str(message) } @@ -281,6 +302,12 @@ impl From for AnalysisEngineError { } } +impl From for AnalysisEngineError { + fn from(error: InterpretationError) -> Self { + Self::Interpretation(error) + } +} + /// Execute the cutoff-safe temporal evidence readiness analysis. /// /// Evidence whose `available_time` is later than the request cutoff is excluded @@ -415,6 +442,7 @@ mod tests { AnalysisEngineError, AnalysisEvidenceUnit, MAX_ANALYSIS_IDENTIFIER_BYTES, MAX_EVIDENCE_UNITS, TopicMeasurementError, add_membership_count, execute_analysis_run, }; + use interpretation_gateway::InterpretationError; use temporal_core::{AvailableTime, EventTime}; use tepp_api::{AnalysisRunAccepted, AnalysisRunRequest, AnalysisRunTerminalState, ApiError}; @@ -681,6 +709,14 @@ mod tests { AnalysisEngineError::InvalidTopicLineageArtifact, "invalid topic lineage artifact", ), + ( + AnalysisEngineError::InvalidInterpreterVerifierArtifact, + "invalid interpreter verifier artifact", + ), + ( + AnalysisEngineError::Interpretation(InterpretationError::MissingEvidenceSpan), + "interpretation is missing an evidence span", + ), ]; for (error, message) in messages { assert_eq!(error.to_string(), message); @@ -689,6 +725,12 @@ mod tests { assert_eq!(converted.to_string(), "invalid API wire payload"); let from_topic: AnalysisEngineError = TopicMeasurementError::DidNotConverge.into(); assert_eq!(from_topic.to_string(), "topic estimator did not converge"); + let from_interpretation: AnalysisEngineError = + InterpretationError::MissingEvidenceSpan.into(); + assert_eq!( + from_interpretation.to_string(), + "interpretation is missing an evidence span" + ); assert_eq!( add_membership_count(u64::MAX, 1), Err(AnalysisEngineError::ArithmeticOverflow) diff --git a/crates/analysis_engine/tests/interpreter_verifier_execution_contract.rs b/crates/analysis_engine/tests/interpreter_verifier_execution_contract.rs new file mode 100644 index 000000000..b02dc1750 --- /dev/null +++ b/crates/analysis_engine/tests/interpreter_verifier_execution_contract.rs @@ -0,0 +1,201 @@ +//! End-to-end contract for cutoff-safe interpreter/verifier composition. + +use analysis_engine::{ + AnalysisEngineError, INTERPRETER_VERIFIER_ARTIFACT_SCHEMA_VERSION, + INTERPRETER_VERIFIER_MODEL_CONTRACT_VERSION, INTERPRETER_VERIFIER_OUTPUT_PROFILE, + InterpreterVerifierExecution, InterpreterVerifierInput, execute_interpreter_verifier_run, +}; +use interpretation_gateway::{ClaimSupport, InterpretationError, InterpretationId}; +use temporal_core::KnowledgeCutoff; +use tepp_api::{AnalysisRunAccepted, AnalysisRunRequest, AnalysisRunTerminalState}; +use uuid::Uuid; + +fn cutoff() -> KnowledgeCutoff { + KnowledgeCutoff::parse_rfc3339("2026-02-01T00:00:00Z").expect("cutoff") +} + +fn request() -> AnalysisRunRequest { + AnalysisRunRequest { + contract_version: 1, + idempotency_key: "interpreter-verifier-idem".into(), + tenant_workspace_id: "tenant-workspace".into(), + snapshot_id: "snapshot-interpreter-verifier".into(), + knowledge_cutoff: "2026-02-01T00:00:00Z".into(), + model_contract_version: INTERPRETER_VERIFIER_MODEL_CONTRACT_VERSION.into(), + output_profile: INTERPRETER_VERIFIER_OUTPUT_PROFILE.into(), + } +} + +fn accepted(request: &AnalysisRunRequest) -> AnalysisRunAccepted { + AnalysisRunAccepted::new( + "run-interpreter-verifier", + "accepted", + &request.idempotency_key, + ) + .expect("accepted") +} + +fn cited_input() -> InterpreterVerifierInput { + InterpreterVerifierInput::new( + InterpretationId::from_uuid(Uuid::from_u128(2)), + vec![Uuid::from_u128(7)], + vec![ + ClaimSupport::Unsupported, + ClaimSupport::Unsupported, + ClaimSupport::Supported, + ], + vec![ + ClaimSupport::Unsupported, + ClaimSupport::Unsupported, + ClaimSupport::Supported, + ], + ) +} + +fn uncited_promotion_input() -> InterpreterVerifierInput { + InterpreterVerifierInput::new( + InterpretationId::from_uuid(Uuid::from_u128(2)), + vec![Uuid::from_u128(7)], + vec![ + ClaimSupport::Unsupported, + ClaimSupport::Unsupported, + ClaimSupport::Supported, + ], + vec![ + ClaimSupport::Supported, + ClaimSupport::Supported, + ClaimSupport::Supported, + ], + ) +} + +fn execute( + request: &AnalysisRunRequest, + input: &InterpreterVerifierInput, +) -> Result { + execute_interpreter_verifier_run( + request, + &accepted(request), + "snapshot-interpreter-verifier", + cutoff(), + input, + "2026-02-02T00:00:00Z", + ) +} + +#[test] +fn cited_interpretation_stays_hypothetical_and_records_zero_unsupported_rate() { + let request = request(); + let execution = execute(&request, &cited_input()).expect("execution"); + assert_eq!( + execution.artifact.schema_version, + INTERPRETER_VERIFIER_ARTIFACT_SCHEMA_VERSION + ); + assert_eq!(execution.artifact.cited_span_count, 1); + assert!((execution.artifact.unsupported_claim_rate - 0.0).abs() < f64::EPSILON); + assert!(execution.artifact.estimator_result_refused); + assert!(execution.artifact.observed_fact_refused); + assert_eq!(execution.artifact.interpretation_status, "hypothetical"); + assert_eq!( + execution.artifact.inference_status, + "hypothetical_interpretation_not_scientific_authority" + ); + assert_eq!( + execution.terminal_result.run_state, + AnalysisRunTerminalState::Succeeded + ); + assert_eq!( + execution.terminal_result.result_sha256.as_deref(), + Some(execution.artifact.sha256().expect("digest").as_str()) + ); + assert_eq!( + execution.terminal_result.result_schema_version.as_deref(), + Some(INTERPRETER_VERIFIER_ARTIFACT_SCHEMA_VERSION) + ); + let summary = execution.terminal_result.summary.as_ref().expect("summary"); + assert_eq!(summary.analysis_family, "interpreter_verifier"); + assert_eq!( + summary.validation_status, + "hypothetical_interpretation_not_scientific_authority" + ); +} + +#[test] +fn uncited_promotion_records_unit_rate_and_cannot_become_scientific_authority() { + let execution = execute(&request(), &uncited_promotion_input()).expect("execution"); + assert!((execution.artifact.unsupported_claim_rate - 1.0).abs() < f64::EPSILON); + assert!(execution.artifact.estimator_result_refused); + assert!(execution.artifact.observed_fact_refused); + assert_eq!(execution.artifact.interpretation_status, "hypothetical"); + assert_eq!( + execution.artifact.inference_status, + "hypothetical_interpretation_not_scientific_authority" + ); +} + +#[test] +fn missing_spans_and_invalid_support_payloads_fail_closed() { + let request = request(); + let missing_spans = InterpreterVerifierInput::new( + InterpretationId::from_uuid(Uuid::from_u128(2)), + Vec::new(), + vec![ClaimSupport::Unsupported], + vec![ClaimSupport::Unsupported], + ); + assert_eq!( + execute(&request, &missing_spans), + Err(AnalysisEngineError::Interpretation( + InterpretationError::MissingEvidenceSpan + )) + ); + let invalid_support = InterpreterVerifierInput::new( + InterpretationId::from_uuid(Uuid::from_u128(2)), + vec![Uuid::from_u128(7)], + vec![ClaimSupport::Supported], + vec![ClaimSupport::Supported], + ); + assert_eq!( + execute(&request, &invalid_support), + Err(AnalysisEngineError::Interpretation( + InterpretationError::InvalidSupportPayload + )) + ); +} + +#[test] +fn execution_refuses_snapshot_profile_and_cutoff_mismatch() { + let request = request(); + assert_eq!( + execute_interpreter_verifier_run( + &request, + &accepted(&request), + "other-snapshot", + cutoff(), + &cited_input(), + "2026-02-02T00:00:00Z", + ), + Err(AnalysisEngineError::SnapshotMismatch) + ); + for invalid_request in [ + { + let mut value = request.clone(); + value.knowledge_cutoff = "2026-08-02T00:00:00Z".into(); + value + }, + { + let mut value = request.clone(); + value.model_contract_version = "other-model".into(); + value + }, + { + let mut value = request.clone(); + value.output_profile = "trsl_topic_lineage_v1".into(); + value + }, + ] { + assert_eq!( + execute(&invalid_request, &cited_input()), + Err(AnalysisEngineError::InvalidEvidence) + ); + } +} diff --git a/docs/TRACEABILITY.md b/docs/TRACEABILITY.md index 2b783c2ab..041cffd2a 100644 --- a/docs/TRACEABILITY.md +++ b/docs/TRACEABILITY.md @@ -58,6 +58,7 @@ The full APA 7th standards/literature register remains `docs/research/standards- | versioned service/API contracts and exports | PRD; API contract; ADR 0011/0013 | `tepp_api` analysis-run/export/JSON-LD/GraphML contracts on protected main (PR #21); HTTP service remaining accepted-target | partial | | versioned service/API contracts and exports | PRD; API contract; ADR 0011/0013 | `tepp_api` analysis-run/export/JSON-LD/GraphML contracts on protected main (PR #21); LineageWeave loopback contracts and request-bound terminal result are composed on the active product branch; production TLS remaining | partial | | executable cutoff-safe analysis runs | ADR 0012/0022; temporal research; API terminal-result contract | `analysis_engine` availability cutoff, snapshot binding, multiple-membership aggregation, digest-bound readiness artifact, and `tepp.trsl_topic_lineage.v1` execution through `topic_measurement`; synthetic recovery plus tamper/non-convergence tests and exact coverage on the active product branch | active-PR | +| interpreter/verifier analysis-run composition | ADR 0010/0022/0050; PRD | `analysis_engine` `interpreter_verifier_v1` binds `interpretation_gateway` numerical-authority refusal and evidence-grounded claim rates; hypothetical only; not a live LLM provider and not implemented-main | active-PR | | immutable split/run/reproducibility manifests | ADR 0013; ERD | `tepp_api` reproducibility manifest contract on protected main; `persistence_postgres` append-only SQL insert/lookup for `reproducibility_manifest`, `corpus_split_manifest`, `model_run`, and `model_artifact` (migration `0003`); full physical ERD constraints remaining | partial | | multilingual shared latent semantic space | PRD; ADR 0004; ADR 0020 | `semantic_core` span-grounded units (active-PR); concept dictionary and shared latent estimator remaining | active-PR | | TRSL-TM temporal/relational topic posterior and backend compatibility | ADR 0012; ADR 0004 | `topic_measurement` stable ALR/ILR coordinates and bounded CPU `f64` reference estimator on protected main; `model_selection` fitted candidate-`K` scoring on this PR; calibrated posterior promotion, method effects, persistence, and accelerated backends remaining | partial | diff --git a/docs/adr/0050-interpreter-verifier-analysis-run.md b/docs/adr/0050-interpreter-verifier-analysis-run.md new file mode 100644 index 000000000..538b9a4d7 --- /dev/null +++ b/docs/adr/0050-interpreter-verifier-analysis-run.md @@ -0,0 +1,88 @@ +# ADR 0050 — Interpreter/verifier composition as an analysis-run output profile + +**Decision status:** Accepted +**Implementation maturity:** active-PR — composed on this branch; not implemented-main +**Date:** 2026-08-31 +**Supersedes:** None; complements ADR 0010 (adaptive LLM orchestration) and ADR 0022 (cutoff-safe analysis-run execution). +**Figma File ID:** N/A — this increment changes a Rust service crate and has no user-interface surface. +**Storybook inventory:** N/A — no reusable web object or interaction changed. + +## Context + +Protected main already owns `interpretation_gateway`: an interpretation must +cite at least one evidence span, remains hypothetical, cannot become an +estimator result or observed fact, and records an unsupported-claim rate from +known truth. Operators still cannot request that interpreter/verifier boundary +as a digest-bound analysis-run output. + +Live contextual-orchestrator provider execution, committee/conductor +calibration, and scientific claim promotion remain later GAP-013 work and are +not this slice. An LLM completion must not define numerical authority. + +## Decision + +Add the `interpreter_verifier_v1` analysis-run output profile to +`analysis_engine`. The executor: + +- consumes an interpretation identity, cited evidence spans, and known-truth + support labels; +- requires the request snapshot and knowledge cutoff to match the offered + construction; +- invokes `EvidenceBoundInterpretation::propose`, + `refuse_interpretation_as_estimator_result`, + `refuse_interpretation_as_observed_fact`, and `unsupported_claim_rate` + without reimplementing those gates; +- fails closed when numerical-authority refusal or observed-fact refusal would + ever succeed; +- emits a canonical SHA-256-digested `tepp.interpreter_verifier.v1` artifact + with cited-span count, unsupported-claim rate, interpretation status + `hypothetical`, and inference status + `hypothetical_interpretation_not_scientific_authority`; +- does not invent a live LLM provider, persist rows, or promote scientific + truth. + +This is evidence-bounded interpretation composition, not live orchestration +and not estimator authority. + +## Alternatives considered + +1. Bind another GAP-004 Bayesian sampler or topic birth/split/merge slice — + rejected because fitted candidate-`K` is already a live analysis-run PR and + this hour's unique gap is GAP-013. +2. Invent a live LLM provider inside `analysis_engine` — rejected because + provider execution belongs behind contextual-orchestrator and would embed + model names as scientific meaning. +3. Put interpreter/verifier composition into `tepp_api` — rejected because + transport contracts and interpretation composition would become one service + boundary. +4. Bind the existing `interpretation_gateway` refusals to ADR 0022's + analysis-run profile — accepted. + +## Consequences + +Operators can request cutoff-safe interpreter/verifier composition as a +digest-bound terminal result. The artifact cannot become an estimator result +or observed fact. Snapshot/profile/cutoff mismatch, missing evidence spans, +and invalid support payloads fail closed. Live provider execution, committee +modes, and scientific promotion remain later work. + +## Verification + +The PR includes Rust unit and integration tests for cited-span hypothetical +output, uncited-promotion rate recording without scientific promotion, missing +span refusal, invalid support-payload refusal, snapshot/profile/cutoff +mismatch, and artifact tampering. Run: + +```text +cargo fmt --all -- --check +cargo test -p analysis_engine +cargo clippy -p analysis_engine --all-targets -- -D warnings +python3 scripts/validate_documentation.py +``` + +## Rollback and supersession + +Rollback removes the `interpreter_verifier_v1` profile. No persisted schema +migration is introduced. Supersede only with an ADR that keeps numerical +authority and observed-fact refusal fail-closed and distinct from live LLM +execution. diff --git a/docs/adr/README.md b/docs/adr/README.md index 1254c8079..b942a19f2 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -28,6 +28,7 @@ Read [`ADR_POLICY.md`](ADR_POLICY.md) first. **Decision status and implementatio | [0020](0020-span-grounded-semantic-units.md) | Span-grounded semantic units; language tags are not identity | Accepted | active-PR | First ADR 0004 production slice; concept alignment, invariance, and topic estimation are not claimed. | | [0021](0021-lineageweave-project-history-boundary.md) | LineageWeave project-history service boundary | Accepted | active-PR | Credential-free bounded project-history API preserves LineageWeave authorization ownership. | | [0022](0022-deterministic-analysis-run-execution.md) | Deterministic cutoff-safe analysis-run execution | Accepted | active-PR | Closes the first executable product path from accepted run to digest-bound terminal result without claiming estimator authority. | +| [0050](0050-interpreter-verifier-analysis-run.md) | Interpreter/verifier as an analysis-run profile | Accepted | active-PR | Complements ADR 0010/0022; hypothetical evidence-bounded interpretation, not a live LLM provider and not scientific promotion. | | [0024](0024-lineage-pair-criterion-and-project-journey-posterior.md) | Independent Event Lineage pair criterion and posterior Project Journey | Proposed | active-PR | Strict artifacts preserve criterion/event-time draws, branches, ties, and CPU/GPU receipts without claiming the scientific estimator is complete. | | [0025](0025-macos-native-rust-mlx-metal-boundary.md) | macOS-native Rust-owned MLX Metal execution | Accepted | accepted-target | Compose authenticates to a native host service; Linux never claims Metal, and actual backend/parity receipts fail closed. | | [0023](0023-lineage-criterion-anchor-contract.md) | TEPP-owned Event Lineage criterion anchor | Accepted | active-PR | PR #237 publishes the strict accepted/rejected artifact and identities; estimator execution remains fail-closed future work. | @@ -138,6 +139,7 @@ Use the narrowest owning ADR when decisions overlap: - **project-history wire-size symmetry:** ADR 0019. - **LineageWeave project-history service boundary:** ADR 0021. - **accepted-run execution and terminal artifact production:** ADR 0022. +- **interpreter/verifier analysis-run claim boundary:** ADR 0050. - **independent lineage criterion and posterior Project Journey:** ADR 0023. - **macOS-native Rust-owned MLX Metal execution:** ADR 0024. diff --git a/docs/doctoring/interpreter-verifier-analysis-run.md b/docs/doctoring/interpreter-verifier-analysis-run.md new file mode 100644 index 000000000..797383ad8 --- /dev/null +++ b/docs/doctoring/interpreter-verifier-analysis-run.md @@ -0,0 +1,16 @@ +# Interpreter/verifier analysis-run composition + +**Active slice:** ADR 0050 / `interpreter_verifier_v1` +**Protected-main status:** not implemented-main + +`interpretation_gateway` already refuses to treat an interpretation as an +estimator result or observed fact, requires cited evidence spans, and records +unsupported-claim rates from known truth. This slice binds those gates to a +cutoff-safe analysis-run profile so an operator can request a digest-bound +terminal result. + +The executor does not call a live LLM provider and cannot promote scientific +truth. Live committee/conductor execution remains later GAP-013 work. + +Exact-head Checks and two independent approvals are required before any +implemented-main claim.