diff --git a/CHANGELOG.md b/CHANGELOG.md index 062a69412..4504dd938 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -38,6 +38,8 @@ All notable changes to TEPP are documented here. The format follows Keep a Chang ## [Unreleased] +- **Location-membership analysis-run profile**: `analysis_engine` binds existing `location_membership::refuse_location_as_entity_identity` and `refuse_location_as_language_channel` to cutoff-safe `location_membership_v1` (`tepp.location_membership.v1`) with explicit document availability, a shared 100,000-document execution bound, five accurately reported census statistics, and inference status `location_is_not_entity_identity_not_language_channel`. `identity_recovery_rate` stays library-side. Not membership-posterior ICC, not copied-text, not citation-edge, not corpus-background, not GPU, not MCMC, and not topic birth/split/merge. + - `event_core` adds bounded Allen interval-consistency classification, atomic path-consistency closure, contradiction/resource refusals, and an explicit dependency-error fallback without claiming unrestricted global satisfiability. - `psychometric_core` recovers the Driver, Oud, and Voelkle (2017, Table 2, p. 12 `MANIFESTTRAITVAR`; §7.1, p. 19; p. 16 `MANIFESTTRAITVARstd`; footnote 4; 2017-era ctsem `summary.ctsemFit.R`; JSS PDF re-opened 2026-08-27T14:20Z from https://www.jstatsoft.org/index.php/jss/article/download/v077i05/1104) scalar standardised manifest-trait variance on current main after `0ce16e8` dropped the pre-consolidation code while research notes already named the map (register items 83–84). Table 2 names `MANIFESTTRAITVAR` `Ψ_τ` the additional time-invariant variance-covariance on the measurement level and sets it `NULL` when there is no manifest trait. Equation 5 writes `Γ ~ N(τ, Ψ)` and names that covariance the manifest traits. Section 7.1 names manifest traits stable individual differences in indicator levels, distinct from process-level `TRAITVAR` `φ_ξ`. Page 16 prints standardised matrices with the suffix `std` when appropriate. The printed example on p. 16 is `discreteDRIFTstd`, not `MANIFESTTRAITVARstd`. Footnote 4 standardises using only the relevant variance, not the total. The relevant variance for that named indicator-level correlation is `MANIFESTTRAITVAR`, not process-level `TRAITVAR` and not residual `MANIFESTVAR` `θ`. The 2017-era source forms `MANIFESTTRAITVARstd` only when `MANIFESTTRAITVAR != 0`, as `solve(sqrt(diag(MANIFESTTRAITVAR) + ridging)) %&% MANIFESTTRAITVAR` when `verbose = TRUE`. OpenMx `%&%` is `t(A) %*% B %*% A`. Unlike `TRAITVARstd`, that formation adds `diag(c(ridging), n.manifest)`. The default `ridging = FALSE` adds 0, not `0.0001`; that ridge is a numerical hack and is not this exact map. The scalar correlation is `ψ / ψ = 1` after strictly positive `MANIFESTTRAITVAR`. Form strictly positive `ψ` first, then `1 / √ψ`, then `(1 / √ψ) ψ (1 / √ψ)`. Unstandardised `MANIFESTTRAITVAR` is defined for a zero trait; standardised `MANIFESTTRAITVAR` is not. Zero `MANIFESTTRAITVAR` skips forming `MANIFESTTRAITVARstd` in the 2017-era source and fails closed here. Indicator-level trait variance is an event-time structural quantity, so a non-event clock fails closed. `MANIFESTTRAITVAR` does not require stable `a < 0`. Distinct positive `ψ` recover the same 1. `trait / trait = 1` is `TRAITVARstd` and recovers the same number and remains a distinct named quantity. `θ` is `MANIFESTVAR` and is measurement error, not this correlation. Meredith (1993) remains unread (web search 2026-08-27T14:20Z: Springer/Cambridge Core paywalled; Unpaywall historically `is_oa: false`; Springer `content/pdf` is an HTML stub). Mislevy (1991, *Psychometrika, 56*, 177–196) remains unread on the same terms (DOI `10.1007/bf02294457`). Still not a Kalman filter, not a matrix `expm`, not ESEM estimation, not DSEM, and not ctsem estimation. diff --git a/Cargo.lock b/Cargo.lock index 454a7d612..a24602d70 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -73,6 +73,7 @@ version = "0.2.0" dependencies = [ "corpus_split", "event_core", + "location_membership", "membership_core", "relation_graph", "serde", diff --git a/DOCUMENTATION.md b/DOCUMENTATION.md index 6fa4b9683..23fb2b300 100644 --- a/DOCUMENTATION.md +++ b/DOCUMENTATION.md @@ -71,6 +71,7 @@ TEPP's approved PRD v0.4 and implementation plan are the primary product baselin | Hourly NIM OpenCode doctoring | [`docs/doctoring/hourly-nim-opencode-development.md`](docs/doctoring/hourly-nim-opencode-development.md) | | Analysis engine v1 doctoring | [`docs/doctoring/analysis-engine-v1.md`](docs/doctoring/analysis-engine-v1.md) | | Analysis engine gap-closure doctoring | [`docs/doctoring/analysis-engine-gap-closure.md`](docs/doctoring/analysis-engine-gap-closure.md) | +| Location-membership analysis-run doctoring | [`docs/doctoring/location-membership-analysis-run.md`](docs/doctoring/location-membership-analysis-run.md) | | Corpus-split leakage-audit wire doctoring | [`docs/research/corpus-split-manifest-wire.md`](docs/research/corpus-split-manifest-wire.md) | | Unicode canonical-identity doctoring | [`docs/research/unicode-canonical-identity.md`](docs/research/unicode-canonical-identity.md) | | Change history | [`CHANGELOG.md`](CHANGELOG.md) | diff --git a/crates/analysis_engine/Cargo.toml b/crates/analysis_engine/Cargo.toml index 7322212b2..799a13761 100644 --- a/crates/analysis_engine/Cargo.toml +++ b/crates/analysis_engine/Cargo.toml @@ -21,9 +21,11 @@ sha2 = { workspace = true } tepp_api = { path = "../tepp_api", version = "0.2.0" } temporal_core = { path = "../temporal_core", version = "0.2.0" } topic_measurement = { path = "../topic_measurement", version = "0.2.0" } +location_membership = { path = "../location_membership", version = "0.2.0" } uuid.workspace = true [dev-dependencies] +location_membership = { path = "../location_membership", version = "0.2.0" } corpus_split = { path = "../corpus_split", version = "0.2.0" } membership_core = { path = "../membership_core", version = "0.2.0" } relation_graph = { path = "../relation_graph", version = "0.2.0" } diff --git a/crates/analysis_engine/src/lib.rs b/crates/analysis_engine/src/lib.rs index 72bd5854c..a5079f66c 100644 --- a/crates/analysis_engine/src/lib.rs +++ b/crates/analysis_engine/src/lib.rs @@ -12,6 +12,7 @@ mod case_deletion_refit; mod lineage_criterion; +mod location_membership_artifact; mod topic_context_posterior; mod topic_lineage_artifact; @@ -46,6 +47,13 @@ pub use lineage_criterion::{ LineageCriterionFit, LineageCriterionFitError, LineageCriterionObservation, fit_lineage_criterion_posteriors, }; +/// Location-membership artifact and execution contracts from this engine. +pub use location_membership_artifact::{ + LOCATION_MEMBERSHIP_ARTIFACT_BYTE_LIMIT, LOCATION_MEMBERSHIP_ARTIFACT_SCHEMA_VERSION, + LOCATION_MEMBERSHIP_MODEL_CONTRACT_VERSION, LOCATION_MEMBERSHIP_OUTPUT_PROFILE, + LocationMembershipArtifact, LocationMembershipDocument, LocationMembershipExecution, + execute_location_membership_run, +}; /// Bounded posterior topic-context producer contract and record types. pub use topic_context_posterior::{ TOPIC_CONTEXT_POSTERIOR_BYTE_LIMIT, TOPIC_CONTEXT_POSTERIOR_SCHEMA_VERSION, @@ -248,6 +256,8 @@ pub enum AnalysisEngineError { TopicMeasurement(TopicMeasurementError), /// A topic-lineage artifact violated its bounded schema or count invariants. InvalidTopicLineageArtifact, + /// A location-membership artifact violated its bounded schema or count invariants. + InvalidLocationMembershipArtifact, } impl fmt::Display for AnalysisEngineError { @@ -262,6 +272,7 @@ impl fmt::Display for AnalysisEngineError { Self::LimitExceeded => "analysis corpus exceeded its execution bound", Self::TopicMeasurement(error) => return error.fmt(formatter), Self::InvalidTopicLineageArtifact => "invalid topic lineage artifact", + Self::InvalidLocationMembershipArtifact => "invalid location-membership artifact", }; formatter.write_str(message) } @@ -681,6 +692,10 @@ mod tests { AnalysisEngineError::InvalidTopicLineageArtifact, "invalid topic lineage artifact", ), + ( + AnalysisEngineError::InvalidLocationMembershipArtifact, + "invalid location-membership artifact", + ), ]; for (error, message) in messages { assert_eq!(error.to_string(), message); diff --git a/crates/analysis_engine/src/location_membership_artifact.rs b/crates/analysis_engine/src/location_membership_artifact.rs new file mode 100644 index 000000000..6d40bfa65 --- /dev/null +++ b/crates/analysis_engine/src/location_membership_artifact.rs @@ -0,0 +1,415 @@ +//! Digest-bound location-membership refusals as an analysis-run profile. + +use location_membership::{ + LocationKind, refuse_location_as_entity_identity, refuse_location_as_language_channel, +}; +use serde::{Deserialize, Serialize}; +use sha2::{Digest, Sha256}; +use temporal_core::{AvailableTime, KnowledgeCutoff}; +use tepp_api::{ + AnalysisResultSummary, AnalysisRunAccepted, AnalysisRunRequest, AnalysisRunTerminalResult, +}; + +use crate::{ + AnalysisEngineError, MAX_EVIDENCE_UNITS, format_digest, require_receipt_identity, + valid_identifier, +}; + +/// Versioned schema for a completed location-membership artifact. +pub const LOCATION_MEMBERSHIP_ARTIFACT_SCHEMA_VERSION: &str = "tepp.location_membership.v1"; +/// Model contract required by the location-membership execution path. +pub const LOCATION_MEMBERSHIP_MODEL_CONTRACT_VERSION: &str = "location_membership_v1"; +/// Analysis-run output profile required for a location-membership artifact. +pub const LOCATION_MEMBERSHIP_OUTPUT_PROFILE: &str = "location_membership_v1"; +/// Maximum canonical artifact JSON size. +pub const LOCATION_MEMBERSHIP_ARTIFACT_BYTE_LIMIT: usize = 256 * 1024; +const LOCATION_MEMBERSHIP_INFERENCE_STATUS: &str = + "location_is_not_entity_identity_not_language_channel"; + +/// One cutoff-admitted membership treatment with a closed location kind. +#[derive(Clone, Debug, Eq, PartialEq)] +pub struct LocationMembershipDocument { + document_id: String, + kind: LocationKind, + available_time: AvailableTime, +} + +impl LocationMembershipDocument { + /// Construct a bounded location-membership document. + /// + /// # Errors + /// + /// Returns [`AnalysisEngineError::InvalidEvidence`] when the document + /// identity is empty or oversized. Availability is carried by a validated + /// temporal clock type and cannot be inferred from event time. + pub fn new( + document_id: impl Into, + kind: LocationKind, + available_time: AvailableTime, + ) -> Result { + let document_id = document_id.into(); + if !valid_identifier(&document_id) { + return Err(AnalysisEngineError::InvalidEvidence); + } + Ok(Self { + document_id, + kind, + available_time, + }) + } + + /// Return the opaque document identity. + #[must_use] + pub fn document_id(&self) -> &str { + &self.document_id + } + + /// Return the closed location kind. + #[must_use] + pub const fn kind(&self) -> LocationKind { + self.kind + } + + /// Return when this document became available for analysis. + #[must_use] + pub const fn available_time(&self) -> AvailableTime { + self.available_time + } +} + +/// Completed, bounded location-membership census for analysis-run clients. +#[derive(Clone, Debug, Deserialize, PartialEq, Serialize)] +#[serde(deny_unknown_fields)] +pub struct LocationMembershipArtifact { + /// Exact versioned schema identity. + pub schema_version: String, + /// Opaque accepted-run identity. + pub run_id: String, + /// Immutable source snapshot identity. + pub snapshot_id: String, + /// Historical evidence cutoff used to admit documents. + pub knowledge_cutoff: String, + /// Number of documents admitted at the cutoff. + pub document_count: u64, + /// Time-varying location memberships admitted at the cutoff. + pub location_count: u64, + /// Permanent entity-identity treatments admitted at the cutoff. + pub entity_identity_count: u64, + /// Language-channel treatments admitted at the cutoff. + pub language_channel_count: u64, + /// Location memberships refused as permanent entity identity. + pub refused_as_entity_identity_count: u64, + /// Location memberships refused as a language channel. + pub refused_as_language_channel_count: u64, + /// Fixed claim boundary for consumer copy. + pub inference_status: String, +} + +impl LocationMembershipArtifact { + /// Parse and fully validate a bounded artifact JSON payload. + /// + /// # Errors + /// + /// Returns [`AnalysisEngineError::InvalidLocationMembershipArtifact`] when + /// the schema, identifiers, counts, or claim boundary fail. + pub fn from_json(payload: &str) -> Result { + if payload.len() > LOCATION_MEMBERSHIP_ARTIFACT_BYTE_LIMIT { + return Err(AnalysisEngineError::LimitExceeded); + } + let artifact: Self = serde_json::from_str(payload) + .map_err(|_| AnalysisEngineError::InvalidLocationMembershipArtifact)?; + artifact.validate()?; + Ok(artifact) + } + + /// Serialize canonical validated artifact JSON. + /// + /// # Errors + /// + /// Returns a typed validation, serialization, or size failure. + pub fn to_json(&self) -> Result { + self.validate()?; + let payload = + serde_json::to_string(self).map_err(|_| AnalysisEngineError::SerializationFailure)?; + if payload.len() > LOCATION_MEMBERSHIP_ARTIFACT_BYTE_LIMIT { + return Err(AnalysisEngineError::LimitExceeded); + } + Ok(payload) + } + + /// Return the lowercase SHA-256 digest of canonical artifact JSON. + /// + /// # Errors + /// + /// Returns a typed validation or serialization failure. + pub fn sha256(&self) -> Result { + self.to_json() + .map(|json| format_digest(Sha256::digest(json.into_bytes()))) + } + + fn validate(&self) -> Result<(), AnalysisEngineError> { + let kind_sum = self + .location_count + .checked_add(self.entity_identity_count) + .and_then(|value| value.checked_add(self.language_channel_count)); + let non_location = self + .entity_identity_count + .checked_add(self.language_channel_count); + if self.schema_version != LOCATION_MEMBERSHIP_ARTIFACT_SCHEMA_VERSION + || !valid_identifier(&self.run_id) + || !valid_identifier(&self.snapshot_id) + || KnowledgeCutoff::parse_rfc3339(&self.knowledge_cutoff).is_err() + || self.document_count < 2 + || self.document_count > MAX_EVIDENCE_UNITS as u64 + || self.location_count == 0 + || non_location == Some(0) + || kind_sum != Some(self.document_count) + || self.refused_as_entity_identity_count != self.location_count + || self.refused_as_language_channel_count != self.location_count + || self.inference_status != LOCATION_MEMBERSHIP_INFERENCE_STATUS + { + return Err(AnalysisEngineError::InvalidLocationMembershipArtifact); + } + Ok(()) + } +} + +/// One completed location-membership artifact and its terminal result. +#[derive(Clone, Debug, PartialEq)] +pub struct LocationMembershipExecution { + /// Digest-bound completed location-membership census. + pub artifact: LocationMembershipArtifact, + /// Terminal result carrying the artifact identity, digest, and schema. + pub terminal_result: AnalysisRunTerminalResult, +} + +/// Execute cutoff-safe location-membership refusals as one analysis-run profile. +/// +/// The executor invokes [`refuse_location_as_entity_identity`] and +/// [`refuse_location_as_language_channel`] already on protected main. +/// It does not emit `identity_recovery_rate`, a `scientific_acceptance` +/// inspect metric, GPU kernels, MCMC, or topic birth/split/merge events. +/// +/// # Errors +/// +/// Returns a request/receipt/snapshot/cutoff/profile error, empty or +/// single-kind corpus, duplicate document identity, or invalid artifact error. +pub fn execute_location_membership_run( + request: &AnalysisRunRequest, + accepted: &AnalysisRunAccepted, + snapshot_id: &str, + knowledge_cutoff: KnowledgeCutoff, + documents: &[LocationMembershipDocument], + completed_at: impl Into, +) -> Result { + request.to_json()?; + accepted.to_json()?; + require_receipt_identity(request, accepted)?; + if request.snapshot_id != snapshot_id { + return Err(AnalysisEngineError::SnapshotMismatch); + } + let request_cutoff = KnowledgeCutoff::parse_rfc3339(&request.knowledge_cutoff) + .map_err(|_| AnalysisEngineError::InvalidEvidence)?; + if request_cutoff.instant() != knowledge_cutoff.instant() + || request.model_contract_version != LOCATION_MEMBERSHIP_MODEL_CONTRACT_VERSION + || request.output_profile != LOCATION_MEMBERSHIP_OUTPUT_PROFILE + { + return Err(AnalysisEngineError::InvalidEvidence); + } + if documents.len() > MAX_EVIDENCE_UNITS { + return Err(AnalysisEngineError::LimitExceeded); + } + + let mut seen = std::collections::BTreeSet::new(); + let mut location_count = 0_u64; + let mut entity_identity_count = 0_u64; + let mut language_channel_count = 0_u64; + let mut refused_as_entity_identity_count = 0_u64; + let mut refused_as_language_channel_count = 0_u64; + for document in documents { + if document.available_time().instant() > knowledge_cutoff.instant() { + return Err(AnalysisEngineError::InvalidEvidence); + } + if !seen.insert(document.document_id()) { + return Err(AnalysisEngineError::DuplicateEvidence); + } + match document.kind() { + LocationKind::Location => { + let _ = refuse_location_as_entity_identity(document.kind()); + let _ = refuse_location_as_language_channel(document.kind()); + refused_as_entity_identity_count += 1; + refused_as_language_channel_count += 1; + location_count += 1; + } + LocationKind::EntityIdentity => { + entity_identity_count += 1; + } + LocationKind::LanguageChannel => { + language_channel_count += 1; + } + } + } + let document_count = documents.len() as u64; + if document_count < 2 + || location_count == 0 + || entity_identity_count + .checked_add(language_channel_count) + .unwrap_or(0) + == 0 + { + return Err(AnalysisEngineError::InvalidEvidence); + } + + let artifact = LocationMembershipArtifact { + schema_version: LOCATION_MEMBERSHIP_ARTIFACT_SCHEMA_VERSION.into(), + run_id: accepted.run_id.clone(), + snapshot_id: snapshot_id.to_owned(), + knowledge_cutoff: knowledge_cutoff.to_rfc3339(), + document_count, + location_count, + entity_identity_count, + language_channel_count, + refused_as_entity_identity_count, + refused_as_language_channel_count, + inference_status: LOCATION_MEMBERSHIP_INFERENCE_STATUS.into(), + }; + let digest = artifact.sha256()?; + let summary = AnalysisResultSummary { + analysis_family: "location_membership".into(), + evidence_count: document_count, + statistic_count: 5, + validation_status: LOCATION_MEMBERSHIP_INFERENCE_STATUS.into(), + }; + let terminal_result = AnalysisRunTerminalResult::succeeded( + request, + accepted, + format!("location_membership_artifact_{}", &digest[..16]), + digest, + LOCATION_MEMBERSHIP_ARTIFACT_SCHEMA_VERSION, + completed_at, + summary, + )?; + Ok(LocationMembershipExecution { + artifact, + terminal_result, + }) +} + +#[cfg(test)] +mod tests { + use super::{ + LOCATION_MEMBERSHIP_ARTIFACT_BYTE_LIMIT, LOCATION_MEMBERSHIP_ARTIFACT_SCHEMA_VERSION, + LOCATION_MEMBERSHIP_INFERENCE_STATUS, LocationMembershipArtifact, + }; + use crate::AnalysisEngineError; + + fn artifact() -> LocationMembershipArtifact { + LocationMembershipArtifact { + schema_version: LOCATION_MEMBERSHIP_ARTIFACT_SCHEMA_VERSION.into(), + run_id: "run-1".into(), + snapshot_id: "snapshot-1".into(), + knowledge_cutoff: "2026-08-01T00:00:00Z".into(), + document_count: 3, + location_count: 1, + entity_identity_count: 1, + language_channel_count: 1, + refused_as_entity_identity_count: 1, + refused_as_language_channel_count: 1, + inference_status: LOCATION_MEMBERSHIP_INFERENCE_STATUS.into(), + } + } + + fn assert_invalid(artifact: &LocationMembershipArtifact) { + assert_eq!( + artifact.to_json(), + Err(AnalysisEngineError::InvalidLocationMembershipArtifact) + ); + } + + #[test] + fn artifact_round_trip_and_size_bounds_fail_closed() { + let artifact = artifact(); + let payload = artifact.to_json().expect("json"); + assert_eq!( + LocationMembershipArtifact::from_json(&payload), + Ok(artifact.clone()) + ); + assert_eq!(artifact.sha256().expect("digest").len(), 64); + assert_eq!( + LocationMembershipArtifact::from_json("{}"), + Err(AnalysisEngineError::InvalidLocationMembershipArtifact) + ); + assert_eq!( + LocationMembershipArtifact::from_json( + &"x".repeat(LOCATION_MEMBERSHIP_ARTIFACT_BYTE_LIMIT + 1) + ), + Err(AnalysisEngineError::LimitExceeded) + ); + } + + #[test] + fn artifact_metadata_tampering_fails_closed() { + let artifact = artifact(); + let invalid_artifacts = [ + { + let mut value = artifact.clone(); + value.schema_version.clear(); + value + }, + { + let mut value = artifact.clone(); + value.run_id.clear(); + value + }, + { + let mut value = artifact.clone(); + value.snapshot_id.clear(); + value + }, + { + let mut value = artifact.clone(); + value.knowledge_cutoff = "invalid".into(); + value + }, + { + let mut value = artifact.clone(); + value.document_count = 1; + value + }, + { + let mut value = artifact.clone(); + value.location_count = 0; + value + }, + { + let mut value = artifact.clone(); + value.entity_identity_count = 0; + value.language_channel_count = 0; + value + }, + { + let mut value = artifact.clone(); + value.refused_as_entity_identity_count = 0; + value + }, + { + let mut value = artifact.clone(); + value.refused_as_language_channel_count = 0; + value + }, + { + let mut value = artifact.clone(); + value.location_count = u64::MAX; + value + }, + { + let mut value = artifact.clone(); + value.inference_status.clear(); + value + }, + ]; + for invalid in invalid_artifacts { + assert_invalid(&invalid); + } + } +} diff --git a/crates/analysis_engine/tests/location_membership_artifact_bound_contract.rs b/crates/analysis_engine/tests/location_membership_artifact_bound_contract.rs new file mode 100644 index 000000000..80d0ed06d --- /dev/null +++ b/crates/analysis_engine/tests/location_membership_artifact_bound_contract.rs @@ -0,0 +1,35 @@ +//! Serialization bound for the location-membership analysis artifact. + +use analysis_engine::{ + AnalysisEngineError, LocationMembershipArtifact, MAX_EVIDENCE_UNITS, + LOCATION_MEMBERSHIP_ARTIFACT_SCHEMA_VERSION, +}; + +#[test] +fn compact_oversized_location_artifact_fails_closed() { + let document_count = MAX_EVIDENCE_UNITS as u64 + 1; + let location_count = document_count - 2; + let artifact = LocationMembershipArtifact { + schema_version: LOCATION_MEMBERSHIP_ARTIFACT_SCHEMA_VERSION.into(), + run_id: "run-compact-oversize".into(), + snapshot_id: "snapshot-compact-oversize".into(), + knowledge_cutoff: "2026-08-01T00:00:00Z".into(), + document_count, + location_count, + entity_identity_count: 1, + language_channel_count: 1, + refused_as_entity_identity_count: location_count, + refused_as_language_channel_count: location_count, + inference_status: "location_is_not_entity_identity_not_language_channel".into(), + }; + let raw_payload = serde_json::to_string(&artifact).expect("raw json"); + + assert_eq!( + artifact.to_json(), + Err(AnalysisEngineError::InvalidLocationMembershipArtifact) + ); + assert_eq!( + LocationMembershipArtifact::from_json(&raw_payload), + Err(AnalysisEngineError::InvalidLocationMembershipArtifact) + ); +} diff --git a/crates/analysis_engine/tests/location_membership_cutoff_equivalence_contract.rs b/crates/analysis_engine/tests/location_membership_cutoff_equivalence_contract.rs new file mode 100644 index 000000000..c0d5e351c --- /dev/null +++ b/crates/analysis_engine/tests/location_membership_cutoff_equivalence_contract.rs @@ -0,0 +1,57 @@ +//! Regression for semantically equivalent RFC 3339 cutoff spellings. + +use analysis_engine::{ + LOCATION_MEMBERSHIP_MODEL_CONTRACT_VERSION, LOCATION_MEMBERSHIP_OUTPUT_PROFILE, + LocationMembershipDocument, execute_location_membership_run, +}; +use location_membership::LocationKind; +use temporal_core::{AvailableTime, KnowledgeCutoff}; +use tepp_api::{AnalysisRunAccepted, AnalysisRunRequest}; + +fn available(value: &str) -> AvailableTime { + AvailableTime::parse_rfc3339(value).expect("availability") +} + +#[test] +fn equivalent_offset_cutoff_is_the_same_analysis_instant() { + let request = AnalysisRunRequest { + contract_version: 1, + idempotency_key: "location-membership-cutoff-equivalence".into(), + tenant_workspace_id: "tenant-workspace".into(), + snapshot_id: "snapshot-location-membership".into(), + knowledge_cutoff: "2026-08-01T09:00:00+09:00".into(), + model_contract_version: LOCATION_MEMBERSHIP_MODEL_CONTRACT_VERSION.into(), + output_profile: LOCATION_MEMBERSHIP_OUTPUT_PROFILE.into(), + }; + let accepted = AnalysisRunAccepted::new( + "run-location-membership-cutoff-equivalence", + "accepted", + &request.idempotency_key, + ) + .expect("accepted"); + let cutoff = KnowledgeCutoff::parse_rfc3339("2026-08-01T00:00:00Z").expect("cutoff"); + let documents = vec![ + LocationMembershipDocument::new( + "loc-a", + LocationKind::Location, + available("2026-07-31T23:59:59Z"), + ) + .expect("location"), + LocationMembershipDocument::new( + "ent-b", + LocationKind::EntityIdentity, + available("2026-07-31T23:59:59Z"), + ) + .expect("entity"), + ]; + + execute_location_membership_run( + &request, + &accepted, + "snapshot-location-membership", + cutoff, + &documents, + "2026-08-02T00:00:00Z", + ) + .expect("equivalent cutoff instants must be admitted"); +} diff --git a/crates/analysis_engine/tests/location_membership_execution_contract.rs b/crates/analysis_engine/tests/location_membership_execution_contract.rs new file mode 100644 index 000000000..879385db4 --- /dev/null +++ b/crates/analysis_engine/tests/location_membership_execution_contract.rs @@ -0,0 +1,284 @@ +//! End-to-end contract for cutoff-safe location-membership refusals. + +use analysis_engine::{ + AnalysisEngineError, LOCATION_MEMBERSHIP_ARTIFACT_SCHEMA_VERSION, + LOCATION_MEMBERSHIP_MODEL_CONTRACT_VERSION, LOCATION_MEMBERSHIP_OUTPUT_PROFILE, + LocationMembershipDocument, execute_location_membership_run, +}; +use location_membership::LocationKind; +use temporal_core::{AvailableTime, KnowledgeCutoff}; +use tepp_api::{AnalysisRunAccepted, AnalysisRunRequest, AnalysisRunTerminalState}; + +fn cutoff() -> KnowledgeCutoff { + KnowledgeCutoff::parse_rfc3339("2026-08-01T00:00:00Z").expect("cutoff") +} + +fn available(value: &str) -> AvailableTime { + AvailableTime::parse_rfc3339(value).expect("availability") +} + +fn document(id: &str, kind: LocationKind) -> LocationMembershipDocument { + LocationMembershipDocument::new(id, kind, available("2026-07-31T23:59:59Z")).expect("document") +} + +fn request() -> AnalysisRunRequest { + AnalysisRunRequest { + contract_version: 1, + idempotency_key: "location-membership-idem".into(), + tenant_workspace_id: "tenant-workspace".into(), + snapshot_id: "snapshot-location-membership".into(), + knowledge_cutoff: "2026-08-01T00:00:00Z".into(), + model_contract_version: LOCATION_MEMBERSHIP_MODEL_CONTRACT_VERSION.into(), + output_profile: LOCATION_MEMBERSHIP_OUTPUT_PROFILE.into(), + } +} + +fn accepted(request: &AnalysisRunRequest) -> AnalysisRunAccepted { + AnalysisRunAccepted::new( + "run-location-membership", + "accepted", + &request.idempotency_key, + ) + .expect("accepted") +} + +fn mixed_documents() -> Vec { + vec![ + document("loc-a", LocationKind::Location), + document("ent-b", LocationKind::EntityIdentity), + document("lang-c", LocationKind::LanguageChannel), + ] +} + +fn execute( + request: &AnalysisRunRequest, + documents: &[LocationMembershipDocument], +) -> Result { + execute_location_membership_run( + request, + &accepted(request), + "snapshot-location-membership", + cutoff(), + documents, + "2026-08-02T00:00:00Z", + ) +} + +#[test] +fn mixed_location_kinds_emit_digest_bound_refusals_without_recovery_metric() { + let request = request(); + let execution = execute(&request, &mixed_documents()).expect("execution"); + assert_eq!( + execution.artifact.schema_version, + LOCATION_MEMBERSHIP_ARTIFACT_SCHEMA_VERSION + ); + assert_eq!(execution.artifact.document_count, 3); + assert_eq!(execution.artifact.location_count, 1); + assert_eq!(execution.artifact.entity_identity_count, 1); + assert_eq!(execution.artifact.language_channel_count, 1); + assert_eq!(execution.artifact.refused_as_entity_identity_count, 1); + assert_eq!(execution.artifact.refused_as_language_channel_count, 1); + assert_eq!( + execution.artifact.inference_status, + "location_is_not_entity_identity_not_language_channel" + ); + let payload = execution.artifact.to_json().expect("json"); + assert!(!payload.contains("identity_recovery_rate")); + assert!(!payload.contains("scientific_acceptance")); + assert_eq!( + execution.terminal_result.run_state, + AnalysisRunTerminalState::Succeeded + ); + assert_eq!( + execution.terminal_result.result_sha256.as_deref(), + Some(execution.artifact.sha256().expect("digest").as_str()) + ); + assert_eq!( + execution.terminal_result.result_schema_version.as_deref(), + Some(LOCATION_MEMBERSHIP_ARTIFACT_SCHEMA_VERSION) + ); + assert_eq!( + execution + .terminal_result + .summary + .as_ref() + .expect("summary") + .statistic_count, + 5 + ); +} + +#[test] +fn empty_single_kind_and_duplicate_identities_fail_closed() { + let request = request(); + assert_eq!( + execute(&request, &[]), + Err(AnalysisEngineError::InvalidEvidence) + ); + let location_only = vec![ + document("loc-a", LocationKind::Location), + document("loc-b", LocationKind::Location), + ]; + assert_eq!( + execute(&request, &location_only), + Err(AnalysisEngineError::InvalidEvidence) + ); + let entity_only = vec![ + document("ent-a", LocationKind::EntityIdentity), + document("ent-b", LocationKind::EntityIdentity), + ]; + assert_eq!( + execute(&request, &entity_only), + Err(AnalysisEngineError::InvalidEvidence) + ); + let language_only = vec![ + document("lang-a", LocationKind::LanguageChannel), + document("lang-b", LocationKind::LanguageChannel), + ]; + assert_eq!( + execute(&request, &language_only), + Err(AnalysisEngineError::InvalidEvidence) + ); + let no_location = vec![ + document("ent-a", LocationKind::EntityIdentity), + document("lang-b", LocationKind::LanguageChannel), + ]; + assert_eq!( + execute(&request, &no_location), + Err(AnalysisEngineError::InvalidEvidence) + ); + let duplicates = vec![ + document("same", LocationKind::Location), + document("same", LocationKind::EntityIdentity), + ]; + assert_eq!( + execute(&request, &duplicates), + Err(AnalysisEngineError::DuplicateEvidence) + ); + assert_eq!( + LocationMembershipDocument::new( + "", + LocationKind::Location, + available("2026-07-31T23:59:59Z"), + ), + Err(AnalysisEngineError::InvalidEvidence) + ); +} + +#[test] +fn availability_cutoff_and_document_limit_fail_closed() { + let request = request(); + let mut documents = mixed_documents(); + documents[0] = LocationMembershipDocument::new( + "loc-a", + LocationKind::Location, + available("2026-08-01T00:00:00Z"), + ) + .expect("at cutoff"); + execute(&request, &documents).expect("availability at cutoff"); + + documents[0] = LocationMembershipDocument::new( + "loc-a", + LocationKind::Location, + available("2026-08-01T00:00:00.000000001Z"), + ) + .expect("after cutoff"); + assert_eq!( + execute(&request, &documents), + Err(AnalysisEngineError::InvalidEvidence) + ); + + let oversized = (0..=analysis_engine::MAX_EVIDENCE_UNITS) + .map(|index| document(&format!("document-{index}"), LocationKind::Location)) + .collect::>(); + assert_eq!( + execute(&request, &oversized), + Err(AnalysisEngineError::LimitExceeded) + ); +} + +#[test] +fn execution_refuses_snapshot_profile_and_cutoff_mismatch() { + let request = request(); + let documents = mixed_documents(); + assert_eq!( + execute_location_membership_run( + &request, + &accepted(&request), + "other-snapshot", + cutoff(), + &documents, + "2026-08-02T00:00:00Z", + ), + Err(AnalysisEngineError::SnapshotMismatch) + ); + let mut mismatched = request.clone(); + mismatched.knowledge_cutoff = "2026-07-01T00:00:00Z".into(); + assert_eq!( + execute_location_membership_run( + &mismatched, + &accepted(&mismatched), + "snapshot-location-membership", + cutoff(), + &documents, + "2026-08-02T00:00:00Z", + ), + Err(AnalysisEngineError::InvalidEvidence) + ); + let mut mismatched_model = request.clone(); + mismatched_model.model_contract_version = "other-model".into(); + assert_eq!( + execute_location_membership_run( + &mismatched_model, + &accepted(&mismatched_model), + "snapshot-location-membership", + cutoff(), + &documents, + "2026-08-02T00:00:00Z", + ), + Err(AnalysisEngineError::InvalidEvidence) + ); + for profile in [ + "trsl_topic_lineage_v1", + "fitted_candidate_k_v1", + "pareto_candidate_k_v1", + "joint_posterior_draws_v1", + "method_effects_v1", + "copy_identity_v1", + "style_source_v1", + "prompt_source_v1", + "modality_source_v1", + "corpus_background_v1", + "citation_edge_v1", + "copied_text_v1", + "lineage_criterion_v1", + "composed_fitted_lineage_v1", + "case_deletion_refit_v1", + "topic_activity_v1", + ] { + let mut reused = request.clone(); + reused.output_profile = profile.into(); + assert_eq!( + execute_location_membership_run( + &reused, + &accepted(&reused), + "snapshot-location-membership", + cutoff(), + &documents, + "2026-08-02T00:00:00Z", + ), + Err(AnalysisEngineError::InvalidEvidence) + ); + } + assert!( + execute_location_membership_run( + &request, + &accepted(&request), + "snapshot-location-membership", + cutoff(), + &documents, + "not-a-time", + ) + .is_err() + ); +} diff --git a/docs/TRACEABILITY.md b/docs/TRACEABILITY.md index 2b783c2ab..2d9bcb004 100644 --- a/docs/TRACEABILITY.md +++ b/docs/TRACEABILITY.md @@ -58,6 +58,7 @@ The full APA 7th standards/literature register remains `docs/research/standards- | versioned service/API contracts and exports | PRD; API contract; ADR 0011/0013 | `tepp_api` analysis-run/export/JSON-LD/GraphML contracts on protected main (PR #21); HTTP service remaining accepted-target | partial | | versioned service/API contracts and exports | PRD; API contract; ADR 0011/0013 | `tepp_api` analysis-run/export/JSON-LD/GraphML contracts on protected main (PR #21); LineageWeave loopback contracts and request-bound terminal result are composed on the active product branch; production TLS remaining | partial | | executable cutoff-safe analysis runs | ADR 0012/0022; temporal research; API terminal-result contract | `analysis_engine` availability cutoff, snapshot binding, multiple-membership aggregation, digest-bound readiness artifact, and `tepp.trsl_topic_lineage.v1` execution through `topic_measurement`; synthetic recovery plus tamper/non-convergence tests and exact coverage on the active product branch | active-PR | +| location-membership analysis-run profile | ADR 0003/0022/0066; LocationKind Location/EntityIdentity/LanguageChannel | `analysis_engine` `location_membership_v1` binds explicit availability and bounded input to `refuse_location_as_entity_identity` and `refuse_location_as_language_channel`; digest-bound five-statistic refusals, not `identity_recovery_rate` inspect metric, not membership-posterior ICC, not GPU, not MCMC, not topic birth/split/merge; not implemented-main | active-PR | | immutable split/run/reproducibility manifests | ADR 0013; ERD | `tepp_api` reproducibility manifest contract on protected main; `persistence_postgres` append-only SQL insert/lookup for `reproducibility_manifest`, `corpus_split_manifest`, `model_run`, and `model_artifact` (migration `0003`); full physical ERD constraints remaining | partial | | multilingual shared latent semantic space | PRD; ADR 0004; ADR 0020 | `semantic_core` span-grounded units (active-PR); concept dictionary and shared latent estimator remaining | active-PR | | TRSL-TM temporal/relational topic posterior and backend compatibility | ADR 0012; ADR 0004 | `topic_measurement` stable ALR/ILR coordinates and bounded CPU `f64` reference estimator on protected main; `model_selection` fitted candidate-`K` scoring on this PR; calibrated posterior promotion, method effects, persistence, and accelerated backends remaining | partial | diff --git a/docs/adr/0066-location-membership-analysis-run.md b/docs/adr/0066-location-membership-analysis-run.md new file mode 100644 index 000000000..7e35ae4f6 --- /dev/null +++ b/docs/adr/0066-location-membership-analysis-run.md @@ -0,0 +1,93 @@ +# ADR 0066 — Location-membership refusals as an analysis-run output profile + +**Decision status:** Accepted +**Implementation maturity:** active-PR — composed on this branch; not implemented-main +**Date:** 2026-08-31 +**Supersedes:** None; complements ADR 0003 (location is a time-varying market membership, not entity identity and not a language channel) and ADR 0022 (cutoff-safe analysis-run execution). Does not reuse ADR 0065 (copied-text residue), ADR 0064 (citation-edge provenance-is-not-transition), ADR 0063 (lineage-criterion fitting), ADR 0062 (corpus-background), or ADR 0058 (copy-identity / template-copy). +**Figma File ID:** N/A — this increment changes a Rust service crate and has no user-interface surface. +**Storybook inventory:** N/A — no reusable web object or interaction changed. + +## Context + +Protected main already refuses to treat location membership as permanent +entity identity or as a language channel via +`location_membership::refuse_location_as_entity_identity` and +`refuse_location_as_language_channel`. Operators still cannot request +that refusal census as a digest-bound analysis-run output. +Membership-posterior ICC (#398) binds a psychometric ICC estimator and +does not replace `location_membership`. Copied-text refusals (#427 / +ADR 0065) bind unique-content/stopword vocabulary and do not replace +location-versus-entity identity. + +`identity_recovery_rate` stays library-side. This slice does not put a +`scientific_acceptance` metric on inspect payloads. + +GPU kernels, MCMC, and topic birth/split/merge remain later GAP-004 work +and are not this slice. + +## Decision + +Add the `location_membership_v1` analysis-run output profile to +`analysis_engine`. The executor: + +- consumes already-validated `LocationMembershipDocument` rows with + closed `LocationKind` values and explicit availability clocks; +- requires the request snapshot and knowledge cutoff to match the offered + input construction; +- rejects post-cutoff documents and inputs above the shared 100,000-document + in-memory execution bound before aggregation; +- invokes `refuse_location_as_entity_identity` and + `refuse_location_as_language_channel` without reimplementing the + location/entity/language vocabulary; +- requires at least one location membership and at least one + non-location treatment so the census is mixed; +- emits a canonical SHA-256-digested `tepp.location_membership.v1` + artifact with five reported census statistics (three per-kind counts and + two matching refusal counts), and inference + status `location_is_not_entity_identity_not_language_channel`; +- does not emit `identity_recovery_rate`, invent MCMC, select GPU + backends, or emit topic birth/split/merge events. + +## Alternatives considered + +1. Duplicate membership-posterior ICC (#398) — rejected because that + profile binds a psychometric ICC estimator and does not bind + `location_membership`. +2. Duplicate copied-text refusals (#427) — rejected because that + profile binds unique-content/stopword vocabulary, not + location-versus-entity identity. +3. Put `identity_recovery_rate` on the operator artifact — rejected + because inspect payloads stay metric-free and + `tepp.scientific_acceptance.v1` never appears. +4. Bind the existing location-membership refusals to ADR 0022's + analysis-run profile — accepted. + +## Consequences + +Operators can request cutoff-safe location-membership refusals as a +digest-bound terminal result. The artifact does not claim MCMC, GPU +parity, membership-posterior ICC, copied-text, citation-edge, +corpus-background, method-effect estimation, or topic birth/split/merge. +Snapshot/profile/cutoff mismatch, future evidence, oversized input, empty or +single-kind corpora, and duplicate document identities fail closed. + +## Verification + +The PR includes Rust unit and integration tests for mixed +location/entity/language corpora, empty/single-kind/duplicate refusal, +availability immediately at/after cutoff, oversized input, snapshot / profile / +cutoff mismatch, and artifact tampering. Run: + +```text +cargo fmt --all -- --check +cargo test -p analysis_engine +cargo clippy -p analysis_engine --all-targets -- -D warnings +python3 scripts/validate_documentation.py +``` + +## Rollback and supersession + +Rollback removes the `location_membership_v1` profile. No persisted +schema migration is introduced. Supersede only with an ADR that keeps +location membership distinct from entity identity, language channels, +and `identity_recovery_rate` inspect metrics. diff --git a/docs/adr/README.md b/docs/adr/README.md index 1254c8079..7a8a27476 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -30,6 +30,7 @@ Read [`ADR_POLICY.md`](ADR_POLICY.md) first. **Decision status and implementatio | [0022](0022-deterministic-analysis-run-execution.md) | Deterministic cutoff-safe analysis-run execution | Accepted | active-PR | Closes the first executable product path from accepted run to digest-bound terminal result without claiming estimator authority. | | [0024](0024-lineage-pair-criterion-and-project-journey-posterior.md) | Independent Event Lineage pair criterion and posterior Project Journey | Proposed | active-PR | Strict artifacts preserve criterion/event-time draws, branches, ties, and CPU/GPU receipts without claiming the scientific estimator is complete. | | [0025](0025-macos-native-rust-mlx-metal-boundary.md) | macOS-native Rust-owned MLX Metal execution | Accepted | accepted-target | Compose authenticates to a native host service; Linux never claims Metal, and actual backend/parity receipts fail closed. | +| [0066](0066-location-membership-analysis-run.md) | Location-membership refusals as an analysis-run profile | Accepted | active-PR | Complements ADR 0003/0022; `refuse_location_as_entity_identity` + `refuse_location_as_language_channel`, not membership-posterior ICC. | | [0023](0023-lineage-criterion-anchor-contract.md) | TEPP-owned Event Lineage criterion anchor | Accepted | active-PR | PR #237 publishes the strict accepted/rejected artifact and identities; estimator execution remains fail-closed future work. | | [0024](0024-independent-topic-importance-anchor.md) | Posterior topic-context producer contract | Accepted | contract-only active-PR | Strict DTO/schema only; the current estimator does not emit it. fast-mlsirm owns case-deletion influence. | | [0001](0001-rust-first-modular-msa.md) | Rust-first numerical core and CPU `f64` reference | Accepted | partial | ADR 0011 owns cross-service/MSA authority; 0001 retains numerical/backend authority. | @@ -138,6 +139,7 @@ Use the narrowest owning ADR when decisions overlap: - **project-history wire-size symmetry:** ADR 0019. - **LineageWeave project-history service boundary:** ADR 0021. - **accepted-run execution and terminal artifact production:** ADR 0022. +- **location-membership analysis-run profile:** ADR 0066. - **independent lineage criterion and posterior Project Journey:** ADR 0023. - **macOS-native Rust-owned MLX Metal execution:** ADR 0024. diff --git a/docs/doctoring/location-membership-analysis-run.md b/docs/doctoring/location-membership-analysis-run.md new file mode 100644 index 000000000..099d2a66b --- /dev/null +++ b/docs/doctoring/location-membership-analysis-run.md @@ -0,0 +1,18 @@ +# Location-membership analysis-run composition + +**Active slice:** ADR 0066 / `location_membership_v1` +**Protected-main status:** not implemented-main + +`location_membership` already refuses to treat location membership as +permanent entity identity or as a language channel. This slice binds +those refusals to a cutoff-safe analysis-run profile so operators can +request a digest-bound location-membership refusal artifact. Every input +document carries a validated availability clock; post-cutoff evidence and +corpora above the shared 100,000-document execution bound fail closed before +counting. The terminal summary reports the artifact's five census statistics. + +The artifact inference status is +`location_is_not_entity_identity_not_language_channel`. +`identity_recovery_rate` stays library-side. This is not +membership-posterior ICC, not copied-text, not citation-edge, not +corpus-background, not GPU, not MCMC, and not topic birth/split/merge.