Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions ARCHITECTURE.md

Large diffs are not rendered by default.

2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,8 @@ All notable changes to TEPP are documented here. The format follows Keep a Chang

## [Unreleased]

- `psychometric_core` names the grand-mean pooled OLS slope on clustered scores as the total \(\hat\beta_t\) of Enders and Tofighi (2007, p. 125; PDF opened 2026-08-31T00:20Z from OSF `mt5z3`). That slope ignores cluster membership. The opened PDF quotes Raudenbush and Bryk (2002, pp. 137–139): the total OLS coefficient is a weighted combination of the within-cluster and between-cluster slopes and is generally an uninterpretable blend of those two; the hierarchical estimator under grand-mean centering is likewise neither \(\beta_w\) nor \(\beta_b\). Raudenbush and Bryk (2002) remains unread; the quotation is from the opened Enders PDF. Grand-mean centering the coordinates does not change an OLS slope, so this is also the OLS analogue of a CGM Level-1 slope that does not include cluster means. It is not CWC \(\gamma_{10}\), not the between-cluster slope, not the CWC contextual effect, and not their multilevel maximum-likelihood CGM \(\gamma_{10}\). Treating that pooled slope as a within-cluster effect fails closed. Empty, singleton, one-cluster, non-finite, singular, and overflowing designs fail closed. Still not DSEM, not RI-CLPM, not a random-effects sampler, and not a second invariance crate.

- `event_core` adds bounded Allen interval-consistency classification, atomic path-consistency closure, contradiction/resource refusals, and an explicit dependency-error fallback without claiming unrestricted global satisfiability.

- `psychometric_core` recovers the Driver, Oud, and Voelkle (2017, Table 2, p. 12 `MANIFESTTRAITVAR`; §7.1, p. 19; p. 16 `MANIFESTTRAITVARstd`; footnote 4; 2017-era ctsem `summary.ctsemFit.R`; JSS PDF re-opened 2026-08-27T14:20Z from https://www.jstatsoft.org/index.php/jss/article/download/v077i05/1104) scalar standardised manifest-trait variance on current main after `0ce16e8` dropped the pre-consolidation code while research notes already named the map (register items 83–84). Table 2 names `MANIFESTTRAITVAR` `Ψ_τ` the additional time-invariant variance-covariance on the measurement level and sets it `NULL` when there is no manifest trait. Equation 5 writes `Γ ~ N(τ, Ψ)` and names that covariance the manifest traits. Section 7.1 names manifest traits stable individual differences in indicator levels, distinct from process-level `TRAITVAR` `φ_ξ`. Page 16 prints standardised matrices with the suffix `std` when appropriate. The printed example on p. 16 is `discreteDRIFTstd`, not `MANIFESTTRAITVARstd`. Footnote 4 standardises using only the relevant variance, not the total. The relevant variance for that named indicator-level correlation is `MANIFESTTRAITVAR`, not process-level `TRAITVAR` and not residual `MANIFESTVAR` `θ`. The 2017-era source forms `MANIFESTTRAITVARstd` only when `MANIFESTTRAITVAR != 0`, as `solve(sqrt(diag(MANIFESTTRAITVAR) + ridging)) %&% MANIFESTTRAITVAR` when `verbose = TRUE`. OpenMx `%&%` is `t(A) %*% B %*% A`. Unlike `TRAITVARstd`, that formation adds `diag(c(ridging), n.manifest)`. The default `ridging = FALSE` adds 0, not `0.0001`; that ridge is a numerical hack and is not this exact map. The scalar correlation is `ψ / ψ = 1` after strictly positive `MANIFESTTRAITVAR`. Form strictly positive `ψ` first, then `1 / √ψ`, then `(1 / √ψ) ψ (1 / √ψ)`. Unstandardised `MANIFESTTRAITVAR` is defined for a zero trait; standardised `MANIFESTTRAITVAR` is not. Zero `MANIFESTTRAITVAR` skips forming `MANIFESTTRAITVARstd` in the 2017-era source and fails closed here. Indicator-level trait variance is an event-time structural quantity, so a non-event clock fails closed. `MANIFESTTRAITVAR` does not require stable `a < 0`. Distinct positive `ψ` recover the same 1. `trait / trait = 1` is `TRAITVARstd` and recovers the same number and remains a distinct named quantity. `θ` is `MANIFESTVAR` and is measurement error, not this correlation. Meredith (1993) remains unread (web search 2026-08-27T14:20Z: Springer/Cambridge Core paywalled; Unpaywall historically `is_oa: false`; Springer `content/pdf` is an HTML stub). Mislevy (1991, *Psychometrika, 56*, 177–196) remains unread on the same terms (DOI `10.1007/bf02294457`). Still not a Kalman filter, not a matrix `expm`, not ESEM estimation, not DSEM, and not ctsem estimation.
Expand Down
211 changes: 201 additions & 10 deletions crates/psychometric_core/src/cluster_mean.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,12 @@
//! Enders and Tofighi (2007, Table 2, pp. 124–127) separate the
//! **within-cluster** slope, the **between-cluster** slope, and the
//! **contextual** effect. The CWC cluster-mean coefficient is the contextual
//! effect (`between − within`), not the between-cluster effect.
//! effect (`between − within`), not the between-cluster effect. The
//! grand-mean pooled OLS slope is the total slope that ignores nesting
//! (Enders & Tofighi, 2007, p. 125) and is generally an uninterpretable
//! blend of within and between.

use std::collections::BTreeMap;
use std::collections::{BTreeMap, BTreeSet};

use crate::error::PsychometricError;
use crate::indicator::require_finite;
Expand Down Expand Up @@ -39,9 +42,10 @@ pub struct WithinBetweenSlopes {
/// Recover within-cluster and between-cluster OLS slopes after CWC.
///
/// Between components use unweighted cluster means. Within components use the
/// stacked cluster-mean-centered residuals. A grand-mean pooled slope is not
/// returned because it confounds the two (Enders & Tofighi, 2007; Curran &
/// Bauer, 2011; Hamaker, Kuiper, & Grasman, 2015).
/// stacked cluster-mean-centered residuals. The grand-mean pooled slope is a
/// separate named estimand ([`recover_grand_mean_pooled_slope`]) because it
/// confounds the two (Enders & Tofighi, 2007, p. 125; Curran & Bauer, 2011;
/// Hamaker, Kuiper, & Grasman, 2015).
///
/// `contextual_effect` is `between_slope − within_slope`. Enders and Tofighi
/// (2007, Table 2, pp. 124–127) show that this is the cluster-mean coefficient
Expand Down Expand Up @@ -109,6 +113,73 @@ pub fn recover_cluster_mean_within_between_slopes(
})
}

/// Recover the grand-mean pooled OLS slope that ignores cluster membership.
///
/// Enders and Tofighi (2007, p. 125; PDF opened 2026-08-31T00:20Z from OSF
/// `mt5z3`) write that the total OLS slope on clustered data is a weighted
/// combination of the within-cluster and between-cluster slopes, quoting
/// Raudenbush and Bryk (2002, p. 137). They quote p. 138: that total slope
/// "is generally an uninterpretable blend of" the within and between
/// coefficients. They then quote p. 139: the hierarchical estimator under
/// grand-mean centering is an inappropriate estimator of the person-level
/// (Level 1) effect; it too is an uninterpretable blend, neither within nor
/// between. This helper is the OLS analogue of that total slope: ordinary
/// least squares of outcome on predictor, ignoring `cluster_key`. Grand-mean
/// centering the coordinates does not change an OLS slope, so this is also
/// the OLS analogue of a CGM Level-1 slope that does not include cluster
/// means. It is not CWC `γ10`, not the between-cluster slope, not the CWC
/// contextual effect, and not the multilevel maximum-likelihood CGM `γ10`
/// Enders and Tofighi estimate. Raudenbush and Bryk (2002) remains unread;
/// the quotations are from the opened Enders PDF.
///
/// # Errors
///
/// Returns [`PsychometricError::InvalidNumericInput`] for empty, singleton, or
/// non-finite rows, [`PsychometricError::InsufficientClusters`] when fewer
/// than two clusters are present, and [`PsychometricError::SingularDesign`]
/// when the pooled predictor has zero variance.
pub fn recover_grand_mean_pooled_slope(rows: &[ClusteredScore]) -> Result<f64, PsychometricError> {
if rows.len() < 2 {
return Err(PsychometricError::InvalidNumericInput);
}
let mut cluster_keys = BTreeSet::new();
let mut predictors = Vec::with_capacity(rows.len());
let mut outcomes = Vec::with_capacity(rows.len());
for row in rows {
if !row.predictor.is_finite() || !row.outcome.is_finite() {
return Err(PsychometricError::InvalidNumericInput);
}
cluster_keys.insert(row.cluster_key);
predictors.push(row.predictor);
outcomes.push(row.outcome);
}
if cluster_keys.len() < 2 {
return Err(PsychometricError::InsufficientClusters);
}
ordinary_least_squares_slope(&predictors, &outcomes)
}

/// Refuse treating a grand-mean pooled OLS slope as a within-cluster effect.
///
/// Enders and Tofighi (2007, p. 125) quote Raudenbush and Bryk (2002, p. 139):
/// the hierarchical estimator under grand-mean centering is an inappropriate
/// estimator of the person-level (Level 1) effect. It too is an uninterpretable
/// blend: neither the within-cluster slope nor the between-cluster slope. The
/// OLS analogue [`recover_grand_mean_pooled_slope`] is that total slope, not
/// CWC `γ10`.
///
/// # Errors
///
/// Always returns
/// [`PsychometricError::GrandMeanPooledSlopeIsNotWithinClusterEffect`].
pub fn refuse_grand_mean_pooled_slope_as_within_cluster_effect(
pooled_slope: f64,
within_slope: f64,
) -> Result<f64, PsychometricError> {
let _ = (pooled_slope, within_slope);
Err(PsychometricError::GrandMeanPooledSlopeIsNotWithinClusterEffect)
}

/// Enders and Tofighi (2007, p. 127) identity: CWC `γ01 = β_between − β_within`.
///
/// This helper is crate-visible so overflow of the subtraction can be recovered
Expand Down Expand Up @@ -203,13 +274,13 @@ pub fn recover_kish_weighted_slope(
mod tests {
use super::{
ClusteredScore, contextual_effect_from_slopes, kish_effective_sample_size,
recover_cluster_mean_within_between_slopes, recover_kish_weighted_slope,
recover_cluster_mean_within_between_slopes, recover_grand_mean_pooled_slope,
recover_kish_weighted_slope, refuse_grand_mean_pooled_slope_as_within_cluster_effect,
};
use crate::error::PsychometricError;

#[test]
fn noiseless_cwc_recovers_distinct_within_between_and_contextual() {
let rows = [
fn noiseless_two_cluster_rows() -> [ClusteredScore; 4] {
[
ClusteredScore {
cluster_key: 1,
predictor: 0.0,
Expand All @@ -230,7 +301,12 @@ mod tests {
predictor: 6.0,
outcome: 11.0,
},
];
]
}

#[test]
fn noiseless_cwc_recovers_distinct_within_between_and_contextual() {
let rows = noiseless_two_cluster_rows();
// cluster 1 mean x=1 y=2.5; cluster 2 mean x=5 y=10.5 → between = 2
// within: (-1,-0.5),(1,0.5) and (-1,-0.5),(1,0.5) → within = 0.5
// contextual = 2 − 0.5 = 1.5 (CWC γ01; not the between slope)
Expand All @@ -246,6 +322,121 @@ mod tests {
);
}

#[test]
fn noiseless_grand_mean_pooled_slope_is_an_uninterpretable_blend() {
let rows = noiseless_two_cluster_rows();
// total OLS: x={0,2,4,6} ȳ=6.5 x̄=3 → slope = 34/20 = 1.7
// Enders & Tofighi (2007, p. 125): βt is generally an uninterpretable
// blend of βw=0.5 and βb=2; 1.7 lies strictly between them.
let pooled = recover_grand_mean_pooled_slope(&rows).expect("pooled");
let recovered = recover_cluster_mean_within_between_slopes(&rows).expect("cwc");
assert!((pooled - 1.7).abs() < 1e-12);
assert!((pooled - recovered.within_slope).abs() > 1e-9);
assert!((pooled - recovered.between_slope).abs() > 1e-9);
assert!((pooled - recovered.contextual_effect).abs() > 1e-9);
assert!(pooled > recovered.within_slope);
assert!(pooled < recovered.between_slope);
assert_eq!(
refuse_grand_mean_pooled_slope_as_within_cluster_effect(pooled, recovered.within_slope),
Err(PsychometricError::GrandMeanPooledSlopeIsNotWithinClusterEffect)
);
assert_eq!(
refuse_grand_mean_pooled_slope_as_within_cluster_effect(0.5, 0.5),
Err(PsychometricError::GrandMeanPooledSlopeIsNotWithinClusterEffect)
);
}

#[test]
fn grand_mean_pooled_slope_fails_closed_on_empty_one_cluster_nonfinite_singular() {
assert_eq!(
recover_grand_mean_pooled_slope(&[]),
Err(PsychometricError::InvalidNumericInput)
);
assert_eq!(
recover_grand_mean_pooled_slope(&[ClusteredScore {
cluster_key: 1,
predictor: 0.0,
outcome: 1.0,
}]),
Err(PsychometricError::InvalidNumericInput)
);
assert_eq!(
recover_grand_mean_pooled_slope(&[
ClusteredScore {
cluster_key: 1,
predictor: 0.0,
outcome: 1.0,
},
ClusteredScore {
cluster_key: 1,
predictor: 1.0,
outcome: 2.0,
},
]),
Err(PsychometricError::InsufficientClusters)
);
assert_eq!(
recover_grand_mean_pooled_slope(&[
ClusteredScore {
cluster_key: 1,
predictor: f64::NAN,
outcome: 1.0,
},
ClusteredScore {
cluster_key: 2,
predictor: 1.0,
outcome: 2.0,
},
]),
Err(PsychometricError::InvalidNumericInput)
);
assert_eq!(
recover_grand_mean_pooled_slope(&[
ClusteredScore {
cluster_key: 1,
predictor: 0.0,
outcome: f64::INFINITY,
},
ClusteredScore {
cluster_key: 2,
predictor: 1.0,
outcome: 2.0,
},
]),
Err(PsychometricError::InvalidNumericInput)
);
assert_eq!(
recover_grand_mean_pooled_slope(&[
ClusteredScore {
cluster_key: 1,
predictor: 1.0,
outcome: 2.0,
},
ClusteredScore {
cluster_key: 2,
predictor: 1.0,
outcome: 3.0,
},
]),
Err(PsychometricError::SingularDesign)
);
assert_eq!(
recover_grand_mean_pooled_slope(&[
ClusteredScore {
cluster_key: 1,
predictor: 0.0,
outcome: 0.0,
},
ClusteredScore {
cluster_key: 2,
predictor: f64::MAX,
outcome: f64::MAX,
},
]),
Err(PsychometricError::InvalidNumericInput)
);
}

#[test]
fn overflowing_contextual_subtraction_fails_closed() {
assert_eq!(
Expand Down
12 changes: 12 additions & 0 deletions crates/psychometric_core/src/error.rs
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,11 @@ pub enum PsychometricError {
UnmatchedTimeVaryingInterval,
/// Fewer than two clusters were supplied for a within/between decomposition.
InsufficientClusters,
/// Enders and Tofighi (2007, p. 125) grand-mean pooled OLS slope was
/// treated as the within-cluster (Level 1) effect. That total slope is
/// generally an uninterpretable blend of the within-cluster and
/// between-cluster slopes (quoting Raudenbush & Bryk, 2002, pp. 138–139).
GrandMeanPooledSlopeIsNotWithinClusterEffect,
/// A membership or survey weight is empty, negative, or non-finite.
InvalidWeight,
/// An event-time interval is non-positive.
Expand Down Expand Up @@ -739,6 +744,9 @@ impl fmt::Display for PsychometricError {
"time-varying predictor discrete effect requires matching sampling and constancy intervals"
}
Self::InsufficientClusters => "within/between recovery requires at least two clusters",
Self::GrandMeanPooledSlopeIsNotWithinClusterEffect => {
"grand-mean pooled OLS slope is not the within-cluster effect"
}
Self::InvalidWeight => "invalid non-negative finite psychometric weight",
Self::NonPositiveInterval => "event-time interval must be strictly positive",
Self::InsufficientDraws => {
Expand Down Expand Up @@ -1292,6 +1300,10 @@ mod tests {
PsychometricError::InsufficientClusters.to_string(),
"within/between recovery requires at least two clusters"
);
assert_eq!(
PsychometricError::GrandMeanPooledSlopeIsNotWithinClusterEffect.to_string(),
"grand-mean pooled OLS slope is not the within-cluster effect"
);
assert_eq!(
PsychometricError::InvalidWeight.to_string(),
"invalid non-negative finite psychometric weight"
Expand Down
7 changes: 7 additions & 0 deletions crates/psychometric_core/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,9 @@
//! estimates across posterior draws on a CPU `f64` path without claiming Rubin
//! uncertainty pooling, combines draw-level OLS loadings with Rubin `T`,
//! decomposes cluster-mean within/between OLS and the CWC contextual effect,
//! names the grand-mean pooled OLS slope as the uninterpretable blend of
//! those two (Enders & Tofighi, 2007, p. 125) and refuses it as a
//! within-cluster effect,
//! maps event-time discrete lags through the exact scalar exponential, maps
//! already-centered irregular residuals without re-centering, remaps discrete
//! lags across unequal event intervals through that log-rate, recovers the
Expand Down Expand Up @@ -287,8 +290,12 @@ pub use cluster_mean::WithinBetweenSlopes;
pub use cluster_mean::kish_effective_sample_size;
/// Cluster-mean within/between OLS after CWC, plus the contextual effect.
pub use cluster_mean::recover_cluster_mean_within_between_slopes;
/// Grand-mean pooled OLS slope that ignores cluster membership.
pub use cluster_mean::recover_grand_mean_pooled_slope;
/// Kish-weighted least-squares slope.
pub use cluster_mean::recover_kish_weighted_slope;
/// Refuse treating that pooled slope as a within-cluster effect.
pub use cluster_mean::refuse_grand_mean_pooled_slope_as_within_cluster_effect;
/// Higher-order construct class.
pub use construct::ConstructClass;
/// Typed invariance evidence required before a latent-mean comparison.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -307,6 +307,10 @@ fn finite_alr_correlation_and_error_messages_are_stable() {
PsychometricError::InsufficientClusters.to_string(),
"within/between recovery requires at least two clusters"
);
assert_eq!(
PsychometricError::GrandMeanPooledSlopeIsNotWithinClusterEffect.to_string(),
"grand-mean pooled OLS slope is not the within-cluster effect"
);
assert_eq!(
PsychometricError::InvalidWeight.to_string(),
"invalid non-negative finite psychometric weight"
Expand Down
Loading
Loading