Skip to content
Closed
Show file tree
Hide file tree
Changes from 38 commits
Commits
Show all changes
61 commits
Select commit Hold shift + click to select a range
d337259
docs: add canonical product requirements
seonghobae Aug 9, 2026
a5f9f8f
docs: add canonical technical requirements
seonghobae Aug 9, 2026
c982f01
docs: add canonical architecture description
seonghobae Aug 9, 2026
f37fb22
docs: add architecture decision record index
seonghobae Aug 9, 2026
c272670
docs(adr): define measurement ownership boundary
seonghobae Aug 9, 2026
73e84be
docs(adr): define Rust-first numerical ownership
seonghobae Aug 9, 2026
08aa056
docs(adr): define content-addressed measurement contracts
seonghobae Aug 9, 2026
186f448
docs(adr): define governed rubric item-bank lifecycle
seonghobae Aug 9, 2026
0779678
docs(adr): define automated scoring rater model
seonghobae Aug 9, 2026
7075c84
docs(adr): define relation-safe model selection
seonghobae Aug 9, 2026
046f816
docs(adr): define multilevel and temporal architecture
seonghobae Aug 9, 2026
cc383b4
docs(adr): define true-parameter recovery evidence
seonghobae Aug 9, 2026
2332f2f
docs(adr): define adaptive rotation selection
seonghobae Aug 9, 2026
7ba3d34
docs(adr): define LLM orchestration credential boundary
seonghobae Aug 9, 2026
599be48
docs(uml): add component architecture diagram
seonghobae Aug 9, 2026
07bffed
docs(uml): add governed scoring sequence
seonghobae Aug 9, 2026
3d450a3
docs(uml): add model selection sequence
seonghobae Aug 9, 2026
f94bce4
docs(erd): add logical domain model
seonghobae Aug 9, 2026
667b331
docs(uml): add deployment composition diagram
seonghobae Aug 9, 2026
3b771bf
docs(uml): add item lifecycle state diagram
seonghobae Aug 9, 2026
8e52e0e
docs(uml): add diagram index
seonghobae Aug 9, 2026
d1b7306
docs(traceability): map requirements to implementation
seonghobae Aug 9, 2026
c245f7a
docs(traceability): record research to architecture basis
seonghobae Aug 9, 2026
8229066
docs(uml): add canonical item-bank state view
seonghobae Aug 9, 2026
a1f497a
docs: deprecate stale PRD TRD summary
seonghobae Aug 9, 2026
8f4e9e7
test(docs): pin canonical architecture documentation
seonghobae Aug 9, 2026
24eeec3
docs(changelog): record canonical architecture baseline
seonghobae Aug 9, 2026
66f915c
docs: add canonical documentation authority map
seonghobae Aug 9, 2026
93671dd
docs(security): add reusable-core threat model
seonghobae Aug 9, 2026
ee1aeb3
docs: add architecture documentation completeness matrix
seonghobae Aug 9, 2026
22fc5c1
docs(adr): define canonical PyO3 public export registry
seonghobae Aug 9, 2026
219a203
docs(adr): define purpose-limited sensitive-data handling
seonghobae Aug 9, 2026
fe5371e
docs(adr): index PyO3 and sensitive-data decisions
seonghobae Aug 9, 2026
82e4d25
docs(traceability): add security privacy binding and scientific inter…
seonghobae Aug 9, 2026
bb9deb1
test(docs): enforce security completeness and interpretation invariants
seonghobae Aug 9, 2026
c5a9139
docs(changelog): include security and documentation governance baseline
seonghobae Aug 9, 2026
847ab1a
docs(adr): add canonical decision-record template
seonghobae Aug 9, 2026
ca49a77
docs(adr): index canonical ADR template
seonghobae Aug 9, 2026
2f82315
docs(architecture): add standards watch registry
seonghobae Aug 9, 2026
3e231bf
docs(validation): add canonical V&V plan
seonghobae Aug 9, 2026
8e69581
docs(adr): record continuous execution governance
seonghobae Aug 9, 2026
fe402bc
docs(adr): index continuous execution governance
seonghobae Aug 9, 2026
941456e
fix(docs): conform ADR 0013 status contract
seonghobae Aug 9, 2026
1e1a61f
test(docs): pin standards and V&V architecture spine
seonghobae Aug 9, 2026
88c6127
docs(architecture): index standards and V&V authorities
seonghobae Aug 9, 2026
c49eea2
test(docs): require standards and V&V index links
seonghobae Aug 9, 2026
9323166
docs(adr): align architecture reference edition
seonghobae Aug 9, 2026
61842ee
docs(architecture): track standards and V&V completeness
seonghobae Aug 9, 2026
a5259f5
test(docs): reject nested PlantUML lifecycle alias
seonghobae Aug 9, 2026
df4bb5d
fix(docs): avoid nested PlantUML lifecycle document
seonghobae Aug 9, 2026
8ea339d
test(docs): require execution-governance traceability
seonghobae Aug 9, 2026
ec4f0a3
docs: consolidate orchestration research doctoring
seonghobae Aug 9, 2026
08353f4
docs: make active standards revisions explicit
seonghobae Aug 9, 2026
7923ae7
docs: align rotation maturity with protected main
seonghobae Aug 9, 2026
b893560
docs: accept protected-main adaptive rotation policy
seonghobae Aug 9, 2026
17ae968
docs: align ADR index with implemented rotation
seonghobae Aug 9, 2026
232a2c6
docs: align traceability with protected-main rotation
seonghobae Aug 9, 2026
92d3fe5
test(docs): pin implemented rotation maturity
seonghobae Aug 9, 2026
02569a3
docs(adr): exclude template from live decision index
seonghobae Aug 9, 2026
b24aefd
fix(docs): complete canonical architecture contract
seonghobae Aug 11, 2026
a672a6c
Merge remote-tracking branch 'origin/main' into codex/pr-604-canonica…
seonghobae Aug 11, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
342 changes: 342 additions & 0 deletions ARCHITECTURE.md

Large diffs are not rendered by default.

281 changes: 281 additions & 0 deletions docs/PRD.md

Large diffs are not rendered by default.

46 changes: 46 additions & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# fast-mlsirm documentation authority

This index distinguishes governing product/architecture documents from implementation history and method-local evidence.

## Canonical architecture package

| Document | Governing purpose |
|---|---|
| [`../ARCHITECTURE.md`](../ARCHITECTURE.md) | System of interest, bounded contexts, dependency direction, component/data/deployment/scientific views |
| [`PRD.md`](PRD.md) | Product requirements, users, workflows, non-goals, acceptance boundaries |
| [`TRD.md`](TRD.md) | Technical realization, numerical/runtime/security/resource/release requirements |
| [`adr/README.md`](adr/README.md) | Durable architecture/scientific decision log and status history |
| [`uml/README.md`](uml/README.md) | PlantUML component, sequence, lifecycle, model-selection and deployment views |
| [`erd/domain-model.puml`](erd/domain-model.puml) | Logical reusable-domain artifact relationships; **not** a hosted ORM schema |
| [`traceability/requirements-matrix.md`](traceability/requirements-matrix.md) | PRD/TRD/ADR -> protected-main implementation/evidence maturity |
| [`traceability/research-basis.md`](traceability/research-basis.md) | Scientific/standards evidence and APA 7 reference mapping |
| [`documentation_coverage.md`](documentation_coverage.md) | Documentation completeness states, remaining P0/P1/P2 gaps and maintenance gate |
| [`security/threat-model.md`](security/threat-model.md) | Reusable-core trust/threat model; hosted product threats remain downstream |
| [`doctoring/`](doctoring/) | Method/security/interoperability evidence and conservative implementation boundaries |
| [`../AGENTS.md`](../AGENTS.md), [`../CLAUDE.md`](../CLAUDE.md) | Agent/developer operating rules aligned to this architecture |
| [`../CHANGELOG.md`](../CHANGELOG.md) | User-visible released/unreleased change history |

`prd_trd_summary.md` is historical and must not compete with `PRD.md` and `TRD.md` as a requirements source.

## Authority and status

1. Protected-main source/tests define executable behavior.
2. Accepted ADRs define governing architecture/scientific decisions.
3. PRD/TRD define product/technical requirements and non-claims.
4. `ARCHITECTURE.md`, UML/ERD and the threat model define coherent system views.
5. Method-specific doctoring and primary literature justify local scientific/interoperability details.
6. Proposed ADRs, open PRs/issues and plans describe future/active work and are not released capability merely because they exist.

A conversation or PR body is discovery evidence until the durable decision is captured in the documents above.

## Implementation history

`docs/superpowers/specs/` and `docs/superpowers/plans/` preserve bounded design/implementation history. They do not automatically remain normative after implementation. If a plan creates a durable product/architecture/scientific decision, update the canonical PRD/TRD/ADR/traceability set.

## Completeness gate

A material change is incomplete if it creates a contradiction among code, accepted ADRs, PRD/TRD, architecture diagrams, security/threat model, traceability, doctoring or release evidence. The documentation-contract test and `documentation_coverage.md` make these gaps visible; a missing or stale canonical artifact is release-maintenance debt rather than harmless prose drift.

## Cross-repository boundary

`fast-mlsirm` is the standalone reusable measurement/psychometric core. Psychometrics Commons or another owning downstream service is responsible for hosted HTTP/session/consent/tenant/RBAC/UI/database/deployment lifecycle. Architecture documents here may define interoperable reusable artifacts and versioned handoffs without creating a shared application database or reverse product dependency.
257 changes: 257 additions & 0 deletions docs/TRD.md

Large diffs are not rendered by default.

80 changes: 80 additions & 0 deletions docs/adr/0000-template.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
# ADR-NNNN: Decision title

Status: **Proposed**
Comment thread
seonghobae marked this conversation as resolved.
Outdated
Date: YYYY-MM-DD
Supersedes: none
Superseded by: none

## Context

Describe the observed product/scientific/technical problem and current protected-main behavior. Separate facts from assumptions, active PR work, and future plans.

## Decision drivers

- Driver one.
- Driver two.

## Ownership and dependency direction

State the owning bounded context/repository and which dependencies are allowed or forbidden. Explicitly note whether the decision changes the `fast-mlsirm` reusable-core vs Psychometrics Commons hosted-product boundary.

## Decision

State the decision precisely enough that code/tests/documentation can determine compliance. If only part of the decision is implemented, mark the ADR Proposed or distinguish the accepted invariant from proposed implementation.

## Invariants / acceptance evidence

1. Invariant tied to a test, recovery study, security check, or other exact evidence.
2. Invariant tied to a failure/degraded/recovery rule.

## Non-goals and claims not made

List adjacent capabilities or interpretations this decision does not authorize.

## Consequences and trade-offs

### Benefits

- Benefit.

### Costs / risks

- Cost or risk.

## Alternatives considered

### Alternative A

Why it was considered and rejected/deferred.

### Alternative B

Why it was considered and rejected/deferred.

## Failure, degraded, and recovery behavior

Describe fail-closed/fallback behavior, retry/idempotency if applicable, operator evidence, recovery/rollback, and how a failed migration or rollout is handled.

## Security and privacy implications

Cover new credentials/permissions, data classification, PII/sensitive evidence, native/provider trust, supply-chain surface, retention, auditability, and threat-model changes where applicable.

## Compatibility, migration, and rollback

Define public API/schema/artifact compatibility, migration steps, old-artifact interpretation, rollback and how supersession preserves decision history.

## Verification and release evidence

List required unit/property/fuzz/security tests, Rust↔Python or CPU↔GPU parity, true-parameter recovery/coverage, documentation/traceability, package/release evidence, or downstream contract checks before the implementation may be called Accepted/released.

## Research and standards basis

Use APA 7 references to primary peer-reviewed methods and current official standards/specifications where material. Mark preprints as preprints; do not use legacy software output as the scientific oracle when the original method is available.

## Follow-ups

Record deliberately deferred bounded work, with owning issue/PR only as a tracking aid rather than the decision authority.

## Reversal / supersession conditions

State what evidence, standard, product-boundary change, or implementation failure should trigger a new superseding ADR instead of silently editing this accepted history.
72 changes: 72 additions & 0 deletions docs/adr/0001-domain-neutral-measurement-boundary.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
# ADR-0001: Domain-neutral measurement boundary

Status: **Accepted**
Date: 2026-08-09

## Context

`fast-mlsirm` has grown from an MLS2PLM-focused toolkit into a reusable measurement layer with assessment/rubric/scoring contracts, automated-scoring adapters, psychometric diagnostics, item/rater calibration and release evidence. ContextualWisdomLab also has downstream hosted/application bounded contexts, especially Psychometrics Commons.

Without an explicit boundary, the library can accidentally absorb participant/session state, identity, hosted persistence, UI and deployment logic. That would make the numerical core harder to install independently, create circular repository dependencies, and duplicate ownership already assigned to other CWL services.

## Decision

`fast-mlsirm` owns reusable domain-neutral measurement contracts and scientific/numerical capabilities:

- Assessment/Rubric/Scoring contracts and observations;
- calibration, CTT/IRT/MIRT, MLSIRM/MLS2PLM, facets, testlets and related kernels;
- model diagnostics, linking, DIF/invariance/fairness, CAT/ATA and G-theory;
- factor retention/model comparison, bifactor scoreability, rotation and recovery;
- automated-scoring and LLM-judge measurement primitives;
- governed rubric/item-bank primitives and portable scientific/audit reports.

The hosted product boundary belongs downstream. In particular `ContextualWisdomLab/psychometrics-commons` owns hosted/public/admin APIs, participant/session/response/consent/result lifecycle, product databases/migrations, tenant/resource authorization, UI/reference client behavior, deployment composition and research-release orchestration.

The dependency direction is:

```text
hosted/downstream product -> fast-mlsirm
```

and never:

```text
fast-mlsirm -> hosted product
```

Other CWL services are explicit optional integrations, not hidden implementation dependencies.

## Invariants and evidence

- The package must install and execute without Psychometrics Commons source or runtime.
- No product ORM/database, HTTP route, session/consent or UI type may become a required `fast_mlsirm` dependency.
- Cross-repository composition uses versioned APIs/contracts/immutable artifacts.
- Hosted product state must not be recreated under a library-local assessment runtime service.
- `AGENTS.md` and `CLAUDE.md` carry the same boundary.

## Consequences

Benefits:

- independent adoption and testing;
- clear MSA ownership and security authority;
- lower coupling between scientific evolution and product deployment;
- reusable measurement contracts across essay, RAG, enterprise-issue and other domains.

Costs:

- downstream adapters are required for persistence/transport;
- some end-to-end features require cross-repository integration tests rather than one monolith.

## Alternatives considered

1. **Make fast-mlsirm the hosted platform.** Rejected because it couples scientific kernels to application infrastructure.
2. **Keep only numerical functions and move all contracts downstream.** Rejected because versioned measurement/scoring contracts are reusable scientific primitives and must stay adjacent to the numerical interpretation they govern.

## Reversal conditions

Supersede this ADR only if the organization intentionally redefines repository bounded contexts and provides a migration plan preserving independent scientific/numerical reuse.

## References

International Organization for Standardization, International Electrotechnical Commission, & Institute of Electrical and Electronics Engineers. (2022). *ISO/IEC/IEEE 42010:2022 Software, systems and enterprise—Architecture description*.
60 changes: 60 additions & 0 deletions docs/adr/0002-rust-first-numerical-ownership.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
# ADR-0002: Rust-first numerical ownership

Status: **Accepted**
Date: 2026-08-09

## Context

The repository exposes Python APIs while supporting computationally intensive psychometric estimation, diagnostics, calibration and simulation. Maintaining independent Python and Rust production formulas creates drift risk, doubles verification burden and makes CPU/GPU ownership ambiguous. At the same time, transparent NumPy paths are valuable for parity, research inspection and controlled fallback.

## Decision

Rust is the production source of truth for mathematically material psychometric computation, including likelihoods, gradients, Hessians/information matrices, optimization, scoring/ranking, item/factor information and other numerical kernels when promoted to production capability.

Python may:

- validate and bound inputs;
- marshal NumPy arrays;
- orchestrate domains/providers;
- expose typed results;
- render reports;
- retain governed reference/fallback calculations where a parity contract exists.

Python shall not become an independently evolving second production formula.

The public backend architecture may expose `auto`, `rust` and governed `numpy` reference/fallback choices for APIs that currently support them. Rust is the preferred resolved production backend when the extension is available. GPU is a Rust device path, not a third psychometric formula implementation.

The canonical PyO3 layer must support feature growth without independent PRs overwriting initialization/export structure. Secondary module symbols, if retained, must be registered through one auditable binding architecture.

## Numerical invariants

- Formula changes update Rust and any governed reference path together.
- Analytic derivatives are checked against independent finite-difference or equivalent oracles where practical.
- CPU/GPU parity uses identification-aware invariants rather than raw non-identifiable coordinates.
- Caller-controlled allocation sizes are bounded before allocation.
- Non-finite numerical boundaries fail closed where the model requires finite values.
- Computationally material CPU parallelism should minimize task/thread context-switch overhead.

## Consequences

Benefits:

- one production arithmetic authority;
- safer GPU/CPU parity and performance work;
- clearer auditability and reproducibility;
- Python remains ergonomic without owning scientific numerics.

Costs:

- new mathematical features require Rust/PyO3 work before product release;
- pure-Python prototypes cannot be declared production estimators without migration.

## Alternatives considered

1. **NumPy as primary backend, Rust as optional accelerator.** Rejected as the long-term architecture because production behavior can diverge and high-cost computations remain Python-owned.
2. **Rust-only API.** Rejected because Python remains the principal research/product integration surface.
3. **Independent CPU/GPU model implementations.** Rejected; GPU must share the same mathematical contract and parity evidence.

## Reversal conditions

A different numerical owner requires an ADR demonstrating equivalent or stronger correctness, performance, packaging, parity and scientific-evidence guarantees.
73 changes: 73 additions & 0 deletions docs/adr/0003-content-addressed-measurement-contracts.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
# ADR-0003: Content-addressed measurement contracts

Status: **Accepted**
Date: 2026-08-09

## Context

Psychometric interpretation depends on exact construct, rubric, task, response, rater/engine, calibration and software revisions. Human-readable IDs alone do not prove content identity, while mutable objects can make later audits falsely appear to reproduce the original analysis.

## Decision

Reusable measurement artifacts use versioned, canonical and content-addressed contracts wherever semantic identity affects interpretation.

Key rules:

1. Schema/wire version and semantic/domain revision are separate fields.
2. Authoritative content identity uses deterministic canonical serialization and a full cryptographic digest where the contract exposes fingerprints.
3. Short public handles may aid display but never replace full fingerprint comparison at a trust boundary.
4. Aggregate artifacts replay/verify package-owned child objects when crossing a trust boundary rather than trusting parent caches or display IDs.
Comment thread
seonghobae marked this conversation as resolved.
5. Logical task ID and exact task revision remain separate.
6. Stable respondent/system identity and response artifact/revision remain separate.
7. Rater/engine descriptor identity and individual rater identity remain separately representable where many-facet interpretation requires them.
8. Published/approved artifacts are immutable; corrections create superseding revisions.
9. Caller/provider text is bounded and sensitive content is not embedded in error identifiers or audit hashes beyond the explicit canonical input.

## Primary contract family

```text
RubricSpecification
|
v
AssessmentSpec / policy references
|
+--> task + exact task revision
|
v
ScoringRequest
|
v
ScoreObservation / ScoringResult
|
v
Calibration design/report
```

Rubric-centered item generation extends the same chain:

```text
RubricSpecification -> ItemBlueprint -> GenerationContract
-> Candidate -> Screening/Calibration -> Item-bank revision
```

## Invariants

- A one-byte semantic change that participates in canonical content changes the fingerprint.
- Reusing an identifier with changed canonical content fails closed at governed boundaries.
- Source/evidence spans are verified against the exact source revision when source evidence is part of the contract.
- Unknown major schema versions do not silently downgrade.
- Serialization is deterministic across input ordering where ordering is not semantically meaningful and preserves declared ordering where it is meaningful.

## Consequences

This increases object/schema discipline and migration work but provides replay protection, reproducible research, audit evidence, safe caching and reliable cross-service composition.

## Alternatives considered

- **Database integer IDs as identity.** Rejected because they identify rows, not semantic content, and are not portable across independent deployments.
- **Mutable named revisions.** Rejected because an old result could silently point at new content.
- **Hash everything including uncontrolled raw text in every report.** Rejected; only contract-relevant content is included, with privacy/data-minimization boundaries.

## Reversal conditions

A replacement must provide equal or stronger deterministic replay, compatibility, privacy and cross-deployment identity guarantees.
Loading
Loading