-
Notifications
You must be signed in to change notification settings - Fork 0
feat(simulation): exclude delayed documents before they are available #62
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
43e3d10
55247f3
5b5d88b
3688b6f
1951ab4
7a16e57
0446878
7ba0f09
a14db9e
fe29d2a
d9dffae
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
|
|
@@ -36,6 +36,7 @@ TEPP's approved PRD v0.4 and implementation plan are the primary product baselin | |||||
| | Hourly NIM product-development operations | [`docs/operations/HOURLY_NIM_PRODUCT_DEVELOPMENT.md`](docs/operations/HOURLY_NIM_PRODUCT_DEVELOPMENT.md) | | ||||||
| | Actions workflow fleet audit | [`docs/operations/ACTIONS_WORKFLOW_FLEET.md`](docs/operations/ACTIONS_WORKFLOW_FLEET.md) | | ||||||
| | Actions fleet research doctoring | [`docs/research/actions-workflow-fleet.md`](docs/research/actions-workflow-fleet.md) | | ||||||
| | Simulation cutoff-eligibility doctoring | [`docs/research/simulation-cutoff-eligibility.md`](docs/research/simulation-cutoff-eligibility.md) | | ||||||
| | Posterior ESEM/DSEM input-gate doctoring | [`docs/research/posterior-esem-input-gates.md`](docs/research/posterior-esem-input-gates.md) | | ||||||
| | Multilevel/event-time recovery doctoring | [`docs/research/multilevel-event-time-recovery.md`](docs/research/multilevel-event-time-recovery.md) | | ||||||
| | Rubin total-variance doctoring | [`docs/research/rubin-total-variance.md`](docs/research/rubin-total-variance.md) | | ||||||
|
|
@@ -53,9 +54,11 @@ TEPP's approved PRD v0.4 and implementation plan are the primary product baselin | |||||
| | Entity/project target SQL doctoring | [`docs/research/entity-project-sql.md`](docs/research/entity-project-sql.md) | | ||||||
| | Scientific claim-promotion gate doctoring | [`docs/research/scientific-claim-promotion-gates.md`](docs/research/scientific-claim-promotion-gates.md) | | ||||||
| | Retention/deletion/legal-hold doctoring | [`docs/research/retention-deletion-legal-hold.md`](docs/research/retention-deletion-legal-hold.md) | | ||||||
| | Stopword-deletion doctoring | [`docs/research/stopword-deletion.md`](docs/research/stopword-deletion.md) | | ||||||
| | Provider-payload minimization doctoring | [`docs/research/provider-payload-minimization.md`](docs/research/provider-payload-minimization.md) | | ||||||
| | Adaptive orchestration router doctoring | [`docs/research/adaptive-orchestration-router.md`](docs/research/adaptive-orchestration-router.md) | | ||||||
| | Mention-confidence Brier doctoring | [`docs/research/mention-confidence-brier.md`](docs/research/mention-confidence-brier.md) | | ||||||
| | Event-intelligence status-gate doctoring | [`docs/research/event-intelligence-status-gates.md`](docs/research/event-intelligence-status-gates.md) | | ||||||
|
Comment on lines
+59
to
+60
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🟡 Documentation map lists two entries twice The
Suggested change
Was this helpful? React with 👍 or 👎 to provide feedback. |
||||||
| | Stopword-deletion doctoring | [`docs/research/stopword-deletion.md`](docs/research/stopword-deletion.md) | | ||||||
| | Topic log-ratio coordinate doctoring | [`docs/research/topic-logratio-coordinates.md`](docs/research/topic-logratio-coordinates.md) | | ||||||
| | Hourly NIM OpenCode doctoring | [`docs/doctoring/hourly-nim-opencode-development.md`](docs/doctoring/hourly-nim-opencode-development.md) | | ||||||
| | Analysis engine v1 doctoring | [`docs/doctoring/analysis-engine-v1.md`](docs/doctoring/analysis-engine-v1.md) | | ||||||
|
|
||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,50 @@ | ||
| //! Delayed-reporting documents cannot enter a historical fit before they exist. | ||
|
|
||
| use temporal_core::KnowledgeCutoff; | ||
| use tepp_simulation::{SimulationConfig, SimulationError, generate, refuse_unavailable_document}; | ||
|
|
||
| #[test] | ||
| fn delayed_documents_are_excluded_and_counts_match_known_truth() { | ||
| let config = SimulationConfig::new(21, 4, 1, 2, 48, 24, 0, 0, 0, 0, 0, 0).expect("cfg"); | ||
| let manifest = generate(config).expect("corpus"); | ||
| let cutoff = KnowledgeCutoff::parse_rfc3339("2026-01-03T00:00:00Z").expect("cutoff"); | ||
|
|
||
| let eligible = manifest.documents_eligible_at_cutoff(&cutoff); | ||
| let truth_count = manifest | ||
| .documents() | ||
| .iter() | ||
| .filter(|document| document.available_time().instant() <= cutoff.instant()) | ||
| .count(); | ||
| assert_eq!(eligible.len(), truth_count); | ||
|
Comment on lines
+13
to
+18
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 📝 Info: Cutoff eligibility test mirrors the implementation The test computes its expected Was this helpful? React with 👍 or 👎 to provide feedback. |
||
| assert!( | ||
| truth_count < manifest.document_count(), | ||
| "cutoff must exclude at least one delayed document" | ||
| ); | ||
|
|
||
| let boundary_document = manifest.documents().first().expect("boundary document"); | ||
| let boundary_cutoff = | ||
| KnowledgeCutoff::parse_rfc3339(&boundary_document.available_time().to_rfc3339()) | ||
| .expect("boundary cutoff"); | ||
| let boundary_eligible = manifest.documents_eligible_at_cutoff(&boundary_cutoff); | ||
| assert!( | ||
| boundary_eligible | ||
| .iter() | ||
| .any(|document| document.document_id() == boundary_document.document_id()), | ||
| "a document available exactly at cutoff must be eligible" | ||
| ); | ||
| refuse_unavailable_document(boundary_document, &boundary_cutoff) | ||
| .expect("availability equal to cutoff is valid"); | ||
|
|
||
| for document in eligible { | ||
| refuse_unavailable_document(document, &cutoff).expect("eligible"); | ||
| } | ||
| let late = manifest | ||
| .documents() | ||
| .iter() | ||
| .find(|document| document.available_time().instant() > cutoff.instant()) | ||
| .expect("late document"); | ||
| assert_eq!( | ||
| refuse_unavailable_document(late, &cutoff), | ||
| Err(SimulationError::TemporalInvariantViolation) | ||
| ); | ||
| } | ||
|
seonghobae marked this conversation as resolved.
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,27 @@ | ||
| # Simulation cutoff eligibility | ||
|
|
||
| ## Scope | ||
|
|
||
| This note documents the `tepp_simulation` historical-fit filter: | ||
|
|
||
| 1. a delayed-reporting document is eligible only when `available_time <= knowledge_cutoff`; | ||
| 2. eligible counts must match the known-truth count computed from generated clocks; | ||
| 3. a late document fails closed. | ||
|
|
||
| No database migration is allocated. Interval-aware cutoff on uncertain availability remains a separate follow-up capability; no open PR currently claims it. | ||
|
|
||
| ## Authoritative sources | ||
|
|
||
| Jensen, C. S., & Snodgrass, R. T. (1999). Temporal data management. *IEEE Transactions on Knowledge and Data Engineering, 11*(1), 36–44. https://doi.org/10.1109/69.755613 | ||
|
|
||
| Dyreson, C. E., & Snodgrass, R. T. (1998). Supporting valid-time indeterminacy. *ACM Transactions on Database Systems, 23*(1), 1–57. https://doi.org/10.1145/288086.288087 | ||
|
|
||
| ## Application | ||
|
|
||
| Jensen and Snodgrass (1999) distinguish valid time from transaction time. TEPP defines `available_time` as an application-specific knowledge-availability clock for historical fits. This mapping is specified by ADR 0002 and the `tepp_simulation` eligibility and fail-closed validation APIs. A document written at `document_time` but released later cannot enter a cutoff that precedes `available_time` (Jensen & Snodgrass, 1999; Dyreson & Snodgrass, 1998). | ||
|
|
||
| ## Verification | ||
|
|
||
| - a delayed corpus has fewer eligible documents than total documents at an early cutoff; | ||
| - eligible count equals the known-truth count from generated clocks; | ||
| - `refuse_unavailable_document` admits a document whose availability is exactly the cutoff and denies late ones with `TemporalInvariantViolation`. |
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🔍 Documentation map file is duplicated end-to-end
DOCUMENTATION.md contains the whole map twice (headers at line 1 and line 83). This predates the PR, but the PR edits only the first copy, so the two now diverge. Worth consolidating.
Was this helpful? React with 👍 or 👎 to provide feedback.