Skip to content

test(intake): stabilize ATIF ingest tests against the 30-day read lookback - #174

Merged
marcusds merged 1 commit into
mainfrom
fix-atif-ingest-experiment-rollup/mschwab
Jun 4, 2026
Merged

test(intake): stabilize ATIF ingest tests against the 30-day read lookback#174
marcusds merged 1 commit into
mainfrom
fix-atif-ingest-experiment-rollup/mschwab

Conversation

@marcusds

@marcusds marcusds commented Jun 3, 2026

Copy link
Copy Markdown
Contributor

What

Pins the historical span/trace reads in services/intake/tests/integration/spans/test_atif_ingest.py to an explicit started_at lower bound.

Why

test_atif_ingest_accepts_example_trajectory_and_reconstructs_read_side_data and test_atif_trace_tokens_do_not_double_count_when_trajectory_and_steps_both_carry_metrics ingest spans with fixed 2026-05-04 timestamps and read them back with no time filter. The spans list/traces endpoints default started_at to now − 30 days (DEFAULT_LIST_LOOKBACK_DAYS). Once wall-clock passed ~30 days after the fixtures (≈ 2026-06-03T18:57Z), the reads returned nothing → assert 0 == 7 / StopIteration. This is a calendar time-bomb in the tests, not an ingest regression — verified locally (Docker/ClickHouse): the 7 spans are written fine; only the read filtered them out by date.

This is currently red on main and every open PR (the integration check is non-blocking, so PRs kept merging).

Fix

Add an explicit filter[started_at][gte] (constant _HISTORICAL_GTE = 2024-01-01) to each historical list read so the suite no longer depends on the current date. Timestamps and reconstruction assertions are unchanged. Product behavior (recent-by-default lookback) untouched.

Testing

pytest services/intake/tests/integration/spans/test_atif_ingest.py → 16 passed (was 2 failed). ruff + ty clean.

Summary by CodeRabbit

  • Tests
    • Improved test reliability and determinism for span and trace retrieval by applying a fixed time-range filter to all relevant test queries.
    • This change stabilizes example-trajectory, evaluation-scoped, cross-run, same-session, and trace-rollup checks to reduce intermittent failures.

@marcusds
marcusds requested review from a team as code owners June 3, 2026 23:59
@coderabbitai

coderabbitai Bot commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 1f057633-f772-4dac-bc18-334541709b80

📥 Commits

Reviewing files that changed from the base of the PR and between 41b857c and 3a9bc49.

📒 Files selected for processing (1)
  • services/intake/tests/integration/spans/test_atif_ingest.py
🚧 Files skipped from review as they are similar to previous changes (1)
  • services/intake/tests/integration/spans/test_atif_ingest.py

📝 Walkthrough

Walkthrough

Adds a module-level _HISTORICAL_GTE timestamp and applies filter[started_at][gte]=_HISTORICAL_GTE to all /spans and /traces test queries in the ATIF ingest integration test to fix time-dependent result windows.

Changes

Test Timestamp Pinning

Layer / File(s) Summary
Timestamp constant and query filters
services/intake/tests/integration/spans/test_atif_ingest.py
Added _HISTORICAL_GTE = "2024-01-01T00:00:00Z" and appended filter[started_at][gte]=_HISTORICAL_GTE to the example-trajectory, evaluation-scoped, per-field, evaluation-root, cross-run, same-session, and trace-rollup /spans and /traces queries.
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title directly and specifically describes the main change: stabilizing ATIF ingest tests against the 30-day read lookback by adding explicit time filters to historical span/trace reads.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix-atif-ingest-experiment-rollup/mschwab

Comment @coderabbitai help to get the list of available commands and usage tips.

These tests ingest spans with fixed 2026-05 timestamps and read them back
without a time filter. The spans list/traces endpoints default started_at to
now-30d, so once wall-clock passes ~30 days after the fixtures the reads return
nothing and the suite fails (assert 0 == 7 / StopIteration) — unrelated to any
ingest logic.

Pin each historical list read to an explicit started_at lower bound so the
suite is stable regardless of the current date.

Signed-off-by: mschwab <mschwab@nvidia.com>
@marcusds
marcusds force-pushed the fix-atif-ingest-experiment-rollup/mschwab branch from 41b857c to 3a9bc49 Compare June 4, 2026 00:02
@github-actions

github-actions Bot commented Jun 4, 2026

Copy link
Copy Markdown
Contributor
Suite Lines Covered Line Rate Branch Rate
Unit Tests 18647/24682 75.5% 62.1%
Integration Tests 11967/23459 51.0% 26.2%

@marcusds
marcusds enabled auto-merge June 4, 2026 00:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants