Skip to content

fix(ci): paginate nightly scorecard trend lookup - #3922

Merged
jyaunches merged 2 commits into
NVIDIA:mainfrom
jyaunches:fix/scorecard-prior-day-trend
May 20, 2026
Merged

fix(ci): paginate nightly scorecard trend lookup#3922
jyaunches merged 2 commits into
NVIDIA:mainfrom
jyaunches:fix/scorecard-prior-day-trend

Conversation

@jyaunches

@jyaunches jyaunches commented May 20, 2026

Copy link
Copy Markdown
Contributor

Summary

  • add a regression test for scorecard trend lookup when selective dispatch volume fills the first workflow-runs API page
  • paginate the prior scheduled-run lookup in the nightly scorecard
  • keep the lookup bounded while allowing the previous scheduled run to be found beyond page one

Validation

  • npm test -- test/nightly-scorecard.test.ts test/validate-e2e-coverage.test.ts test/e2e-advisor-dispatch.test.ts

Summary by CodeRabbit

  • Tests

    • Added test coverage for detecting prior scheduled workflow runs with filtering and pagination.
  • Chores

    • Improved nightly test workflow to use optimized pagination when fetching prior workflow runs for trend comparison.

Review Change Stack

jyaunches added 2 commits May 20, 2026 15:41
This test demonstrates the bug where the nightly scorecard only checks the first workflow-runs API page for prior scheduled runs. Expected: find the prior scheduled run even when selective dispatches fill page one. Actual: returns no prior-day data.
Resolves the nightly scorecard reporting no prior-day data when selective dispatch volume pushes the previous scheduled run beyond the first workflow-runs API page. The fix checks additional pages until it finds a prior scheduled run or reaches a bounded page limit.
@coderabbitai

coderabbitai Bot commented May 20, 2026

Copy link
Copy Markdown
Contributor

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 56e19e0d-9051-4325-afb7-c47cf66f1ddb

📥 Commits

Reviewing files that changed from the base of the PR and between e122450 and 1a4eddc.

📒 Files selected for processing (2)
  • .github/workflows/nightly-e2e.yaml
  • test/nightly-scorecard.test.ts

📝 Walkthrough

Walkthrough

The nightly-e2e workflow's scorecard job now fetches prior-day scheduled runs using pagination (up to 10 pages, 100 items per page) to improve trend comparison lookup, replacing a fixed single-page call. A new test suite validates the core pagination filtering logic.

Changes

Nightly Scorecard Prior-Run Pagination

Layer / File(s) Summary
Workflow pagination implementation
.github/workflows/nightly-e2e.yaml
Scorecard job's prior-run fetch replaces a single listWorkflowRuns call (per_page: 50) with a paginated loop (10 pages, per_page: 100) that stops once qualified completed scheduled runs older than 24h are found.
Pagination helper test
test/nightly-scorecard.test.ts
New Vitest suite adds findPriorScheduledRunFromPages helper that filters paginated workflow runs by completion status, schedule event, and creation time; test confirms correct run selection across multiple pages.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Possibly related PRs

  • NVIDIA/NemoClaw#3735: Both PRs modify .github/workflows/nightly-e2e.yaml's nightly scorecard script around how "prior-day" workflow runs are looked up for trend comparison.

Suggested labels

CI/CD, fix, E2E, enhancement: testing

Suggested reviewers

  • ericksoa
  • cv

Poem

🐰 A paginated sprint through workflow runs,
Finding scheduled friends from days before,
With tests to prove each loop is true,
The scorecard trends shine bright once more! ✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main change: implementing pagination for the nightly scorecard trend lookup to find prior scheduled runs across multiple API pages instead of just the first page.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 ESLint

If the error stems from missing dependencies, add them to the package.json file. For unrecoverable errors (e.g., due to private dependencies), disable the tool in the CodeRabbit configuration.

ESLint skipped: no ESLint configuration detected in root package.json. To enable, add eslint to devDependencies.


Comment @coderabbitai help to get the list of available commands and usage tips.

@jyaunches
jyaunches merged commit 5622c3c into NVIDIA:main May 20, 2026
30 checks passed
@wscurran wscurran added the bug-fix PR fixes a bug or regression label Jun 8, 2026
@jyaunches
jyaunches deleted the fix/scorecard-prior-day-trend branch June 22, 2026 13:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug-fix PR fixes a bug or regression

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants