[DQ-06] Add data quality center - #2402
Conversation
|
Bugbot is not enabled for your account, so this pull request was not reviewed. Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs. |
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
📝 WalkthroughWalkthroughAdds a server data-quality overview for five check types, exposes it through tRPC, and adds web and mobile review centers. Adds navigation routes, inline source-overlap indicators, tests, and Storybook states. ChangesData quality feature
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant User
participant DataQualityPage
participant DataQualityScreen
participant processingRouter
participant DataQualityRepository
participant DataQualityCenter
User->>DataQualityPage: open web data-quality route
User->>DataQualityScreen: open mobile data-quality screen
DataQualityPage->>processingRouter: query dataQuality
DataQualityScreen->>processingRouter: query dataQuality
processingRouter->>DataQualityRepository: request overview
DataQualityRepository-->>processingRouter: return DataQualityOverview
processingRouter-->>DataQualityPage: return validated overview
processingRouter-->>DataQualityScreen: return validated overview
DataQualityPage->>DataQualityCenter: render checks and review links
DataQualityScreen->>DataQualityCenter: render checks and review actions
Assessment against linked issues
Possibly related PRs
Suggested labels: 🚥 Pre-merge checks | ✅ 2✅ Passed checks (2 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Reviewer's GuideImplements a server-owned data quality overview endpoint and wires it into new cross-platform Data Quality centers on web and mobile, plus inline source-overlap indicators on activity and nutrition surfaces, with tests and stories for the new behavior. Sequence diagram for processingRouter.dataQuality end-to-end flowsequenceDiagram
actor User
participant WebApp as Web_or_Mobile_UI
participant TRPC as trpc.processing.dataQuality
participant Router as processingRouter.dataQuality
participant Sync as ensureProvidersRegistered
participant DQRepo as DataQualityRepository
participant ProcRepo as ProcessingRepository
participant NutRepo as NutritionAnalyticsRepository
participant ActRepo as ActivitiesCalendarRepository
participant AnomRepo as AnomalyDetectionRepository
participant JournalRepo as JournalRepository
User->>WebApp: Navigate to data quality
WebApp->>TRPC: useQuery({ endDate })
TRPC->>Router: dataQuality({ endDate })
Router->>Sync: ensureProvidersRegistered()
Sync-->>Router: undefined
Router->>DQRepo: overview(endDate)
par Processing status
DQRepo->>ProcRepo: status({})
ProcRepo-->>DQRepo: processing status
and Nutrition quality
DQRepo->>NutRepo: getMicronutrientDataQuality(DATA_QUALITY_WINDOW_DAYS)
NutRepo-->>DQRepo: nutrition quality
and Activity overlap
DQRepo->>ActRepo: getWeekList({ weeks: 1, endDate, includeProviderAbsent: true })
ActRepo-->>DQRepo: activityDays
and Anomalies
DQRepo->>AnomRepo: getHistory(DATA_QUALITY_WINDOW_DAYS, endDate)
AnomRepo-->>DQRepo: anomalies
and Journal entries
DQRepo->>JournalRepo: listEntries(DATA_QUALITY_WINDOW_DAYS)
JournalRepo-->>DQRepo: journalEntries
end
DQRepo-->>Router: DataQualityOverview
Router-->>TRPC: DataQualityOverview
TRPC-->>WebApp: DataQualityOverview
WebApp-->>User: Render DataQualityCenter with checks
File-Level Changes
Assessment against linked issues
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
|
🤖 Review skipped: Repository rate limit exceeded. Free accounts are limited to 2 reviews per 4 hours per repository. Upgrade to a paid plan for unlimited reviews. |
PR Summary by QodoAdd server-owned Data Quality center across server, web, and mobile
AI Description
Diagram
High-Level Assessment
Files changed (28)
|
Mobile PreviewScan to open on device:
To test on device:
|
Code Review by Qodo
Context used✅ Compliance rules (platform):
194 rules✅ Skills:
fix-provider, write-tests, cloudflare 1.
|
![qodo-code-review[bot]](https://avatars.githubusercontent.com/in/484649?s=60&v=4){kind=small}
|
Storybook previews for This comment updates automatically on each PR push. |
|
CI diagnosis (run 30744242217): |
|
🤖 Review skipped: Repository rate limit exceeded. Free accounts are limited to 2 reviews per 4 hours per repository. Upgrade to a paid plan for unlimited reviews. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 17
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/mobile/components/DataQualityCenter.stories.tsx`:
- Around line 53-55: Add local Storybook stories in
DataQualityCenter.stories.tsx that use fixture data containing healthy and
informational check statuses, so both statusStyle and statusLabel branches are
rendered. Keep the existing default, Loading, and Empty stories unchanged, and
define the new fixtures within this file rather than sharing them with the web
story.
In `@packages/mobile/components/DataQualityCenter.test.tsx`:
- Around line 35-50: Extend the DataQualityCenter test suite to cover the !data
empty state and the onReview interaction path. Add the appropriate vi mock and
renderer-compatible user-event helper, render with a review callback, activate
the existing “Review nutrition” button, and assert the callback receives the
expected check; also verify the empty-state output when data is omitted.
In `@packages/mobile/components/DataQualityCenter.tsx`:
- Around line 68-72: Both clients incorrectly construct count text from the
non-countable server label; update
packages/mobile/components/DataQualityCenter.tsx lines 68-72 and
packages/web/src/components/DataQualityCenter.tsx lines 58-62 to remove those
count Text blocks or render the server-authored grammatical message/countable
label, keeping both platforms consistent and server-owned.
- Around line 82-89: Conditionally render the review Pressable in
DataQualityCenter only when onReview is provided, while preserving its current
label, styles, and callback behavior. Update the corresponding test to supply an
onReview spy and assert it is invoked instead of expecting a button without a
handler.
- Around line 2-6: Remove the direct server-source type import from
DataQualityCenter and obtain DataQualityCheck, DataQualityCheckKey, and
DataQualityOverview from an appropriate shared domain package or the tRPC router
output type. Update all usages in DataQualityCenter to reference the new shared
or inferred types while preserving their existing shapes and behavior.
- Around line 184-193: Update the healthy and attention styles in the
data-quality status configuration to use the existing themed colors. Map healthy
to colors.positiveSubtle and colors.positive, and attention to
colors.warningSubtle and colors.warning, then update the matching web badge
classes to use the same theme tokens for mobile/web parity.
In `@packages/server/src/repositories/data-quality-repository.test.ts`:
- Around line 101-146: Expand the DataQualityRepository test suite with cases
covering sourceOverlapCheck for nutrition-only overlap and exactly one nutrition
overlap day, coverageCheck when daysWithData exceeds the selected window,
latestDate with empty date lists returning null for manual_edits and outliers,
and check() filtering empty detail strings. Assert each branch’s public result
so the mutation score reaches the configured threshold.
In `@packages/server/src/repositories/data-quality-repository.ts`:
- Around line 255-257: Verb agreement is inconsistent across data-quality
messages. In packages/server/src/repositories/data-quality-repository.ts:157-159
and 255-257, add a small verb-selection helper beside pluralize and use it to
select “was/were” for anomalies.length and “needs/need” for attentionCount;
update the corresponding expected overallMessage assertions in
packages/server/src/repositories/data-quality-repository.test.ts:349-352 and the
same assertion at line 455 to use “needs” for a count of one.
- Around line 115-147: Update processingMessage and syncFreshnessCheck so the
message category is derived from the non-ready dataset statuses, including
failed or blocked datasets even when overallStatus is "ready"; retain
delayed/cancelled and still-updating behavior for their respective cases. Type
overallStatus with the existing ProcessingRepository.status return type (or its
shared status type) instead of string to preserve exhaustiveness, and update the
failed-dataset regression assertion in the related test to expect “could not be
updated.”
- Around line 209-215: The activity query in the source-overlap flow must cover
the same 30-day period reported by overview() and used by
DATA_QUALITY_WINDOW_DAYS. Update the getWeekList call on
ActivitiesCalendarRepository to request an equivalent 30-day window, using the
existing API’s week-based input or its supported day-based alternative, while
preserving endDate and includeProviderAbsent.
- Around line 58-77: Update coverageCheck and its caller in
getMicronutrientDataQuality to use nutrition.selectedWindowDays as the effective
window instead of DATA_QUALITY_WINDOW_DAYS, including missing-day calculation,
messages, and details while preserving the existing status behavior.
In `@packages/server/src/routers/processing.test.ts`:
- Around line 133-150: Add a second test for caller.dataQuality({}) that
verifies the procedure invokes ensureProvidersRegistered(), passes the resolved
YYYY-MM-DD default end date to mockDataQuality/overview(), and preserves the
expected response shape. Keep the existing explicit-endDate assertion while
covering the omitted-input default path.
In `@packages/server/src/routers/processing.ts`:
- Around line 103-123: Update dataQualityOutputSchema to reuse dateStringSchema
for window.endDate and checks.lastObservedDate, and timestampStringSchema for
generatedAt, removing the duplicated regex and deprecated z.string().datetime()
validation. Preserve the existing nullable behavior for lastObservedDate and
ensure the shared schemas are imported from src/lib/typed-sql.ts.
In `@packages/web/src/components/DataQualityCenter.test.tsx`:
- Around line 73-81: Update the test case “renders server-authored checks,
evidence, and review links” to assert each review link’s destination href, using
the Link mock’s rendered href and the routes defined by reviewDestinations. Keep
the existing label assertions, and verify that the nutrition review link points
to its expected route rather than only checking its accessible name.
In `@packages/web/src/components/DataQualityCenter.tsx`:
- Around line 89-97: Update the empty-state section in DataQualityCenter so it
is labelled by the existing “Data quality” heading rather than the
developer-facing aria-label. Use the appropriate heading-reference accessibility
attribute on the section and preserve the current heading and empty-state
content.
In `@packages/web/src/pages/DataQualityPage.tsx`:
- Around line 7-42: Add colocated tests for
packages/web/src/pages/DataQualityPage.tsx lines 7-42 covering loading, initial
error with retry, empty, successful data, and refresh failure while retaining
data; exercise the query state and retry behavior around DataQualityPage. Also
add a colocated screen test for packages/mobile/app/data-quality.tsx lines 18-61
covering the same query states and verifying each DataQualityCheckKey navigates
to its review destination.
In `@packages/web/src/routeTree.gen.ts`:
- Around line 156-160: Remove the `as any` suppression from the generated
`DataQualityRoute` definition and fix the generator or route-tree plugin
responsible for emitting these `update(...).update(...)` casts, so regenerated
route trees are correctly typed without `as any`. Apply the fix consistently to
the generator output rather than editing only `routeTree.gen.ts`, and regenerate
the file to verify the casts are gone.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: df1972f4-dcb1-4e11-97f6-dc1d65d2019a
📒 Files selected for processing (28)
packages/mobile/app/(tabs)/activities.test.tsxpackages/mobile/app/(tabs)/activities.tsxpackages/mobile/app/_layout.tsxpackages/mobile/app/data-quality.tsxpackages/mobile/app/more.test.tsxpackages/mobile/app/more.tsxpackages/mobile/components/DataQualityCenter.stories.tsxpackages/mobile/components/DataQualityCenter.test.tsxpackages/mobile/components/DataQualityCenter.tsxpackages/mobile/components/NutritionDataQualityPanel.test.tsxpackages/mobile/components/NutritionDataQualityPanel.tsxpackages/server/src/repositories/data-quality-repository.test.tspackages/server/src/repositories/data-quality-repository.tspackages/server/src/routers/processing.test.tspackages/server/src/routers/processing.tspackages/web/src/components/ActivityCardContent.test.tsxpackages/web/src/components/ActivityCardContent.tsxpackages/web/src/components/AppHeader.tsxpackages/web/src/components/DataQualityCenter.stories.tsxpackages/web/src/components/DataQualityCenter.test.tsxpackages/web/src/components/DataQualityCenter.tsxpackages/web/src/components/NutritionDataQualityPanel.test.tsxpackages/web/src/components/NutritionDataQualityPanel.tsxpackages/web/src/pages/DataQualityPage.tsxpackages/web/src/pages/MorePage.test.tsxpackages/web/src/pages/MorePage.tsxpackages/web/src/routeTree.gen.tspackages/web/src/routes/data-quality.tsx
|
Hosted run 30744850810 also had a shard-4 integration failure (job 91488810479), independent of the mutation changes: |
|
🤖 Review skipped: Repository rate limit exceeded. Free accounts are limited to 2 reviews per 4 hours per repository. Upgrade to a paid plan for unlimited reviews. |
|
Fresh CI run |
|
Addressed the actionable CodeRabbit review items in 6b17970.\n\n- Centralized status/destination mappings in the shared format domain package and kept web/mobile rendering in parity.\n- Removed client-authored count grammar; the server overview owns count/message text, including singular attention grammar and failed-dataset wording.\n- Added duplicate-detail key regressions, full 30-day overlap coverage, restricted-window intersection coverage, typed date/timestamp schema coverage, processing forwarding/default coverage, and repository mutation-focused boundary coverage.\n- Added web/mobile Data Quality screen query-state tests, review href/destination assertions, empty-state accessibility coverage, optional mobile review-button coverage, and Healthy/Informational mobile Storybook variants.\n- Patched both installed TanStack router-generator versions and regenerated routeTree.gen.ts with typed route assertions; no generated unsafe any assertions remain.\n\nFocused validation: 41 unit tests, 10 mobile tests, package typechecks, lint:sandbox, Biome, and focused Stryker (191/196 effective mutants killed; 0 no-coverage/errors; 97.45%). |
|
🤖 Review skipped: Repository rate limit exceeded. Free accounts are limited to 2 reviews per 4 hours per repository. Upgrade to a paid plan for unlimited reviews. |
2 similar comments
|
🤖 Review skipped: Repository rate limit exceeded. Free accounts are limited to 2 reviews per 4 hours per repository. Upgrade to a paid plan for unlimited reviews. |
|
🤖 Review skipped: Repository rate limit exceeded. Free accounts are limited to 2 reviews per 4 hours per repository. Upgrade to a paid plan for unlimited reviews. |
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
packages/server/src/repositories/data-quality-repository.ts (1)
188-209: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winUse a shared Dofek provider-id constant for manual-entry checks.
DOFEK_PROVIDER_IDis not exported fromjournal-repository.ts, andpackages/server/src/repositories/contains several independent local"dofek"constants. Move the canonical provider id to a shared constants module and import it here, then replace"dofek"with that constant so this check cannot drift from the manual-entry writer.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/server/src/repositories/data-quality-repository.ts` around lines 188 - 209, Move the canonical Dofek provider ID into a shared constants module, export it, and update the manual-entry writer and manualEntriesCheck to import and reuse DOFEK_PROVIDER_ID. Replace the local "dofek" comparison in manualEntriesCheck while preserving the existing filtering behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/format/src/data-quality.ts`:
- Around line 50-55: Update DATA_QUALITY_REVIEWS so source_overlap findings have
a review action that covers both nutrition and activity, or split the check into
domain-specific keys. Extend DataQualityReviewDestination and update the
corresponding web and mobile route maps to resolve the new shared contract,
ensuring activity-only overlaps navigate to affected records instead of the
nutrition review.
In `@packages/mobile/components/DataQualityCenter.test.tsx`:
- Around line 67-82: Update the duplicate-key regression assertions in
DataQualityCenter.test.tsx for
packages/mobile/components/DataQualityCenter.test.tsx:67-82 and
packages/web/src/components/DataQualityCenter.test.tsx:110-125 to inspect all
console.error call arguments by flattening consoleError.mock.calls before
checking for “same key”; preserve the existing repeated-detail rendering
assertions and cleanup.
---
Outside diff comments:
In `@packages/server/src/repositories/data-quality-repository.ts`:
- Around line 188-209: Move the canonical Dofek provider ID into a shared
constants module, export it, and update the manual-entry writer and
manualEntriesCheck to import and reuse DOFEK_PROVIDER_ID. Replace the local
"dofek" comparison in manualEntriesCheck while preserving the existing filtering
behavior.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 1ce734db-43d2-4502-a9af-2e0dc84038e1
⛔ Files ignored due to path filters (1)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (20)
packages/format/package.jsonpackages/format/src/data-quality.test.tspackages/format/src/data-quality.tspackages/mobile/app/data-quality.test.tsxpackages/mobile/app/data-quality.tsxpackages/mobile/components/DataQualityCenter.stories.tsxpackages/mobile/components/DataQualityCenter.test.tsxpackages/mobile/components/DataQualityCenter.tsxpackages/server/src/repositories/data-quality-repository.test.tspackages/server/src/repositories/data-quality-repository.tspackages/server/src/routers/processing.test.tspackages/server/src/routers/processing.tspackages/web/src/components/DataQualityCenter.stories.tsxpackages/web/src/components/DataQualityCenter.test.tsxpackages/web/src/components/DataQualityCenter.tsxpackages/web/src/pages/DataQualityPage.test.tsxpackages/web/src/routeTree.gen.tspatches/@tanstack__router-generator@1.167.17.patchpatches/@tanstack__router-generator@1.167.18.patchpnpm-workspace.yaml
|
🤖 Review skipped: Repository rate limit exceeded. Free accounts are limited to 2 reviews per 4 hours per repository. Upgrade to a paid plan for unlimited reviews. |
1 similar comment
|
🤖 Review skipped: Repository rate limit exceeded. Free accounts are limited to 2 reviews per 4 hours per repository. Upgrade to a paid plan for unlimited reviews. |
|
Addressed the review feedback in 34aec45. Data-quality source overlap is now split into nutrition_source_overlap and activity_source_overlap so each finding maps to the correct destination and review copy; shared format types, server schema, web route, mobile route, tests, and stories were updated. Duplicate-key assertions now flatten all console.error arguments on both platforms. Validation: 47 focused tests passed, Biome passed, and server/web/mobile typechecks passed. |
|
🤖 Review skipped: Repository rate limit exceeded. Free accounts are limited to 2 reviews per 4 hours per repository. Upgrade to a paid plan for unlimited reviews. |
Share review metadata and detail keys across clients while making the overview window honor access limits and full activity history.\n\nCloses #2079
45a91be to
4605d6d
Compare
|
🤖 Review skipped: Repository rate limit exceeded. Free accounts are limited to 2 reviews per 4 hours per repository. Upgrade to a paid plan for unlimited reviews. |
Summary
processing.dataQualityoverview combining existing coverage, source-overlap, processing freshness, anomaly, and journal signals.Validation
pnpm test— 1,029 files passed; 15,735 tests passed; 21 skipped.pnpm typecheck— passed.pnpm --dir packages/server exec tsc --noEmit— passed.pnpm --dir packages/web typecheck— passed.pnpm --dir packages/mobile typecheck— passed.Closes #2079
Fixes #2079
Summary by Sourcery
Introduce a server-composed data quality overview and surface it across web and mobile for users to review data reliability over the last 30 days.
New Features:
Enhancements:
Tests:
Summary by cubic
Adds a cross-platform Data Quality Center that rolls up coverage, source overlap (including activities), sync freshness, outliers, and manual edits into a server-owned overview with new web and iOS screens and a
/data-qualityroute. Delivers #2079 so users can trust data before interpreting metrics.New Features
DataQualityRepositorycomposes signals and exposes cachedprocessing.dataQualitywith runtime validation; ensures providers are registered; tests for composition and router output.DataQualityCenterwith loading/empty states and per-check review links; header and More page link to/data-quality; activity cards show a “Source overlap” pill; nutrition panel adds “Source overlap to review”; stories and tests./data-qualityscreen andDataQualityCenterwith loading/empty states and per-check review actions; More menu link; activities show a “Source overlap” pill; nutrition panel adds “Source overlap to review”; stories and tests.Bug Fixes
@dofek/format/data-qualityunifies status labels, review destinations, and stable detail keys across web and iOS; addsactivity_source_overlaprouted to Activities; tests added.@tanstack/router-generatorto fix route type generation inrouteTree.gen.ts.Written for commit 4605d6d. Summary will update on new commits.
Summary by CodeRabbit
New Features
Tests