feat: sync GPS route data from HealthKit workouts - #764
Conversation
Activities synced from HealthKit via the iOS app were missing GPS map data because the native module never queried HKWorkoutRoute. This adds end-to-end support for fetching workout route locations and storing them as sensor_sample rows. - Swift: request HKSeriesType.workoutRoute() permission, add queryWorkoutRoutes function that fetches CLLocation data - Server: add pushWorkoutRoutes tRPC endpoint that looks up the activity by workout UUID and inserts lat/lng/altitude/speed/ gps_accuracy sensor samples - Mobile sync: after pushing workouts, query routes for each and push to the new endpoint - Tests: server unit tests for route processing, mobile sync tests for route fetching/pushing, updated mocks across all test files Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
Storybook preview for This comment updates automatically on each PR push. |
There was a problem hiding this comment.
Pull request overview
Adds end-to-end syncing of HealthKit workout GPS route data (via HKWorkoutRoute) so activities imported from the iOS app can populate the existing route/map rendering pipeline on web/mobile.
Changes:
- Server: add
pushWorkoutRoutestRPC endpoint and store route points asfitness.sensor_samplerows (lat/lng/altitude/speed/gps_accuracy). - iOS HealthKit module: request
HKSeriesType.workoutRoute()read permission and implementqueryWorkoutRoutes. - Mobile sync: query routes after pushing workouts and send them to the server; update mocks and unit tests accordingly.
Reviewed changes
Copilot reviewed 13 out of 13 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| packages/server/src/routers/health-kit-sync.ts | Adds route schemas, processWorkoutRoutes, and pushWorkoutRoutes endpoint. |
| packages/server/src/routers/health-kit-sync.test.ts | Adds unit tests for pushWorkoutRoutes. |
| packages/mobile/test-setup.ts | Extends HealthKit native module mock with queryWorkoutRoutes. |
| packages/mobile/modules/health-kit/ios/HealthKitTypes.swift | Adds HKSeriesType.workoutRoute() to requested read types. |
| packages/mobile/modules/health-kit/ios/HealthKitModule.swift | Implements queryWorkoutRoutes by reading HKWorkoutRoute locations. |
| packages/mobile/modules/health-kit/index.ts | Exposes RouteLocation type and queryWorkoutRoutes wrapper. |
| packages/mobile/lib/useAutoSync.ts | Passes queryWorkoutRoutes into auto-sync adapter. |
| packages/mobile/lib/useAutoSync.test.ts | Updates module mock to include queryWorkoutRoutes. |
| packages/mobile/lib/health-kit-sync.ts | Queries per-workout routes and calls pushWorkoutRoutes. |
| packages/mobile/lib/health-kit-sync.test.ts | Adds tests for route querying/pushing behavior. |
| packages/mobile/lib/background-health-kit-sync.ts | Includes queryWorkoutRoutes in background sync adapter. |
| packages/mobile/lib/background-health-kit-sync.test.ts | Updates mocks to include pushWorkoutRoutes and queryWorkoutRoutes. |
| packages/mobile/app/_layout.tsx | Wires pushWorkoutRoutes into the injected sync tRPC client. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| // Insert each location as sensor_sample rows (one per channel) | ||
| for (const location of route.locations) { | ||
| for (const { channel, getValue, round } of ROUTE_CHANNELS) { | ||
| const value = getValue(location); | ||
| if (value == null) continue; | ||
|
|
||
| const scalar = round ? Math.round(value) : value; | ||
| await db.execute( | ||
| sql`INSERT INTO fitness.sensor_sample | ||
| (recorded_at, user_id, provider_id, activity_id, device_id, source_type, channel, scalar) | ||
| VALUES ( | ||
| ${location.date}::timestamptz, | ||
| ${userId}, | ||
| ${PROVIDER_ID}, | ||
| ${activityId}::uuid, | ||
| ${route.sourceName ?? null}, | ||
| ${"api"}, | ||
| ${channel}, | ||
| ${scalar}::real | ||
| )`, | ||
| ); | ||
| inserted++; | ||
| } | ||
| } |
There was a problem hiding this comment.
processWorkoutRoutes performs one INSERT per (location × channel), inside nested for loops. For workout routes with thousands of points this becomes tens of thousands of separate DB round-trips and will be very slow / likely to time out. Consider batching inserts (e.g., build VALUES ${sql.join(...)} per channel and per BATCH_SIZE, similar to ActivityRecordingRepository.saveActivity) so each batch is inserted with a single statement.
| // Insert each location as sensor_sample rows (one per channel) | |
| for (const location of route.locations) { | |
| for (const { channel, getValue, round } of ROUTE_CHANNELS) { | |
| const value = getValue(location); | |
| if (value == null) continue; | |
| const scalar = round ? Math.round(value) : value; | |
| await db.execute( | |
| sql`INSERT INTO fitness.sensor_sample | |
| (recorded_at, user_id, provider_id, activity_id, device_id, source_type, channel, scalar) | |
| VALUES ( | |
| ${location.date}::timestamptz, | |
| ${userId}, | |
| ${PROVIDER_ID}, | |
| ${activityId}::uuid, | |
| ${route.sourceName ?? null}, | |
| ${"api"}, | |
| ${channel}, | |
| ${scalar}::real | |
| )`, | |
| ); | |
| inserted++; | |
| } | |
| } | |
| const pendingValues: ReturnType<typeof sql>[] = []; | |
| const flushPendingValues = async () => { | |
| if (pendingValues.length === 0) return; | |
| await db.execute( | |
| sql`INSERT INTO fitness.sensor_sample | |
| (recorded_at, user_id, provider_id, activity_id, device_id, source_type, channel, scalar) | |
| VALUES ${sql.join(pendingValues, sql`, `)}`, | |
| ); | |
| inserted += pendingValues.length; | |
| pendingValues.length = 0; | |
| }; | |
| // Insert each location as sensor_sample rows, batching rows to reduce DB round-trips | |
| for (const location of route.locations) { | |
| for (const { channel, getValue, round } of ROUTE_CHANNELS) { | |
| const value = getValue(location); | |
| if (value == null) continue; | |
| const scalar = round ? Math.round(value) : value; | |
| pendingValues.push( | |
| sql`( | |
| ${location.date}::timestamptz, | |
| ${userId}, | |
| ${PROVIDER_ID}, | |
| ${activityId}::uuid, | |
| ${route.sourceName ?? null}, | |
| ${"api"}, | |
| ${channel}, | |
| ${scalar}::real | |
| )`, | |
| ); | |
| if (pendingValues.length >= BATCH_SIZE) { | |
| await flushPendingValues(); | |
| } | |
| } | |
| } | |
| await flushPendingValues(); |
| for (const route of routes) { | ||
| if (route.locations.length === 0) continue; | ||
|
|
||
| // Look up the activity by external_id | ||
| const externalId = `hk:workout:${route.workoutUuid}`; | ||
| const activityIdSchema = z.object({ id: z.string() }); | ||
| const activityRows = await executeWithSchema( | ||
| db, | ||
| activityIdSchema, | ||
| sql`SELECT id FROM fitness.activity | ||
| WHERE user_id = ${userId} | ||
| AND provider_id = ${PROVIDER_ID} | ||
| AND external_id = ${externalId} | ||
| LIMIT 1`, | ||
| ); | ||
|
|
||
| const activityId = activityRows[0]?.id ?? null; | ||
|
|
There was a problem hiding this comment.
processWorkoutRoutes does an activity lookup query for each route (SELECT id ... LIMIT 1). If multiple routes are pushed in one request, this creates an N+1 query pattern. Consider resolving all workoutUuid -> activityId mappings in one query (e.g., WHERE external_id IN (...)) before inserting samples, then insert route samples using the resolved IDs.
| for (const route of routes) { | |
| if (route.locations.length === 0) continue; | |
| // Look up the activity by external_id | |
| const externalId = `hk:workout:${route.workoutUuid}`; | |
| const activityIdSchema = z.object({ id: z.string() }); | |
| const activityRows = await executeWithSchema( | |
| db, | |
| activityIdSchema, | |
| sql`SELECT id FROM fitness.activity | |
| WHERE user_id = ${userId} | |
| AND provider_id = ${PROVIDER_ID} | |
| AND external_id = ${externalId} | |
| LIMIT 1`, | |
| ); | |
| const activityId = activityRows[0]?.id ?? null; | |
| const externalIds = Array.from( | |
| new Set( | |
| routes | |
| .filter((route) => route.locations.length > 0) | |
| .map((route) => `hk:workout:${route.workoutUuid}`), | |
| ), | |
| ); | |
| const activityIdByExternalId = new Map<string, string>(); | |
| if (externalIds.length > 0) { | |
| const activityRowSchema = z.object({ | |
| id: z.string(), | |
| external_id: z.string(), | |
| }); | |
| const activityRows = await executeWithSchema( | |
| db, | |
| activityRowSchema, | |
| sql`SELECT id, external_id FROM fitness.activity | |
| WHERE user_id = ${userId} | |
| AND provider_id = ${PROVIDER_ID} | |
| AND external_id IN (${sql.join( | |
| externalIds.map((externalId) => sql`${externalId}`), | |
| sql`, `, | |
| )})`, | |
| ); | |
| for (const activityRow of activityRows) { | |
| activityIdByExternalId.set(activityRow.external_id, activityRow.id); | |
| } | |
| } | |
| for (const route of routes) { | |
| if (route.locations.length === 0) continue; | |
| const externalId = `hk:workout:${route.workoutUuid}`; | |
| const activityId = activityIdByExternalId.get(externalId) ?? null; |
| let locationQuery = HKWorkoutRouteQuery(route: route) { _, locations, done, locationError in | ||
| if locationError != nil { | ||
| if done { group.leave() } | ||
| return | ||
| } | ||
| guard let locations = locations else { | ||
| if done { group.leave() } | ||
| return | ||
| } |
There was a problem hiding this comment.
HKWorkoutRouteQuery error handling can leave the DispatchGroup unbalanced: when locationError != nil and done is false, the closure returns without calling group.leave(), so group.notify may never fire and the JS promise can hang indefinitely. Ensure the group is left exactly once per route even on error (and consider resolving/rejecting with a partial result rather than hanging).
| // Collect all locations from all routes | ||
| var allLocations: [[String: Any]] = [] | ||
| let group = DispatchGroup() | ||
|
|
||
| for route in routes { | ||
| group.enter() | ||
| var routeLocations: [[String: Any]] = [] | ||
|
|
||
| let locationQuery = HKWorkoutRouteQuery(route: route) { _, locations, done, locationError in | ||
| if locationError != nil { | ||
| if done { group.leave() } | ||
| return | ||
| } | ||
| guard let locations = locations else { | ||
| if done { group.leave() } | ||
| return | ||
| } | ||
| for location in locations { | ||
| var dict: [String: Any] = [ | ||
| "date": HealthKitQueries.formatDate(location.timestamp), | ||
| "lat": location.coordinate.latitude, | ||
| "lng": location.coordinate.longitude, | ||
| ] | ||
| if location.altitude >= 0 || location.verticalAccuracy >= 0 { | ||
| dict["altitude"] = location.altitude | ||
| } | ||
| if location.speed >= 0 { | ||
| dict["speed"] = location.speed | ||
| } | ||
| if location.horizontalAccuracy >= 0 { | ||
| dict["horizontalAccuracy"] = location.horizontalAccuracy | ||
| } | ||
| routeLocations.append(dict) | ||
| } | ||
| if done { | ||
| allLocations.append(contentsOf: routeLocations) | ||
| group.leave() | ||
| } |
There was a problem hiding this comment.
allLocations is mutated from multiple HKWorkoutRouteQuery callbacks. These callbacks are not guaranteed to run on the main queue, so allLocations.append(...) can race across routes. Consider collecting results on a serial queue (or dispatching the append to .main) and also sorting the final locations by timestamp before resolving so the returned polyline order is deterministic.
| "lat": location.coordinate.latitude, | ||
| "lng": location.coordinate.longitude, | ||
| ] | ||
| if location.altitude >= 0 || location.verticalAccuracy >= 0 { |
There was a problem hiding this comment.
Altitude inclusion check if location.altitude >= 0 || location.verticalAccuracy >= 0 will include altitude values even when verticalAccuracy is negative (invalid), as long as altitude happens to be non-negative. This can store invalid altitude samples. Prefer gating altitude on verticalAccuracy >= 0 (and allow negative altitude when accuracy is valid).
| if location.altitude >= 0 || location.verticalAccuracy >= 0 { | |
| if location.verticalAccuracy >= 0 { |
| const routes: WorkoutRoutePayload[] = []; | ||
| for (const workout of workouts) { | ||
| const locations = await healthKit.queryWorkoutRoutes(workout.uuid); | ||
| if (locations.length > 0) { | ||
| routes.push({ | ||
| workoutUuid: workout.uuid, | ||
| sourceName: workout.sourceName, | ||
| locations, | ||
| }); | ||
| } | ||
| } |
There was a problem hiding this comment.
syncHealthKitToServer queries routes sequentially (await inside a for loop). With many workouts this can significantly extend sync time and delay subsequent steps. Consider parallelizing with a bounded concurrency (e.g., Promise.allSettled with a small pool) and/or short-circuiting after some max number of workouts/points to avoid very large route payloads.
| const routes: WorkoutRoutePayload[] = []; | |
| for (const workout of workouts) { | |
| const locations = await healthKit.queryWorkoutRoutes(workout.uuid); | |
| if (locations.length > 0) { | |
| routes.push({ | |
| workoutUuid: workout.uuid, | |
| sourceName: workout.sourceName, | |
| locations, | |
| }); | |
| } | |
| } | |
| const routeQueryConcurrency = Math.min(4, workouts.length); | |
| const routeGroups = await Promise.all( | |
| Array.from({ length: routeQueryConcurrency }, async (_, workerIndex) => { | |
| const workerRoutes: WorkoutRoutePayload[] = []; | |
| for (let workoutIndex = workerIndex; workoutIndex < workouts.length; workoutIndex += routeQueryConcurrency) { | |
| const workout = workouts[workoutIndex]; | |
| const locations = await healthKit.queryWorkoutRoutes(workout.uuid); | |
| if (locations.length > 0) { | |
| workerRoutes.push({ | |
| workoutUuid: workout.uuid, | |
| sourceName: workout.sourceName, | |
| locations, | |
| }); | |
| } | |
| } | |
| return workerRoutes; | |
| }), | |
| ); | |
| const routes = routeGroups.flat(); |
| const locations = await healthKit.queryWorkoutRoutes(workout.uuid); | ||
| if (locations.length > 0) { | ||
| routes.push({ | ||
| workoutUuid: workout.uuid, | ||
| sourceName: workout.sourceName, | ||
| locations, | ||
| }); | ||
| } | ||
| } | ||
|
|
||
| if (routes.length > 0) { | ||
| onProgress?.(`Pushing ${routes.length} workout routes...`); | ||
| const routeResult = await trpcClient.healthKitSync.pushWorkoutRoutes.mutate({ routes }); | ||
| totalInserted += routeResult.inserted; |
There was a problem hiding this comment.
queryWorkoutRoutes failures will currently throw and abort the entire sync (there’s no try/catch around the per-workout route query loop, and route errors aren’t appended to the errors array like quantity push errors are). Consider handling route query/push errors as non-fatal (record in errors and continue) so a route permission/query issue doesn’t prevent workout/sleep syncing.
| const locations = await healthKit.queryWorkoutRoutes(workout.uuid); | |
| if (locations.length > 0) { | |
| routes.push({ | |
| workoutUuid: workout.uuid, | |
| sourceName: workout.sourceName, | |
| locations, | |
| }); | |
| } | |
| } | |
| if (routes.length > 0) { | |
| onProgress?.(`Pushing ${routes.length} workout routes...`); | |
| const routeResult = await trpcClient.healthKitSync.pushWorkoutRoutes.mutate({ routes }); | |
| totalInserted += routeResult.inserted; | |
| try { | |
| const locations = await healthKit.queryWorkoutRoutes(workout.uuid); | |
| if (locations.length > 0) { | |
| routes.push({ | |
| workoutUuid: workout.uuid, | |
| sourceName: workout.sourceName, | |
| locations, | |
| }); | |
| } | |
| } catch (error) { | |
| const message = error instanceof Error ? error.message : String(error); | |
| errors.push(`Failed to query workout routes for workout ${workout.uuid}: ${message}`); | |
| } | |
| } | |
| if (routes.length > 0) { | |
| onProgress?.(`Pushing ${routes.length} workout routes...`); | |
| try { | |
| const routeResult = await trpcClient.healthKitSync.pushWorkoutRoutes.mutate({ routes }); | |
| totalInserted += routeResult.inserted; | |
| } catch (error) { | |
| const message = error instanceof Error ? error.message : String(error); | |
| errors.push(`Failed to push workout routes: ${message}`); | |
| } |
| // Find the gps_accuracy insert and verify the value is rounded | ||
| const gpsAccuracyCall = execute.mock.calls.find((call: unknown[]) => { | ||
| const serialized = JSON.stringify(call[0]); | ||
| return ( | ||
| serialized.includes("INSERT INTO fitness.sensor_sample") && | ||
| serialized.includes("gps_accuracy") | ||
| ); | ||
| }); | ||
| expect(gpsAccuracyCall).toBeDefined(); | ||
| }); |
There was a problem hiding this comment.
The "rounds gps_accuracy to integer" test only asserts that an insert call mentioning gps_accuracy exists, but it doesn't actually verify that the rounded scalar value is what gets inserted. Strengthen this test to assert the inserted value (e.g., by inspecting the SQL parameters / bindings) so regressions in rounding behavior are caught.
The testReadTypesTotalCount assertion expected 57 types but adding HKSeriesType.workoutRoute() bumped it to 58. Also adds an explicit test for the workout route type. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Preview Environment
|
Coros workouts include a fitUrl field pointing to a pre-signed FIT file. Download and parse it using the existing FIT parser to get GPS (lat/lng), heart rate, power, cadence, altitude, and all other channels — matching what Wahoo already does. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Preview Environment
|
Write a reusable TCX parser (src/tcx/parser.ts) that extracts GPS coordinates, altitude, heart rate, cadence, speed, and power from Training Center XML files. Integrate with Fitbit: when an activity has a tcxLink, download and parse the TCX file to store sensor_sample rows with GPS + metrics. Previously Fitbit only stored activity summaries. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Polar: download TCX for exercises with has_route=true, parse with the
shared TCX parser for GPS + HR + cadence data.
Suunto: download FIT files via the /v2/workout/exportFit/{key} endpoint,
parse with the existing FIT parser for full time-series including GPS.
Both previously only stored activity summaries.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Preview Environment
|
- Move fitRecordsToMetricStream to src/fit/records.ts to fix import boundary violation (Coros and Suunto were importing from wahoo/parsers) - Re-export from wahoo/parsers.ts for backward compatibility - Add "trackpoint"/"trackpoints" to cspell dictionary - Fix coros-extra.test.ts mock to include returning() and delete().where() chains needed by the FIT file download path Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Preview Environment
|
- Add MSW handler for Polar TCX export endpoint in integration test (exercises with has_route=true now trigger TCX download) - Add SAX callback names and TCX element names to cspell dictionary Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Server (processWorkoutRoutes): - Batch INSERT: accumulate VALUES and flush per BATCH_SIZE instead of one INSERT per channel per location (was O(n*5) round-trips) - Bulk activity lookup: resolve all workoutUuid→activityId in one IN query instead of N+1 SELECTs Swift (queryWorkoutRoutes): - Fix DispatchGroup imbalance: always call group.leave() on done=true regardless of error state, preventing promise hangs - Thread safety: collect route locations on a serial queue - Sort final locations by timestamp for deterministic polyline order - Gate altitude on verticalAccuracy >= 0 only (not altitude >= 0) Mobile sync: - Parallelize route queries with bounded concurrency (4 workers) - Wrap route query/push in try/catch so failures are non-fatal errors instead of aborting the entire sync Tests: - Add tests for route query and push error handling - Strengthen gps_accuracy rounding test to verify actual value - Update mock shapes for batched insert + bulk lookup Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The Suunto sync now tries to download FIT files for each workout. Add an MSW handler that returns 404 (gracefully skipped by the provider) to prevent unhandled request errors in tests. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Previous preview server never became reachable (cloud-init completed but HTTPS health check timed out). Re-run reused the stale server. Empty commit forces Terraform to recreate with fresh user_data. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The preview server was destroyed and recreated on every push because commit_sha was embedded in user_data, changing it each time. This nuked Docker volumes (including Caddy's ACME certs), hitting Let's Encrypt rate limits after 5 deploys per week. Now follows the production deploy-config pattern: - user_data is stable (commit_sha hardcoded to "initial") so the server is only created once per PR - A null_resource with remote-exec SSHes in on each push to pull the latest image and restart containers - Docker volumes (and ACME certs) persist across deploys - SSH setup added to the preview workflow for the provisioner Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The null_resource only pulled images but didn't update the compose or Caddyfile on disk. If those files change in the PR, the server wouldn't pick them up. Now uploads both via file provisioner before restarting, matching how prod's deploy-config uploads config before running deploy.sh. Also triggers on compose/caddy content hash changes, not just commit_sha. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Split the 1136-line monolith into focused modules following the established Wahoo provider pattern: - client.ts (~160 lines): Zod schemas + FitbitClient API client - parsers.ts (~140 lines): Pure parse functions + activity type mapping - persisters.ts (~160 lines): DB upsert functions (activity, sleep, daily metrics, body measurements) — eliminates duplication between sync() and syncWebhookEvent() - provider.ts (~350 lines): FitbitProvider orchestration, OAuth, webhooks - index.ts: Barrel re-exports for backward compatibility This speeds up Stryker mutation testing by letting it target smaller files with proportionally fewer tests per mutant. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Mobile Preview
To test on device:
|
The workspace export for dofek/providers/fitbit still pointed to the deleted fitbit.ts. Update to fitbit/index.ts. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Mobile Preview
To test on device:
|
Delete index.ts barrel and have all consumers import directly from client.ts, parsers.ts, or provider.ts. This makes the dependency graph explicit and matches how Stryker traces mutations to tests. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Mobile Preview
To test on device:
|
Summary
HKWorkoutRoutesrc/tcx/parser.ts) for XML-based activity file formatsProviders with GPS support added
HKWorkoutRoutequeryfitUrl)tcxLink)has_route)Providers that already had GPS (unchanged)
Strava, Garmin, Wahoo, Ride With GPS, Zwift, Apple Health (XML export)
Key changes
src/tcx/parser.ts): SAX-based parser extracting GPS, HR, cadence, power, speed, altitude from TCX filesHealthKitModule.swift):queryWorkoutRoutesfunction usingHKWorkoutRoute+HKWorkoutRouteQueryhealth-kit-sync.ts):pushWorkoutRoutestRPC endpoint for iOS route dataFuture work
Komoot, Decathlon, MapMyFitness, and Peloton outdoor have GPS data available but their APIs need per-provider investigation.
Test plan
🤖 Generated with Claude Code