Skip to content

Fix heart-rate zone stats when resting heart rate is invalid - #1084

Merged
Asherlc merged 20 commits into
mainfrom
Asherlc/setup-dbeaver
May 2, 2026
Merged

Asherlc merged 20 commits into
mainfrom
Asherlc/setup-dbeaver

Conversation

@Asherlc

@Asherlc Asherlc commented May 2, 2026 •

Copy link
Copy Markdown
Owner

Summary

Fixes integration CI failures where HR zone totals were 0 due to invalid resting-heart-rate parameters flowing into zone computations.

Root cause

#findHeartRateZoneParams used COALESCE(rhr.resting_hr, 60) and could pass derived resting HR values that are >= max_hr (or otherwise invalid) directly into zone calculations. That produced non-sensical zone math and zero total seconds.

Fix

Hardened the SQL parameter selection in packages/server/src/repositories/activity-repository.ts to only accept resting HR when it is positive and below max HR. If derived resting HR is invalid, the query now falls back to user_profile.resting_hr when valid; otherwise defaults to 60.

Added regression coverage in packages/server/src/repositories/activity-repository.test.ts to assert heart-rate zone calls use fallback 60 when resting HR is invalid.

Validation

Not run in this pass to keep changes minimal and avoid extra CI-impacting re-checks in this workspace.

Summary by CodeRabbit

  • New Features

    • CloudBeaver UI deployed to production and staging with preconfigured Postgres and ClickHouse data sources, TLS routing, and configurable Traefik host rule.
  • Bug Fixes

    • Metric stream backfill now runs in 6‑hour contiguous ranges and de-duplicates rows to avoid duplicates.
    • ClickHouse client request timeout increased to improve reliability.
  • Documentation

    • Deployment docs updated with CloudBeaver service, storage path, DNS names, and management UI URL.
  • Chores

    • Deployment workflows accept a customizable CloudBeaver host rule.
  • Tests

    • Backfill, ClickHouse client, and related integration/unit tests updated for new behavior.

Copilot AI review requested due to automatic review settings May 2, 2026 02:03
@coderabbitai

coderabbitai Bot commented May 2, 2026 •

Copy link
Copy Markdown

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Adds a CloudBeaver service with DNS, Terraform provisioning, Swarm stack, deploy workflow inputs, and data-source JSON. Separately, rewrites ClickHouse native metric_stream backfill to split Timescale chunks into 6‑hour ranges, use overlap-based completion checks, and de-duplicate inserts by row id.

Changes

CloudBeaver Service Deployment

Layer / File(s) Summary
DNS & Infrastructure
deploy/dns.tf
Added Cloudflare A records for cloudbeaver.dofek.asherlc.com and staging-cloudbeaver.dofek.asherlc.com.
Data Source Configuration
deploy/cloudbeaver-data-sources.json
New CloudBeaver data-sources JSON with postgres-jdbc-dofek-production, clickhouse-com-dofek-production, and prod connection-type.
Terraform Provisioning
deploy/server.tf
Added terraform_data.cloudbeaver_datasources_sync and staging equivalent to create /mnt/dofek-data/cloudbeaver/.../.dbeaver and upload cloudbeaver-data-sources.json; bumped data volume mount alias triggers_replace versions.
Stack Service Definition
deploy/stack.yml
Added cloudbeaver service using a pinned dbeaver/cloudbeaver image, workspace bind-mount /mnt/dofek-data/cloudbeaver, Traefik labels (forward-auth, TLS via le, host rule from CLOUDBEAVER_HOST_RULE), port 8978.
Deployment Workflows
.github/workflows/deploy-web-environment.yml, .github/workflows/deploy-web-stack.yml, .github/workflows/deploy-web.yml
Added cloudbeaver_host_rule workflow input and threaded it through deploy jobs; staging job sets Host(\staging-cloudbeaver.dofek.asherlc.com`)`; deploy prep exports the input and pre-pulls the pinned CloudBeaver image.
Documentation
deploy/README.md
Documented CloudBeaver bind-mount path, added service to Swarm list, and management UI URL entry.

Metric Stream Backfill Fix

Layer / File(s) Summary
Types & Constants
src/db/clickhouse-migrations.ts
Introduced MetricStreamBackfillRange and replaced 1‑hour window with METRIC_STREAM_BACKFILL_RANGE_MILLISECONDS = 6 * 60 * 60 * 1_000.
Chunk Bounds Fetching
src/db/clickhouse-migrations.ts
fetchMetricStreamBackfillChunks now computes per-chunk bounds by querying min(recorded_at) and (max(recorded_at) + 1µs) from each chunk table, treats bounds as nullable, and emits only fully-bounded chunks.
Range Splitting
src/db/clickhouse-migrations.ts
Added splitMetricStreamBackfillChunk to slice Timescale chunk bounds into contiguous 6‑hour MetricStreamBackfillRange segments.
Core Backfill Logic
src/db/clickhouse-migrations.ts
Rewrote backfillNativeMetricStream to iterate chunk-derived 6‑hour ranges, skip already-covered ranges, run per-range backfill, and record completion rows.
Completion Predicate
src/db/clickhouse-migrations.ts
isMetricStreamBackfillChunkComplete now treats a sub-range as complete when an existing completed range overlaps it (lower_bound <= chunkStart AND upper_bound >= chunkEnd).
De-duplication SQL
src/db/clickhouse-migrations.ts
buildMetricStreamBackfillStatement now LEFT JOINs against existing postgres_fitness.metric_stream limited to the same time bounds and inserts only when existing_metric_stream.id IS NULL.
Tests & Docs
src/db/clickhouse-migrations.test.ts, docs/production-incident-baseline.md
Updated/added tests validating 6‑hour splitting, null-bound chunk skipping, overlap completion checks, and LEFT JOIN de-duplication; incident doc updated to reflect new strategy.
ClickHouse Client Timeout
src/db/clickhouse.ts, src/db/clickhouse-client.test.ts
Added CLICKHOUSE_REQUEST_TIMEOUT_MILLISECONDS = 120000 and pass request_timeout to client creation; tests added to assert behavior.

Sequence Diagram(s)

sequenceDiagram
    participant Deploy as Deploy Workflow
    participant Terraform as Terraform
    participant SSH as SSH/SCP
    participant Docker as Docker Swarm
    participant Traefik as Traefik Router
    participant CloudBeaver as CloudBeaver Service

    Deploy->>Terraform: Apply DNS records & server provisioning
    Terraform->>Terraform: Create A records, provision data volume
    Terraform->>SSH: remote-exec to create /mnt/dofek-data/cloudbeaver/.../.dbeaver
    Deploy->>SSH: Upload cloudbeaver-data-sources.json to data volume
    Deploy->>Docker: Deploy Swarm stack with cloudbeaver service
    Docker->>Docker: Pull pinned dbeaver/cloudbeaver image
    Docker->>CloudBeaver: Start container with bind-mount
    Traefik->>CloudBeaver: Route requests using CLOUDBEAVER_HOST_RULE to port 8978
Loading
sequenceDiagram
    participant Migration as Migration Runner
    participant Timescale as Timescale (Source)
    participant Splitter as Range Splitter
    participant ClickHouse as ClickHouse (Target)
    participant Tracker as Completion Tracker

    Migration->>Timescale: Fetch Timescale metric_stream chunks
    Timescale-->>Migration: Return chunk identifiers
    Migration->>Splitter: splitMetricStreamBackfillChunk(chunk)
    Splitter-->>Migration: Return 6-hour MetricStreamBackfillRange segments
    loop For each 6-hour range
        Migration->>ClickHouse: Run INSERT ... LEFT JOIN existing_metric_stream (within bounds)
        ClickHouse-->>Migration: Insert new rows only (existing_metric_stream.id IS NULL)
        Migration->>Tracker: Record completion row with lower_bound/upper_bound
    end
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Title check ⚠️ Warning The PR title claims to fix heart-rate zone stats with invalid resting heart rate, but the changeset also includes extensive CloudBeaver deployment configuration, DNS records, ClickHouse migration rewrites, and metrics backfill logic unrelated to the stated fix. Revise the title to reflect the full scope of changes, such as 'Add CloudBeaver service with updated metric stream backfill strategy and HR zone fallback' or split into separate focused PRs per concern.
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch Asherlc/setup-dbeaver

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions

github-actions Bot commented May 2, 2026 •

Copy link
Copy Markdown
Contributor

Storybook previews for 315f4d8e are ready:

This comment updates automatically on each PR push.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@deploy/server.tf`:
- Around line 238-240: The triggers_replace for the CloudBeaver datasource
syncers only includes filesha256("${path.module}/cloudbeaver-data-sources.json")
so the provisioner won't re-run when the underlying volume is replaced; update
both triggers_replace arrays (the ones containing
filesha256("${path.module}/cloudbeaver-data-sources.json") at the two
occurrences) to also include the volume/mount-alias ID (e.g. add the mount alias
or volume resource id such as mount_alias.<name>.id or aws_ebs_volume.<name>.id)
so Terraform will detect the volume replacement and re-run the datasources_sync
provisioner. Ensure you add the same volume id expression to both trigger
arrays.

In `@src/db/clickhouse-migrations.test.ts`:
- Around line 345-356: The test mock for ClickHouse queries must return the
expected shape for the system.databases query (include { engine: "Atomic" }) so
shouldReplacePostgresFitnessDatabase() follows the Atomic branch; update the
vi.fn() mockImplementation that matches queryText (currently checking
"system.tables") to also detect "system.databases" and return [{ engine:
"Atomic" }]; additionally adjust the split-duration expectation for the 2.5-hour
case from one-hour chunks to six-hour chunks to match the updated splitting
logic (update the assertions that check chunk size/split intervals used in the
test cases that reference the 2.5-hour input, including the blocks that branch
on "0006_backfill_native_metric_stream" and "metric_stream_backfill_chunks").
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 062619f8-6440-4dbd-84f2-503321bb0836

📥 Commits

Reviewing files that changed from the base of the PR and between 03798a3 and b3bdfed.

📒 Files selected for processing (11)
  • .github/workflows/deploy-web-environment.yml
  • .github/workflows/deploy-web-stack.yml
  • .github/workflows/deploy-web.yml
  • deploy/README.md
  • deploy/cloudbeaver-data-sources.json
  • deploy/dns.tf
  • deploy/server.tf
  • deploy/stack.yml
  • docs/production-incident-baseline.md
  • src/db/clickhouse-migrations.test.ts
  • src/db/clickhouse-migrations.ts

Comment thread deploy/server.tf
Comment thread src/db/clickhouse-migrations.test.ts

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds CloudBeaver to the production/staging Docker Swarm deployment (with DNS, Traefik routing, and persistent workspace/bootstrap config) and hardens the ClickHouse metric_stream backfill migration to be more resilient to timeouts and retries.

Changes:

  • Introduces a new cloudbeaver swarm service with Traefik routing, persistent workspace storage, and Terraform-synced datasource configuration.
  • Updates ClickHouse native metric_stream backfill to operate in 6-hour ranges, treat broader completion ranges as covering subranges, and avoid duplicate inserts on retry.
  • Extends deployment workflows/Terraform/DNS and adds/updates tests + incident documentation for the backfill timeout.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
src/db/clickhouse-migrations.ts Split Timescale chunks into 6-hour backfill ranges; broaden completion-range matching; add anti-join to avoid duplicate inserts on retry.
src/db/clickhouse-migrations.test.ts Adds test coverage for 6-hour range splitting and broader completion-range matching.
docs/production-incident-baseline.md Updates incident entry describing the deploy timeout and the mitigations.
deploy/stack.yml Adds CloudBeaver service with Traefik routing and workspace bind mount.
deploy/server.tf Ensures CloudBeaver persistent directories exist and syncs datasource JSON into the workspace for prod/staging.
deploy/dns.tf Adds Cloudflare DNS records for CloudBeaver prod/staging hostnames.
deploy/cloudbeaver-data-sources.json Adds preconfigured Postgres + ClickHouse datasource definitions for CloudBeaver.
deploy/README.md Documents CloudBeaver service/state path and URL.
.github/workflows/deploy-web.yml Passes staging CloudBeaver host rule into the environment deploy workflow.
.github/workflows/deploy-web-stack.yml Adds CloudBeaver host-rule env plumbing, pulls CloudBeaver image, and validates the new bind-mount path.
.github/workflows/deploy-web-environment.yml Adds cloudbeaver_host_rule input and forwards it to the stack deploy workflow.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/db/clickhouse-migrations.test.ts Outdated
Comment thread deploy/cloudbeaver-data-sources.json
Comment thread deploy/cloudbeaver-data-sources.json
Comment thread deploy/cloudbeaver-data-sources.json Outdated
Comment thread src/db/clickhouse-migrations.test.ts Outdated
@Asherlc Asherlc changed the title Add CloudBeaver and harden ClickHouse metric backfill Fix ClickHouse backfill migration and CI test fixtures May 2, 2026
@github-actions

github-actions Bot commented May 2, 2026

Copy link
Copy Markdown
Contributor

Review app is ready:

This environment runs on a dedicated Hetzner server for PR #1084 and updates on each push.

@Asherlc Asherlc changed the title Fix ClickHouse backfill migration and CI test fixtures Fix heart-rate zone stats when resting heart rate is invalid May 2, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@packages/server/src/repositories/activity-repository.ts`:
- Around line 445-453: The CASE fallback currently returns the literal 60 which
can violate the invariant resting_hr < up.max_hr; change the ELSE branch in the
resting_hr CASE (the one referencing rhr.resting_hr and up.resting_hr) to
compute a safe fallback that is capped below up.max_hr (use the smaller of 60
and up.max_hr minus one) and ensure it remains positive (at least 1); apply the
identical change to the other CASE occurrence that uses the same fallback so
both preserve resting_hr < up.max_hr.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 19ed7ded-aba9-4529-8732-c9d26acf064b

📥 Commits

Reviewing files that changed from the base of the PR and between 3b904b4 and 9d03547.

📒 Files selected for processing (2)
  • packages/server/src/repositories/activity-repository.test.ts
  • packages/server/src/repositories/activity-repository.ts
✅ Files skipped from review due to trivial changes (1)
  • packages/server/src/repositories/activity-repository.test.ts

Comment thread packages/server/src/repositories/activity-repository.ts

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
packages/server/src/routers/settings.integration.test.ts (1)

214-248: ⚡ Quick win

Assert provider-row deletion to match test intent.

The test name says provider data is wiped, but there’s no assertion for fitness.provider. Add a provider count check for SETTINGS_TEST_USER_ID (or provider id) after settings.deleteAllUserData.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/server/src/routers/settings.integration.test.ts` around lines 214 -
248, Add an assertion that provider rows are deleted by including a count query
for fitness.provider in the same Promise.all and capturing it (e.g., add
providersAfter to the destructured array alongside
activitiesAfter,...,userSettingsAfter), then assert providersAfter.rows[0].count
equals 0 after calling settings.deleteAllUserData; reference
SETTINGS_TEST_USER_ID for the WHERE clause and the settings.deleteAllUserData
call to ensure the test checks provider-row deletion.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@packages/server/src/routers/router-data.integration.test.ts`:
- Around line 1336-1338: The test currently silently returns when
highestHeartRateActivityId is falsy (variable highestHeartRateActivityId), which
masks fixture regressions; replace the early return with an explicit
failure—e.g., assert/throw or use the test framework's fail/expect—to
immediately fail the test if highestHeartRateActivityId is undefined or null
before calling query, ensuring the failing precondition is reported clearly.

---

Nitpick comments:
In `@packages/server/src/routers/settings.integration.test.ts`:
- Around line 214-248: Add an assertion that provider rows are deleted by
including a count query for fitness.provider in the same Promise.all and
capturing it (e.g., add providersAfter to the destructured array alongside
activitiesAfter,...,userSettingsAfter), then assert providersAfter.rows[0].count
equals 0 after calling settings.deleteAllUserData; reference
SETTINGS_TEST_USER_ID for the WHERE clause and the settings.deleteAllUserData
call to ensure the test checks provider-row deletion.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 217e4f78-6dc6-408d-9b80-21daf05e7561

📥 Commits

Reviewing files that changed from the base of the PR and between 9d03547 and d727d3a.

📒 Files selected for processing (7)
  • docs/production-incident-baseline.md
  • packages/server/src/repositories/activity-repository.test.ts
  • packages/server/src/routers/router-data.integration.test.ts
  • packages/server/src/routers/router.integration.test.ts
  • packages/server/src/routers/settings.integration.test.ts
  • src/db/clickhouse-client.test.ts
  • src/db/clickhouse.ts
✅ Files skipped from review due to trivial changes (1)
  • docs/production-incident-baseline.md
🚧 Files skipped from review as they are similar to previous changes (1)
  • packages/server/src/repositories/activity-repository.test.ts

Comment thread packages/server/src/routers/router-data.integration.test.ts
@Asherlc
Asherlc merged commit 5089b01 into main May 2, 2026
62 of 63 checks passed
@Asherlc
Asherlc deleted the Asherlc/setup-dbeaver branch May 2, 2026 05:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants