Skip to content

fixing duplicate migration runs for logstore - #4416

Merged
akshaydeo merged 1 commit into
devfrom
06-15-fixing_duplicate_migration_runs_for_logstore
Jun 15, 2026
Merged

fixing duplicate migration runs for logstore#4416
akshaydeo merged 1 commit into
devfrom
06-15-fixing_duplicate_migration_runs_for_logstore

Conversation

@akshaydeo

@akshaydeo akshaydeo commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

Summary

Removes noisy, redundant logger.Info calls that were logging truncated SQL statement strings immediately before executing those same statements in both configstore and logstore migration functions. Additionally, removes the Postgres-only guard from areThereAnyPendingMigrations in both stores so that the preflight migration check runs for all database dialects (including SQLite), aligning behavior across environments.

Changes

  • Removed ~100 logger.Info calls that logged partial, truncated SQL strings (e.g. "executing ALTER TABLE config_keys ADD COLUMN name VARCHAR(255)\").Error; err != nil { retur") immediately before executing those statements — these logs were misleading and added no diagnostic value.
  • Removed the if db.Dialector.Name() == "postgres" guard from areThereAnyPendingMigrations in configstore/migrations.go, making the pending-migration preflight check dialect-agnostic and consistent with the fail-open behavior already documented in the comment.
  • Extracted areThereAnyPendingMigrations in logstore/migrations.go as a named function (mirroring the configstore pattern) and removed the Postgres-only guard from triggerMigrations, so SQLite deployments also benefit from the preflight skip and double-check-after-lock optimizations.
  • Added structured logging in the logstore areThereAnyPendingMigrations function to enumerate each pending migration ID.
  • Moved the SQLite dialect guard inside the migrationUpdateTimestampFormat migrate closure so the migration step is always registered but skips execution on non-SQLite dialects, keeping the migration table consistent across dialects.

Type of change

  • Bug fix
  • Feature
  • Refactor
  • Documentation
  • Chore/CI

Affected areas

  • Core (Go)
  • Transports (HTTP)
  • Providers/Integrations
  • Plugins
  • UI (React)
  • Docs

How to test

go test ./framework/configstore/... ./framework/logstore/...

Verify that on startup with a fresh SQLite database, migrations run without errors and the migration preflight check correctly skips re-running already-applied steps. Verify the same on Postgres. Confirm that log output no longer contains truncated SQL strings prefixed with "executing".

Breaking changes

  • Yes
  • No

Security considerations

None. Changes are limited to migration orchestration logic and log verbosity.

Checklist

  • I read docs/contributing/README.md and followed the guidelines
  • I added/updated tests where appropriate
  • I updated documentation where needed
  • I verified builds succeed (Go and UI)
  • I verified the CI pipeline passes locally if applicable

@coderabbitai

coderabbitai Bot commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: c033bb42-347d-4410-a068-138187e71aa0

📥 Commits

Reviewing files that changed from the base of the PR and between befeccc and 1211a8f.

📒 Files selected for processing (2)
  • framework/configstore/migrations.go
  • framework/logstore/migrations.go

📝 Walkthrough

Summary by CodeRabbit

  • Bug Fixes

    • Improved database migration reliability with safer SQL execution and better error handling across all database systems.
    • Enhanced migration idempotency for indexes, constraints, and data operations.
  • Chores

    • Refactored migration orchestration and execution logic for better performance and maintainability.

Walkthrough

Migration orchestration in both configstore and logstore is refactored to centralize pending-migration preflight into areThereAnyPendingMigrations, used at two points around the advisory lock. Across dozens of existing migration functions in both files, SQL execution, rollback ordering, backfill conditions, encryption NULL-handling, OAuth deduplication, session_id-based uniqueness, and index/constraint creation are corrected or hardened.

Changes

Migration Orchestration and DDL/DML Hardening

Layer / File(s) Summary
Preflight helper and dual-check lock flow
framework/configstore/migrations.go, framework/logstore/migrations.go
areThereAnyPendingMigrations is introduced in both stores, made dialect-agnostic and fail-open; triggerMigrations uses it before acquiring the advisory lock and again after, so nodes that win the lock skip running when another node already completed migrations.
Early configstore migration SQL corrections
framework/configstore/migrations.go
allow_direct_keys and enable_litellm_fallbacks add/drop paths are corrected; related initialization/backfill statements for subsequent migrations use explicit defaults and remove brittle concatenated SQL strings.
Distributed locks, governance, and model-config FK migrations
framework/configstore/migrations.go
Adds explicit expires_at index to distributed_locks creation; reorders provider governance rollback drops; corrects VK/provider governance-to-model-config FK/nulling SQL; fixes model-config/model-budget FK wiring with pre-cleaning for orphaned rows before cascade constraint.
Aliases JSON migration and encryption fixup
framework/configstore/migrations.go
Corrects deployment-columns-to-aliases_json UPDATE/drop statements; encryption fixup path reliably encrypts plaintext alias JSON and recomputes config_hash values.
Enforce-auth, encryption NULL-handling, and column-widening guards
framework/configstore/migrations.go
Enforce-auth DML uses consistent UPDATE statements; hash columns are set to NULL via dialect-safe SQL; azure_api_version widening is guarded with a column-existence check.
Compat columns, pricing dedup, calendar-aligned guards, and MCP URL DDL
framework/configstore/migrations.go
Updates replace_enable_litellm_with_compat_columns data migration/rollback; adjusts add_model_pricing_unique_index to explicit DELETE/GROUP BY dedup; adds column-introspection guards to migrate_calendar_aligned; corrects MCP external base URL column add/drop/backfill DDL.
OAuth auth-mode dedup, NOT NULL, partial indexes, and session_id migration
framework/configstore/migrations.go
Reworks OAuth auth-mode migration with backfill, identity-less row deletion, NOT NULL enforcement on Postgres, window-function deduplication, and partial unique indexes. Switches OAuth session schema to session_id-backed uniqueness; non-vk cleanup deletes rows where discriminator is NULL or not vk.
Late configstore migrations: column drops, access-profile, customer-budget
framework/configstore/migrations.go
Drops legacy calendar_aligned columns via dropColumnSQL; refines access-profile and per-user header credential index/column drops; adjusts customer-budget nulling backfill while preserving forward-only semantics.
Logstore migration logging cleanup and batched backfill
framework/logstore/migrations.go
Removes malformed logger.Info strings across many migrations while preserving all tx.Exec calls. Changes migrationAddRequestIDColumnToMCPToolLogs to batched ctid+SKIP LOCKED backfill. Restructures migrationUpdateTimestampFormat with a dialect variable.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

  • maximhq/bifrost#4093: Modifies framework/configstore/migrations.go around governance/provider-governance migration idempotency and calendar_aligned/legacy column drop handling with different guard approaches.
  • maximhq/bifrost#4327: Modifies framework/logstore/migrations.go migration orchestration and triggerMigrations/pending-migration preflight logic with logger plumbing and backfill logging updates.
  • maximhq/bifrost#4051: Modifies framework/configstore/migrations.go specifically around triggerMigrations preflight/lock flow and FK-cascade migration steps.

Suggested reviewers

  • danpiths
  • roroghost17

🐇 Hops through migrations, one fix at a time,
No more garbled logs, no more half-baked crime,
The advisory lock checks twice — just to be sure,
OAuth deduped clean, session IDs pure,
With idempotent guards and a rollback in line,
These migrations now march in a tidy parade! 🎉

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Linked Issues check ⚠️ Warning The linked issue #123 'Files API Support' describes implementation of file upload APIs for providers like OpenAI/Anthropic, which is completely unrelated to the PR's focus on fixing duplicate migration runs and refactoring migration orchestration in configstore/logstore. Remove the unrelated #123 issue from linked issues and link to issues that actually describe the migration orchestration problems this PR addresses.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title 'fixing duplicate migration runs for logstore' is specific and directly related to the core change (refactoring migration orchestration in logstore), but the raw summary and PR description indicate broader changes across both configstore and logstore, making the title incomplete.
Description check ✅ Passed The PR description is well-structured, comprehensive, and addresses all required template sections with clear details about changes, rationale, testing approach, and affected areas.
Out of Scope Changes check ✅ Passed All changes in configstore/migrations.go and logstore/migrations.go are directly related to the stated objectives of refactoring migration orchestration, removing redundant logging, and ensuring dialect-agnostic behavior.
Docstring Coverage ✅ Passed Docstring coverage is 97.18% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch 06-15-fixing_duplicate_migration_runs_for_logstore

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 golangci-lint (2.12.2)

level=error msg="[linters_context] typechecking error: pattern ./...: directory prefix . does not contain main module or its selected dependencies"


Comment @coderabbitai help to get the list of available commands and usage tips.

@akshaydeo
akshaydeo marked this pull request as ready for review June 15, 2026 16:36

Copy link
Copy Markdown
Contributor Author

This stack of pull requests is managed by Graphite. Learn more about stacking.

@coderabbitai
coderabbitai Bot requested review from danpiths and roroghost17 June 15, 2026 16:38
@greptile-apps

greptile-apps Bot commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

Confidence Score: 4/5

Safe to merge for correctness; the core migration-deduplication fix is sound and the configstore change is a clean extension to SQLite. The only remaining rough edge is cosmetic debug logging left in logstore's new helper.

The root-cause fix is well-targeted: moving the dialect check inside the Migrate callback is exactly how gormigrate expects dialect-conditional no-ops to be expressed. The double-check-lock pattern is correct. One non-blocking issue remains: areThereAnyPendingMigrations in logstore unconditionally logs every pending migration ID and is called twice per boot, producing redundant or empty log lines that the configstore counterpart avoids entirely.

framework/logstore/migrations.go — the new areThereAnyPendingMigrations helper has leftover per-ID debug logging worth cleaning up before the next release.

Important Files Changed

Filename Overview
framework/logstore/migrations.go Extracts areThereAnyPendingMigrations helper and refactors triggerMigrations into a double-check-lock pattern; fixes migrationUpdateTimestampFormat to record itself in the migrations table for non-SQLite dialects (the root cause of duplicate runs); removes many truncated debug logger.Info calls. Minor issue: the new helper unconditionally logs every pending migration ID on each of its two calls per startup.
framework/configstore/migrations.go Removes the Postgres-only guard from areThereAnyPendingMigrations, extending the skip-if-no-pending optimisation to SQLite deployments; strips a large number of truncated debug logger.Info calls added in a previous pass. No behavioral regressions for Postgres; SQLite now benefits from the same early-exit path.

Reviews (1): Last reviewed commit: "fixing duplicate migration runs for logs..." | Re-trigger Greptile

Comment thread framework/logstore/migrations.go
@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

akshaydeo commented Jun 15, 2026

Copy link
Copy Markdown
Contributor Author

Merge activity

  • Jun 15, 5:06 PM UTC: A user started a stack merge that includes this pull request via Graphite.
  • Jun 15, 5:07 PM UTC: @akshaydeo merged this pull request with Graphite.

@akshaydeo
akshaydeo merged commit 3c84cdc into dev Jun 15, 2026
15 of 16 checks passed
@akshaydeo
akshaydeo deleted the 06-15-fixing_duplicate_migration_runs_for_logstore branch June 15, 2026 17:07
akshaydeo added a commit that referenced this pull request Jun 15, 2026
## Summary

Removes noisy, redundant `logger.Info` calls that were logging truncated SQL statement strings immediately before executing those same statements in both `configstore` and `logstore` migration functions. Additionally, removes the Postgres-only guard from `areThereAnyPendingMigrations` in both stores so that the preflight migration check runs for all database dialects (including SQLite), aligning behavior across environments.

## Changes

- Removed ~100 `logger.Info` calls that logged partial, truncated SQL strings (e.g. `"executing ALTER TABLE config_keys ADD COLUMN name VARCHAR(255)\").Error; err != nil { retur"`) immediately before executing those statements — these logs were misleading and added no diagnostic value.
- Removed the `if db.Dialector.Name() == "postgres"` guard from `areThereAnyPendingMigrations` in `configstore/migrations.go`, making the pending-migration preflight check dialect-agnostic and consistent with the fail-open behavior already documented in the comment.
- Extracted `areThereAnyPendingMigrations` in `logstore/migrations.go` as a named function (mirroring the configstore pattern) and removed the Postgres-only guard from `triggerMigrations`, so SQLite deployments also benefit from the preflight skip and double-check-after-lock optimizations.
- Added structured logging in the logstore `areThereAnyPendingMigrations` function to enumerate each pending migration ID.
- Moved the SQLite dialect guard inside the `migrationUpdateTimestampFormat` migrate closure so the migration step is always registered but skips execution on non-SQLite dialects, keeping the migration table consistent across dialects.

## Type of change

- [ ] Bug fix
- [ ] Feature
- [x] Refactor
- [ ] Documentation
- [ ] Chore/CI

## Affected areas

- [x] Core (Go)
- [ ] Transports (HTTP)
- [ ] Providers/Integrations
- [ ] Plugins
- [ ] UI (React)
- [ ] Docs

## How to test

```sh
go test ./framework/configstore/... ./framework/logstore/...
```

Verify that on startup with a fresh SQLite database, migrations run without errors and the migration preflight check correctly skips re-running already-applied steps. Verify the same on Postgres. Confirm that log output no longer contains truncated SQL strings prefixed with `"executing"`.

## Breaking changes

- [ ] Yes
- [x] No

## Security considerations

None. Changes are limited to migration orchestration logic and log verbosity.

## Checklist

- [ ] I read `docs/contributing/README.md` and followed the guidelines
- [ ] I added/updated tests where appropriate
- [ ] I updated documentation where needed
- [ ] I verified builds succeed (Go and UI)
- [ ] I verified the CI pipeline passes locally if applicable
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants