Skip to content

ci: exercise migrations against MySQL and PostgreSQL #1629

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
tada5hi merged 6 commits into from
May 21, 2026
Merged

ci: exercise migrations against MySQL and PostgreSQL #1629

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
cae2458
ci: exercise migrations against MySQL and PostgreSQL
tada5hi May 20, 2026
8d332d0
ci(migrations): auth mysqladmin probe; document full local flow
tada5hi May 20, 2026
253b981
build: regenerate package-lock.json
tada5hi May 20, 2026
beddd86
fix(build): bump stale kit/server-kit refs to ^0.9.0
tada5hi May 20, 2026
3d1c593
ci(migrations): pipefail-safe count and unauthed-warning-free mysqladmin
tada5hi May 21, 2026
2806760
ci(migrations): use GHA services: blocks instead of docker compose
tada5hi May 21, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 44 additions & 0 deletions .agents/testing.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -231,3 +231,47 @@ Existing fakes to reuse:
- `FakeAnalysisMetadataRecalculator` / `FakeAnalysisNodeMetadataRecalculator` / `FakeAnalysisFileMetadataRecalculator` — record `recalc()` and `recalcDebounced()` calls

When a dependency has no fake yet, create one in the entity's test directory (`test/unit/core/entities/<entity>/`) implementing the port interface.

## Migration Tests

The `test` CI job runs the integration suite against MySQL, Postgres, and SQLite, but the schema is built via `dataSource.synchronize()` (see `apps/<service>/test/app/database.ts`) — migration files in `apps/<service>/src/adapters/database/migrations/{mysql,postgres}/` are **not** exercised by that suite.

A separate `tests-migrations` CI job runs the migration CLI end-to-end for `server-core`, `server-storage`, and `server-telemetry` against a fresh MySQL and Postgres container:

1. `migration run` — applies all migrations forward
2. `migration revert` × N — undoes every migration in reverse order (verifies every `down()` works)
3. `migration run` — re-applies the full chain (verifies idempotency)

This catches SQL syntax errors, cross-DB type mismatches, and `down()` regressions across every migration. It does **not** catch data-correctness bugs in `UPDATE`/`INSERT` migrations against pre-existing rows — those still require manual smoke-testing against a populated database.

The job pre-flights with a sanity check that the compiled migrations exist under `apps/<service>/dist/adapters/database/migrations/{mysql,postgres}/` — without this guard, running the CLI from the wrong working directory results in typeorm silently reporting "No migrations are pending" with exit code 0, masking the failure.

Locally, run the same `run → revert × N → run` flow against a running compose stack:

```bash
# MySQL
docker compose up -d mysql
cd apps/server-core # or: server-storage / server-telemetry

export DB_TYPE=mysql DB_HOST=127.0.0.1 DB_PORT=3306 \
DB_USERNAME=root DB_PASSWORD=start123 DB_DATABASE=app

node dist/cli/index.mjs migration run
node dist/cli/index.mjs migration revert # repeat once per migration
node dist/cli/index.mjs migration run # idempotency replay
```

```bash
# Postgres
docker compose up -d postgres
cd apps/server-core

export DB_TYPE=postgres DB_HOST=127.0.0.1 DB_PORT=5432 \
DB_USERNAME=postgres DB_PASSWORD=start123 DB_DATABASE=app

node dist/cli/index.mjs migration run
node dist/cli/index.mjs migration revert
node dist/cli/index.mjs migration run
```

The CLI auto-creates the target database if it does not exist.
92 changes: 92 additions & 0 deletions .github/workflows/main.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -136,6 +136,98 @@ jobs:
run: |
npm run test

tests-migrations:
name: Test Migrations
needs: [build]
runs-on: ubuntu-latest

strategy:
fail-fast: false
matrix:
service: [ server-core, server-storage, server-telemetry ]
db: [ mysql, postgres ]
include:
- db: mysql
port: 3306
user: root
- db: postgres
port: 5432
user: postgres

env:
DB_TYPE: ${{ matrix.db }}
DB_HOST: 127.0.0.1
DB_PORT: ${{ matrix.port }}
DB_USERNAME: ${{ matrix.user }}
DB_PASSWORD: start123
DB_DATABASE: app

Comment on lines +139 to +164
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot May 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

fd -type f -name "main.yml" | head -20

Repository: PrivateAIM/hub

Length of output: 229

🏁 Script executed:

fd "main.yml" .github/workflows/

Repository: PrivateAIM/hub

Length of output: 85

🏁 Script executed:

cat -n .github/workflows/main.yml | sed -n '135,170p'

Repository: PrivateAIM/hub

Length of output: 1349

🏁 Script executed:

cat -n .github/workflows/main.yml | sed -n '165,230p'

Repository: PrivateAIM/hub

Length of output: 3149

Add explicit least-privilege permissions for tests-migrations.

This job currently inherits default token permissions. Set an explicit minimal block with contents: read to avoid accidental over-privilege drift. The job only requires repository read access for the checkout step.

Suggested hardening 
 jobs:
   tests-migrations:
+    permissions:
+      contents: read
     name: Test Migrations
🧰 Tools
🪛 zizmor (1.25.2)

[warning] 139-213: overly broad permissions (excessive-permissions): default permissions used due to no permissions: block

(excessive-permissions)

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/main.yml around lines 139 - 164, The tests-migrations job
is inheriting default token permissions; add an explicit least-privilege
permissions block to that job by inserting a permissions: mapping under the
tests-migrations job (the job named tests-migrations in the workflow) with at
minimum contents: read so the job only has repository read access for checkout;
ensure the permissions mapping is at the same indentation level as
name/needs/runs-on and does not grant any other scopes.

services:
mysql:
image: mysql:9
env:
MYSQL_ROOT_PASSWORD: start123
ports:
- 3306:3306
options: >-
--health-cmd="mysqladmin ping -h localhost -pstart123"
--health-interval=10s
--health-timeout=5s
--health-retries=5

postgres:
image: postgres:18
env:
POSTGRES_PASSWORD: start123
ports:
- 5432:5432
options: >-
--health-cmd="pg_isready -U postgres"
--health-interval=10s
--health-timeout=5s
--health-retries=5

steps:
- name: Checkout
uses: actions/checkout@v6
Comment on lines +191 to +192
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot May 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

cat -n .github/workflows/main.yml | sed -n '160,175p'

Repository: PrivateAIM/hub

Length of output: 654

🌐 Web query:

GitHub Actions checkout action security best practices pinning commit SHA

💡 Result:

Pinning GitHub Actions to a full-length commit SHA is considered the industry standard for supply chain security [1][2][3]. Unlike version tags (e.g., @v4) or branch references (e.g., @main), which are mutable and can be moved or compromised to point to malicious code, a full-length commit SHA provides an immutable, cryptographically verifiable reference [4][5][6]. Why Pinning to SHA is Critical 1. Prevents Tag Hijacking: Attackers can gain control of a repository or its maintainer accounts and move existing version tags to point to "imposter" commits containing malicious payloads, such as credential scrapers [7][8][3]. Workflows using tag-based references will automatically execute this malicious code without warning [7][3]. 2. Ensures Immutability: Once a commit exists in Git, its SHA cannot be altered without changing the code itself [5]. Pinning to a SHA guarantees that your workflow executes the exact, validated code every time [4][5]. 3. Enables Policy Enforcement: GitHub now allows administrators to enforce full commit SHA pinning at the repository and organization levels [9][10]. Workflows that do not use a full SHA will fail, preventing the accidental use of unpinned actions [5][10][6]. Best Practices for Implementation - Use Full-Length SHAs: Always use the full-length commit SHA, not a truncated version [1][2]. - Include Descriptive Comments: To maintain readability and ease of updates, include the version tag in a comment on the same line [11][12][9]. Example: uses: actions/checkout@b4ffde6 # v4.1.1 - Automate Updates: Manually tracking SHAs can be burdensome [5][3]. Use automated dependency management tools like Dependabot or Renovate to monitor for updates and create pull requests when new, safe versions are available [11][3][6]. - Adopt Cooldown Periods: To further mitigate risk, implement a "cooldown" or minimum age policy (e.g., 7-14 days) before adopting new action versions [6]. This waiting period helps ensure that any malicious code introduced via a new release is discovered and remediated before you incorporate it into your environment [6]. - Monitor Transitive Dependencies: Even if your top-level action is pinned, ensure the action itself does not pull in unpinned dependencies [5][6]. Organizations may enforce this by applying SHA-pinning policies that inspect the entire action dependency tree [5]. If you find an action that is not pinned, you can typically retrieve its current SHA using the command: git ls-remote https://github.com/[owner]/[repo] [tag/branch] [11]

Citations:

🌐 Web query:

GitHub Actions checkout action persist-credentials false security best practice

💡 Result:

Setting persist-credentials: false in the actions/checkout action is considered a security best practice to prevent the automatic storage of Git authentication credentials (such as the GITHUB_TOKEN) within the runner's local Git configuration [1][2][3]. By default, actions/checkout configures Git to use a provided token (often the GITHUB_TOKEN) to authenticate subsequent Git operations (e.g., git fetch, git push) [1][4]. When persist-credentials is true (the default), these credentials are saved in a way that makes them accessible to any other process, step, or action running on the same runner [4][3][5]. If a workflow step or a third-party action is compromised, an attacker could potentially scrape these credentials from the disk [4][2][5]. Key considerations include: - Purpose of persist-credentials: false: It ensures that sensitive tokens are not stored in the environment where they could be inadvertently exposed, especially if the repository or artifacts are archived or accessed by untrusted code [2][3]. - Modern Mitigation: While recent updates to actions/checkout have improved security by moving credential storage to a separate, isolated location under $RUNNER_TEMP (using Git's includeIf mechanism) rather than the main .git/config file, explicitly setting persist-credentials: false remains the recommended best practice for workflows where authenticated Git operations are not required [1][6][7]. - Defense in Depth: Because the GITHUB_TOKEN is automatically available to many workflows, simply setting persist-credentials: false is not a complete security solution [4][5]. You should also enforce the principle of least privilege by explicitly defining the required permissions for your jobs at the workflow or job level (e.g., setting contents: read rather than the default, more permissive settings) [4][8][9]. To implement this, add the parameter to your checkout step: yaml - uses: actions/checkout@v4 with: persist-credentials: false Using this setting ensures that your workflow does not leave unnecessary, persistent credentials on the runner disk [2][3].

Citations:

Harden checkout: pin to a commit SHA and disable credential persistence.

Using a floating action tag (@v6) and persisted credentials weakens supply-chain security. Commit SHA pinning prevents tag hijacking attacks where compromised maintainer accounts can move version tags to malicious code, while disabling credential persistence prevents token exposure if workflow steps are compromised.

Suggested hardening 
-            -   name: Checkout
-                uses: actions/checkout@v6
+            -   name: Checkout
+                uses: actions/checkout@<pinned_commit_sha_for_v6>
+                with:
+                    persist-credentials: false
🧰 Tools
🪛 zizmor (1.25.2)

[warning] 166-167: credential persistence through GitHub Actions artifacts (artipacked): does not set persist-credentials: false

(artipacked)

[error] 167-167: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy)

(unpinned-uses)

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/main.yml around lines 166 - 167, The workflow uses a
floating actions/checkout tag ("uses: actions/checkout@v6") which is vulnerable
to tag hijacking and leaves credentials persisted; update the Checkout step
("name: Checkout") to pin actions/checkout to a specific commit SHA (replace `@v6`
with @<commit-sha>) and add a with: persist-credentials: false setting (and
optionally fetch-depth: 1) so credentials are not persisted to the workspace.


- name: Install
uses: ./.github/actions/install
with:
node-version: ${{ env.NODE_VERSION }}
node-registry: ${{ env.NODE_REGISTRY }}

- name: Build
uses: ./.github/actions/build

- name: Verify compiled migrations exist
working-directory: apps/${{ matrix.service }}
run: |
shopt -s nullglob
files=(dist/adapters/database/migrations/${{ matrix.db }}/*.mjs)
count=${#files[@]}
if [ "$count" = "0" ]; then
echo "ERROR: no compiled migrations in apps/${{ matrix.service }}/dist/adapters/database/migrations/${{ matrix.db }}/"
exit 1
fi
echo "Found $count migration(s)"
echo "MIGRATION_COUNT=$count" >> "$GITHUB_ENV"

- name: Run migrations forward
working-directory: apps/${{ matrix.service }}
run: node dist/cli/index.mjs migration run

- name: Revert all migrations
working-directory: apps/${{ matrix.service }}
run: |
for ((i = 1; i <= MIGRATION_COUNT; i++)); do
node dist/cli/index.mjs migration revert
done

- name: Re-apply all migrations (idempotency)
working-directory: apps/${{ matrix.service }}
run: node dist/cli/index.mjs migration run

lint:
needs: [build]
runs-on: ubuntu-latest
Expand Down
4 changes: 2 additions & 2 deletions docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
version: '3.9'
services:
mysql:
image: mysql
image: mysql:9
restart: always
healthcheck:
test: [ "CMD", "mysqladmin" ,"ping", "-h", "localhost" ]
Expand All @@ -14,7 +14,7 @@ services:
ports:
- '3306:3306'
postgres:
image: postgres
image: postgres:18
restart: always
healthcheck:
test: [ "CMD", "pg_isready" ]
Expand Down
12 changes: 6 additions & 6 deletions package-lock.json
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 4 additions & 4 deletions packages/server-storage-kit/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,13 +19,13 @@
"license": "Apache-2.0",
"description": "This package contains server side db helpers & utilities.",
"devDependencies": {
"@privateaim/kit": "^0.8.21",
"@privateaim/server-kit": "^0.8.21",
"@privateaim/kit": "^0.9.0",
"@privateaim/server-kit": "^0.9.0",
"@privateaim/storage-kit": "^0.8.21"
},
"peerDependencies": {
"@privateaim/kit": "^0.8.21",
"@privateaim/server-kit": "^0.8.21",
"@privateaim/kit": "^0.9.0",
"@privateaim/server-kit": "^0.9.0",
"@privateaim/storage-kit": "^0.8.21"
},
"scripts": {
Expand Down
4 changes: 2 additions & 2 deletions packages/server-test-kit/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,13 +22,13 @@
"@authup/access": "^1.0.0-beta.36",
"@authup/core-kit": "^1.0.0-beta.36",
"@privateaim/errors": "^0.8.42",
"@privateaim/server-kit": "^0.8.37"
"@privateaim/server-kit": "^0.9.0"
},
"peerDependencies": {
"@authup/access": "^1.0.0-beta.36",
"@authup/core-kit": "^1.0.0-beta.36",
"@privateaim/errors": "^0.8.42",
"@privateaim/server-kit": "^0.8.37"
"@privateaim/server-kit": "^0.9.0"
},
"scripts": {
"build": "rimraf ./dist && tsdown"
Expand Down
Loading