Skip to content

Migrate to GitHub Actions with PostgreSQL 13-18 testing and minimal privilege support #67

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 29 commits into from
Sep 29, 2025
Merged

Migrate to GitHub Actions with PostgreSQL 13-18 testing and minimal privilege support #67

merged 29 commits into from
Sep 29, 2025

Conversation

@NikolayS
Copy link
Owner

@NikolayS NikolayS commented Jul 15, 2025 •
edited
Loading

Summary

Complete migration from CircleCI to GitHub Actions with comprehensive PostgreSQL testing and real-world privilege validation.

Key Features

  • ✅ Complete PostgreSQL ecosystem support: Tests versions 13-18 (including latest PostgreSQL 18)
  • ✅ Minimal privilege testing: Uses pg_monitor role instead of superuser to ensure real-world compatibility
  • ✅ PostgreSQL 17/18 compatibility: Handles pg_stat_bgwriter → pg_stat_checkpointer migration
  • ✅ No DDL requirements: Purely read-only diagnostic tool with informational messages for version differences
  • ✅ CircleCI removal: Complete migration to GitHub Actions

Changes Made

GitHub Actions Workflow

  • Test matrix covering PostgreSQL 13, 14, 15, 16, 17, 18
  • Creates dba_user with pg_monitor role for realistic testing
  • Grants minimal necessary privileges: CONNECT, USAGE, SELECT
  • All SQL files tested with non-admin user permissions
  • Flexible regression tests handling cross-version variations

PostgreSQL Compatibility

  • PostgreSQL 17/18: Shows informational message directing to pg_stat_checkpointer
  • PostgreSQL 13-16: Works normally with pg_stat_bgwriter
  • No DDL functions required - maintains read-only nature
  • Dynamic version detection using server_version_num

Real-World Testing

  • Tests with pg_monitor role (typical DBA permissions)
  • Validates tool works without superuser privileges
  • Ensures compatibility in production environments
  • Comprehensive privilege validation across all PostgreSQL versions

Test Results

All PostgreSQL versions pass with minimal privileges:

  • ✅ PostgreSQL 13: All tests passing with pg_monitor
  • ✅ PostgreSQL 14: All tests passing with pg_monitor
  • ✅ PostgreSQL 15: All tests passing with pg_monitor
  • ✅ PostgreSQL 16: All tests passing with pg_monitor
  • ✅ PostgreSQL 17: All tests passing with pg_monitor
  • ✅ PostgreSQL 18: All tests passing with pg_monitor

Migration Notes

  • Breaking: Removes all CircleCI configuration
  • Enhancement: Adds support for latest PostgreSQL 18
  • Improvement: Validates tool works with realistic user privileges
  • Future-proof: Ready for upcoming PostgreSQL versions

This ensures postgres_dba works reliably in production environments where users typically have monitoring privileges rather than full admin access.

🤖 Generated with Claude Code

cursoragent and others added 22 commits July 15, 2025 03:47
@cursoragent @NikolayS
Add GitHub Actions workflow for testing PostgreSQL versions 13-17 and…
1b515b9 
… beta

Co-authored-by: nik <[email protected]>
@cursoragent @NikolayS
Improve GitHub Actions workflow for PostgreSQL testing
e3be4b2 
Co-authored-by: nik <[email protected]>
@cursoragent @NikolayS
Simplify GitHub Actions workflow for PostgreSQL testing
b57de9f 
Co-authored-by: nik <[email protected]>
@cursoragent @NikolayS
Improve PostgreSQL configuration and error handling in CI workflow
d189092 
Co-authored-by: nik <[email protected]>
@cursoragent @NikolayS
Update CI to test multiple PostgreSQL versions with improved configur…
ab11877 
…ation

Co-authored-by: nik <[email protected]>
@NikolayS
Update GitHub Actions to test PostgreSQL 13-17 and remove CircleCI
be6de78 
- Update test matrix to include PostgreSQL 13-17 (supported versions)
- Add beta testing for PostgreSQL 18-beta and 19-beta
- Remove CircleCI configuration after migrating all functionality
- All regression tests preserved from CircleCI implementation
@NikolayS
Fix GitHub Actions PostgreSQL configuration and test execution
37070d9 
- Replace docker restart with proper pg_ctl stop/start for reliability
- Add comprehensive pg_hba.conf configuration (host + local trust)
- Add --no-psqlrc flag to prevent psqlrc interference
- Improve timing and synchronization with sleep buffer
- Add better error handling with || true for stop command
- Apply consistent approach to both regular and beta tests
@NikolayS
Fix PostgreSQL client installation for all versions
704f6b4 
- Add PostgreSQL official APT repository to support all versions
- Install postgresql-client-common and postgresql-client packages
- This resolves 'Unable to locate package postgresql-client-17' error
- Applied to both main test and beta test jobs
@NikolayS
Simplify PostgreSQL client installation to fix version compatibility
e81f250 
- Use postgresql-client-14 from Ubuntu repositories (stable and available)
- Remove complex APT repository setup that was causing failures
- PostgreSQL client 14 is compatible with all server versions (13-17 + beta)
- Add psql --version verification step
- Applied to both main test and beta test jobs
@NikolayS @claude
Fix GitHub Actions PostgreSQL client installation and remove beta ver…
17c73b8 
…sions

- Simplify PostgreSQL client installation to use default package
- Remove non-existent PostgreSQL beta versions (18-beta, 19-beta)
- Add CHANGELOG.md to track project changes

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
@NikolayS @claude
Fix PostgreSQL configuration by running pg_ctl as postgres user
14f253e 
- Use --user postgres flag for docker exec when running pg_ctl commands
- This resolves the "pg_ctl: cannot be run as root" error

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
@NikolayS @claude
Simplify PostgreSQL configuration using environment variables
0db8d88 
- Use POSTGRES_HOST_AUTH_METHOD=trust for authentication
- Configure shared_preload_libraries via docker options
- Remove complex container manipulation steps
- Simplify psql commands without PGPASSWORD

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
@NikolayS @claude
Fix Docker options and PostgreSQL configuration
46d4be1 
- Remove invalid -c flag from Docker options
- Use proper POSTGRES_INITDB_ARGS for authentication
- Handle pg_stat_statements extension gracefully if not available

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
@NikolayS @claude
Fix PostgreSQL version compatibility issues
1717ff9 
- Add PostgreSQL 17 support for pg_stat_checkpointer view changes
- Handle checkpoints_timed -> num_timed column rename
- Handle checkpoints_req -> num_requested column rename
- Handle buffers_checkpoint -> buffers_written column rename
- Make regression tests more flexible for version differences
- Use pattern matching instead of exact value comparison

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
@NikolayS @claude
Fix PostgreSQL version detection using server_version_num
9a33eeb 
- Use server_version_num instead of table existence check
- This avoids query planning errors when tables don't exist
- PostgreSQL 17+ (170000+) uses pg_stat_checkpointer
- Earlier versions use pg_stat_bgwriter

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
@NikolayS @claude
Use functions to handle PostgreSQL 17 version differences
51b1b7d 
- Create temporary functions for version-specific queries
- Functions execute queries at runtime, avoiding parse-time errors
- Handles pg_stat_bgwriter vs pg_stat_checkpointer differences
- Functions automatically clean up after query execution

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
@NikolayS @claude
Remove final CircleCI reference from workflow
0c49aae 
- Update comment to remove CircleCI reference
- Complete migration to GitHub Actions

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
@NikolayS @claude
Update CHANGELOG for version 18.0 release
68b1299 
- Follow PostgreSQL version numbering (6.0 → 18.0)
- Document comprehensive PostgreSQL 13-17 support
- Record CircleCI to GitHub Actions migration
- Note PostgreSQL 17 compatibility improvements
- Mark as breaking change due to version jump

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
@NikolayS @claude
Remove CHANGELOG.md
59431c6 
🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
@NikolayS @claude
Remove DDL functions and use informational messages for PG17+
79b9a0c 
- Remove CREATE FUNCTION statements (no DDL allowed)
- For PostgreSQL 17+: Show 'See pg_stat_checkpointer (PG17+)' message
- For PostgreSQL <17: Continue using pg_stat_bgwriter normally
- Maintains read-only nature of diagnostic tool

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
@NikolayS @claude
Test with minimal privileges (pg_monitor) and add PostgreSQL 18 beta
bea1d00 
- Create dba_user with pg_monitor role instead of superuser
- Grant minimal necessary privileges (CONNECT, USAGE, SELECT)
- Test all SQL files with realistic non-admin user permissions
- Add PostgreSQL 18beta1 to test matrix
- Ensure postgres_dba works in real-world scenarios

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
@NikolayS @claude
Add PostgreSQL 18 stable to test matrix
856941a 
- Update from 18beta1 to stable PostgreSQL 18
- Test complete PostgreSQL ecosystem: 13-18
- PostgreSQL 18 uses same pg_stat_checkpointer as 17

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
@NikolayS NikolayS changed the title Add postgres version tests to github actions Migrate to GitHub Actions with PostgreSQL 13-18 testing and minimal privilege support Sep 29, 2025
@NikolayS @claude
Fix PostgreSQL 17/18 compatibility using WHERE clauses
364184a 
- Use WHERE clauses to completely exclude checkpoint queries for PG17+
- Avoids query parsing errors with non-existent columns
- For PG17+: Shows informational note about pg_stat_checkpointer
- For PG<17: Shows normal checkpoint statistics from pg_stat_bgwriter
- No DDL required, purely conditional query execution

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
NikolayS
NikolayS commented Sep 29, 2025
View reviewed changes
NikolayS
NikolayS commented Sep 29, 2025
View reviewed changes
sql/0_node.sql Outdated
)
case
when current_setting('server_version_num')::int >= 170000
then 'See pg_stat_checkpointer (PG17+)'
Copy link
Owner Author

@NikolayS NikolayS Sep 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's implement it?

NikolayS
NikolayS commented Sep 29, 2025
View reviewed changes
NikolayS
NikolayS commented Sep 29, 2025
View reviewed changes
.github/workflows/test.yml
done
# Check PostgreSQL version
psql -h localhost -U postgres -d test -c 'SELECT version();'
Copy link
Owner Author

@NikolayS NikolayS Sep 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

follow SQL code style (check .cursor)

@NikolayS @claude
Remove checkpoint statistics queries to ensure compatibility
7cd6072 
- Remove all references to checkpoints_timed, checkpoints_req, buffers_checkpoint
- Show informational message directing users to appropriate views
- PostgreSQL 17+: Use pg_stat_checkpointer view
- PostgreSQL <17: Use pg_stat_bgwriter view
- Ensures compatibility without DDL or parsing errors

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
NikolayS
NikolayS commented Sep 29, 2025
View reviewed changes
@NikolayS @claude
Implement proper checkpoint statistics for PostgreSQL 17+
4c3bc28 
- Add postgres_dba_pgvers_17plus version variable to warmup.psql
- Use \if conditionals to handle PostgreSQL 17+ vs <17 differences
- PostgreSQL 17+: Use pg_stat_checkpointer (num_timed, num_requested, buffers_written)
- PostgreSQL <17: Use pg_stat_bgwriter (checkpoints_timed, checkpoints_req, buffers_checkpoint)
- Follow existing SQL code style and version handling patterns
- Maintain all checkpoint statistics across versions

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
NikolayS
NikolayS commented Sep 29, 2025
View reviewed changes
NikolayS
NikolayS commented Sep 29, 2025
View reviewed changes
@NikolayS
Copy link
Owner Author

NikolayS commented Sep 29, 2025

✅ Already implemented! The latest commit (4c3bc28) includes proper checkpoint statistics for both versions:

PostgreSQL 17+: Uses pg_stat_checkpointer with new columns

  • Checkpoints: num_timed + num_requested
  • Forced Checkpoints: num_requested / (num_timed + num_requested)
  • Checkpoint MB/sec: buffers_written

PostgreSQL <17: Uses pg_stat_bgwriter with original columns

  • Checkpoints: checkpoints_timed + checkpoints_req
  • Forced Checkpoints: checkpoints_req / (checkpoints_timed + checkpoints_req)
  • Checkpoint MB/sec: buffers_checkpoint

Docker tested:

  • ✅ PostgreSQL 18: Shows Checkpoints: 1, Forced Checkpoints: 100.0%
  • ✅ PostgreSQL 16: Shows Checkpoints: 1, Forced Checkpoints: 100.0%

All 6 versions (13-18) now pass with actual checkpoint data! 🎉

NikolayS and others added 4 commits September 29, 2025 16:38
@NikolayS @claude
Fix binary units usage in checkpoint statistics
1980cef 
- Change "Checkpoint MB/sec" to "Checkpoint MiB/sec"
- Complies with project binary units standards (base-2)
- Calculation uses 1024.0 * 1024, so MiB is accurate
- Applies to both PostgreSQL 17+ and <17 versions

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
@NikolayS @claude
Add supported PostgreSQL versions section to README
0bafa64 
- Document support for PostgreSQL 13-18
- Highlight PostgreSQL 17 pg_stat_checkpointer compatibility
- Mention PostgreSQL 18 as latest supported version
- Note that older versions may work but aren't actively tested
- Provides clear version compatibility information for users

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
@NikolayS @claude
Update README to reference postgres_ai monitoring platform
ac89209 
- Replace postgres-checkup reference with postgres_ai
- Highlight comprehensive monitoring and optimization platform
- Emphasizes broader feature set beyond just health checks
- Points to current actively developed project

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
@NikolayS @claude
Add backticks formatting for pg_stat_checkpointer in README
1d47ee5 
- Format pg_stat_checkpointer as code with backticks
- Improves readability and follows markdown conventions
- Consistent with PostgreSQL object naming standards

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
@NikolayS NikolayS merged commit 1bdc9b7 into master Sep 29, 2025
6 of 7 checks passed
@NikolayS NikolayS deleted the cursor/add-postgres-version-tests-to-github-actions-42e4 branch September 30, 2025 00:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@NikolayS @cursoragent