Skip to content

feat: replace github actions cache with s3#623

Closed
sublimator wants to merge 1 commit intodevfrom
actions-cache-flattened
Closed

feat: replace github actions cache with s3#623
sublimator wants to merge 1 commit intodevfrom
actions-cache-flattened

Conversation

@sublimator
Copy link
Copy Markdown
Collaborator

Replaces actions/cache with custom S3-based caching to avoid upcoming
GitHub Actions cache eviction policy changes.

Changes:

  • New S3 cache actions (xahau-ga-cache-restore/save)
  • zstd compression (ccache: clang=323/gcc=609 MB, Conan: clang=1.1/gcc=1.9 GB)
  • Immutability (first-write-wins)
  • Bootstrap mode (creates empty dir if no cache)

Cache clearing tag:

  • [ci-ga-clear-cache] - Clear all caches for this job
  • [ci-ga-clear-cache:ccache] - Clear only ccache
  • [ci-ga-clear-cache:conan] - Clear only conan

Configuration ordering fixes:

  • ccache config applied AFTER cache restore (prevents stale cached config)
  • Conan profile created AFTER cache restore (prevents stale cached profile)

ccache improvements:

Conan improvements:

Breaking changes:

  • Workflows must pass AWS credentials (aws-access-key-id, aws-secret-access-key)

S3 setup:

  • Bucket: xahaud-github-actions-cache-niq (us-east-1)
  • Credentials already configured in GitHub secrets

@sublimator sublimator marked this pull request as draft November 1, 2025 02:57
Replaces actions/cache with custom S3-based caching to avoid upcoming
GitHub Actions cache eviction policy changes.

Changes:
- New S3 cache actions (xahau-ga-cache-restore/save)
- zstd compression (ccache: clang=323/gcc=609 MB, Conan: clang=1.1/gcc=1.9 GB)
- Immutability (first-write-wins)
- Bootstrap mode (creates empty dir if no cache)

Cache clearing tag:
- [ci-ga-clear-cache] - Clear all caches for this job
- [ci-ga-clear-cache:ccache] - Clear only ccache
- [ci-ga-clear-cache:conan] - Clear only conan

Configuration ordering fixes:
- ccache config applied AFTER cache restore (prevents stale cached config)
- Conan profile created AFTER cache restore (prevents stale cached profile)

ccache improvements:
- Single cache directory (~/.ccache)
- Wrapper toolchain (enables ccache without affecting Conan builds)
- Verbose build output (-v flag)
- Fixes #620

Conan improvements:
- Removed branch comparison logic for cache saves
- Cache keys don't include branch names, comparison was ineffective
- Fixes #618

Breaking changes:
- Workflows must pass AWS credentials (aws-access-key-id, aws-secret-access-key)

S3 setup:
- Bucket: xahaud-github-actions-cache-niq (us-east-1)
- Credentials already configured in GitHub secrets
@sublimator sublimator force-pushed the actions-cache-flattened branch from 5f8ebd0 to 8b39d09 Compare November 3, 2025 07:04
@sublimator sublimator marked this pull request as ready for review November 3, 2025 07:09
sublimator added a commit that referenced this pull request Nov 24, 2025
Reverts to GitHub Actions cache (from custom S3 implementation) while
preserving all bug fixes and improvements from PR #623.

Key changes:
- Replace custom S3 cache actions with actions/cache@v4
- Remove AWS credentials from workflows
- Restore dual-cache system (main + feature branch with secondary_storage)
- Add cache clearing via GitHub API (replaces S3 delete operations)
- Create reusable xahau-ga-clear-cache action (DRY)
- Fix conanfile.txt → conanfile.py in cache keys

Bug fixes preserved:
- Wrapper toolchain: prevents Conan from overriding ccache settings
- Configuration ordering: ccache/Conan config applied AFTER cache restore
- Single ccache directory with secondary_storage for feature branches
- Verbose build flag support: [ci-ga-cmake-verbose]
- Branch comparison logic removed for Conan cache saves

Cache clearing features:
- [ci-ga-clear-cache] - Clear all caches for this job
- [ci-ga-clear-cache:ccache] - Clear only if key contains "ccache"
- [ci-ga-clear-cache:conan] - Clear only if key contains "conan"
- [ci-ga-clear-cache:gcc Debug] - Clear if key contains both terms (AND logic)
- Uses gh CLI with github.token (no additional credentials needed)

Rationale:
GitHub increased cache limits, eliminating need for S3. This approach
maintains all performance improvements while simplifying infrastructure.

Fixes #620
Fixes #618
@sublimator
Copy link
Copy Markdown
Collaborator Author

Closed in favour of #633

@sublimator sublimator closed this Nov 25, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Nix CI ccache caches are empty Nix CI not caching conan on feature branches where conanfile hasn't changed yet main is empty

1 participant