Repository navigation
ci: add Platform testnet sync status reporting #3958
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
infraclaw-dash
wants to merge
7
commits into
dashpay:v5.1-dev
Choose a base branch
from
infraclaw-dash:infraclaw/latest-core-testnet-sync-worker
base: v5.1-dev
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 5 commits
Commits
Show all changes
7 commits
Select commit
Hold shift + click to select a range
1749712
ci: add latest core testnet sync worker
infraclaw-dash 4069ab1
ci: harden latest Core sync worker
infraclaw-dash 64a8fca
ci: record latest Core status publish outcome
infraclaw-dash f6783a6
ci: harden latest Core branch handling
infraclaw-dash 06c6405
ci: clarify latest Core baseline readiness
infraclaw-dash 07c7e22
ci: make Platform testnet sync naming explicit
infraclaw-dash 3546112
ci: keep Platform sync server code out of repo
infraclaw-dash File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,131 @@ | ||
| name: "Latest public Core testnet sync status" | ||
|
|
||
| "on": | ||
| repository_dispatch: | ||
| types: | ||
| - latest-core-testnet-sync-completed | ||
| workflow_dispatch: | ||
| inputs: | ||
| status: | ||
| description: Final completed run status | ||
| required: true | ||
| type: choice | ||
| options: | ||
| - sync_passed | ||
| - build_failed | ||
| - sync_failed | ||
| target_sha: | ||
| description: Platform commit SHA to report against | ||
| required: false | ||
| type: string | ||
| target_url: | ||
| description: URL for run logs or status details | ||
| required: false | ||
| type: string | ||
| core_version: | ||
| description: Public Core version used by the run | ||
| required: false | ||
| type: string | ||
| platform_sha: | ||
| description: Platform commit SHA built and synced | ||
| required: false | ||
| type: string | ||
| completed_at: | ||
| description: Completion timestamp, preferably ISO 8601 UTC | ||
| required: false | ||
| type: string | ||
|
|
||
| permissions: | ||
| contents: read | ||
| statuses: write | ||
|
|
||
| jobs: | ||
| report: | ||
| name: Report latest completed sync status | ||
| runs-on: ubuntu-24.04 | ||
| steps: | ||
| - name: Publish commit status | ||
| uses: actions/github-script@f28e40c7f34bde8b3046d885e986cb6290c5673b # v7.1.0 | ||
| env: | ||
| CLIENT_PAYLOAD: ${{ toJson(github.event.client_payload) }} | ||
| with: | ||
| script: | | ||
| const dispatchPayload = JSON.parse(process.env.CLIENT_PAYLOAD || '{}'); | ||
| const payload = context.eventName === 'workflow_dispatch' | ||
| ? context.payload.inputs | ||
| : dispatchPayload; | ||
|
|
||
| if (!payload || typeof payload !== 'object' || Array.isArray(payload)) { | ||
| core.setFailed('Missing repository_dispatch client_payload'); | ||
| return; | ||
| } | ||
|
|
||
| const status = payload.status; | ||
| const labels = { | ||
| sync_passed: 'Sync Passed', | ||
| build_failed: 'Build Failed', | ||
| sync_failed: 'Sync Failed', | ||
| }; | ||
|
|
||
| if (!Object.prototype.hasOwnProperty.call(labels, status)) { | ||
| core.setFailed(`Unsupported status: ${status || '<empty>'}`); | ||
| return; | ||
| } | ||
|
|
||
| const targetSha = String(payload.target_sha || payload.platform_sha || '').trim(); | ||
| if (!/^[0-9a-f]{40}$/i.test(targetSha)) { | ||
| core.setFailed('target_sha or platform_sha must be an explicit 40-character commit SHA'); | ||
| return; | ||
| } | ||
|
|
||
| await github.rest.repos.getCommit({ | ||
| owner: context.repo.owner, | ||
| repo: context.repo.repo, | ||
| ref: targetSha, | ||
| }); | ||
|
|
||
| let targetUrl; | ||
| if (payload.target_url) { | ||
| let parsedUrl; | ||
| try { | ||
| parsedUrl = new URL(payload.target_url); | ||
| } catch (error) { | ||
| core.setFailed(`target_url must be a valid URL: ${error.message}`); | ||
| return; | ||
| } | ||
|
|
||
| if (parsedUrl.origin !== 'https://github.com') { | ||
| core.setFailed('target_url must use the https://github.com origin'); | ||
| return; | ||
| } | ||
| targetUrl = parsedUrl.toString(); | ||
| } | ||
|
|
||
| const state = status === 'sync_passed' ? 'success' : 'failure'; | ||
| const label = labels[status]; | ||
| const details = [ | ||
| payload.core_version, | ||
| payload.platform_sha ? payload.platform_sha.slice(0, 12) : null, | ||
| payload.completed_at, | ||
| ].filter(Boolean).join(' '); | ||
| const description = `${label}${details ? ` - ${details}` : ''}`.slice(0, 140); | ||
|
|
||
| await github.rest.repos.createCommitStatus({ | ||
| owner: context.repo.owner, | ||
| repo: context.repo.repo, | ||
| sha: targetSha, | ||
| state, | ||
| context: 'Latest public Core testnet sync', | ||
| description, | ||
| target_url: targetUrl || undefined, | ||
| }); | ||
|
|
||
| await core.summary | ||
| .addHeading('Latest public Core testnet sync') | ||
| .addRaw(`Status: ${label}\n\n`) | ||
| .addRaw(`Target SHA: ${targetSha}\n\n`) | ||
| .addRaw(`Core version: ${payload.core_version || 'not provided'}\n\n`) | ||
| .addRaw(`Platform SHA: ${payload.platform_sha || targetSha}\n\n`) | ||
| .addRaw(`Completed at: ${payload.completed_at || 'not provided'}\n\n`) | ||
| .addRaw(`Details: ${targetUrl || 'not provided'}\n`) | ||
| .write(); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,112 @@ | ||
| # Nightly Latest Core Testnet Sync | ||
|
|
||
| This document defines the first repo-side contract for reporting Platform sync against the latest public Dash Core release on testnet. | ||
|
|
||
| The visible GitHub status is intentionally the last completed run only. A currently running build or sync must not replace the useful completed result with a running state. | ||
|
|
||
| ## Visible Status | ||
|
|
||
| The external orchestrator reports one commit status on the tested Platform commit: | ||
|
|
||
| - Context: `Latest public Core testnet sync` | ||
| - Success state: | ||
| - `Sync Passed` | ||
| - Failure states: | ||
| - `Build Failed` | ||
| - `Sync Failed` | ||
|
|
||
| Detailed build, Core, and sync logs belong behind the status `target_url`, not in the status description. | ||
|
|
||
| ## System Phases | ||
|
|
||
| 1. Keep a long-lived testnet Core node synced on the latest public Core release. | ||
| 2. Build Platform for the target `dashpay/platform` commit on a fast disposable builder. | ||
| 3. Run Platform sync on a normal sync runner using the synced Core baseline and freshly built Platform artifacts. | ||
| 4. Report only the final completed result back to this repository. | ||
|
|
||
| Core baseline maintenance should be owned by a preconfigured, stateful, warm Core system. Platform build should be disposable and cache-heavy. Platform sync should be reproducible and log-rich. | ||
|
|
||
| ## Scheduling | ||
|
|
||
| The long-running sync is owned by a persistent worker, not by a GitHub-hosted runner. Platform sync can take 16+ hours, which is too long for GitHub-hosted runner limits and too expensive to keep idle while waiting for chain progress. | ||
|
|
||
| The worker assets live in `ops/latest-core-testnet-sync/`: | ||
|
|
||
| - `install-worker.sh` | ||
| - `run-worker.sh` | ||
| - `latest-core-testnet-sync.service` | ||
| - `latest-core-testnet-sync.timer` | ||
| - `latest-core-testnet-sync.env.example` | ||
|
|
||
| The default timer runs nightly at 01:30 UTC with a 30 minute randomized delay. The service uses a lock file, so overlapping runs exit without changing the visible GitHub status. | ||
|
|
||
| The worker does not publish a pending or running commit status. It leaves the previous completed status in place while the new run is active, then publishes one final completed result when the run finishes. | ||
|
|
||
| The Platform sync worker assumes the latest-Core testnet node/baseline already exists. It may run a readiness command to verify that baseline before building and syncing Platform, but it should not perform a full Core chain sync itself. | ||
|
|
||
| The worker delegates host-specific work to environment variables: | ||
|
|
||
| - `LATEST_CORE_TESTNET_CORE_READY_COMMAND` | ||
| - `LATEST_CORE_TESTNET_PLATFORM_BUILD_COMMAND` | ||
| - `LATEST_CORE_TESTNET_PLATFORM_SYNC_COMMAND` | ||
|
|
||
| Optional variables: | ||
|
|
||
| - `LATEST_CORE_TESTNET_CORE_VERSION_COMMAND` overrides release discovery. It must print the Core version/tag on stdout. | ||
| - `LATEST_CORE_TESTNET_PHASE_TIMEOUT_MINUTES` overrides the per-phase timeout. Defaults to 1440 minutes. | ||
| - `LATEST_CORE_TESTNET_RESOLVE_TIMEOUT_MINUTES` overrides the Core-version resolution command timeout. Defaults to 30 minutes. | ||
| - `LATEST_CORE_RELEASE_REPO` overrides the GitHub release source. Defaults to `dashpay/dash`. | ||
| - `LATEST_CORE_TESTNET_LOG_DIR` controls where run logs and metadata are written. | ||
| - `LATEST_CORE_TESTNET_TARGET_URL` links the GitHub status to durable logs or a run page. | ||
|
|
||
| Each phase command receives: | ||
|
|
||
| - `LATEST_CORE_VERSION` | ||
| - `PLATFORM_SHA` | ||
| - `LATEST_CORE_TESTNET_SYNC_RUN_DIR` | ||
|
|
||
| The run directory should be used as the durable artifact location for phase logs and metadata. | ||
|
|
||
| The GitHub token is used only by the parent worker process for final status publication. It is intentionally stripped from the environment passed to Core readiness, Platform build, and Platform sync commands. | ||
|
|
||
| ## Reporting Contract | ||
|
|
||
| When a run completes, the orchestrator sends a `repository_dispatch` event: | ||
|
|
||
| ```json | ||
| { | ||
| "event_type": "latest-core-testnet-sync-completed", | ||
| "client_payload": { | ||
| "status": "sync_passed", | ||
| "target_sha": "0000000000000000000000000000000000000000", | ||
| "target_url": "https://github.com/dashpay/platform/actions/runs/0000000000", | ||
| "core_version": "vX.Y.Z", | ||
| "platform_sha": "0000000000000000000000000000000000000000", | ||
| "completed_at": "2026-06-24T05:30:00Z" | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| Allowed `status` values: | ||
|
|
||
| - `sync_passed` | ||
| - `build_failed` | ||
| - `sync_failed` | ||
|
|
||
| Manual status testing is available through the `Latest public Core testnet sync status` workflow's `workflow_dispatch` trigger with the same fields. The normal worker path reports directly through the GitHub commit statuses API. | ||
|
|
||
| The dispatch credential should be held only by the persistent sync worker/operator account. The status workflow requires an explicit 40-character commit SHA that exists in `dashpay/platform`, and workflow-provided `target_url` values are limited to the `https://github.com` origin. | ||
|
|
||
| ## Log and Artifact Expectations | ||
|
|
||
| The `target_url` should lead to a run page or artifact bundle containing: | ||
|
|
||
| - selected public Core version and binary/image digest | ||
| - target Platform commit and image digests | ||
| - Core tip height/hash and sync health at run start | ||
| - build logs | ||
| - Platform service logs | ||
| - Platform sync progress and terminal state | ||
| - concise failure phase and reason | ||
|
|
||
| The GitHub status description stays short so the repo panel remains readable. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,81 @@ | ||
| # Latest Public Core Testnet Sync Worker | ||
|
|
||
| The Platform sync can run for 16+ hours, so the long-running work belongs on a persistent worker instead of a GitHub-hosted runner. | ||
|
|
||
| The worker: | ||
|
|
||
| 1. Updates and cleans a dedicated `dashpay/platform` checkout. | ||
| 2. Resolves the latest public Dash Core release. | ||
| 3. Verifies the preconfigured latest-Core testnet baseline, then runs Platform build and Platform sync commands. | ||
| 4. Publishes one final commit status to the tested Platform commit: | ||
| - `Sync Passed` | ||
| - `Build Failed` | ||
| - `Sync Failed` | ||
|
|
||
| It does not publish a running status. The previous completed GitHub status remains visible while a new worker run is active. | ||
|
|
||
| ## Install | ||
|
|
||
| On the sync worker, clone a dedicated checkout first: | ||
|
|
||
| ```bash | ||
| sudo useradd --system --create-home --shell /usr/sbin/nologin platform-sync | ||
| sudo git clone https://github.com/dashpay/platform.git /opt/dash-platform | ||
| sudo chown -R platform-sync:platform-sync /opt/dash-platform | ||
| ``` | ||
|
|
||
| Then install the units: | ||
|
|
||
| ```bash | ||
| sudo /opt/dash-platform/ops/latest-core-testnet-sync/install-worker.sh | ||
|
|
||
| sudo editor /etc/latest-core-testnet-sync.env | ||
| sudo systemctl enable --now latest-core-testnet-sync.timer | ||
| ``` | ||
|
|
||
| The checkout should be dedicated to this worker because the harness resets and cleans it to `origin/$PLATFORM_BRANCH` before every run. | ||
| The installer validates that `PLATFORM_REPO_DIR` already exists as a writable git checkout for the `platform-sync` user. | ||
|
|
||
| ## Required Configuration | ||
|
|
||
| Set these in `/etc/latest-core-testnet-sync.env`: | ||
|
|
||
| - `GITHUB_TOKEN` | ||
| - `LATEST_CORE_TESTNET_CORE_READY_COMMAND` | ||
| - `LATEST_CORE_TESTNET_PLATFORM_BUILD_COMMAND` | ||
| - `LATEST_CORE_TESTNET_PLATFORM_SYNC_COMMAND` | ||
|
|
||
| `LATEST_CORE_TESTNET_CORE_READY_COMMAND` should verify that the preconfigured Core node or baseline is on the selected `LATEST_CORE_VERSION` and synced far enough for the Platform sync run. It is not intended to do a full Core chain sync on the Platform worker. | ||
|
|
||
| The worker commands run from `PLATFORM_REPO_DIR` and receive: | ||
|
|
||
| - `LATEST_CORE_VERSION` | ||
| - `PLATFORM_SHA` | ||
| - `LATEST_CORE_TESTNET_SYNC_RUN_DIR` | ||
|
|
||
| Write logs or machine-readable metadata into `LATEST_CORE_TESTNET_SYNC_RUN_DIR` so failures can be inspected without overloading the GitHub status panel. | ||
|
|
||
| `GITHUB_TOKEN` and `GH_TOKEN` are stripped from worker command environments. They remain available only to the worker harness for publishing the final commit status. | ||
|
|
||
| ## Operations | ||
|
|
||
| Run immediately: | ||
|
|
||
| ```bash | ||
| sudo systemctl start latest-core-testnet-sync.service | ||
| ``` | ||
|
|
||
| Check timer: | ||
|
|
||
| ```bash | ||
| systemctl list-timers latest-core-testnet-sync.timer | ||
| ``` | ||
|
|
||
| Follow logs: | ||
|
|
||
| ```bash | ||
| journalctl -u latest-core-testnet-sync.service -f | ||
| ``` | ||
|
|
||
| Run artifacts are stored under `LATEST_CORE_TESTNET_LOG_DIR`. | ||
| Run directories older than `LATEST_CORE_TESTNET_LOG_RETENTION_DAYS` are pruned by `run-worker.sh`; the default is 30 days. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,65 @@ | ||
| #!/usr/bin/env bash | ||
|
|
||
| set -euo pipefail | ||
|
|
||
| : "${PLATFORM_REPO_DIR:=/opt/dash-platform}" | ||
| PLATFORM_SYNC_USER=platform-sync | ||
| PLATFORM_SYNC_GROUP=platform-sync | ||
| SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)" | ||
|
|
||
| if [[ "${EUID}" -ne 0 ]]; then | ||
| echo "install-worker.sh must be run as root" >&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| if ! getent group "${PLATFORM_SYNC_GROUP}" >/dev/null 2>&1; then | ||
| groupadd --system "${PLATFORM_SYNC_GROUP}" | ||
| fi | ||
|
|
||
| if ! id "${PLATFORM_SYNC_USER}" >/dev/null 2>&1; then | ||
| useradd \ | ||
| --system \ | ||
| --create-home \ | ||
| --shell /usr/sbin/nologin \ | ||
| --gid "${PLATFORM_SYNC_GROUP}" \ | ||
| "${PLATFORM_SYNC_USER}" | ||
| fi | ||
|
|
||
| if [[ ! -d "${PLATFORM_REPO_DIR}/.git" ]]; then | ||
| echo "PLATFORM_REPO_DIR must be an existing git checkout: ${PLATFORM_REPO_DIR}" >&2 | ||
| echo "Clone dashpay/platform there before running this installer." >&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| if ! runuser -u "${PLATFORM_SYNC_USER}" -- test -w "${PLATFORM_REPO_DIR}"; then | ||
| echo "PLATFORM_REPO_DIR must be writable by ${PLATFORM_SYNC_USER}: ${PLATFORM_REPO_DIR}" >&2 | ||
| echo "Run: chown -R ${PLATFORM_SYNC_USER}:${PLATFORM_SYNC_GROUP} ${PLATFORM_REPO_DIR}" >&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| install -d -o "${PLATFORM_SYNC_USER}" -g "${PLATFORM_SYNC_GROUP}" /var/lib/latest-core-testnet-sync | ||
| install -d -o "${PLATFORM_SYNC_USER}" -g "${PLATFORM_SYNC_GROUP}" /var/log/latest-core-testnet-sync | ||
|
|
||
| install -m 0644 \ | ||
| "${SCRIPT_DIR}/latest-core-testnet-sync.service" \ | ||
| /etc/systemd/system/latest-core-testnet-sync.service | ||
| install -m 0644 \ | ||
| "${SCRIPT_DIR}/latest-core-testnet-sync.timer" \ | ||
| /etc/systemd/system/latest-core-testnet-sync.timer | ||
|
|
||
| if [[ ! -f /etc/latest-core-testnet-sync.env ]]; then | ||
| install -m 0600 \ | ||
| "${SCRIPT_DIR}/latest-core-testnet-sync.env.example" \ | ||
| /etc/latest-core-testnet-sync.env | ||
| echo "Created /etc/latest-core-testnet-sync.env; fill it before enabling the timer." | ||
| else | ||
| chmod 0600 /etc/latest-core-testnet-sync.env | ||
| fi | ||
|
|
||
| systemctl daemon-reload | ||
|
|
||
| echo "Installed latest Core testnet sync worker units." | ||
| echo "Next:" | ||
| echo " 1. Edit /etc/latest-core-testnet-sync.env" | ||
| echo " 2. Run: systemctl start latest-core-testnet-sync.service" | ||
| echo " 3. Enable nightly timer: systemctl enable --now latest-core-testnet-sync.timer" |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why it exists? who needs that? When I, as a user or developer, should use that tool?