Skip to content
Open
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
496 changes: 496 additions & 0 deletions .github/scripts/latest-core-testnet-sync/run.cjs

Large diffs are not rendered by default.

131 changes: 131 additions & 0 deletions .github/workflows/latest-core-testnet-sync-status.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,131 @@
name: "Latest public Core testnet sync status"

"on":
repository_dispatch:
types:
- latest-core-testnet-sync-completed
workflow_dispatch:
inputs:
status:
description: Final completed run status
required: true
type: choice
options:
- sync_passed
- build_failed
- sync_failed
target_sha:
description: Platform commit SHA to report against
required: false
type: string
target_url:
description: URL for run logs or status details
required: false
type: string
core_version:
description: Public Core version used by the run
required: false
type: string
platform_sha:
description: Platform commit SHA built and synced
required: false
type: string
completed_at:
description: Completion timestamp, preferably ISO 8601 UTC
required: false
type: string

permissions:
contents: read
statuses: write

jobs:
report:
name: Report latest completed sync status
runs-on: ubuntu-24.04
steps:
- name: Publish commit status
uses: actions/github-script@f28e40c7f34bde8b3046d885e986cb6290c5673b # v7.1.0
env:
CLIENT_PAYLOAD: ${{ toJson(github.event.client_payload) }}
with:
script: |
const dispatchPayload = JSON.parse(process.env.CLIENT_PAYLOAD || '{}');
const payload = context.eventName === 'workflow_dispatch'
? context.payload.inputs
: dispatchPayload;

if (!payload || typeof payload !== 'object' || Array.isArray(payload)) {
core.setFailed('Missing repository_dispatch client_payload');
return;
}

const status = payload.status;
const labels = {
sync_passed: 'Sync Passed',
build_failed: 'Build Failed',
sync_failed: 'Sync Failed',
};

if (!Object.prototype.hasOwnProperty.call(labels, status)) {
core.setFailed(`Unsupported status: ${status || '<empty>'}`);
return;
}

const targetSha = String(payload.target_sha || payload.platform_sha || '').trim();
if (!/^[0-9a-f]{40}$/i.test(targetSha)) {
core.setFailed('target_sha or platform_sha must be an explicit 40-character commit SHA');
return;
}

await github.rest.repos.getCommit({
owner: context.repo.owner,
repo: context.repo.repo,
ref: targetSha,
});

let targetUrl;
if (payload.target_url) {
let parsedUrl;
try {
parsedUrl = new URL(payload.target_url);
} catch (error) {
core.setFailed(`target_url must be a valid URL: ${error.message}`);
return;
}

if (parsedUrl.origin !== 'https://github.com') {
core.setFailed('target_url must use the https://github.com origin');
return;
}
targetUrl = parsedUrl.toString();
}

const state = status === 'sync_passed' ? 'success' : 'failure';
const label = labels[status];
const details = [
payload.core_version,
payload.platform_sha ? payload.platform_sha.slice(0, 12) : null,
payload.completed_at,
].filter(Boolean).join(' ');
const description = `${label}${details ? ` - ${details}` : ''}`.slice(0, 140);

await github.rest.repos.createCommitStatus({
owner: context.repo.owner,
repo: context.repo.repo,
sha: targetSha,
state,
context: 'Latest public Core testnet sync',
description,
target_url: targetUrl || undefined,
});

await core.summary
.addHeading('Latest public Core testnet sync')
.addRaw(`Status: ${label}\n\n`)
.addRaw(`Target SHA: ${targetSha}\n\n`)
.addRaw(`Core version: ${payload.core_version || 'not provided'}\n\n`)
.addRaw(`Platform SHA: ${payload.platform_sha || targetSha}\n\n`)
.addRaw(`Completed at: ${payload.completed_at || 'not provided'}\n\n`)
.addRaw(`Details: ${targetUrl || 'not provided'}\n`)
.write();
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -102,3 +102,6 @@ __pycache__/

# Security audit reports (local-only, not committed)
audits/

# Latest public Core testnet sync local artifacts
.latest-core-testnet-sync/
112 changes: 112 additions & 0 deletions docs/NIGHTLY_LATEST_CORE_TESTNET_SYNC.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
# Nightly Latest Core Testnet Sync

This document defines the first repo-side contract for reporting Platform sync against the latest public Dash Core release on testnet.

The visible GitHub status is intentionally the last completed run only. A currently running build or sync must not replace the useful completed result with a running state.

## Visible Status

The external orchestrator reports one commit status on the tested Platform commit:

- Context: `Latest public Core testnet sync`
- Success state:
- `Sync Passed`
- Failure states:
- `Build Failed`
- `Sync Failed`

Detailed build, Core, and sync logs belong behind the status `target_url`, not in the status description.

## System Phases

1. Keep a long-lived testnet Core node synced on the latest public Core release.
2. Build Platform for the target `dashpay/platform` commit on a fast disposable builder.
3. Run Platform sync on a normal sync runner using the synced Core baseline and freshly built Platform artifacts.
4. Report only the final completed result back to this repository.

Core baseline maintenance should be owned by a preconfigured, stateful, warm Core system. Platform build should be disposable and cache-heavy. Platform sync should be reproducible and log-rich.

## Scheduling

The long-running sync is owned by a persistent worker, not by a GitHub-hosted runner. Platform sync can take 16+ hours, which is too long for GitHub-hosted runner limits and too expensive to keep idle while waiting for chain progress.

The worker assets live in `ops/latest-core-testnet-sync/`:

- `install-worker.sh`
- `run-worker.sh`
- `latest-core-testnet-sync.service`
- `latest-core-testnet-sync.timer`
- `latest-core-testnet-sync.env.example`

The default timer runs nightly at 01:30 UTC with a 30 minute randomized delay. The service uses a lock file, so overlapping runs exit without changing the visible GitHub status.

The worker does not publish a pending or running commit status. It leaves the previous completed status in place while the new run is active, then publishes one final completed result when the run finishes.

The Platform sync worker assumes the latest-Core testnet node/baseline already exists. It may run a readiness command to verify that baseline before building and syncing Platform, but it should not perform a full Core chain sync itself.

The worker delegates host-specific work to environment variables:

- `LATEST_CORE_TESTNET_CORE_READY_COMMAND`
- `LATEST_CORE_TESTNET_PLATFORM_BUILD_COMMAND`
- `LATEST_CORE_TESTNET_PLATFORM_SYNC_COMMAND`

Optional variables:

- `LATEST_CORE_TESTNET_CORE_VERSION_COMMAND` overrides release discovery. It must print the Core version/tag on stdout.
- `LATEST_CORE_TESTNET_PHASE_TIMEOUT_MINUTES` overrides the per-phase timeout. Defaults to 1440 minutes.
- `LATEST_CORE_TESTNET_RESOLVE_TIMEOUT_MINUTES` overrides the Core-version resolution command timeout. Defaults to 30 minutes.
- `LATEST_CORE_RELEASE_REPO` overrides the GitHub release source. Defaults to `dashpay/dash`.
- `LATEST_CORE_TESTNET_LOG_DIR` controls where run logs and metadata are written.
- `LATEST_CORE_TESTNET_TARGET_URL` links the GitHub status to durable logs or a run page.

Each phase command receives:

- `LATEST_CORE_VERSION`
- `PLATFORM_SHA`
- `LATEST_CORE_TESTNET_SYNC_RUN_DIR`

The run directory should be used as the durable artifact location for phase logs and metadata.

The GitHub token is used only by the parent worker process for final status publication. It is intentionally stripped from the environment passed to Core readiness, Platform build, and Platform sync commands.

## Reporting Contract

When a run completes, the orchestrator sends a `repository_dispatch` event:

```json
{
"event_type": "latest-core-testnet-sync-completed",
"client_payload": {
"status": "sync_passed",
"target_sha": "0000000000000000000000000000000000000000",
"target_url": "https://github.com/dashpay/platform/actions/runs/0000000000",
"core_version": "vX.Y.Z",
"platform_sha": "0000000000000000000000000000000000000000",
"completed_at": "2026-06-24T05:30:00Z"
}
}
```

Allowed `status` values:

- `sync_passed`
- `build_failed`
- `sync_failed`

Manual status testing is available through the `Latest public Core testnet sync status` workflow's `workflow_dispatch` trigger with the same fields. The normal worker path reports directly through the GitHub commit statuses API.

The dispatch credential should be held only by the persistent sync worker/operator account. The status workflow requires an explicit 40-character commit SHA that exists in `dashpay/platform`, and workflow-provided `target_url` values are limited to the `https://github.com` origin.

## Log and Artifact Expectations

The `target_url` should lead to a run page or artifact bundle containing:

- selected public Core version and binary/image digest
- target Platform commit and image digests
- Core tip height/hash and sync health at run start
- build logs
- Platform service logs
- Platform sync progress and terminal state
- concise failure phase and reason

The GitHub status description stays short so the repo panel remains readable.
81 changes: 81 additions & 0 deletions ops/latest-core-testnet-sync/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
# Latest Public Core Testnet Sync Worker

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why it exists? who needs that? When I, as a user or developer, should use that tool?


The Platform sync can run for 16+ hours, so the long-running work belongs on a persistent worker instead of a GitHub-hosted runner.

The worker:

1. Updates and cleans a dedicated `dashpay/platform` checkout.
2. Resolves the latest public Dash Core release.
3. Verifies the preconfigured latest-Core testnet baseline, then runs Platform build and Platform sync commands.
4. Publishes one final commit status to the tested Platform commit:
- `Sync Passed`
- `Build Failed`
- `Sync Failed`

It does not publish a running status. The previous completed GitHub status remains visible while a new worker run is active.

## Install

On the sync worker, clone a dedicated checkout first:

```bash
sudo useradd --system --create-home --shell /usr/sbin/nologin platform-sync
sudo git clone https://github.com/dashpay/platform.git /opt/dash-platform
sudo chown -R platform-sync:platform-sync /opt/dash-platform
```

Then install the units:

```bash
sudo /opt/dash-platform/ops/latest-core-testnet-sync/install-worker.sh

sudo editor /etc/latest-core-testnet-sync.env
sudo systemctl enable --now latest-core-testnet-sync.timer
```

The checkout should be dedicated to this worker because the harness resets and cleans it to `origin/$PLATFORM_BRANCH` before every run.
The installer validates that `PLATFORM_REPO_DIR` already exists as a writable git checkout for the `platform-sync` user.

## Required Configuration

Set these in `/etc/latest-core-testnet-sync.env`:

- `GITHUB_TOKEN`
- `LATEST_CORE_TESTNET_CORE_READY_COMMAND`
- `LATEST_CORE_TESTNET_PLATFORM_BUILD_COMMAND`
- `LATEST_CORE_TESTNET_PLATFORM_SYNC_COMMAND`

`LATEST_CORE_TESTNET_CORE_READY_COMMAND` should verify that the preconfigured Core node or baseline is on the selected `LATEST_CORE_VERSION` and synced far enough for the Platform sync run. It is not intended to do a full Core chain sync on the Platform worker.

The worker commands run from `PLATFORM_REPO_DIR` and receive:

- `LATEST_CORE_VERSION`
- `PLATFORM_SHA`
- `LATEST_CORE_TESTNET_SYNC_RUN_DIR`

Write logs or machine-readable metadata into `LATEST_CORE_TESTNET_SYNC_RUN_DIR` so failures can be inspected without overloading the GitHub status panel.

`GITHUB_TOKEN` and `GH_TOKEN` are stripped from worker command environments. They remain available only to the worker harness for publishing the final commit status.

## Operations

Run immediately:

```bash
sudo systemctl start latest-core-testnet-sync.service
```

Check timer:

```bash
systemctl list-timers latest-core-testnet-sync.timer
```

Follow logs:

```bash
journalctl -u latest-core-testnet-sync.service -f
```

Run artifacts are stored under `LATEST_CORE_TESTNET_LOG_DIR`.
Run directories older than `LATEST_CORE_TESTNET_LOG_RETENTION_DAYS` are pruned by `run-worker.sh`; the default is 30 days.
65 changes: 65 additions & 0 deletions ops/latest-core-testnet-sync/install-worker.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
#!/usr/bin/env bash

set -euo pipefail

: "${PLATFORM_REPO_DIR:=/opt/dash-platform}"
PLATFORM_SYNC_USER=platform-sync
PLATFORM_SYNC_GROUP=platform-sync
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"

if [[ "${EUID}" -ne 0 ]]; then
echo "install-worker.sh must be run as root" >&2
exit 1
fi

if ! getent group "${PLATFORM_SYNC_GROUP}" >/dev/null 2>&1; then
groupadd --system "${PLATFORM_SYNC_GROUP}"
fi

if ! id "${PLATFORM_SYNC_USER}" >/dev/null 2>&1; then
useradd \
--system \
--create-home \
--shell /usr/sbin/nologin \
--gid "${PLATFORM_SYNC_GROUP}" \
"${PLATFORM_SYNC_USER}"
fi

if [[ ! -d "${PLATFORM_REPO_DIR}/.git" ]]; then
echo "PLATFORM_REPO_DIR must be an existing git checkout: ${PLATFORM_REPO_DIR}" >&2
echo "Clone dashpay/platform there before running this installer." >&2
exit 1
fi

if ! runuser -u "${PLATFORM_SYNC_USER}" -- test -w "${PLATFORM_REPO_DIR}"; then
echo "PLATFORM_REPO_DIR must be writable by ${PLATFORM_SYNC_USER}: ${PLATFORM_REPO_DIR}" >&2
echo "Run: chown -R ${PLATFORM_SYNC_USER}:${PLATFORM_SYNC_GROUP} ${PLATFORM_REPO_DIR}" >&2
exit 1
fi

install -d -o "${PLATFORM_SYNC_USER}" -g "${PLATFORM_SYNC_GROUP}" /var/lib/latest-core-testnet-sync
install -d -o "${PLATFORM_SYNC_USER}" -g "${PLATFORM_SYNC_GROUP}" /var/log/latest-core-testnet-sync

install -m 0644 \
"${SCRIPT_DIR}/latest-core-testnet-sync.service" \
/etc/systemd/system/latest-core-testnet-sync.service
install -m 0644 \
"${SCRIPT_DIR}/latest-core-testnet-sync.timer" \
/etc/systemd/system/latest-core-testnet-sync.timer

if [[ ! -f /etc/latest-core-testnet-sync.env ]]; then
install -m 0600 \
"${SCRIPT_DIR}/latest-core-testnet-sync.env.example" \
/etc/latest-core-testnet-sync.env
echo "Created /etc/latest-core-testnet-sync.env; fill it before enabling the timer."
else
chmod 0600 /etc/latest-core-testnet-sync.env
fi

systemctl daemon-reload

echo "Installed latest Core testnet sync worker units."
echo "Next:"
echo " 1. Edit /etc/latest-core-testnet-sync.env"
echo " 2. Run: systemctl start latest-core-testnet-sync.service"
echo " 3. Enable nightly timer: systemctl enable --now latest-core-testnet-sync.timer"
Loading
Loading