Skip to content

Commit 7ba1739

Browse files
samuelburnhamsamuelburnham
authored
ci: Add Solidity compatibility check (#363)
* ci: Add Solidity compatibility check * Fix CI * Test breaking change comment * Test breaking change issue * Prep for review * Address feedback & clean up * Address feedback * Test breaking change issue creation * Prep for final review
1 parent 6133454 commit 7ba1739

File tree

3 files changed

+179
-0
lines changed

3 files changed

+179
-0
lines changed

‎.github/SOLIDITY_COMPAT_ISSUE.md

+5
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
Compatibility with the [Arecibo](https://github.com/lurk-lab/arecibo) dependency has been broken by commit [`__COMMIT__`](__COMMIT_URL__) from __PR_URL__.
2+
3+
Check the [Solidity compatibility workflow run](__WORKFLOW_URL__) for details.
4+
5+
This issue was raised by the workflow at __WORKFLOW_FILE__.

‎.github/workflows/solidity.yml

+172
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,172 @@
1+
# TODO: Reusify and combine with the `check-lurk-compiles` action and/or make a reusable open-issue action
2+
#
3+
# This workflow runs autogenerated `solidity-verifier` compatibility tests on Arecibo PRs and notifies if compatibility is broken
4+
# The workflow is intended to be a required status check only to make sure the Rust test & basic job steps work
5+
# It is NOT intended to block breaking changes from merging, only to noisily surface them for review
6+
#
7+
# If the Rust template fails to generate the Solidity test or the job errors for another reason, the workflow fails immediately
8+
# If the Solidity test fails on `pull_request`, it writes a PR comment to ensure the author/reviewer are notified
9+
# If the Solidity test fails on `merge_group`, it opens an issue in `solidity-verifier` downstream that compatibility has been broken
10+
# `merge_group` failures should only happen intentionally when breaking changes need to be merged in Arecibo
11+
#
12+
# Implementation note:
13+
# `falnyr/replace-env-vars-action`, `micalevisk/last-issue-action` and `peter-evans/create-issue-from-file` replace
14+
# equivalent functionality in `JasonEtco/create-an-issue`. We can't use the latter because it doesn't allow creating
15+
# the issue in another repo. See https://github.com/JasonEtco/create-an-issue/issues/40
16+
name: Test `solidity-verifier` compatibility
17+
18+
on:
19+
merge_group:
20+
pull_request:
21+
types: [opened, synchronize, reopened, ready_for_review]
22+
branches:
23+
- dev
24+
- 'feat/**'
25+
- release-candidate
26+
27+
concurrency:
28+
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
29+
cancel-in-progress: true
30+
31+
jobs:
32+
solidity-compat:
33+
runs-on: buildjet-16vcpu-ubuntu-2204
34+
steps:
35+
- uses: actions/checkout@v4
36+
with:
37+
repository: lurk-lab/ci-workflows
38+
- uses: ./.github/actions/ci-env
39+
- uses: actions/checkout@v4
40+
- uses: dtolnay/rust-toolchain@stable
41+
- uses: taiki-e/install-action@nextest
42+
- uses: Swatinem/rust-cache@v2
43+
- name: Run Solidity test generator
44+
run: |
45+
# Runs all tests with the `test_solidity_compatibility` prefix, e.g. `test_solidity_compatibility_ipa`
46+
cargo nextest run -E 'test(test_solidity_compatibility)' --release --run-ignored all --nocapture > test-output
47+
working-directory: ${{ github.workspace }}
48+
- name: Check out `solidity-verifier` for tests
49+
uses: actions/checkout@v4
50+
with:
51+
repository: lurk-lab/solidity-verifier
52+
path: solidity-verifier
53+
submodules: recursive
54+
- name: Install Foundry
55+
uses: foundry-rs/foundry-toolchain@v1
56+
with:
57+
version: nightly
58+
- name: Prep Solidity test files
59+
run: |
60+
# Get test names from output, extracting the final word after the final `_` in the test output
61+
# E.g. `test provider::tests::ipa_pc::test::test_solidity_compatibility_ipa ... ok` returns `ipa`
62+
# Expects all tests to live in the `provider` module, but can be changed/strengthened in future
63+
TEST_NAMES=$(grep 'test provider::*' test-output | awk -F'[_(.*?)\b...]' '{ print $(NF-3) }')
64+
echo "$TEST_NAMES"
65+
66+
# Print output of each test to `<test_name>.t.sol` file
67+
awk -v names="$TEST_NAMES" 'BEGIN {
68+
file_counter = 0
69+
buffer = ""
70+
# Convert test names to array
71+
split(names, elements, " ")
72+
for (i in elements) {
73+
print "Element:", elements[i]
74+
}
75+
}
76+
77+
/running 1 test/ {
78+
between = 1
79+
buffer = ""
80+
}
81+
82+
between {
83+
buffer = buffer $0 ORS
84+
}
85+
86+
/^test provider.*$/ {
87+
between = 0
88+
89+
if (buffer != "") {
90+
++file_counter
91+
print buffer > elements[file_counter]".t.sol"
92+
buffer = ""
93+
}
94+
}' test-output
95+
96+
# Clean up
97+
shopt -s nullglob
98+
for file in *.t.sol; do
99+
cat $file | sed '1d' | head -n -2 > tmp.file && mv tmp.file solidity-verifier/test/$file
100+
done
101+
shopt -u nullglob
102+
working-directory: ${{ github.workspace }}
103+
- name: Run Forge tests
104+
id: solidity-test
105+
run: forge test -vvv
106+
working-directory: ${{ github.workspace }}/solidity-verifier
107+
continue-on-error: true
108+
# Prepares env vars for use in a PR comment or issue in `solidity-verifier`
109+
- name: Set env vars
110+
if: steps.solidity-test.outcome != 'success'
111+
run: |
112+
if [[ "${{ github.event_name }}" == "pull_request" ]]; then
113+
COMMIT=$(echo ${{ github.event.pull_request.head.sha }} | cut -c -7)
114+
PR_NUMBER=${{ github.event.pull_request.number }}
115+
else
116+
COMMIT=$(echo ${{ github.event.merge_group.head_sha }} | cut -c -7)
117+
PR_NUMBER=$(echo ${{ github.event.merge_group.head_ref }} | sed -e 's/.*pr-\(.*\)-.*/\1/')
118+
fi
119+
GITHUB_URL=https://github.com/${{ github.repository }}
120+
WORKFLOW_URL=$GITHUB_URL/actions/runs/${{ github.run_id }}
121+
echo "WORKFLOW_FILE=$WORKFLOW_URL/workflow" | tee -a $GITHUB_ENV
122+
echo "WORKFLOW_URL=$WORKFLOW_URL" | tee -a $GITHUB_ENV
123+
echo "COMMIT_URL=$GITHUB_URL/commit/$COMMIT" | tee -a $GITHUB_ENV
124+
echo "PR_URL=$GITHUB_URL/pull/$PR_NUMBER" | tee -a $GITHUB_ENV
125+
echo "COMMIT=$COMMIT" | tee -a $GITHUB_ENV
126+
# Comment on PR when test fails on `pull_request`
127+
- name: Comment on failing run
128+
if: steps.solidity-test.outcome != 'success' && github.event_name == 'pull_request'
129+
uses: peter-evans/create-or-update-comment@v4
130+
with:
131+
issue-number: ${{ github.event.pull_request.number }}
132+
body: |
133+
`solidity-verifier` compatibility test failed :x:
134+
135+
${{ env.WORKFLOW_URL }}
136+
# Substitutes env vars for their values in `SOLIDITY_COMPAT_ISSUE.md`
137+
- uses: falnyr/replace-env-vars-action@master
138+
if: steps.solidity-test.outcome != 'success' && (github.event_name != 'pull_request' || github.event.action == 'enqueued')
139+
env:
140+
WORKFLOW_URL: ${{ env.WORKFLOW_URL }}
141+
WORKFLOW_FILE: ${{ env.WORKFLOW_FILE }}
142+
COMMIT: ${{ env.COMMIT }}
143+
COMMIT_URL: ${{ env.COMMIT_URL }}
144+
PR_URL: ${{ env.PR_URL }}
145+
with:
146+
filename: .github/SOLIDITY_COMPAT_ISSUE.md
147+
# Finds the last open issue matching given labels
148+
- name: Find the last open compatibility issue
149+
id: last-issue
150+
if: steps.solidity-test.outcome != 'success' && (github.event_name != 'pull_request' || github.event.action == 'enqueued')
151+
uses: micalevisk/last-issue-action@v2
152+
with:
153+
repository: lurk-lab/solidity-verifier
154+
state: open
155+
# Find the last updated open issue that has these labels:
156+
labels: |
157+
compatibility
158+
debt
159+
automated issue
160+
# Update existing issue in `solidity-verifier` or create new one
161+
- uses: peter-evans/create-issue-from-file@v5
162+
if: steps.solidity-test.outcome != 'success' && (github.event_name != 'pull_request' || github.event.action == 'enqueued')
163+
with:
164+
token: ${{ secrets.REPO_TOKEN }}
165+
repository: lurk-lab/solidity-verifier
166+
issue-number: ${{ steps.last-issue.outputs.issue-number }}
167+
title: ":rotating_light: Arecibo compatibility is broken"
168+
content-filepath: .github/SOLIDITY_COMPAT_ISSUE.md
169+
labels: |
170+
compatibility
171+
debt
172+
automated issue

‎.lycheeignore

+2
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
__COMMIT_URL__
2+
__WORKFLOW_URL__

0 commit comments

Comments
 (0)