diff --git a/op-acceptance-tests/README.md b/op-acceptance-tests/README.md new file mode 100644 index 0000000000000..f11b4beaf1c7d --- /dev/null +++ b/op-acceptance-tests/README.md @@ -0,0 +1,56 @@ +# OP Stack Acceptance Tests + +## Overview + +This directory contains the acceptance tests and configuration for the OP Stack. These tests are executed by `op-acceptor`, which serves as an automated gatekeeper for OP Stack network promotions. + +Think of acceptance testing as Gandalf 🧙, standing at the gates and shouting, "You shall not pass!" to networks that don't meet our standards. It enforces the "Don't trust, verify" principle by: + +- Running automated acceptance tests +- Providing clear pass/fail results (and tracking these over time) +- Gating network promotions based on test results +- Providing insight into test feature/functional coverage + +The `op-acceptor` ensures network quality and readiness by running a comprehensive suite of acceptance tests before features can advance through the promotion pipeline: + +Localnet -> Alphanet → Betanet → Testnet + +This process helps maintain high-quality standards across all networks in the OP Stack ecosystem. + +## Usage + +The tests can be run using the `just` command runner: + +```bash +# Run the default acceptance tests against a simple devnet +just + +# Run the acceptance tests against a specific devnet and gate +just acceptance-test + +# Run the acceptance tests using a specific version of op-acceptor +ACCEPTOR_IMAGE=op-acceptor:latest just acceptance-test +``` + +### Configuration + +- `acceptance-tests.yaml`: Defines the validation gates and the suites and tests that should be run for each gate. +- `justfile`: Contains the commands for running the acceptance tests. + +## Adding New Tests + +To add new acceptance tests: + +1. Create your test in the appropriate Go package (as a regular Go test) +2. Register the test in `acceptance-tests.yaml` under the appropriate gate +3. Follow the existing pattern for test registration: + ```yaml + - name: YourTestName + package: github.com/ethereum-optimism/optimism/your/package/path + ``` + +## Further Information + +For more details about `op-acceptor` and the acceptance testing process, refer to the main documentation or ask the team for guidance. + +The source code for `op-acceptor` is available at [github.com/ethereum-optimism/infra/op-acceptor](https://github.com/ethereum-optimism/infra/tree/main/op-acceptor). If you discover any bugs or have feature requests, please open an issue in that repository. \ No newline at end of file diff --git a/op-acceptance-tests/acceptance-tests.yaml b/op-acceptance-tests/acceptance-tests.yaml new file mode 100644 index 0000000000000..36fe719a7aef9 --- /dev/null +++ b/op-acceptance-tests/acceptance-tests.yaml @@ -0,0 +1,17 @@ +# Configuration file for acceptance tests (op-acceptor) +# +# All acceptance tests need to be registered here for op-acceptor to run them. +# +# Note: The current acceptance tests are placeholders; real ones to come soon. + +gates: + - id: localnet + description: "Localnet validation gate" + tests: + - name: TestFindRPCEndpoints + package: github.com/ethereum-optimism/optimism/kurtosis-devnet/pkg/kurtosis/api/run + - name: TestSystemWrapETH + package: github.com/ethereum-optimism/optimism/kurtosis-devnet/tests/interop + - name: TestInteropSystemNoop + package: github.com/ethereum-optimism/optimism/kurtosis-devnet/tests/interop + diff --git a/op-acceptance-tests/justfile b/op-acceptance-tests/justfile new file mode 100644 index 0000000000000..4967b2efc4176 --- /dev/null +++ b/op-acceptance-tests/justfile @@ -0,0 +1,29 @@ +REPO_ROOT := `realpath ..` +KURTOSIS_DIR := REPO_ROOT + "/kurtosis-devnet" +ACCEPTOR_IMAGE := env_var_or_default("ACCEPTOR_IMAGE", "us-docker.pkg.dev/oplabs-tools-artifacts/images/op-acceptor:v0.1.1") + +# Default recipe - runs the acceptance tests with default parameters +default: + @just acceptance-test + +# Run acceptance tests against a devnet +acceptance-test devnet="simple" gate="localnet": + #!/usr/bin/env bash + set -euo pipefail + + # First run the appropriate devnet from the kurtosis-devnet directory if needed. + # We ignore failures here because if the devnet is already running then this command will fail. + just {{KURTOSIS_DIR}}/{{ devnet }}-devnet || true + + # Print which image is being used (for debugging) + echo "Using acceptor image: {{ACCEPTOR_IMAGE}}" + + # Run op-acceptor with the repository mounted at the correct Go module path + docker run \ + -v "$(pwd)/acceptance-tests.yaml:/acceptance-tests.yaml" \ + -v "{{REPO_ROOT}}:/go/src/github.com/ethereum-optimism/optimism" \ + {{ACCEPTOR_IMAGE}} \ + --testdir "/go/src/github.com/ethereum-optimism/optimism" \ + --gate {{gate}} \ + --validators /acceptance-tests.yaml \ + --log.level info