Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 56 additions & 0 deletions op-acceptance-tests/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# OP Stack Acceptance Tests

## Overview

This directory contains the acceptance tests and configuration for the OP Stack. These tests are executed by `op-acceptor`, which serves as an automated gatekeeper for OP Stack network promotions.

Think of acceptance testing as Gandalf 🧙, standing at the gates and shouting, "You shall not pass!" to networks that don't meet our standards. It enforces the "Don't trust, verify" principle by:

- Running automated acceptance tests
- Providing clear pass/fail results (and tracking these over time)
- Gating network promotions based on test results
- Providing insight into test feature/functional coverage

The `op-acceptor` ensures network quality and readiness by running a comprehensive suite of acceptance tests before features can advance through the promotion pipeline:

Localnet -> Alphanet → Betanet → Testnet

This process helps maintain high-quality standards across all networks in the OP Stack ecosystem.

## Usage

The tests can be run using the `just` command runner:

```bash
# Run the default acceptance tests against a simple devnet
just

# Run the acceptance tests against a specific devnet and gate
just acceptance-test <devnet> <gate>

# Run the acceptance tests using a specific version of op-acceptor
ACCEPTOR_IMAGE=op-acceptor:latest just acceptance-test
```

### Configuration

- `acceptance-tests.yaml`: Defines the validation gates and the suites and tests that should be run for each gate.
- `justfile`: Contains the commands for running the acceptance tests.

## Adding New Tests

To add new acceptance tests:

1. Create your test in the appropriate Go package (as a regular Go test)
2. Register the test in `acceptance-tests.yaml` under the appropriate gate
3. Follow the existing pattern for test registration:
```yaml
- name: YourTestName
package: github.com/ethereum-optimism/optimism/your/package/path
```

## Further Information

For more details about `op-acceptor` and the acceptance testing process, refer to the main documentation or ask the team for guidance.

The source code for `op-acceptor` is available at [github.com/ethereum-optimism/infra/op-acceptor](https://github.com/ethereum-optimism/infra/tree/main/op-acceptor). If you discover any bugs or have feature requests, please open an issue in that repository.
17 changes: 17 additions & 0 deletions op-acceptance-tests/acceptance-tests.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Configuration file for acceptance tests (op-acceptor)
#
# All acceptance tests need to be registered here for op-acceptor to run them.
#
# Note: The current acceptance tests are placeholders; real ones to come soon.

gates:
- id: localnet
description: "Localnet validation gate"
tests:
- name: TestFindRPCEndpoints
package: github.com/ethereum-optimism/optimism/kurtosis-devnet/pkg/kurtosis/api/run
- name: TestSystemWrapETH
package: github.com/ethereum-optimism/optimism/kurtosis-devnet/tests/interop
- name: TestInteropSystemNoop
package: github.com/ethereum-optimism/optimism/kurtosis-devnet/tests/interop

29 changes: 29 additions & 0 deletions op-acceptance-tests/justfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
REPO_ROOT := `realpath ..`
KURTOSIS_DIR := REPO_ROOT + "/kurtosis-devnet"
ACCEPTOR_IMAGE := env_var_or_default("ACCEPTOR_IMAGE", "us-docker.pkg.dev/oplabs-tools-artifacts/images/op-acceptor:v0.1.1")

# Default recipe - runs the acceptance tests with default parameters
default:
@just acceptance-test

# Run acceptance tests against a devnet
acceptance-test devnet="simple" gate="localnet":
#!/usr/bin/env bash
set -euo pipefail

# First run the appropriate devnet from the kurtosis-devnet directory if needed.
# We ignore failures here because if the devnet is already running then this command will fail.
just {{KURTOSIS_DIR}}/{{ devnet }}-devnet || true

# Print which image is being used (for debugging)
echo "Using acceptor image: {{ACCEPTOR_IMAGE}}"

# Run op-acceptor with the repository mounted at the correct Go module path
docker run \
-v "$(pwd)/acceptance-tests.yaml:/acceptance-tests.yaml" \
-v "{{REPO_ROOT}}:/go/src/github.com/ethereum-optimism/optimism" \
{{ACCEPTOR_IMAGE}} \
--testdir "/go/src/github.com/ethereum-optimism/optimism" \
--gate {{gate}} \
--validators /acceptance-tests.yaml \
--log.level info