Skip to content
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
c33c122
squash commits
spypsy Dec 5, 2024
d9cab7b
resolve conflicts
spypsy Dec 5, 2024
d049c0e
some missed changes
spypsy Dec 5, 2024
02fcc11
update network_test
spypsy Dec 5, 2024
6141a50
Merge branch 'master' into spy/sepolia-spartan-updates
spypsy Dec 5, 2024
97d1248
apparently conflicts
spypsy Dec 5, 2024
c5e0e7c
undo irrelevant changes
spypsy Dec 5, 2024
9633ab9
undo change
spypsy Dec 5, 2024
b9dc642
merge conflicts
spypsy Dec 9, 2024
9cc7cc9
github actions
spypsy Dec 10, 2024
fe15b83
revert comment
spypsy Dec 10, 2024
9216a74
formatting ig
spypsy Dec 10, 2024
44c375c
set externalHost + 20sec bot interval
spypsy Dec 11, 2024
cb9f57e
merge conflicts
spypsy Dec 11, 2024
37d3d38
correctly pass list to helm release
spypsy Dec 12, 2024
1eca57d
Merge branch 'master' into spy/sepolia-spartan-updates
spypsy Dec 12, 2024
6e08c18
sepolia-exp-1
spypsy Dec 12, 2024
afefb5f
bootstrap sepolia
spypsy Dec 12, 2024
3717fb1
quote-wrap mnemonic
spypsy Dec 12, 2024
dc983f9
safer quote wrapping
spypsy Dec 12, 2024
bf04092
use eval
spypsy Dec 12, 2024
6ab38e3
Merge branch 'master' into spy/sepolia-spartan-updates
spypsy Dec 13, 2024
ca4e910
just check for externalHost
spypsy Dec 16, 2024
5debe14
PR Fixes
spypsy Dec 16, 2024
e3b8a7d
consider more empty sepolia vars
spypsy Dec 16, 2024
281ce7e
restore default prover publisher pk
spypsy Dec 16, 2024
be47200
exp-2.yaml
spypsy Dec 16, 2024
21bce8c
guard all pk & mnemonic vars
spypsy Dec 16, 2024
d72d954
more tf var defaults
spypsy Dec 17, 2024
2be454b
Merge branch 'master' into spy/sepolia-spartan-updates
spypsy Dec 19, 2024
49ee379
Merge branch 'master' into spy/sepolia-spartan-updates
spypsy Dec 20, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion cspell.json
Original file line number Diff line number Diff line change
Expand Up @@ -322,4 +322,4 @@
"flagWords": [
"anonymous"
]
}
}
22 changes: 20 additions & 2 deletions scripts/run_interleaved.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,24 @@ set -eu
# propagate errors inside while loop pipe
set -o pipefail

# Usage: run_interleaved.sh <main command> <background commands>...
# Usage: run_interleaved.sh [-w "condition command"] <main command> <background commands>...
# Runs the main command with output logging and background commands without logging.
# Finishes when the main command exits.
# -w: Optional wait condition command that must succeed before starting next command

# Parse options
WAIT_CMD=""
while getopts "w:" opt; do
case $opt in
w) WAIT_CMD="$OPTARG";;
\?) echo "Invalid option -$OPTARG" >&2; exit 1;;
esac
done
shift $((OPTIND-1))

# Check if at least two commands are provided (otherwise what is the point)
if [ "$#" -lt 2 ]; then
echo "Usage: $0 <main-command> <background commands>..."
echo "Usage: $0 [-w 'condition command'] <main-command> <background commands>..."
exit 1
fi

Expand Down Expand Up @@ -51,6 +62,13 @@ function run_command() {
# Run background commands without logging output
i=0
for cmd in "$@"; do
if [ $i -gt 0 ] && [ -n "$WAIT_CMD" ]; then
echo "Waiting for condition before starting next command..."
until eval "$WAIT_CMD"; do
sleep 1
done
fi

(run_command "$cmd" "${colors[$((i % ${#colors[@]}))]}" || [ $FINISHED = true ] || (echo "$cmd causing terminate" && kill 0) ) &
((i++)) || true # annoyingly considered a failure based on result
done
Expand Down
3 changes: 3 additions & 0 deletions spartan/aztec-network/files/config/deploy-l1-contracts.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,16 @@ set -exu

CHAIN_ID=$1

# Use default account, it is funded on our dev machine
export PRIVATE_KEY=${PRIVATE_KEY:-"0xac0974bec39a17e36ba4a6b4d238ff944bacb478cbed5efcae784d7bf4f2ff80"}

# Run the deploy-l1-contracts command and capture the output
output=""
MAX_RETRIES=5
RETRY_DELAY=60
for attempt in $(seq 1 $MAX_RETRIES); do
# if INIT_VALIDATORS is true, then we need to pass the validators flag to the deploy-l1-contracts command

if [ "${INIT_VALIDATORS:-false}" = "true" ]; then
output=$(node --no-warnings /usr/src/yarn-project/aztec/dest/bin/index.js deploy-l1-contracts --mnemonic "$MNEMONIC" --validators $2 --l1-chain-id $CHAIN_ID) && break
else
Expand Down
4 changes: 2 additions & 2 deletions spartan/aztec-network/files/config/setup-service-addresses.sh
Original file line number Diff line number Diff line change
Expand Up @@ -53,8 +53,8 @@ get_service_address() {
}

# Configure Ethereum address
if [ "${ETHEREUM_EXTERNAL_HOST}" != "" ]; then
ETHEREUM_ADDR="${ETHEREUM_EXTERNAL_HOST}"
if [ "${EXTERNAL_ETHEREUM_HOST}" != "" ]; then

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry about this one. Hope you didn't lose much time.

ETHEREUM_ADDR="${EXTERNAL_ETHEREUM_HOST}"
elif [ "${NETWORK_PUBLIC}" = "true" ]; then
ETHEREUM_ADDR=$(get_service_address "ethereum" "${ETHEREUM_PORT}")
else
Expand Down
16 changes: 13 additions & 3 deletions spartan/aztec-network/templates/boot-node.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -36,8 +36,8 @@ spec:
cat /shared/config/service-addresses
echo "Awaiting ethereum node at ${ETHEREUM_HOST}"
until curl -s -X POST -H 'Content-Type: application/json' \
-d '{"jsonrpc":"2.0","method":"web3_clientVersion","params":[],"id":67}' \
${ETHEREUM_HOST} | grep -q reth; do
-d '{"jsonrpc":"2.0","method":"eth_chainId","params":[],"id":67}' \
${ETHEREUM_HOST} | grep 0x; do
echo "Waiting for Ethereum node ${ETHEREUM_HOST}..."
sleep 5
done
Expand Down Expand Up @@ -99,7 +99,7 @@ spec:
source /shared/p2p/p2p-addresses && \
source /shared/config/service-addresses && \
env && \
node --no-warnings /usr/src/yarn-project/aztec/dest/bin/index.js start --node --archiver --sequencer --pxe

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why drop the PXE?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wasn't doing anything right? I believe it was only used for the get-node-info call from other services to check boo-node is up but this has now been added to Aztec Node API

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm yes, but I think this now conflicts with spartan/aztec-network/files/config/config-validator-env.sh, since it is asking for a PXE url.

It is convenient in that case because when "dynamic boot node" is enabled, the validators use each other to bootstrap, so they use the PXE service, which is connected to the boot node if the network is public, and the validator service otherwise.

So I think the answer is to update the wait-for-services init container in the validators to use echo "{{ include "aztec-network.validatorUrl" . }}" > /shared/boot_node/boot_node_url, and then change back the config-validator-env to use the node url.

Could/should just be a separate issue though.

node --no-warnings /usr/src/yarn-project/aztec/dest/bin/index.js start --node --archiver --sequencer
startupProbe:
httpGet:
path: /status
Expand Down Expand Up @@ -154,6 +154,8 @@ spec:
value: "{{ .Values.bootNode.sequencer.minTxsPerBlock }}"
- name: VALIDATOR_PRIVATE_KEY
value: "0xac0974bec39a17e36ba4a6b4d238ff944bacb478cbed5efcae784d7bf4f2ff80"
- name: SEQ_PUBLISHER_PRIVATE_KEY
value: "{{ .Values.bootNode.seqPublisherPrivateKey }}"
- name: OTEL_RESOURCE_ATTRIBUTES
value: service.name={{ .Release.Name }},service.namespace={{ .Release.Namespace }},service.version={{ .Chart.AppVersion }},environment={{ .Values.environment | default "production" }}
- name: PROVER_REAL_PROOFS
Expand All @@ -168,6 +170,14 @@ spec:
value: "{{ .Values.aztec.epochDuration }}"
- name: AZTEC_EPOCH_PROOF_CLAIM_WINDOW_IN_L2_SLOTS
value: "{{ .Values.aztec.epochProofClaimWindow }}"
- name: ARCHIVER_POLLING_INTERVAL_MS
value: {{ .Values.bootNode.archiverPollingInterval | quote }}
- name: ARCHIVER_VIEM_POLLING_INTERVAL_MS
value: {{ .Values.bootNode.archiverViemPollingInterval | quote }}
- name: L1_READER_VIEM_POLLING_INTERVAL_MS
value: {{ .Values.bootNode.archiverViemPollingInterval | quote }}
- name: SEQ_VIEM_POLLING_INTERVAL_MS
value: {{ .Values.bootNode.viemPollingInterval | quote }}
- name: PEER_ID_PRIVATE_KEY
value: "{{ .Values.bootNode.peerIdPrivateKey }}"
ports:
Expand Down
8 changes: 5 additions & 3 deletions spartan/aztec-network/templates/prover-node.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -34,8 +34,8 @@ spec:
- |
source /shared/config/service-addresses
until curl -s -X POST -H 'Content-Type: application/json' \
-d '{"jsonrpc":"2.0","method":"web3_clientVersion","params":[],"id":67}' \
${ETHEREUM_HOST} | grep -q reth; do
-d '{"jsonrpc":"2.0","method":"eth_chainId","params":[],"id":67}' \
${ETHEREUM_HOST} | grep 0x; do
echo "Waiting for Ethereum node ${ETHEREUM_HOST}..."
sleep 5
done
Expand Down Expand Up @@ -140,7 +140,7 @@ spec:
- name: PROVER_BROKER_DATA_DIRECTORY
value: "{{ .Values.proverNode.proverBroker.dataDirectory }}"
- name: PROVER_PUBLISHER_PRIVATE_KEY
value: "0xac0974bec39a17e36ba4a6b4d238ff944bacb478cbed5efcae784d7bf4f2ff80"
value: "{{ .Values.proverNode.proverPublisherPrivateKey }}"
- name: OTEL_RESOURCE_ATTRIBUTES
value: service.name={{ .Release.Name }},service.namespace={{ .Release.Namespace }},service.version={{ .Chart.AppVersion }},environment={{ .Values.environment | default "production" }}
- name: L1_CHAIN_ID
Expand All @@ -159,6 +159,8 @@ spec:
value: "{{ .Values.aztec.epochDuration }}"
- name: AZTEC_EPOCH_PROOF_CLAIM_WINDOW_IN_L2_SLOTS
value: "{{ .Values.aztec.epochProofClaimWindow }}"
- name: PROVER_VIEM_POLLING_INTERVAL_MS
value: {{ .Values.proverNode.viemPollingInterval | quote }}
ports:
- containerPort: {{ .Values.proverNode.service.nodePort }}
- containerPort: {{ .Values.proverNode.service.p2pTcpPort }}
Expand Down
4 changes: 3 additions & 1 deletion spartan/aztec-network/templates/reth.yaml
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
{{- if not .Values.network.disableEthNode }}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

instead of adding a new flag, could this be replaced by checking if .Values.ethereum.externalHost is set? If we wanted a flag, we could create a helper based off checking that value?

apiVersion: apps/v1
kind: Deployment
metadata:
Expand Down Expand Up @@ -135,4 +136,5 @@ spec:
requests:
storage: {{ .Values.ethereum.storage }}
{{- end }}
---
---
{{ end }}
14 changes: 12 additions & 2 deletions spartan/aztec-network/templates/validator.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,8 @@ spec:
cat /shared/config/service-addresses
# First check ethereum node
until curl -s -X POST -H 'Content-Type: application/json' \
-d '{"jsonrpc":"2.0","method":"web3_clientVersion","params":[],"id":67}' \
$ETHEREUM_HOST | grep -q reth; do
-d '{"jsonrpc":"2.0","method":"eth_chainId","params":[],"id":67}' \
$ETHEREUM_HOST | grep 0x; do
echo "Waiting for Ethereum node ${ETHEREUM_HOST}..."
sleep 5
done
Expand Down Expand Up @@ -175,6 +175,16 @@ spec:
value: "{{ .Values.aztec.epochDuration }}"
- name: AZTEC_EPOCH_PROOF_CLAIM_WINDOW_IN_L2_SLOTS
value: "{{ .Values.aztec.epochProofClaimWindow }}"
- name: VALIDATOR_ATTESTATIONS_POLLING_INTERVAL_MS
value: {{ .Values.validator.attestationPollingInterval | quote }}
- name: ARCHIVER_POLLING_INTERVAL_MS
value: {{ .Values.validator.archiverPollingInterval | quote }}
- name: ARCHIVER_VIEM_POLLING_INTERVAL_MS
value: {{ .Values.validator.viemPollingInterval | quote }}
- name: L1_READER_VIEM_POLLING_INTERVAL_MS
value: {{ .Values.validator.viemPollingInterval | quote }}
- name: SEQ_VIEM_POLLING_INTERVAL_MS
value: {{ .Values.validator.viemPollingInterval | quote }}
ports:
- containerPort: {{ .Values.validator.service.nodePort }}
- containerPort: {{ .Values.validator.service.p2pTcpPort }}
Expand Down
28 changes: 28 additions & 0 deletions spartan/aztec-network/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ aztec:
l1DeploymentMnemonic: "test test test test test test test test test test test junk" # the mnemonic used when deploying contracts

bootNode:
seqPublisherPrivateKey: ""
peerIdPrivateKey: ""
externalHost: ""
replicas: 1
Expand Down Expand Up @@ -72,6 +73,9 @@ bootNode:
feeJuiceAddress: ""
feeJuicePortalAddress: ""
storage: "8Gi"
archiverPollingInterval: 1000
archiverViemPollingInterval: 1000
viemPollingInterval: 1000

validator:
# If true, the validator will use its peers to serve as the boot node.
Expand Down Expand Up @@ -108,8 +112,13 @@ validator:
requests:
memory: "2Gi"
cpu: "200m"
archiverPollingInterval: 1000
archiverViemPollingInterval: 1000
attestationPollingInterval: 1000
viemPollingInterval: 1000

proverNode:
proverPublisherPrivateKey: "0xac0974bec39a17e36ba4a6b4d238ff944bacb478cbed5efcae784d7bf4f2ff80"
externalHost: ""
replicas: 1
p2pEnabled: true
Expand All @@ -134,6 +143,10 @@ proverNode:
memory: "2Gi"
cpu: "200m"
storage: "8Gi"
archiverPollingInterval: 1000
archiverViemPollingInterval: 1000
pollInterval: 1000
viemPollingInterval: 1000

pxe:
logLevel: "debug"
Expand Down Expand Up @@ -219,6 +232,21 @@ proverAgent:
hardwareConcurrency: ""
nodeSelector: {}
resources: {}
pollInterval: 200

proverBroker:
service:
nodePort: 8084
enabled: true
replicas: 1
jobTimeoutMs: 30000
pollIntervalMs: 1000
jobMaxRetries: 3
dataDirectory: ""
logLevel: "debug"
debug: "aztec:*,-aztec:avm_simulator*,-aztec:libp2p_service*,-aztec:circuits:artifact_hash,-json-rpc*,-aztec:world-state:database,-aztec:l2_block_stream*"
nodeSelector: {}
resources: {}

proverBroker:
service:
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
telemetry:
enabled: true
otelCollectorEndpoint: http://metrics-opentelemetry-collector.metrics:4318

network:
setupL2Contracts: false
disableEthNode: true
public: false

ethereum:
externalHost: "https://sepolia.infura.io/v3/${INFURA_API_KEY}"
chainId: "11155111"

validator:
replicas: 3
validatorKeys: ${VALIDATOR_KEYS}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How do these get set? I didn't think you could pull from env like this in a values file.

validatorAddresses: ${VALIDATOR_ADDRESSES}
validator:
disabled: false

bootNode:
seqPublisherPrivateKey: ${SEQ_PUBLISHER_PRIVATE_KEY}
validator:
disabled: true

proverNode:
proverPublisherPrivateKey: ${PROVER_PUBLISHER_PRIVATE_KEY}
14 changes: 11 additions & 3 deletions yarn-project/end-to-end/scripts/native-network/validators.sh
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,15 @@ if [ "$NUM_VALIDATORS" -eq 1 ]; then
echo "Running single validator directly"
eval "${CMD[0]}"
else
echo "Running $NUM_VALIDATORS validators interleaved"
# Execute the run_interleaved.sh script with the commands
"$(git rev-parse --show-toplevel)/scripts/run_interleaved.sh" "${CMD[@]}"
echo "Running $NUM_VALIDATORS validators sequentially, interleaved"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

supernit: no longer sequentially

FIRST_PORT=8081

# check if we're running against anvil
if curl -s -H "Content-Type: application/json" -X POST --data '{"method":"web3_clientVersion","params":[],"id":49,"jsonrpc":"2.0"}' $ETHEREUM_HOST | jq .result | grep -q anvil; then
"$(git rev-parse --show-toplevel)/scripts/run_interleaved.sh" "${CMD[@]}"
else
# Use run_interleaved with a wait condition
WAIT_CONDITION="curl -s http://127.0.0.1:$FIRST_PORT/status >/dev/null"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

was this wait condition added to prevent them being added with the same key? #10327 this pr pre adds the validators at contract deployment time so may invalidate the need for this work around

"$(git rev-parse --show-toplevel)/scripts/run_interleaved.sh" -w "$WAIT_CONDITION" "${CMD[@]}"
fi
fi
Loading