Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,14 @@

**/tmp/

# Heavy test/data assets not needed by the build (saves ~1.4 GB context).
.git/
fixtures/blockchain/
fixtures/blobs/
fixtures/cache/
fixtures/rsp/
fixtures/hive/

# These are backup files generated by rustfmt
**/*.rs.bk

Expand All @@ -20,6 +28,7 @@ docs/
hive/
ethereum-package/
tooling/ef_tests/blockchain/vectors
tooling/ef_tests/blockchain/vectors_zkevm
tooling/ef_tests/state/vectors
tooling/sync/multisync_logs/
dev_ethrex_l1/
Expand Down
6 changes: 5 additions & 1 deletion .github/actions/build-docker/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,11 @@ runs:
type=registry,ref=${{ inputs.registry }}/${{ github.repository }}:cache-${{ steps.cache-config.outputs.cache_scope }}-${{ inputs.variant }}-${{ steps.cache-config.outputs.platform_safe }}
type=registry,ref=${{ inputs.registry }}/${{ github.repository }}:cache-main-${{ inputs.variant }}-${{ steps.cache-config.outputs.platform_safe }}
cache-to: ${{ inputs.cache_write == 'true' && format('type=registry,ref={0}/{1}:cache-{2}-{3}-{4},mode=max', inputs.registry, github.repository, steps.cache-config.outputs.cache_scope, inputs.variant, steps.cache-config.outputs.platform_safe) || format('type=gha,mode=max,scope={0}-{1}-{2}', steps.cache-config.outputs.cache_scope, inputs.variant, steps.cache-config.outputs.platform_safe) }}
build-args: ${{ inputs.build_args }}
build-args: |
${{ inputs.build_args }}
GIT_SHA=${{ github.sha }}
GIT_BRANCH=${{ github.head_ref || github.ref_name }}
VERSION=${{ github.ref_type == 'tag' && github.ref_name || format('dev-{0}', github.sha) }}
platforms: ${{ inputs.platforms }}

# Since we're exporting the image as a tar, we need to load it manually as well
Expand Down
6 changes: 6 additions & 0 deletions .github/actions/snapsync-run/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -50,8 +50,14 @@ runs:
IMAGE_TAG: ${{ inputs.ethrex_tag }}
run: |
echo "Building ethrex with profile: ${BUILD_PROFILE}"
GIT_SHA=$(git rev-parse HEAD 2>/dev/null || echo unknown)
GIT_BRANCH=$(git rev-parse --abbrev-ref HEAD 2>/dev/null || echo unknown)
VERSION=$(git describe --tags --always --dirty 2>/dev/null || echo dev)
docker build \
--build-arg PROFILE="${BUILD_PROFILE}" \
--build-arg GIT_SHA="${GIT_SHA}" \
--build-arg GIT_BRANCH="${GIT_BRANCH}" \
--build-arg VERSION="${VERSION}" \
-t ethrex-local:${IMAGE_TAG} \
-f Dockerfile .

Expand Down
183 changes: 113 additions & 70 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,100 +1,143 @@
FROM rust:1.91 AS chef

RUN apt-get update && apt-get install -y \
build-essential \
libclang-dev \
libc6 \
libssl-dev \
ca-certificates \
&& rm -rf /var/lib/apt/lists/*
RUN cargo install cargo-chef
# syntax=docker/dockerfile:1.10

# --- Chef base ---
# Slim rust image + apt deps needed to compile native crates (rocksdb, openssl-sys, bindgen).
FROM rust:1.91-slim-bookworm AS chef

RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
--mount=type=cache,target=/var/lib/apt,sharing=locked \
apt-get update && apt-get install -y --no-install-recommends \
build-essential \
libclang-dev \
libssl-dev \
pkg-config \
ca-certificates \
curl \
git

# Force cargo to fetch git deps via the git CLI instead of libgit2. The bundled
# libgit2 hangs on some hosts/networks inside containers; the CLI also supports
# single-commit fetches for rev-pinned deps.
ENV CARGO_NET_GIT_FETCH_WITH_CLI=true

# Install cargo-chef via prebuilt binary (cargo-binstall) — avoids ~2 min source build.
# cargo-binstall pinned for reproducibility; bump deliberately.
ARG CARGO_BINSTALL_VERSION=v1.19.1
RUN --mount=type=cache,target=/usr/local/cargo/registry \
--mount=type=cache,target=/usr/local/cargo/git \
curl -fsSL https://github.com/cargo-bins/cargo-binstall/releases/download/${CARGO_BINSTALL_VERSION}/cargo-binstall-$(uname -m)-unknown-linux-musl.tgz \
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

$(uname -m) here is correct and should stay — but it reads as inconsistent next to the solc download below, which the PR (rightly) switched to $TARGETARCH. The difference is intentional: cargo-binstall installs cargo-chef, a build-platform tool that runs during the build, so it must match the stage's execution architecture (uname -m), not the image's target arch ($TARGETARCH). Using $TARGETARCH here would download the wrong binstall for emulated cross-builds (e.g. building linux/arm64 on an amd64 runner).

Worth a one-line comment so a future "consistency" pass doesn't change it:

# uname -m (NOT $TARGETARCH): cargo-chef is a build-platform tool, must match
# the stage's execution arch, not the target image arch.

| tar -xz -C /usr/local/cargo/bin \
&& cargo binstall --no-confirm cargo-chef

WORKDIR /ethrex


# --- Planner Stage ---
# Copy all source code to calculate the dependency recipe.
# This layer is fast and will be invalidated on any source change.
# --- Planner ---
# Compute the dependency recipe. Fast, invalidated on any source change.
FROM chef AS planner

COPY benches ./benches
COPY crates ./crates
COPY metrics ./metrics
COPY cmd ./cmd
COPY test ./test
COPY tooling ./tooling
COPY Cargo.* .
COPY .cargo/ ./.cargo
COPY --link benches ./benches
COPY --link crates ./crates
COPY --link metrics ./metrics
COPY --link cmd ./cmd
COPY --link test ./test
COPY --link tooling/repl ./tooling/repl
COPY --link tooling/monitor ./tooling/monitor
COPY --link Cargo.toml Cargo.lock ./
COPY --link .cargo ./.cargo

RUN cargo chef prepare --recipe-path recipe.json


# --- Builder Stage ---
# Build the dependencies. This is the most time-consuming step.
# This layer will be cached and only re-run if the recipe.json from the
# previous stage has changed, which only happens when dependencies change.
# --- Builder ---
# Cook deps first (cached unless recipe.json changes), then build the app.
FROM chef AS builder

# Build configuration
# PROFILE: Cargo profile to use (release, release-with-debug-assertions, etc.)
# BUILD_FLAGS: Additional cargo flags (features, etc.)
ARG PROFILE="release"
ARG PROFILE=release
ARG BUILD_FLAGS=""

COPY --from=planner /ethrex/recipe.json recipe.json
RUN cargo chef cook --release --recipe-path recipe.json $BUILD_FLAGS

RUN if [ "$(uname -m)" = aarch64 ]; \
then \
SOLC_URL=https://github.com/ethereum/solidity/releases/download/v0.8.31/solc-static-linux-arm;\
else \
SOLC_URL=https://github.com/ethereum/solidity/releases/download/v0.8.31/solc-static-linux; \
fi \
&& curl -L -o /usr/bin/solc $SOLC_URL \
ARG TARGETARCH

# vergen-git2 reads .git unless these env vars are set. Pass via build args
# so we don't ship the 1 GB .git directory into the build context.
ARG GIT_BRANCH=unknown
ARG GIT_SHA=unknown
ENV VERGEN_GIT_BRANCH=$GIT_BRANCH \
VERGEN_GIT_SHA=$GIT_SHA \
VERGEN_IDEMPOTENT=1
Comment thread
edg-l marked this conversation as resolved.

COPY --from=planner --link /ethrex/recipe.json recipe.json

RUN --mount=type=cache,target=/usr/local/cargo/registry \
--mount=type=cache,target=/usr/local/cargo/git \
--mount=type=cache,target=/ethrex/target,id=ethrex-target-${TARGETARCH} \
cargo chef cook --profile $PROFILE --recipe-path recipe.json $BUILD_FLAGS

# Fetch solc using buildx's TARGETARCH (no shell uname).
RUN case "$TARGETARCH" in \
arm64) SOLC_URL=https://github.com/ethereum/solidity/releases/download/v0.8.31/solc-static-linux-arm ;; \
amd64) SOLC_URL=https://github.com/ethereum/solidity/releases/download/v0.8.31/solc-static-linux ;; \
*) echo "unsupported TARGETARCH=$TARGETARCH" >&2; exit 1 ;; \
esac \
&& curl -fsSL -o /usr/bin/solc "$SOLC_URL" \
&& chmod +x /usr/bin/solc

COPY benches ./benches
COPY crates ./crates
COPY cmd ./cmd
COPY metrics ./metrics
COPY tooling ./tooling
COPY fixtures/genesis ./fixtures/genesis
COPY .git ./.git
COPY Cargo.* ./
COPY fixtures ./fixtures
COPY .cargo/ ./.cargo
COPY --link benches ./benches
COPY --link crates ./crates
COPY --link cmd ./cmd
COPY --link metrics ./metrics
COPY --link test ./test
COPY --link tooling/repl ./tooling/repl
COPY --link tooling/monitor ./tooling/monitor
COPY --link Cargo.toml Cargo.lock ./
COPY --link .cargo ./.cargo
# Only these subdirs are referenced by include_str!/include_bytes! in workspace
# crates; the rest of fixtures/ is test data not needed at build time.
COPY --link fixtures/genesis ./fixtures/genesis
COPY --link fixtures/keys ./fixtures/keys

ENV COMPILE_CONTRACTS=true

RUN cargo build --profile $PROFILE $BUILD_FLAGS
# Combine build + extract in one RUN so the target cache mount is still mounted.
RUN --mount=type=cache,target=/usr/local/cargo/registry \
--mount=type=cache,target=/usr/local/cargo/git \
--mount=type=cache,target=/ethrex/target,id=ethrex-target-${TARGETARCH} \
cargo build --profile $PROFILE $BUILD_FLAGS \
&& mkdir -p /ethrex/bin \
&& cp /ethrex/target/${PROFILE}/ethrex /ethrex/bin/ethrex

RUN mkdir -p /ethrex/bin && \
cp /ethrex/target/${PROFILE}/ethrex /ethrex/bin/ethrex

# --- Final Image ---
# Copy the ethrex binary into a minimalist image to reduce bloat size.
# This image must have glibc and libssl
# --- Runtime ---
# ubuntu:24.04 keeps glibc + libssl3 available. Network genesis/bootnodes are
# embedded into the binary via include_str!, so no extra files are needed.
FROM ubuntu:24.04
WORKDIR /usr/local/bin

RUN apt-get update && apt-get install -y --no-install-recommends libssl3
ARG GIT_SHA=unknown
ARG VERSION=dev

LABEL org.opencontainers.image.title="ethrex" \
org.opencontainers.image.description="Rust Ethereum execution client" \
org.opencontainers.image.source="https://github.com/lambdaclass/ethrex" \
org.opencontainers.image.licenses="MIT OR Apache-2.0" \
org.opencontainers.image.revision="${GIT_SHA}" \
org.opencontainers.image.version="${VERSION}"

COPY cmd/ethrex/networks ./cmd/ethrex/networks
COPY --from=builder /ethrex/bin/ethrex .
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
--mount=type=cache,target=/var/lib/apt,sharing=locked \
apt-get update && apt-get install -y --no-install-recommends \
libssl3 \
ca-certificates

WORKDIR /usr/local/bin

COPY --from=builder --link /ethrex/bin/ethrex /usr/local/bin/ethrex

# Common ports:
# - 8545: RPC
# - 8551: EngineAPI
# - 30303: Discovery
# - 30303: Discovery (tcp+udp)
# - 9090: Metrics
# - 1729: L2 RPC
# - 3900: L2 Proof Coordinator
EXPOSE 8545
EXPOSE 8551
EXPOSE 30303/tcp
EXPOSE 30303/udp
EXPOSE 9090
EXPOSE 1729
EXPOSE 3900

ENTRYPOINT [ "./ethrex" ]
EXPOSE 8545 8551 9090 1729 3900 30303/tcp 30303/udp

ENTRYPOINT ["ethrex"]
24 changes: 19 additions & 5 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -42,15 +42,29 @@ clean: clean-vectors ## 🧹 Remove build artifacts
cargo clean
rm -rf hive

# Docker image tag (override with `make build-image TAG=foo`).
TAG ?= local
IMAGE := ethrex:$(TAG)

# Git metadata baked into the image via Dockerfile ARGs. Falls back to "unknown"
# / "dev" if not in a git checkout.
GIT_SHA := $(shell git rev-parse HEAD 2>/dev/null || echo unknown)
GIT_BRANCH := $(shell git rev-parse --abbrev-ref HEAD 2>/dev/null || echo unknown)
VERSION := $(shell git describe --tags --always --dirty 2>/dev/null || echo dev)

STAMP_FILE := .docker_build_stamp
$(STAMP_FILE): $(shell find crates cmd -type f -name '*.rs') Cargo.toml Dockerfile
docker build -t ethrex:local .
$(STAMP_FILE): $(shell find crates cmd tooling/repl tooling/monitor -type f -name '*.rs') Cargo.toml Dockerfile
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The stamp-file prerequisites don't include the git metadata, so the new GIT_SHA/GIT_BRANCH/VERSION args can go stale. If you switch to a branch whose tracked .rs/Cargo.toml/Dockerfile are identical (or just dirty the tree so git describe --dirty flips), $(STAMP_FILE) is still considered up-to-date and make build-image skips the rebuild — reusing an image baked with the previous SHA/version labels.

Low-impact in practice (cross-branch source usually differs), but since the whole point of this PR is accurate metadata, consider making the stamp depend on the resolved SHA, e.g. write $(GIT_SHA) into a tracked .git_sha_stamp and add it as a prerequisite, or just drop the stamp-file indirection for build-image and always rebuild (BuildKit layer cache makes the no-op rebuild cheap). Non-blocking.

docker build \
--build-arg GIT_SHA=$(GIT_SHA) \
--build-arg GIT_BRANCH=$(GIT_BRANCH) \
--build-arg VERSION=$(VERSION) \
-t $(IMAGE) .
touch $(STAMP_FILE)

build-image: $(STAMP_FILE) ## 🐳 Build the Docker image
build-image: $(STAMP_FILE) ## 🐳 Build the Docker image (override tag with TAG=foo)

run-image: build-image ## 🏃 Run the Docker image
docker run --rm -p 127.0.0.1:8545:8545 ethrex:main --http.addr 0.0.0.0
docker run --rm -p 127.0.0.1:8545:8545 $(IMAGE) --http.addr 0.0.0.0

dev: ## 🏃 Run the ethrex client in DEV_MODE with the InMemory Engine
cargo run $(PROFILING_CFG) --release -- \
Expand Down Expand Up @@ -84,7 +98,7 @@ localnet: build-image checkout-ethereum-package ## 🌐 Start kurtosis network
trap 'printf "\nStopping localnet...\n"; $(MAKE) stop-localnet || true; exit 0' INT TERM HUP QUIT; \
cp metrics/provisioning/grafana/dashboards/common_dashboards/ethrex_l1_perf.json ethereum-package/src/grafana/ethrex_l1_perf.json; \
kurtosis run --enclave $(ENCLAVE) ethereum-package --args-file $(KURTOSIS_CONFIG_FILE); \
CID=$$(docker ps -q --filter ancestor=ethrex:local | head -n1); \
CID=$$(docker ps -q --filter ancestor=$(IMAGE) | head -n1); \
if [ -n "$$CID" ]; then docker logs -f $$CID || true; else echo "No ethrex container found; skipping logs."; fi

stop-localnet: ## 🛑 Stop local network
Expand Down
18 changes: 18 additions & 0 deletions tooling/sync/docker_monitor.py
Original file line number Diff line number Diff line change
Expand Up @@ -386,11 +386,29 @@ def build_docker_image(profile: str, image_tag: str, ethrex_dir: str) -> bool:
"""
print(f"🔨 Building Docker image with profile '{profile}'...")
print(f" Image tag: {image_tag}")

def _git(args: list[str], default: str) -> str:
try:
out = subprocess.run(
["git", "-C", ethrex_dir, *args],
capture_output=True, text=True, check=True,
).stdout.strip()
return out or default
except (subprocess.CalledProcessError, FileNotFoundError):
return default

git_sha = _git(["rev-parse", "HEAD"], "unknown")
git_branch = _git(["rev-parse", "--abbrev-ref", "HEAD"], "unknown")
version = _git(["describe", "--tags", "--always", "--dirty"], "dev")

try:
subprocess.run(
[
"docker", "build",
"--build-arg", f"PROFILE={profile}",
"--build-arg", f"GIT_SHA={git_sha}",
"--build-arg", f"GIT_BRANCH={git_branch}",
"--build-arg", f"VERSION={version}",
"-t", image_tag,
"-f", f"{ethrex_dir}/Dockerfile",
ethrex_dir
Expand Down
Loading