Skip to content

make: add docker-dev-* targets for building dev images #10912

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ZZiigguurraatt wants to merge 1 commit into
base: master
Choose a base branch
from
+115 −4
Open

make: add docker-dev-* targets for building dev images #10912

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 65 additions & 1 deletion Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -213,6 +213,67 @@ docker-tools:
@$(call print, "Building tools docker image.")
docker build -q -t lnd-tools $(TOOLS_DIR)

# Short commit hash of the working tree, with a `-dirty` suffix if there are
# uncommitted changes. Unlike $(COMMIT), this never picks up submodule tags
# like `kvdb/v1.5.1` — `/` is not a valid character in a docker image tag.
# `--match='__no_such_tag__'` forces `git describe` to ignore all tags so
# `--always` falls back to the short hash.
DOCKER_DEV_COMMIT := $(shell git describe --always --dirty --match='__no_such_tag__')

@gemini-code-assist gemini-code-assist Bot Jun 18, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

If make is run outside of a git repository (e.g., from a source archive or in a bare container), git describe will fail and print an error to stderr, resulting in an empty DOCKER_DEV_COMMIT. This causes DOCKER_DEV_IMAGE to be tagged as lnd-dev:, which is an invalid Docker tag. Adding a fallback to latest and redirecting stderr to /dev/null makes the build more robust in non-git environments.

DOCKER_DEV_COMMIT := $(shell git describe --always --dirty --match='__no_such_tag__' 2>/dev/null || echo "latest")

@ZZiigguurraatt ZZiigguurraatt Jun 18, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we care about this since we already have git used by another of other things in the makefile without this safeguard?


# Image (name:tag) produced by `docker-dev-build`. Also consumed by
# `docker-dev-lndinit-build` so the lndinit image is always layered on
# whatever `docker-dev-build` just built. Defaults to `lnd-dev:<short-hash>`
# (e.g. `lnd-dev:f5a093c1f` on commit f5a093c1f).
DOCKER_DEV_IMAGE ?= lnd-dev:$(DOCKER_DEV_COMMIT)

# Repository name for the lndinit dev image; the tag is always
# `lnd-dev-$(DOCKER_DEV_COMMIT)` so it's obvious which docker-dev-build image
# the lndinit image was layered on.
LNDINIT_REPO ?= lndinit

# Build context (path or git URL with optional #ref) for lndinit's dev.Dockerfile.
# NOTE: `#` is escaped as `\#` so make doesn't treat the ref as a comment.
LNDINIT_CONTEXT ?= https://github.com/lightninglabs/lndinit.git\#main

@ZZiigguurraatt ZZiigguurraatt Jun 18, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we want to set this to a release instead of tracking the default branch?

cons are we would have to remember to update here when we have new lndinit releases.


# Full lndinit image name:tag, derived from LNDINIT_REPO and DOCKER_DEV_COMMIT.
LNDINIT_IMAGE := $(LNDINIT_REPO):lnd-dev-$(DOCKER_DEV_COMMIT)

# dev.Dockerfile uses BuildKit cache mounts unconditionally, so these targets
# always go through `docker buildx build`.
#? docker-dev-build: Build a development docker image from dev.Dockerfile (override DOCKER_DEV_IMAGE=<name:tag>)
docker-dev-build:
@$(call print, "Building dev docker image $(DOCKER_DEV_IMAGE).")
docker buildx build -t $(DOCKER_DEV_IMAGE) -f dev.Dockerfile .

@calvinrzachman calvinrzachman Jul 15, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These recipes run docker buildx build with no --platform, so they build for the host's arch. When building on one arch for a different target (e.g. an Apple Silicon Mac for an amd64 host), the image comes out the wrong arch and won't run there. What do you think about a small knob, defaulting to native so existing behavior is unchanged?

DOCKER_DEV_PLATFORM ?= linux/$(shell go env GOARCH)
# in each recipe:
docker buildx build --platform $(DOCKER_DEV_PLATFORM) ...

A cross-build is then just DOCKER_DEV_PLATFORM=linux/amd64. Single-platform on purpose; multi-arch manifests seem reasonably out of scope for a dev image.

@ZZiigguurraatt ZZiigguurraatt Jul 15, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good idea


#? docker-dev-lndinit-build: Build an lndinit dev image layered on the docker-dev-build image (override LNDINIT_REPO=<name>, LNDINIT_CONTEXT=<git ref or path>)
docker-dev-lndinit-build: docker-dev-build

@calvinrzachman calvinrzachman Jul 15, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The lnd -> lndinit chain assumes the default docker driver. docker-dev-build builds the lnd image but doesn't --push or --load it, so this target's FROM ${BASE_IMAGE} only resolves on the default docker driver, where -t lands in the local image store and FROM reads it. On a docker-container driver the base isn't resolvable from the local store even with --load (I verified this: FROM falls back to a registry pull and fails), so the base has to be --pushed to a registry. Worth either defaulting docker-dev-build to --push, or documenting that the local chain requires the docker driver, so it doesn't silently fail on docker-container. Not about multi-arch, just the base being resolvable across setups.

@ZZiigguurraatt ZZiigguurraatt Jul 15, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have not used that workflow, so will look into this.

@calvinrzachman calvinrzachman Jul 15, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To keep this simple: it's fine as-is on the default docker driver. For the docker-container case the lnd image just needs to be pushed to a registry first (--load into the local store isn't enough there — the FROM still resolves from a registry), so no need to support it in the target. Might be worth a one-line note in DOCKER.md that the chain assumes the default docker driver, but that's it.

@$(call print, "Building lndinit docker image $(LNDINIT_IMAGE) on top of $(DOCKER_DEV_IMAGE).")
IMG='$(DOCKER_DEV_IMAGE)'; \
TAG="$${IMG##*:}"; \
case "$$IMG" in \
*:*) case "$$TAG" in */*) HAS_TAG=no ;; *) HAS_TAG=yes ;; esac ;; \
*) HAS_TAG=no ;; \
esac; \
if [ "$$HAS_TAG" != yes ]; then \
echo "DOCKER_DEV_IMAGE=$$IMG has no tag; set DOCKER_DEV_IMAGE=<name>:<tag>" >&2; \
exit 1; \
fi; \
CTX='$(LNDINIT_CONTEXT)'; \
if [ -d "$$CTX" ]; then \
DOCKERFILE="$$CTX/dev.Dockerfile"; \
else \
DOCKERFILE=dev.Dockerfile; \
fi; \
docker buildx build \
--build-arg BASE_IMAGE="$${IMG%:*}" \
--build-arg BASE_IMAGE_VERSION="$$TAG" \
-t $(LNDINIT_IMAGE) -f "$$DOCKERFILE" "$$CTX"
Comment thread
ZZiigguurraatt marked this conversation as resolved.

#? docker-dev-lndinit-build-push: Build the lndinit dev image (via docker-dev-lndinit-build) and push $(LNDINIT_IMAGE). `docker login` to that registry must already be done.
docker-dev-lndinit-build-push: docker-dev-lndinit-build
@$(call print, "Pushing lndinit docker image $(LNDINIT_IMAGE).")
docker push $(LNDINIT_IMAGE)

scratch: build


Expand Down Expand Up @@ -551,4 +612,7 @@ clean-docker-volumes:
android \
mobile \
clean \
clean-docker-volumes
clean-docker-volumes \
docker-dev-build \
docker-dev-lndinit-build \
docker-dev-lndinit-build-push
40 changes: 37 additions & 3 deletions docs/DOCKER.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,45 @@ There are two flavors of Dockerfiles available:

## Development/testing

To build a standalone development image from the local source directory, use the
following command:
To build a standalone development image from the local source directory:

```shell
$ docker build --tag=myrepository/lnd-dev -f dev.Dockerfile .
$ make docker-dev-build
```

The image is tagged `lnd-dev:<short-hash>` by default (with a `-dirty`
suffix if the working tree has uncommitted changes); override with
`make docker-dev-build DOCKER_DEV_IMAGE=myrepository/lnd-dev:mytag`.

To additionally build an `lndinit` image layered on top of the dev image
(pulling `lndinit`'s `dev.Dockerfile` from upstream main):

```shell
$ make docker-dev-lndinit-build
```

The image is tagged `<LNDINIT_REPO>:lnd-dev-<short-hash>` (e.g.
`lndinit:lnd-dev-f5a093c1f`) so the tag makes it obvious which
`docker-dev-build` image it was layered on. `LNDINIT_REPO` defaults to
`lndinit`; override to push to a registry, e.g. `myrepository/lndinit`.
`LNDINIT_CONTEXT` is the build context that `docker buildx` pulls
`lndinit`'s `dev.Dockerfile` from — defaults to
`https://github.com/lightninglabs/lndinit.git#main`, but can be a local path
(e.g. `../lndinit`) or any other git URL with an optional `#<ref>` suffix
(branch, tag, or commit SHA). For example, to build lndinit from a local
checkout under the `myrepository/lndinit` repo name:

```shell
$ make docker-dev-lndinit-build \
LNDINIT_REPO=myrepository/lndinit \
LNDINIT_CONTEXT=../lndinit
```

To build the `lndinit` image and push it to a registry you're already logged
into in one step:

```shell
$ make docker-dev-lndinit-build-push
```

There is also a `docker-compose` setup available for development or testing that
Expand Down
13 changes: 13 additions & 0 deletions docs/release-notes/release-notes-0.22.0.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,19 @@
the build context is a remote git repository because COPY layers are more
smartly compared to cache.

* [New `make` targets for `dev.Dockerfile`](https://github.com/lightningnetwork/lnd/pull/10912):
`docker-dev-build` builds a development image from `dev.Dockerfile`
(tagged `lnd-dev:<short-hash>` by default; override with
`DOCKER_DEV_IMAGE=`). `docker-dev-lndinit-build` layers an
[`lndinit`](https://github.com/lightninglabs/lndinit) image on top of it
(using `lndinit`'s upstream `dev.Dockerfile`; tagged
`<LNDINIT_REPO>:lnd-dev-<short-hash>` so the tag reflects the underlying
dev image — override the repo name with `LNDINIT_REPO=` and the build
context with `LNDINIT_CONTEXT=`). `docker-dev-lndinit-build-push` builds
and then pushes the lndinit image. All three go through
`docker buildx build` so BuildKit cache mounts are used. See
[`docs/DOCKER.md`](../DOCKER.md) for examples.

# Contributors (Alphabetical Order)

* bitromortac
Expand Down
Loading