Skip to content

introduce hitch#8674

Merged
tianon merged 1 commit intodocker-library:masterfrom
gquintard:new_hitch
Apr 2, 2021
Merged

introduce hitch#8674
tianon merged 1 commit intodocker-library:masterfrom
gquintard:new_hitch

Conversation

@gquintard
Copy link
Contributor

@gquintard gquintard commented Sep 4, 2020

Hi,

Here's a new version of #7611, hopefully it'll be more aligned with the guidelines. It reuses a lot of the Varnish image experience and tries to fix the problems pointed out by #7611, notably, the command line is much easier to override and we are using a gpg fingerprint and the "right" keyservers. Oh, and we are not using the new project-built packages instead of the distro ones, so we can keep up with the releases.

Checklist for Review

NOTE: This checklist is intended for the use of the Official Images maintainers both to track the status of your PR and to help inform you and others of where we're at. As such, please leave the "checking" of items to the repository maintainers. If there is a point below for which you would like to provide additional information or note completion, please do so by commenting on the PR. Thanks! (and thanks for staying patient with us ❤️)

  • associated with or contacted upstream? (yes, I'm part of Varnish Software who maintains hitch)
  • does it fit into one of the common categories? ("service", "language stack", "base distribution") (yes, "service")
  • is it reasonably popular, or does it solve a particular use case well? (yes, provides up-to-date versions of the de facto TLS terminator used with Varnish)
  • does a documentation PR exist? (should be reviewed and merged at roughly the same time so that we don't have an empty image page on the Hub for very long) (Adding Hitch docs#1675)
  • dockerization review for best practices and cache gotchas/improvements (ala the official review guidelines)?
  • 2+ dockerization review?
  • existing official images have been considered as a base? (ie, if foobar needs Node.js, has FROM node:... instead of grabbing node via other means been considered?) (no, no relevant base image here)
  • if FROM scratch, tarballs only exist in a single commit within the associated history? (not FROM scratch)
  • passes current tests? any simple new tests that might be appropriate to add? (https://github.com/docker-library/official-images/tree/master/test)

@gquintard gquintard mentioned this pull request Sep 4, 2020
9 tasks
@gquintard gquintard force-pushed the new_hitch branch 4 times, most recently from 20d518f to 51b80cf Compare September 4, 2020 21:31
@gquintard gquintard marked this pull request as draft September 5, 2020 16:24
@gquintard gquintard marked this pull request as ready for review September 5, 2020 16:27
@ThijsFeryn
Copy link

The updated docs PR can be found here: docker-library/docs#1675

@gquintard gquintard mentioned this pull request Sep 29, 2020
@gquintard
Copy link
Contributor Author

@tianon , @yosifkit , sorry to bump this, but is there anything you need here?

@gquintard
Copy link
Contributor Author

Happy Thanksgiving!

I just pushed an update to this PR as we have released hitch 1.7.0 recently. Let me know if you want me to squash the commits

@tianon
Copy link
Member

tianon commented Dec 10, 2020

FWIW, you didn't do yourself any favors by opening a new PR -- the conversation is now split (having to refer back to that previous thread during my review), and you put yourself back at the bottom of the new-image queue. 😅 ❤️

I think some of my comments from #7611 (comment) are still relevant, but I'll try to recapture the salient bits here.


+	cp /etc/hitch/testcert.pem /etc/hitch/certs/default

Having this example certificate pair pre-baked into the image feels a little strange IMO -- what're the details of that certificate? How does it get regenerated? What's it valid for, how long, etc? Is it signed by any other party? (These are all questions that are hard to answer from just an opaque certificate.)

(The docs mention this is a self-signed certificate, but I think more details would probably still be helpful, especially since some systems like Mac OS are picky about the validity period of even self-signed certs.)


What user/UID does this end up running as? Does it matter for that to be consistent? (Is there any persistent storage for things like session/cache that might be affected if it were to change underneath a user?)

user = "_hitch"

If users accidentally omit this bit from their configuration files, will Hitch run as root instead, or does it have some kind of protection against that?

Is it worth considering using USER _hitch or similar instead to enforce this at a higher level? (Usually filesystem permissions and/or "privileged" ports are the reasons setting USER becomes a hassle.)

Did you want to create a dedicated hitch user separate from the one auto-created by the Debian package? (Think about whether that might be something you'll want to consider doing later too, because it's much easier on you to make that sort of user-breaking change now than after we merge this, especially if it ends up getting embedded in user's configuration files too.)

@gquintard
Copy link
Contributor Author

FWIW, you didn't do yourself any favors by opening a new PR -- the conversation is now split (having to refer back to that previous thread during my review), and you put yourself back at the bottom of the new-image queue. sweat_smile heart

Dagnabit! Sorry, I didn't realize that.

(The docs mention this is a self-signed certificate, but I think more details would probably still be helpful, especially since some systems like Mac OS are picky about the validity period of even self-signed certs.)

@ThijsFeryn, would you mind updating the docs to match that we now use the packaged default file, and tell us a bit more about it?

What user/UID does this end up running as? Does it matter for that to be consistent? (Is there any persistent storage for things like session/cache that might be affected if it were to change underneath a user?)

Without user configuration, the hitch user is used (https://github.com/varnish/pkg-hitch/blob/master/systemd/hitch.conf#L97) and is the same as the one created by the debian package, and the only directory we need for persistence will be /var/lib/hitch-ocsp. I think that is what we want.

My instinct here is that we don't need the USER hitch bit here as the process will drop permissions itself and this is a no-home user anyway, so we don't really want to exec as it either. But maybe I'm missing something here.

@gquintard
Copy link
Contributor Author

@tianon, @ThijsFeryn updated the docs PR to explain the test certificate: https://github.com/docker-library/docs/pull/1675/files#diff-001e280f2a3c2c5174bd4d733268cc2f2b4478f3baa6c5fc09ed6c5b7ee3d20cR43-R53

That should address all your concerns, right?

@gquintard
Copy link
Contributor Author

Hi team, anything you need us to do here?

@gquintard
Copy link
Contributor Author

bump @tianon, @yosifkit

@gquintard gquintard mentioned this pull request Mar 23, 2021
@tianon
Copy link
Member

tianon commented Mar 26, 2021

Generally, I think this looks OK. Thank you for your patience. 🙏

A few minor (non-binding) notes/suggestions:

  • apt-transport-https is a transitional package in Buster+, and thus not necessary 😄

  • if hitch creates any files on-disk (cache, etc -- /var/lib/hitch-ocsp?), you might still want to consider pre-creating the package's _hitch user so it can have a known, fixed UID/GID (because that will cause potential upgrade headaches later)

I'm a little confused at the entrypoint control flow -- if HITCH_CONFIG_FILE has a non-empty value, and there are no arguments (or the first argument starts with -, it'll run hitch --config=$HITCH_CONFIG_FILE ..., but if HITCH_CONFIG_FILE is empty in the same case, it'll run hitch /etc/hitch/hitch.conf ... (without --config=); is that the intentional usage?

It seems like maybe something like the following was intended?

#!/bin/sh
set -eu

# this will check if the first argument is a flag
# but only works if all arguments require a hyphenated flag
# -v; -SL; -f arg; etc will work, but not arg1 arg2
if [ "$#" -eq 0 ] || [ "${1#-}" != "$1" ]; then
	if [ -n "${HITCH_CONFIG_FILE:-}" ]; then
		# only add --config=... if HITCH_CONFIG_FILE was set and not empty
		set -- --config="$HITCH_CONFIG_FILE" "$@"
	fi

	set -- hitch "$@"
fi

exec "$@"

(Also, a rebase/squash of the final proposed commit would indeed be appreciated. 👍)

@gquintard
Copy link
Contributor Author

Hi!

So, the logic behind the entrypoint:

  • if you specify a command, we use it verbatim, no question asked
  • otherwise, we inject a --config argument, either using $HITCH_CONFIG_FILE or /etc/hitch/hitch.conf

For example, replacing exec with echo:

$ docker run hitch
hitch --config=/etc/hitch/hitch.conf

$ docker run hitch --some-arg
hitch --config=/etc/hitch/hitch.conf --some-arg

$ docker run hitch some-command
some-command

$ docker run -e HITCH_CONFIG_FILE=/some/other/config.conf hitch
hitch --config=/some/other/config.conf

$ docker run -e HITCH_CONFIG_FILE=/some/other/config.conf hitch --some-arg
hitch --config=/some/other/config.conf --some-arg

$ docker run -e HITCH_CONFIG_FILE=/some/other/config.conf hitch some-command
some-command

The goal was to save the user from setting HTCH_CONFIG_FILE in most cases, but maybe that's confusing?

Thanks for the apt-transport-https, I wasn't aware of it, I'm fixing it right away.

The UID/GID makes sense, I will change it too, is there a rule for picking these IDs, or will any random values do the trick?

@tianon
Copy link
Member

tianon commented Mar 26, 2021

  • if you specify a command, we use it verbatim, no question asked
  • otherwise, we inject a --config argument, either using $HITCH_CONFIG_FILE or /etc/hitch/hitch.conf

Ah, so what you're looking for is something even simpler, like this:

#!/bin/sh
set -eu

# this will check if the first argument is a flag
# but only works if all arguments require a hyphenated flag
# -v; -SL; -f arg; etc will work, but not arg1 arg2
if [ "$#" -eq 0 ] || [ "${1#-}" != "$1" ]; then
	set -- hitch --config="${HITCH_CONFIG_FILE:-/etc/hitch/hitch.conf}" "$@"
fi

exec "$@"

The UID/GID makes sense, I will change it too, is there a rule for picking these IDs, or will any random values do the trick?

We typically choose something in the high 900s because useradd --system will typically pick whatever number comes next in the low range, and non-system will typically start at 1000 and go up from there (so 999 is a common choice).

Another interesting convention we've seen is choosing the "default port" of the service, if it's a reasonable UID/GID that doesn't have a lot of potential for overlap, but given the nature of hitch I don't think that really works here. 😄

Given this image hasn't been published yet, you could also just choose 1000 directly, if you think that's reasonable -- the main goal is to avoid Debian UID/GID changes (however unlikely) from changing your Hitch user's UID/GID and breaking users unexpectedly. 😄

@gquintard
Copy link
Contributor Author

yeah, your code is indeed way more straightforward, I'll go with that.

I did think about going with 443 and expected some resitance, 999 has a nice ring to it though.

I'll push something this afternoon

@tianon
Copy link
Member

tianon commented Mar 26, 2021

I mean, 443 is high enough that it shouldn't have conflicts, and it's kind of cute 😏 -- your call 😄

@gquintard
Copy link
Contributor Author

fixed, rebased and force-pushed!

@tianon
Copy link
Member

tianon commented Mar 26, 2021

If you're using debhelper to build that hitch.deb, you should be able to pre-create the hitch user instead of modifying it post-install (so it's created with the expected UID from the very start), but either way you'll want to ensure the permissions on any hitch-owned directories are correct (notably /var/lib/hitch-ocsp).

I also just left a review with some more notes/discussion on the docs PR: docker-library/docs#1675 (review)

@gquintard
Copy link
Contributor Author

almost there! The packages are the generic ones, so we don't get to tinker with their creation. I didn't realize the OCSP directory was created by the package (I thought it was done at runtime), so I flipped things around and created the user before the package is installed.

Thank you for the docs review, it'll probably wait until @ThijsFeryn is back, but that is great to have your feedback on it

@github-actions
Copy link

Diff for 9ea4d0b:
diff --git a/_bashbrew-cat b/_bashbrew-cat
index bdfae4a..0b6f9fe 100644
--- a/_bashbrew-cat
+++ b/_bashbrew-cat
@@ -1 +1,5 @@
-Maintainers: New Image! :D (@docker-library-bot)
+Maintainers: Thijs Feryn <thijs@varni.sh> (@thijsferyn), Guillaume Quintard <guillaume@varni.sh> (@gquintard)
+GitRepo: https://github.com/varnish/docker-hitch.git
+
+Tags: 1, 1.7, 1.7.0, 1.7.0-1, latest
+GitCommit: d2feb9f1a1a3426da633383c2bac4a31559248bd
diff --git a/_bashbrew-list b/_bashbrew-list
index e69de29..2b55653 100644
--- a/_bashbrew-list
+++ b/_bashbrew-list
@@ -0,0 +1,5 @@
+hitch:1
+hitch:1.7
+hitch:1.7.0
+hitch:1.7.0-1
+hitch:latest
diff --git a/hitch_latest/Dockerfile b/hitch_latest/Dockerfile
new file mode 100644
index 0000000..d0ccdae
--- /dev/null
+++ b/hitch_latest/Dockerfile
@@ -0,0 +1,37 @@
+FROM debian:buster-slim
+
+ENV HITCH_VERSION 1.7.0-1~buster
+
+RUN set -ex; \
+	fetchDeps=" \
+		dirmngr \
+		gnupg \
+	"; \
+	apt-get update; \
+	apt-get install -y --no-install-recommends ca-certificates $fetchDeps; \
+	key=E35824BB706997D9184818E715A7ECE02FE19401; \
+	export GNUPGHOME="$(mktemp -d)"; \
+	gpg --batch --keyserver ha.pool.sks-keyservers.net --recv-keys $key; \
+	gpg --batch --export export $key > /etc/apt/trusted.gpg.d/hitch.gpg; \
+	gpgconf --kill all; \
+	rm -rf $GNUPGHOME; \
+	echo deb https://packagecloud.io/varnishcache/hitch/debian/ buster main > /etc/apt/sources.list.d/hitch.list; \
+	apt-get update; \
+	adduser --quiet --system --no-create-home --uid 443 --group hitch; \
+	groupmod -g 443 hitch; \
+	apt-get install -y --no-install-recommends hitch=$HITCH_VERSION; \
+	apt-get purge -y --auto-remove -o APT::AutoRemove::RecommendsImportant=false $fetchDeps; \
+	rm -rf /var/lib/apt/lists/*; \
+	mkdir /etc/hitch/certs/ /var/lib/hitch/; \
+	cp /etc/hitch/testcert.pem /etc/hitch/certs/default; \
+	sed -i 's/daemon = on/daemon = off/' /etc/hitch/hitch.conf
+
+WORKDIR /etc/hitch
+
+COPY docker-hitch-entrypoint /usr/local/bin/
+
+ENTRYPOINT ["docker-hitch-entrypoint"]
+
+EXPOSE 443
+
+CMD []
diff --git a/hitch_latest/docker-hitch-entrypoint b/hitch_latest/docker-hitch-entrypoint
new file mode 100755
index 0000000..29eaae7
--- /dev/null
+++ b/hitch_latest/docker-hitch-entrypoint
@@ -0,0 +1,11 @@
+#!/bin/sh
+set -e
+
+# this will check if the first argument is a flag
+# but only works if all arguments require a hyphenated flag
+# -v; -SL; -f arg; etc will work, but not arg1 arg2
+if [ "$#" -eq 0 ] || [ "${1#-}" != "$1" ]; then
+    set -- hitch "--config=${HITCH_CONFIG_FILE:-/etc/hitch/hitch.conf}" "$@"
+fi
+
+exec "$@"

@tianon tianon merged commit acc7574 into docker-library:master Apr 2, 2021
@gquintard gquintard deleted the new_hitch branch April 2, 2021 22:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants