Use dedicated port for temporary server

[abitrolly]

This allows pg_isready healtheck to wait until database is initialized

#880 (comment)

[tianon]

This is definitely a breaking change -- we use the default port number specifically so that any tools/scripts users might already have for initializing their databases work as-is without any major changes, and just ensure that the server doesn't listen externally (meaning TCP) until after initialization is successful. Using --host 127.0.0.1 with pg_isready or psql for checking the status of the container from within it is going to be the safed/best supported option (see https://github.com/docker-library/healthcheck/blob/master/postgres/docker-healthcheck for an example that does exactly that).

[abitrolly]

@tianon the problem is that at the time server starts, the initialization is not complete - there is no database, and no SQL schema, so the healthcheck will succeed, but application accessing the server will fail.

[abitrolly]

Neither pg_isready, nor psql is enough to know that initialization is finished. They can only check that a database is created, but not that it is populated. Having a dedicated port for initialization decouples init step from run step for good.

How about switching the logic? Allow to set PGINITPORT for those who need reliable healthchecks?

[yosifkit]

Sorry for the delay, but @tianon's comment already gives a way for a healthcheck to reliably know if the postgres server is finished initializing: --host 127.0.0.1. If you tell pg_isready or psql to connect on something other than the the socket file, it will correctly fail until initialization is done and the remote-available server is started.

https://github.com/docker-library/healthcheck/blob/40afbf64d69cf933af0da4df6383958a29113601/postgres/docker-healthcheck#L10-L11

The temporary server for initialization is explicitly started without any network listening for exactly this purpose:

postgres/docker-entrypoint.sh

Lines 257 to 270 in cdd56d3

	docker_temp_server_start() {
	if [ "$1" = 'postgres' ]; then
	shift
	fi
	# internal start of server in order to allow setup using psql client
	# does not listen on external TCP/IP and waits until start finishes
	set -- "$@" -c listen_addresses='' -p "${PGPORT:-5432}"
	PGUSER="${PGUSER:-$POSTGRES_USER}" \
	pg_ctl -D "$PGDATA" \
	-o "$(printf '%q ' "$@")" \
	-w start
	}

[abitrolly]

@tianon it appears what we are talking different things when speaking about "finished initializing". For me DB finished initializing when all initialization scripts had passed and dumps had restored.

Given that Docker starts entrypoint + cmd process in 1 minute and initialization script takes 5 minutes, when the healtheck passes according to your advice?

[yosifkit]

Given that Docker starts entrypoint + cmd process in 1 minute and initialization script takes 5 minutes, when the healtheck passes according to your advice?

If you use pg_isready --host 127.0.0.1 ... or psql --host 127.0.0.1 ..., then they will not be able to connect until the scripts are done and the "real" server started, i.e. after 6 minutes.

[abitrolly]

@yosifkit and what if 127.0.0.1 is my DATABASE_URL?

[abitrolly]

Ok. I don't remember the original problem. Probably the problem was that after initialization DB needed to be locked to run data migrations, and only after that made available for requests. The dedicated port could probably solve it, but then I am not sure how the main port should be enabled, as DB server doesn't know when migrations are finished.