Assign random ports at configuration start (not later)

[radcortez]

Description

When we set a port configuration like quarkus.http.port to 0, we want Quarkus to assign a random port.

The issue is the particular case of quarkus.http.port, the port generation and assignment are delegated to Vert.x. Why is this an issue?

Because we rely on the Config system to lookup the real quarkus.http.port, we have to mutate the configuration after the configuration is already loaded. We use SystemProperties, but this does not guarantee an override because there may be higher ordinal sources in the Config system that references quarkus.http.port.

Also, we need configuration to be loaded to start Vert.x. Once we load the configuration objects, we cache them for obvious reasons: for performance and to ensure everyone sees the exact copy of the configuration. This becomes another problem when we reference ${quarkus.http.port} as an expression, which at the time of evaluation will be set to 0.

We observed some of these issues in:

• System properties config overrides in tests does not seem to take effect properly in quarkus 3.14.1 #42844
• Move REST Client configuration to use @ConfigMapping #42106 (comment)
• Test profile config properties can bleed into other test profiles #27996
• Improve compatibility of the REST Client configuration #42932 (comment)

Implementation ideas

Ideally, we should move all random port assignments before loading the configuration.

[cescoffier]

Unfortunately, that's not possible. First it's not 0, but 0, -1, -2...

The actual ports are provided by the operating system when we bind to the socket. We cannot do that before.

Approaches that try "lease" ports at some point to use them later as kind of slow (you need to open a server, capture the port, close it) and you have no guarantee that the port will still be available when you start the actual servier.

[gsmet]

First it's not 0, but 0, -1, -2...

Not sure how it's actually a problem.

and you have no guarantee that the port will still be available when you start the actual servier.

So that yes, but my guess is that you would be quite unlucky to have a port of the random range unavailable between reading the config and starting the server.

https://unix.stackexchange.com/a/132524 seems to agree that it wouldn't be that much of a problem in practice.

Sure it could maybe happen but I'm not really sure it's worth having the kind of hacks that we have at the moment. They are costing us too.

I have no idea how slow it is to have a ServerSocket started and if it would be acceptable in practice when running tests (because that's the main use case here).

My problem with the current approach is that you cannot rely on quarkus.http.port. So another option might be that quarkus.http.port can't be consumed anywhere as not reliable. But I'm not sure it's going to make things easier for us.

Especially since it's often used in other configuration properties.

[cescoffier]

Well, I got bitten by this multiple times. Maybe it's an OS issue (I also have issues on CI with this). Multiple times, the ports "leased" got re-assigned before being actually used, which creates a lot of flakiness and friction.

[cescoffier]

Can we have a special config source or something that can retrieve free port using the (brittle) lease strategy? At least it's up to the user to use it and if it works everything is fine.

It would be something like:

%test.quarkus.http.port={$quarkus.random-port.my-first-port:0}

quarkus.random-port.my-first-port will be some special value resolved by opening a socket, retrieving the port, and seeing it back (don't forget to close the socket). It may not work everywhere and everytime, but it should be good enough.

We do something simular for quarkus.uuid if I'm not mistaken.

[radcortez]

Can we have a special config source or something that can retrieve free port using the (brittle) lease strategy? At least it's up to the user to use it and if it works everything is fine.

The annoying part is that users would have to move from setting quarkus.http.port=0 to this new property, correct?

How about we keep the socket open and release it when we know the consumer will need it?

[cescoffier]

You would open, capture the port and release immediately (be aware that it's async for some OS), and then inject the captured port (hopping 1) the port has not been used in between, 2) the socket has released all the resources)

But yes, they would have to use something different than quarkus.http.port=0.

[radcortez]

We could manage it internally. We can create a ConfigSourceFactory which takes the current configuration when the config is still initializing. If quarkus.http.port=0 we capture a random port and rewrite the new port number in a max ordinal source constructed by the factory in quarkus.http.port, so roots and mappings will see the new port assignment.