[RFC 0151] NixOS port allocator

[lucasew]

Rendered

[bjornfor]

Another alternative: just keep track of ports and emit eval error on conflict (no local state).

[lheckemann]

This RFC is now open for shepherd nominations!

[lucasew]

I think I found a bug. Testing lowering the upper range.

https://www.thegeekdiary.com/which-network-ports-are-reserved-by-the-linux-operating-system/amp/

[lheckemann]

The default range of ephemeral ports is 32768 to 60999, and using ports in this range for explicit binding can cause "funny" race conditions when an application connecting to a network port happens to occupy the chosen port. Using ports below 32768 is probably better for this reason.

[kevincox]

Maybe we can tie the default range to the sysctl setting for the ephemeral port range.

[lucasew]

Found out that it's net.ipv4.ip_local_port_range

Cool, #TIL

[lucasew]

It's not nix defined by default so the implementation will need a fallback

Default value is

sysctl net.ipv4.ip_local_port_range
net.ipv4.ip_local_port_range = 32768    60999

[szlend]

Another drawback of this approach is that port numbers are unstable, meaning ports of existing services will change whenever a new service's port is registered, leading to unnecessary restarts of completely unrelated services. I don't have a suggestion on how to address that, just something to note.

[szlend]

I guess it could be mostly mitigated by hashing the values within the available port range. Collisions are very likely in a range as small as this though, so we'd need to be pretty clever about handling those.

[lucasew]

Another drawback of this approach is that port numbers are unstable, meaning ports of existing services will change whenever a new service's port is registered, leading to unnecessary restarts of completely unrelated services. I don't have a suggestion on how to address that, just something to note.

I know, I am having this problem lol

[lucasew]

I guess it could be mostly mitigated by hashing the values within the available port range. Collisions are very likely in a range as small as this though, so we'd need to be pretty clever about handling those.

Well, I think nix has a builtin to hash strings, maybe if I can get the first n letters then use a remainder operator...

[szlend]

If you wanna play around with it, here's a very naive way to map ports:

let
  pkgs = import <nixpkgs> { system = builtins.currentSystem; };
  lib = pkgs.lib;

  inherit (builtins) fromTOML fromJSON toJSON readFile hashString substring;
  inherit (lib) mod;
  inherit (lib.lists) foldl;

  portFor = start: len: hash: start + (mod (fromTOML "x=0x${substring 0 8 hash}").x len);
  portForLocalRange = portFor 32768 28230;

  assignPort = ports: name: hash:
    let
      newHash = hashString "sha1" hash;
      port = portForLocalRange newHash;
    in
    if ports?${toString port}
    then assignPort ports name newHash
    else ports // { ${toString port} = name; };

  data = map (x: x.value) (fromJSON (readFile ./data.json));
in
toJSON (foldl (ports: name: assignPort ports name name) { } data)

$ nix eval --raw -f ports.nix | jq
{
  "32775": "odit-sunt-quidem-voluptatem-ut-harum-et-hic",
  "32798": "voluptates-quis-corrupti-rerum-ullam-sunt",
  "32803": "autem-ab-eum-error-tempore-ut",
  "32807": "aspernatur-qui-omnis-omnis-temporibus-est-eum",
  "32820": "doloremque-quidem-dolorem-occaecati-consequatur-consequatur-possimus-quia-voluptas",
  ...
  "60937": "sed-vero-est-dolores-aspernatur",
  "60960": "ducimus-voluptas-vel-libero-ut-tempore-nobis-magni-labore",
  "60975": "aspernatur-ipsam-earum-beatae-quibusdam",
  "60982": "aut-delectus-totam-labore",
  "60989": "ipsum-voluptas-omnis-magnam-officiis-nam"
}

[szlend]

You can't avoid state while also having stable port numbers. Even if you try to make it stateless with hashing, you'll run into problems when you have collisions.

Yep. Still, if the port registration was lazy (only registered when you actually enable a service), the number of running services would be small in practice so collisions would be very unlikely and they could be handled with some form of backup strategy. This still feels like a weird assumption for an official module though.

Using state is probably more reasonable, though you'd preferably want it at eval-time so you can actually reference the ports without having to resort to runtime wrappers. Not sure how you could do this without resorting to --impure or sandbox-paths though.

I guess if you manage the state file manually (i.e., just append names to it), you technically have automatic port allocation.

[lucasew]

Well, in case of a collision, it's possible to implement a fallback strategy that sums 1 to one of the conflicting ports until there is no conflicts. Seems like a good balance between not triggering a restart and avoiding conflicts.

[kevincox]

You can't avoid state while also having stable port numbers.

It's worth noting that you can have mostly stable port numbers with consistent hashing and some simple collision handling. Yes, if a collision occurs at least one port will change but the average server won't have too many services so collisions will probably be somewhat rare.

I also think it is important to have the ports available at eval time. It is frequently important to configure a reverse proxy, firewall or other config files.

[roberth]

For collision handling, hash tables are relevant, and open addressing in particular: https://en.wikipedia.org/wiki/Hash_table#Open_addressing

You could consider a concept of "priority", which, translated to hash table language, means that entries with high priorities are inserted first, so they get the best spots and are most stable: it takes an entry with a higher priority to cause it to move ports (existence of a colliding entry/entries with an equal or higher priority in either the old or new configuration).

[kevincox]

I'm not sure that we really want priority. It seems overly complicated and error prone. I was thinking sort of two levels of priority. "Required" and "Random". "Required" gives an error on conflict with another "Required" and "Random" is just consistently hashed. I considered "Preferred" but this just seems like it will lead to people assuming that they get the preferred port then causing issues if it gets reassigned, if it is free to be shuffled I think it is best to never pretend to have some sort of stable preferred value.

I wrote more about this here: #151 (comment)

[JulienMalka]

I was discussing the problem of [stable, collision-less and stateless] port assignment with @RaitoBezarius and he suggested that we may take advantage of the fact that the local ip range is 127.0.0.0/8 and instead of allocating ports, we could allocate pairs of (ip, port). This way, instead of working with a space of size ~30000 we can work in a space of size 30000*2^24~10^12. The probability of collision in this space, given the number of services likely to be run on one machine becomes negligible.
This would probably necessitate some changes to this proposition. One that comes in mind is that the module should then export not only the port but also the local ip used so that the value can be used to configure the reverse proxy. Another thing that comes in mind is that I don't know if some services may not offer the option to bind to another ip than 127.0.0.1.

[kevincox]

I think we probably need both. Ideally you could allocate on various IPs. Particularly it would be interesting to allocate on ::0 for public services. However it is also interesting to allocate on localhost for internal services. The allocator could run separately on different IPs. (Although services that bind to all IPs will need to transcend them).

I don't know if some services may not offer the option to bind to another ip than 127.0.0.1.

I think this is quite rare because most services allow you to bind to a specific public IP. So the same code will "accidentally" support binding to a specific internal IP.

[JulienMalka]

I think we probably need both. Ideally you could allocate on various IPs. Particularly it would be interesting to allocate on ::0 for public services. However it is also interesting to allocate on localhost for internal services. The allocator could run separately on different IPs. (Although services that bind to all IPs will need to transcend them).

My point was precisely to allocate on localhost. But localhost is not only 127.0.0.1 but the whole 127.0.0.1/8 range.

[RaitoBezarius]

I was discussing the problem of [stable, collision-less and stateless] port assignment with @RaitoBezarius and he suggested that we may take advantage of the fact that the local ip range is 127.0.0.0/8 and instead of allocating ports, we could allocate pair of (port, ip). This way, instead of working with a space of size 30000 we can work in a space of size 30000*2^2410^12. The probability of collision in this space, given the number of services likely to be run on one machine becomes negligible. This would probably necessitate some changes to this proposition. One that comes in mind is that the module should then export not only the port but also the local ip used so that the value can be used to configure the reverse proxy. Another thing that comes in mind is that I don't know if some services may not offer the option to bind to another ip than 127.0.0.1.

I would also add that IPv6 ULA /48 could also be used this way, which would provide 30_000 * 2^80 if needed at all, for people like me, who prefers to bind over IPv6 anyway.

[Ma27]

Isn't that something you can achieve by using config.services.yourservice.settings.port everywhere (or something similar)? Port collisions of service-ports (i.e. 3000/8000/etc) are - in my experience at least - relatively rare, at least I can't actually recall a single time where I got bitten by this.

[lucasew]

Yes. Now you can set ports but sometimes you don't care which port as long as the two ends are using the same one.

[Ma27]

Considering the wording in here, this is supposed to be opt-in, correct? I.e. this won't be used (or enforced to be used) in other upstream NixOS modules itself, correct?

[lucasew]

Yes. The allocator is opt-in. I specially use for services that are behind nginx so I don't need to think about which port is being used by each service (I got a few weird errors and conflicts before that and wanted to keep ports such as 3000 and 5000 free).

[kevincox]

I wonder if it makes sense to use it by default, however in this case it would probably need to support a "preferred port". I'm thinking something like this:

1. Services set a "preferred port" and register with the allocator.
2. Upon collision all but one of the colliding services are reassigned.
3. There is an option to force a specific port for a service (and get an eval error if there is a conflict).

So the port request options would be something like:

• Prefer(port)
• Require(port)
• NoPreference

The downside would be that if you enable a new service it may shift an existing service which could cause issues if you were relying on the default. (For example you were running Mumble and connecting to mumble://yourhost.example without a port) This could cause runtime issues after switching to the new configuration. This is probably worse than an switch-time failure to start the new service. However for the reverse-proxy case it makes a lot of sense to have this behaviour as you can configure your reverse-proxy using something like config.service.foo.port and it would auto-update.

In fact maybe it would make sense to use the port allocator for all services by default but use Require(port). That way 1. The port allocator won't assign that port to another service and 2. You will get an eval-time error if you enable two services that use the same port. Then the user could configure an override with a different port or with NoPreference. New services that bind to an internal IP could maybe considering using NoPreference by default as it won't cause any backwards incompatibility.

[Ma27]

If this should be actually used by default, another thing to check whether (and how much) this will increase the expensiveness of evaluation a NixOS system.

[lucasew]

I was discussing the problem of [stable, collision-less and stateless] port assignment with @RaitoBezarius and he suggested that we may take advantage of the fact that the local ip range is 127.0.0.0/8 and instead of allocating ports, we could allocate pair of (port, ip). This way, instead of working with a space of size 30000 we can work in a space of size 30000*2^2410^12. The probability of collision in this space, given the number of services likely to be run on one machine becomes negligible. This would probably necessitate some changes to this proposition. One that comes in mind is that the module should then export not only the port but also the local ip used so that the value can be used to configure the reverse proxy. Another thing that comes in mind is that I don't know if some services may not offer the option to bind to another ip than 127.0.0.1.

I would also add that IPv6 ULA /48 could also be used this way, which would provide 30_000 * 2^80 if needed at all, for people like me, who prefers to bind over IPv6 anyway.

Yea, but lib.types.port only accepts 16 bits numbers, no full hosts, so I am afraid this will be out of scope.

And one single machine will very unlikely host more than a few hundreds of services. I don't think it will be ever a problem, at all.

[RaitoBezarius]

This is beyond the point, we usually have a listen host option, both should be used. Or a special listen attr should be standardized. Le dim. 25 juin 2023 à 14:21, Lucas Eduardo ***@***.***> a écrit :

…

I was discussing the problem of [stable, collision-less and stateless] port assignment with @RaitoBezarius <https://github.com/RaitoBezarius> and he suggested that we may take advantage of the fact that the local ip range is 127.0.0.0/8 and instead of allocating ports, we could allocate pair of (port, ip). This way, instead of working with a space of size 30000 we can work in a space of size 30000*2^2410^12. The probability of collision in this space, given the number of services likely to be run on one machine becomes negligible. This would probably necessitate some changes to this proposition. One that comes in mind is that the module should then export not only the port but also the local ip used so that the value can be used to configure the reverse proxy. Another thing that comes in mind is that I don't know if some services may not offer the option to bind to another ip than 127.0.0.1. I would also add that IPv6 ULA /48 could also be used this way, which would provide 30_000 * 2^80 if needed at all, for people like me, who prefers to bind over IPv6 anyway. Yea, but lib.types.port only accepts 16 bits numbers, no full hosts, so I am afraid this will be out of scope. — Reply to this email directly, view it on GitHub <#151 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACMZRBXKHCMB4ULUWGJIDLXNAULPANCNFSM6AAAAAAZBV5HPM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[kevincox]

Suggested change

	request a port by defining `networking.ports.service.enable = true` and get the
	allocated port by referring to `networking.ports.service.port`. The service doesn't
	request a port by defining `networking.ports.${service}.enable = true` and get the
	allocated port by referring to `networking.ports.${service}.port`. The service doesn't

On first read it wasn't clear to me that service was a placeholder. It looks similar to services.foo or systemd.services.foo where it is a literal. I think it would be good to make it more clear.

Alternative suggestions are networking.ports.foo, networking.ports.someservice , networking.ports.${service-name}, networking.ports.*, networking.ports.SERVICE. I think either of these would be more obviously a placeholder.

[kevincox]

To sum up some of the other threads into an alternative proposal/adjustments to the original proposal.

Different Modes

The current proposal only deals with automatically assigned ports. This means that it is unaware of static ports on the system. I think it may be better to make the port allocator (maybe better called a "manager" now) know about all ports to make its decision.

I would add two or three modes to the allocator

1. { port = 1234 }. This is required mode In this mode the specific port is reserved for this service. It will not be assigned to another service. An error is raised at eval time if two services require the same port.
2. {} This is random mode In this mode an otherwise non-allocated port is assigned.
3. { mode = "prefer"; port = 1234 } The port will be assigned to one of the preferred mode services if not taken by a required mode service. If the port is taken by a required mode or another preferred mode service it is given a port as if it was random mode. (IDK if this is necessary, maybe we can skip it for the first pass. The main benefit is aesthetics but it raises the chance of a re-assignment causing a problem if people are relying on the preferred port. It can always be added later.)

Most notably mode = "require" allows existing modules to use the port allocator and remain backwards compatible. This provides good error messages on conflict and avoids collisions between these services and randomly assigned ports. I would recommend migrating all existing services to mode = "require".

Mostly Stable Assignment

I would update the assignment algorithm to be mostly stable. This reduces disruption when changing enabled services. In the vast majority of changes there would be no reassignment. In cases where there would have been a collision only a small number of services (hopefully 1) would change ports.

The most likely method of doing this would be consistent hashing using the port name. On collision a new hash would be run to pick an alternate port.

Reserved Ranges

I think the allowed or reserved ranges should be customizable. By default we should probably exclude 0-1024 (root-only) and the ephemeral port range.

Different IPs

It would also be nice to be able to reserve ports separately on different IPs. This is probably a nice-to-have and can be implemented later. If I reserve some ports on 127.0.0.1 it would be nice if I can also use those on my public IP. Maybe we could even assign IP addresses out of a range like 127.0.0.0/8. But this is probably fairly niche. In most cases assigning ports globally is unlikely to run out and is likely simpler when trying to debug issues on a system as there should be no port reuse so you can mostly ignore IPs.

This would be forwards-compatible to add later. Ports that don't specify an IP are assumed to use :: (and collide with all ports on all addresses).

networking.ports.foo = {
  address = "127.0.0.8";
};

Schema

My current thought is the flowing schema:

networking.ports.assignmentRange.start

The first port number that is eligible for assignment. Defaults to 1.

networking.ports.assignmentRange.end

The first port number that is not eligible for assignment. Defaults to 2^16.

networking.ports.reservedRanges

List of {start = 0; end = 1024} ranges that won't be assigned from. Defaults to [{start = 0; end = 1024} {start = ephemeral_start; end =- ephemeral_end}

networking.ports.port.*

Attrset of ports. Presence indicates that a port should be allocated. (I can't think of a use case to disable, it would likely cause an eval failure for whatever reads the port). This could likely be proxied via services.*.port or similar in many cases.

networking.ports.port.*.address

(Possible future addition). Defaults to ::.

networking.ports.port.*.mode

(Possible future addition). Used if port is set. Enum of "required" or "preferred". Defaults to "required".

networking.ports.port.*.port

Defaults to null. If set this port is fixed and used only for avoidance for other assignments and raising an error on conflict between manually-assigned ports. This is also set by the allocator so that it can be referenced as config.networking.ports.port.foo.port when generating config files for dependent services.

[alyssais]

This should maybe mention socket activation. The canonical solution to running services behind nginx, without having to worry about allocating unique ports to them, is to have the service listen on a unix socket, and point nginx to that. (This also makes access controls easier.) This isn't supported by every service, which is where the value of this RFC comes in, but it's probably a better solution when it is supported.

[kevincox]

Nit: Socket activation is orthogonal to UNIX sockets. You can also do socket activation for IP-based sockets.

I agree that it is worth mentioning UNIX sockets as an alternative ("infinite namespace" prevents collisions, file permissions are useful for security and the performance is better) but I think the downsides of "not all services support UNIX sockets" and "doesn't support remote access" is enough justification for this RFC.

[RaitoBezarius]

Nit² : can't you mount a UNIX socket over the network? :-)

[kevincox]

You can? I've only heard of using something like netcat to proxy a TCP socket into a UNIX socket. Does NFS or something support that? How does file descriptor transfer work? (or does it just fail?)

...but I think we are getting off-topic now.

[alyssais]

Nit: Socket activation is orthogonal to UNIX sockets. You can also do socket activation for IP-based sockets.

I agree that it is worth mentioning UNIX sockets as an alternative ("infinite namespace" prevents collisions, file permissions are useful for security and the performance is better) but I think the downsides of "not all services support UNIX sockets" and "doesn't support remote access" is enough justification for this RFC.

I wouldn't quite call it orthogonal. Theoretically, yes, but in practice, it tends to be the only way to have an application designed for TCP to use a Unix socket.

I think the downsides of "not all services support UNIX sockets" and "doesn't support remote access" is enough justification for this RFC.

Yes, that's why I said it should be mentioned as an alternative considered rather than saying I opposed the RFC!

[roberth]

You could have a "hash function" that takes either a name or a port. If it's a port, it just returns the port.
Then use a high priority, and check that it was possible to allocate them at the desired individual port numbers, throwing an error if a collision happened anyway. (ie "required" mode mentioned before)

[lheckemann]

This RFC has not acquired enough shepherds. This typically shows lack of interest from the community. In order to progress a full shepherd team is required. Consider trying to raise interest by posting in Discourse, talking in Matrix or reaching out to people that you know.

If not enough shepherds can be found in the next month we will close this RFC until we can find enough interested participants. The PR can be reopened at any time if more shepherd nominations are made.

See more info on the Nix RFC process here

[lucasew]

I was thinking about now generalize this concept of allocation of some space. When you deal with NixOS containers you would want to put them in some network but don't need to allocate ports manually for each one.

We may have some kind of value domain like a port interval, static allocations like service X uses port Y so we make sure that this service will not conflict with the other ones.

I need to PoC this stuff. It will be a more elementar abstraction that you can use to create a generic allocator like a port allocator, it may give warnings that some stuff is not reserved or conflicts with something else.

Just a quick question: is IFD an option? Because of __structuredArgs. It may introduce some tradeoffs around evaluation time.

[RaitoBezarius]

I don't think IFD is an option but I'm not sure what is the relation with __structuredArgs or the tradeoffs with evaluation time, it's already quite high so…

[lucasew]

I don't think IFD is an option but I'm not sure what is the relation with __structuredArgs or the tradeoffs with evaluation time, it's already quite high so…

This way the validation can be implemented with another programming language instead of Nix.

Like a conflict solver.

__structuredArgs would serve like the input data and the derivation will give a json file with the results.

[roberth]

Such a program would not be able to maintain state or communicate with the involved system, so IFD doesn't change the equation.

__structuredArgs doesn't permit anything truly new; only makes toJSON a bit less necessary and enables bash arrays and such.

[lucasew]

Rescoped and moved to #159

[bew]

Hi, I very much like what is being discussed here, and the idea of a general purpose allocator system that would allow to do these things for multiple problem spaces like suggested in #159 (although it looks like what @lucasew is actually proposing is more for making problem space static checker, but not really for making a generic allocator..)

What could be the next steps towards an actual generic allocator (if @lucasew doesn't want to go this way) ?
Making yet another RFC ?

[lucasew]

Hi, I very much like what is being discussed here, and the idea of a general purpose allocator system that would allow to do these things for multiple problem spaces like suggested in #159 (although it looks like what @lucasew is actually proposing is more for making problem space static checker, but not really for making a generic allocator..)

What could be the next steps towards an actual generic allocator (if @lucasew doesn't want to go this way) ? Making yet another RFC ?

• How to check for conflicts?
• How to validate values?
• Which values to suggest?

I have a working PoC for the port allocator that I already daily drive but the abstraction is still not a one size fits all.

I think building a list of other use cases is a good start.

[bew]

Which values to suggest?

My view (and the view of the people who recently commented here if I'm not mistaken) is NOT to suggest values that you then have to write statically in the config, but to infer from a specification, compute the values instead when needed, based on contraints.

How to validate values?

What do you mean?
(I'm thinking about a function that can be given to validate something, but I fear I'm not understanding the question correctly)

How to check for conflicts?

That would be the role of a checker, that can do a pass on the generated values to verify invariants? (If I'm understanding the question correctly?)
Or are you asking about the algorithm to calculate the ports, making sure they don't conflicts? If yes I don't know yet

---

Thinking a bit more, I think the two ideas are not conflicting, the generator could have a checker, or even more generic, the checker could be something that can be applied on all used NixOS options (is it feasible?), to validate things in a given problem space (e.g: the values of the generator for ports, or (silly example) path names generated in environment.etc.* match a pattern)

[lucasew]

Which values to suggest?

My view (and the view of the people who recently commented here if I'm not mistaken) is NOT to suggest values that you then have to write statically in the config, but to infer from a specification, compute the values instead when needed, based on contraints.

That's what I am doing. The reference implementation defines suggestedValue that is only used if something don't pass the three validations.

In the port case I have a range then generate the values from top to bottom.

How to validate values?

What do you mean? (I'm thinking about a function that can be given to validate something, but I fear I'm not understanding the question correctly)

It's exactly that. In the port case it's a function that uses lib.types.port.check and the range.

How to check for conflicts?

That would be the role of a checker, that can do a pass on the generated values to verify invariants? (If I'm understanding the question correctly?) Or are you asking about the algorithm to calculate the ports, making sure they don't conflicts? If yes I don't know yet

Thinking a bit more, I think the two ideas are not conflicting, the generator could have a checker, or even more generic, the checker could be something that can be applied on all used NixOS options (is it feasible?), to validate things in a given problem space (e.g: the values of the generator for ports, or (silly example) path names generated in environment.etc.* match a pattern)

mkAllocOption returns a submodule that you can mount in your module system. I think that a "system wide" validation using already defined modules would cause an combinatory explosion. Nix is slow already to evaluate stuff and I don't want to make it worse.

BTW the conflict checking tends to be $O(n^2)$ because you need to check if the previous value has a conflict to then check the next one, it can be even worse if there is some kind of subdivision between values (a subnet allocator maybe). I am biased to keep it simple tho, my conflict resolution strategy so far is to check if a key exists in a attrset but I know it will be leaky for some less trivial usecases.

Nix seems to have the primitives to do tail call optimization for big recursions using builtins.genericClosure [1] but it seems I am not smart enough yet to really grok this stuff.

[1] https://discourse.nixos.org/t/tail-call-optimization-in-nix-today/17763

[lucasew]

BTW you can see in the history "ah what if it also allows you to define ports in loopback IPs". I think that's scope creep. If that would really matter, one can always define their own module using the same primitives.

[bew]

That's what I am doing.

I still don't get it.. 😬 What are you doing exactly?

The reference implementation defines suggestedValue that is only used if something don't pass the three validations.

In your impl, I see that suggestedValue is used in suggestion messages, but nowhere else. So I would have to change my nixos code to make the code evaluate, the port values are not computed at eval time randomly/algorithmically based on a key/etc..

In your examples, you mention about the allocator: Enable automatic port allocation for service ${name}

But what is automatic about the port allocation in this case, if in the end you have to change some values in the config ?

[lucasew]

Initially I did it fully automatic but the problem with that is the shifting problem. As attrsets are always ordered, adding a new service would trigger an unecessary rebuild and restart of other unrelated ones.

This "I want a port" -> "Suggest port" -> "Confirm port" approach was to solve this problem.

If any conflict happens then it just suggest that you change the port of service X to y, and because it suggest only at most one value it's not necessary to process more than one value.

[kevincox]

Note that assignment via consistent hashing has been discussed (example #151 (comment)) and would provide very good stability (rarely moving a single port) and would likely serve as a decent default for services that don't otherwise have a good default.

[lucasew]

Note that assignment via consistent hashing has been discussed (example #151 (comment)) and would provide very good stability (rarely moving a single port) and would likely serve as a decent default for services that don't otherwise have a good default.

I am biased to opt for simpler solutions. The suggestion (succ) function can use hashing optionally to avoid the case that a opt-in module conflict with other existing module.

I think nothing will beat the stability of defining values explicitly, at least we will not get service restarts just because the port changed but the suggestion mechanism can be a bit smarter with this hashing scheme, even more when modules ask for sequence of ports. or the resource space being allocated.

[kevincox]

I'm concerned that the simpler implementation may just be pushing the complexity onto users. I have some modules that are used on various machines and assigning non-conflicting ports is a pain. Especially if I'm using something like Kubernetes where I want to reserve a big port range on some of the servers.

I probably wouldn't recommend using assigned ports by default in NixOS modules but it would be nice to have this option, especially for user-defined services. The complexity would be completely contained in the port allocation module so it is hidden from the user. From the user POV it would be something like:

1. { services.foo.enable = true; services.bar.enable = true; }
2. Oh no, port allocator reports a conflict.
3. services.foo.port.port = "auto-assign";

Then problem solved forever. Or if the users prefer they can manually assign a port and evaluation will fail if they hit another conflict.

[lucasew]

I was trying to PoC this hash thing to see how it would work. I am trying to hash a key that comes from the service name using networking.ports but without having to explicitly set a port.

Unfortunately the bit conversion stuff (to generate a valid port from the hash) in nixpkgs is very incomplete so there is some implementation work that must be done before this can be acceptable in nixpkgs.

[edolstra]

This RFC has not acquired enough shepherds. This typically shows lack of interest from the community. In order to progress a full shepherd team is required. Consider trying to raise interest by posting in Discourse, talking in Matrix or reaching out to people that you know.

If not enough shepherds can be found in the next month we will close this RFC until we can find enough interested participants. The PR can be reopened at any time if more shepherd nominations are made.

See more info on the Nix RFC process here

[lucasew]

I just did the hash based implementation for the allocator

[kevincox]

This RFC is being closed due to lack interest. If enough shepherds are found this issue can be reopened. If you don't have permission to reopen please open an issue for the NixOS RFC Steering Committee linking to this PR.