Add .internal to internal-only hostnames

[nickubels]

ICANN is in the progress of reserving .internal for private use (see this closed consultation). As such I believe that this TLD would be a suitable addition to the list of internal addresses used in SubjectIsInternal.

I based this PR on 6668587 which added .home.arpa to the list.

[mholt]

Interesting. I guess I'm OK with this, but will wait for another team member to approve as well. @francislavoie or @mohammed90 ?

[francislavoie]

Last time we made a change here (adding .home.arpa) it caught out some users which were already using that domain with their own ACME server or something like that. But I don't have a strong opinion about it.

[mholt]

That's true; but if this motion is finalized then I imagine this'll be the right thing to do long-term, despite potential complications one-time.

I might leave this open until it becomes more finalized though. It looks like it goes to "further consideration" at this point, not something that is actually enacted yet.

@nickubels Maybe ping me to remind me about this later after it is finished. 🎗️

[nickubels]

Very good point about possibly breaking workflows and I agree that waiting on finalisation by the ICANN Board would be a smart move.

That prompted me to check if this was already scheduled to be discussed in a board meeting, and to my surprise the board discussed this on 2024/07/29 and approved it as resolution 2024.07.29.06:

Resolved (2024.07.29.06), the Board reserves .INTERNAL from delegation in the DNS root zone permanently to provide for its use in private-use applications. The Board recommends that efforts be undertaken to raise awareness of its reservation for this purpose through the organization's technical outreach.

Theres currently a draft for a RFC: https://datatracker.ietf.org/doc/draft-davies-internal-tld/

While looking around for more information I also stumbled upon RFC 2606 and RFC 6761 which mention the reserved TLDs .test, .example, .invalid and .localhost. The latter is currently already included, but the first three not yet.

I believe that it would be a good idea to add those missing three to create a consistent experience for all four TLDs mentioned in RFC 2606.

For .internal the following question arises from my side: Is the approval of the ICANN Board enough for inclusion, or do we wait until a/the RFC is formalised?

[mholt]

This was discussed previously, I think it was caddyserver/caddy#2006 -- the main reason we were conservative with what domains/TLDs we make internal is because it's an implicit default that's difficult/annoying to override in configuration.

For example, some people have locally-deployed ACME CAs that may issue certs for .test or even .internal, and using Caddy's self-managed CA for that might be a frustration.

[mholt]

Looks like this went forward, and I don't think there will be much conflict with existing systems (if there is, there's config-arounds possible).