RFC 2317 support for /25 and longer IPv4 subnet masks

[peteeckel]

I had a discussion with a friend yesterday, and he made me aware of a special case for PTR record management. I thought I'd ask around whether there is broad demand for this before thinking of a solution (but unfortunately couldn't help doing so anyway before I asked, so I'll share my ideas as well).

The situation is that there are many people and companies who do not have a complete byte-boundary (IPv4) or nibble-boundary (IPv6) subnet delegated to them. I guess the IPv6 problem is much less common, but it is possible that there are actually address assignments of /63 or similar atrocities in the IPv6 world as well - Providers surprise me again and again.

So let'a assume you don't have a full /24 IPv4 subnet assigned to you, and so classic reverse zone delegation does not work - you don't get the in-addr.arpa zone assigned to you, so the current implementation of PTR auto generation within NetBox DNS won't work.

The solution his provider uses for this is quite simple: They assign a non-standard reverse zone to their customers, and then they create CNAMEs to specific records in that zone in the actual in-addr.arpa zone. So the customer manages the PTR records in some other zone (let's call it 16.168.192.ptr.example.com for the sake of this example, but the name is arbitrary), and then CNAME records in the actual reverse zone on the provider's name servers point to entries within that zone, e.g. 17.168.192.in-addr.arpa is a CNAME for 17.16.168.192.ptr.example.com (no worries, I will not post an IPv6 example :-)).

So the question is: How can we provide automatic PTR generation in such a case? And what variants of that schema are floating around? Is that a frequent use case at all, or is it something that is only relevant as an edge case? Honestly I have no idea.

A possible solution within NetBox DNS would be a special type of zone that is flagged as a reverse zone for a given prefix, although it is not one in the classical sense. The information that would be needed is:

1. What prefix should it serve as a reverse zone for? That could be done in CIDR notion for both IPv4 and IPv6, so that a simple check can determine whether or not a given IPv4/v6 address falls within that zones authority. If it does, an entry will be generated, if not, there is no PTR record (and probably a warning about having assigned an address outside the permitted range).
2. What format do the entries have? The canonical way would probably be the suffix of the IP address within the prefix, in the above example that would be 17 for 17.16.168.192.ptr.example.com. but maybe there are other formats in use as well?

Currently, the only solution is to create the PTR records manually, but that's not really satisfying. Should this be something to look into more deeply, or is it such an edge case within the group of possible uses that it does not have high priority?

[LuPo]

If this can help, this specific use case is described in depth in rfc2317 (Classless IN-ADDR.ARPA delegation)

In our institution we are using longer prefixes (>/24) delegated to DNS servers still managed internally (not to outside customer). It serves the purpose to assign address to different zones in the dyndns registration. It is not much common but still used in IPv4. With Ipv6 we will not be using anything longer than /64
Currently netbox-dns places automatic PTR in the longer prefix as I would expect. With IPv4 it is true up to /24. Perhaps the lookup should also be extend to include search in something like 0/26.168.192.in-addr.arpa if this zone is not flagged as delegated to external entities?

[peteeckel]

@LuPo: Thanks, that's an interesting approach - so in fact the subnet-aware ip-addr.arpa reverse zones still have to be delegated to by the still have to be CNAMEd from a subnet-aligned reverse zone in order to work with real world-clients. So to really cover the issue, NetBox DNS could be extended to have an option to create RFC2317 zones for IPv4.

IMHO it doesn't make any sense with IPv6, and we also only need to consider prefixes with more than 24 bits, as all other cases can be solved by providing multiple /24 aligned PTR zones.

For any PTR record there are two sets of properties to consider:

First property:

1. PTR record is in an RFC2317 zone that is maintained in NetBox DNS
2. PTR record is in an RFC2317 zone that is not maintained in NetBox DNS (we can detect that by looking at delegations in the "normal" PTR zone that would contain that record without RFC2317)
3. PTR record is not in an RFC2317 zone

Second property:
(by 'classic reverse zone' I refer to the longest-match in-addr.arpa-Zone containing the PTR record, pun intended :-))

1. The classic reverse zone of the record is maintained in NetBox DNS
2. The classic reverse zone of the record is not maintained in NetBox DNS

For the combinations of properties I think the following actions are required:
1/: Create the PTR record in the RFC2317 zone
2/: Do not create the PTR record at all
1/1: Create the CNAME in the classic reverse zone
2/1: Create the CNAME in the classic reverse zone
3/1: Create the PTR record in the classic reverse zone
3/2: Do not create the PTR record at all

While this is already a bit involved, it really gets funny when we consider a couple of other actions:

• Create a new RFC2317 zone subnetting an existing in-addr.arpa zone
• Remove an RFC2317 zone
• Create a new RFC2317 zone, but fail to delegate to it from an existing in-addr.arpa zone
• Remove an in-addr.arpa zone that has delegations to RFC2317 zones

It's going to be ... interesting (but still makes a lot of sense anyway, so it's probably worth the effort). There are some other examples of similar logic in the code, so it will certainly be doable, but probably not in the next few weeks.

This ISC KB-Article describes a slightly more flexible approach, which doesn't use prefix-based delegation but the slightly more flexible notation of '1-3.0.0.10.in-addr.arpa' for the sub-delegated PTR zones. I see some advantages of that approach: First, you're not limited to prefix boundaries, second, you don't have a '/' in your domain names which may or may not cause some DNS servers and clients to cough up, and third, it's much more convenient to see whether an IP address is within one of the ranges or not.

Taking into account that there is more than one approach of naming the zones maybe we'll find an even better way to represent subnetted /24 reverse zones in NetBox DNS. Using the zone name to determine the range of PTRs delegated to a zone will probably not make everyone happy if they happen to be using a different convention.

[wolfspyre]

another concept is that by declaring the larger/smaller in-addr.arpa zones you can use the relationship to facilitate visualization of the used space..

a use case is delegating a set of /16’s reverse dns to a specific k8s cluster, or some other sdn use that wants to manage its own ip space.

thinking about a zonefile within the context of a view
if it’s declared in that view, the expectation is that it was done so for a reason.

ergo. there aught be the proper relationship management involved.

if a zonefile:
8-10.in-addr.arpa
( or 8-0.0.0.10.in-addr.arpa )

is declared in the same scope as:

16-99.10.in-addr.arpa

then if nothing else, i would expect a relationship to be understood or represented somehow, either with an ns record in 10-8, or having changes replicated somehow. because by allowing the overlapping declarations there are now two distinct sources of truth for a resource that could diverge.

it makes sense to me to adopt the mindset that the intent is to represent truth.
thus it aught be possible, if not preferential, to have changes made within one scope propagate to its declared doppelgängers
this, to me; attempts to maintain coherence and truth across context

[peteeckel]

Hi @wolfspyre, do you think that should be such a zone as 8-10.in-addr.arpa? I don't actually see the use case for this, as the zone can be represented with three standard zones 8.in-addr.arpa, 9.in-addr.arpaand 10.in-addr.arpa so no special handling (particularly, no CNAMEs pointing to PTRs) is required.

The definition of overlapping zones can easily be caught on data entry, and given the fact that allowing overlaps results in tons of confusion while not making any sense I would definitely tend to disallow such a configuration.

Regarding views, they definitely isolate zone sets from each other - an A/AAAA record in a given view will only create PTR records within the same view, so overlapping definitions between views do not matter at all.

Or am I missing something?

[wolfspyre]

sorry. by 8-10 i meant to refer to 10/8
ie the entire 10.x.y.z netblock

depending on implementation, i’ve seen the reverse zone (if using a cidr other than /24) referenced as:

• 8/0.0.0.10.in-addr.arpa
• 8-0.0.0.10.in-addr.arpa
• 8/10.in-addr.arpa
• 8-10.in-addr.arpa

does that help clarify what i meant here?

[peteeckel]

does that help clarify what i meant here?

Not really :-)

But it illustrates pretty exactly why it's impossible to use a naming convention for discrimination which subnetted /24 is actually meant by the zone name. RFC2317 does not define anything and merely makes suggestions, so there is no standard at all and there will definitely different naming conventions for the RFC2317 zones.

The one suggested in the RFC is

$subnet/$prefix_length.x.y.z.in-addr.arpa.

which would results in, for example

0/26.0.0.10.in-addr.arpa.
64/26.0.0.10.in-addr.arpa.
128/26.0.0.10.in-addr.arpa.
192/26.0.0.10.in-addr.arpa.

for the four /26 zones subdelegated from `0.0.10.in-addr.arpa. But there are different conventions I found, most often

$range_start-$range_end.x.y.z.in-addr.arpa.

which gives us

0-63.26.0.0.10.in-addr.arpa.
64-127.0.0.10.in-addr.arpa.
128-191.0.0.10.in-addr.arpa.
192-255.0.0.10.in-addr.arpa.

for the same zones. Additionally, it has the advantage of being able to delegate non-bit-boundary subranges (which, depending on what one likes to achieve, may or may not make sense - at least it avoids the somewhat controversary / in the domain name).

Bottom line: The domain label of the subdelegated domain doesn't help us in the general case (and we do want to solve the general problem, instead of imposing an arbitrary standard on domain names).

Anyway: That means for the NetBox DNS data model that we need to provide for the RFC2317 zones in a general way instead of relying on the domain name. The most general case that I can think of is:

• Domain name (Just for completeness' sake, it's already part of the data model)
• Subdelegating reverse domain (some standard PTR domain name like 0.0.10.in-addr.arpa., or empty for automatic detection)
• Start label of subdelegated range
• End label of subdelegated range or prefix length

The subdelegating reverse domain obviously needs to be uniquely defined so we know where to put the NS records and the CNAMEs to the subdelegated records. It can, however, not necessarily be derived from the domain name, as there might be constructions where, for instance

• 10.in-addr.arpa. is maintained in NetBox DNS
• 1.10.in-addr.arpa.is subdelegated to another organisational unit (and thus not necessarily maintained in NetBox DNS)
• 0-15.28.0.1.10.in-addr.arpa. is again subdelegated to NetBox DNS by that unit
  And yes, that's a constructed case, but it can happen. Automatic detection of the super-zone would find 10.in-addr.arpa., which is clearly wrong, so we need to at least be able to explicitly specify the subdelegating zone. The default can (and should) however be canonical (empty), meaning the first known zone upwards in the zone hierarchy.

Start label and end label are the most flexible way of defining subdelegations, although they can also specified using a prefix length instead of the end label - though this is a less general case. Conflicts can be detected in both cases, most easily using the ipaddressmodule.

With that information attached to a zone NetBox DNS will be able to determine everything that's needed to put the PTR records, the CNAMEs and the NS records for the subdelegation into the proper places:

• NS (delegation) records and CNAMEs go to the subdelegating zone (detected or explicitly specified, although in the latter case they would probably not be maintained in NetBox DNS)
• PTR records are inserted in the zone itself, selected by the zone name (the first three octets of the address are used for this, as it is currently done anyway) and the start and end label of the range.

As you can see, the whole issue is not trivial, and it's important to get it right. But this discussion really helps to put the pieces into their places.

[wolfspyre]

okay lemme try this again....

So.... I have the entire 10.0.0.0/8 network. ( so 10.0.0.0 -> 10.255.255.255 )

I declare a reverse zonefile in view "INTERNALZONES"

8-0.0.0.10.in-addr.arpa

I allocate the 10.0/16 subnet to myself. ( 10.0.0.0 -> 10.0.255.255)
I allocate the subnet 10.1/16 to team foo for a project. ( 10.1.0.0 -> 10.1.255.255)

I allocate the subnet 10.2/16 to team bar for another project. ( 10.2.0.0 -> 10.2.255.255)

the reverse zonefiles aught be:
8-0.0.0.10.in-addr.arpa
16-0.0.0.10.in-addr.arpa
16-0.0.1.10.in-addr.arpa
16-0.0.2.10.in-addr.arpa

I would like to be able to inform netbox-dns somehow that the the three /16 subnet zones are children of the /8...
so any records that are added to the 10.0/16, 10.1/16 or 10.2/16 zonefiles are also propagated to the 10/8 zonefile.

( because the 10/8 network is a superset of 10.0/16,10.1/16, 10.2/16 ... .... .... 10.255/16 )

in the above cases, I'm talking about LARGER subnets than a /24.
now.... in SMALLER subnets ie /28, the way I've always seen those delegated has been the same as route declarations.... you add the network address + the netmask:

28-0.0.0.10.in-addr.arpa
28-16.0.0.10.in-addr.arpa
28-32.0.0.10.in-addr.arpa

which would reference the 10.0.0.0/28, 10.0.0.16/28, and 10.0.0.32/28 subnets

so any record added to the 10.0.0.0/28 in the "INTERNALZONES" view.... (assuming I defined all the above mentioned reverse zones.)

aught be present in the 28-0.0.0.10.in-addr.arpa, the 16-0.0.0.10.in-addr.arpa and 8-0.0.0.10.in-addr.arpa zonefiles......

( as well as the default /24 zonefile: 0.0.0.10.in-addr.arpa. )

as they are each distinctly referencing the same addresses (just with different bounds)

does that make sense?

[peteeckel]

Hi @wolfspyre, I see ...

First of all: We are not talking about subdelegated reverse zones for subnets larger than /24. That is a problem that can be solved with standard DNS means, and it should. There' a reason why RFC2317 starts with

This document describes a way to do IN-ADDR.ARPA delegation on non-octet boundaries for address spaces covering fewer than 256 addresses.

Non-octet boundaries for larger address spaces are not in scope of RFC2317, and since the issue presented by them can be solved in the standard way IMHO it means it should. At least I have the RFC authors on my side :-)

The solution with the subdelegation to RFC2317 zones and the subsequent CNAMEing of PTRs is horrible as it is, and an artifact of the unfortunate situation that more than 20 years after the introduction of IPv6 there are still many institutions and companies around who failed to adopt it (trust me on this, it works).

The other thing that seems to be a subject of misunderstanding is the matter of views. You just cannot delegate zones between views. That's exactly the point of views - provide different namespaces for DNS data. I'm not sure what you want to achieve with views (as you only mention one, 'INTERNALZONES'), but you can't delegate from a hypothetical 'EXTERNALZONES' or the default view to a different one - clients just would not see the zones and records.

What you might be looking for is something along the lines of BINDs $GENERATE statement. That is something that can quite easily be solved with a custom script. Since not all authoritative name servers have this feature and it's most certainly not a standard mechanism it (again) does not make sense to include in the plugin itself - at least not short term, there's other things that are more urgent (such as /24 subdelegation).

We can discuss both issue further, but for the sake of this thread please let's take them out of this discussion. /24 subnetting is bad enough for one discussion :-)

[wolfspyre]

not sure if this helps or not ;)

[peteeckel]

not sure if this helps or not ;)

At least it doesn't show any subnet > /24 :-)

[peteeckel]

Just a quick heads-up: I started implementing RFC2317 today. As always, don't hold your breath - it will take some more days or weeks until something presentable is available.

What's currently working:

• Adding an RFC2317 subnet attribute (for a number of reasons I decided to use prefix notation, not arbitrary ranges, at least for the first shoot) to a zone is possible.
• Additionally it can be defined whether or not NetBox DNS shall manage the parent zone (just a boolean flag). As it turned out, that in combination with the zone status field covers all cases I evaluated above.
• When an RFC2317 attribute is set, the zone name is validated against the subnet specified. Zone names that cannot be sub-delegated to are not accepted (i.e. 1.0.10.in-addr.arpa cannot be used as a parent for 10.0.0.0/25, so 0/25.1.0.10.in-addr.arpa is not a permitted zone name for this prefix - 0/25.0.0.10.in-addr.arpa is, as is everything else ending in .0.0.10.in-addr.arpa.
• Overlapping subnets are rejected. You can't define a zone for e.g. 10.0.0.0/25 and 10.0.0.96/28 at the same time.
• Fake subnets are not accepted either (e.g. 10.0.0.32/26 is not a subnet, but a host).

While not looking too impressive that should be a good basis to go on from. The next steps will be:

• Adjusting the tables so the RFC2317 stuff can be looked at
• Defining filters on the new fields and for parent zones, among other things
• Providing the logic for the creation of PTR records in the correct zones (as well as for updating and deleting them on various occasions)
• Updating the API
• Creating tests (lots of them, the functionality will be quite complex eventually)
• Updating the documentation with the new functionality
• Tests, tests and more tests ...

So as I said - don't hold your breath (yet). The good thing is that I now have a pretty clear idea how the solution will look eventually.

[peteeckel]

Another heads-up: I'm currently experimenting with arbitrarily named zones containing RFC2317 PTR records and how this can be achieved with NetBox DNS. It's a bit longer shot, but it would remove some limitations - on the other hand it might add additional complexity, and this stuff is not really simple to handle to start with.

When we want to enable domains with arbitrary names, we need to add some way to identify PTR records to be pointed to by the CNAME records in the regular .in-addr.arpa-Zones. While this is rather attractive in a way, it's also a bit more confusing. The next step would then be to make the naming convention for the PTR records flexible as well, so for instance we'd get, for instance,

123.0.0.10.in-addr.arpa. IN CNAME 123.0.0.10.in-addr.example.com.

and

123.0.0.10.in-addr.example.com.  IN  PTR  host1.example.com.

as the resolution path for a PTR in an RFC2317-delegated zone.

I'm not sure it is worth the considerable effort to implement, document and maintain this flexible approach - any thoughts on this?

[peteeckel]

HI @wolfspyre, thanks for looking into this, it helps to have someone else validate your own thoughts.

Your unterstanding is pretty similar to mine and to the practice I've seen over the years at various sites. Thanks to @LuPo I know also know that there's an RFC for it :-).

The current implementation (yesterday the functionality for the mechanism you described above was complete, though yet not formally tested and documented) is, if you read further in RFC2317, one way of doing it, and I think it is - or should be - the canonical way: Subdelegate a /24 zone and put CNAMEs in it that point to the subdelegated, non-standard-ARPA zones such as 0/26.0.0.0.in-addr.arpa. In this case the subdelegation NS record is necessary for the resolver to find the target of the CNAME, and that's what I've seen in most cases to be the current practice. The name of the non-standard zone is restricted in this case in order for the resolver to be able to find it, so it must end in the name of the /24 (or larger) zone that contains it. The good news is: That's what I have right now, given or taken a few bugs that are likely to lurk somewhere and that proper testing will hopefully help to get rid of.

I also had the idea of indiscriminately creating all the subnet delegation host entries in the in-addr.arpa. zones when a sub-delegation occurs but decided against that for a couple of reasons, most importantly efficiency: When you create a /25 subnet RFC2317 zone you have to create 126 CNAME records in the parent zone, and when someone gets the idea of creating 1000 such zones programmatically ... you get the idea. In many cases not a single one of these records will be used initially, so it's a huge overhead. Instead the logic is as follows:

• A check for overlapping RFC2317 subnets is performed (you can't define an RFC2317 zone for two overlapping subnets like 10.0.0.0/25 and 10.0.0.64/26)
• A check for the name of the zone is performed (with the RFC2317 setup from Section 4 RFC2317 zones must be delegatable from the parent zone, which requires the parent zone name to be extended to the left to get the RFC2317 zone name). This could be dropped when RFC2317 Section 5.2 zones get supported)
• An RFC2317 zone is created (that is a normal zone with the rfc2317_subnet field containing an netaddr.IPNetwork entry)
• NetBox DNS finds the appropriate standard .in-addr.arpa zone (unless parent management is disabled)
• If it finds one, it inserts the NS entries for the subdelegated zone according to its nameserver records

If the parent zone is not managed within NetBox DNS (which can be controlled by a flag), it's obviously easier to generate all CNAMEs as you described - but since that doesn't happen within NetBox DNS it's not our problem. BIND can easily do it with a $GENERATE statement, so it's not really an issue anyway. But anyway, it's out of scope for this plugin.

Now, when a record is created within the RFC2317 zone, NetBox DNS automatically checks if there is a managed parent zone in the database and if so, it creates the specific CNAME record to the entry in the RFC2317 zone. Pretty easy :-)

Not so easy is handling all the other scenarios ... what happens when the parent zone is deleted, renamed, a new one is created and so on, and the same for RFC2317 zones, for instance when you subnet an existing /24 network and create two or more RFC2317 zones for it - the records need to be migrated properly so the data don't get inconsistent. That's the really interesting part.

RFC2317, however, also outlines an alternative way of doing it: Put the PTR records for the subdelegated /25, /26 ... in an already existing zone, for example example.com, and have the pointers in the real /24 or larger reverse zone point there. That approach gets rid of the need to put delegation NS records in place (as the target zone can be found using the standard DNS resolution mechanism), but adds some complexity and new possibilities to create unexpected errors. The minimum measure to get it a bit safer is to add a reasonable naming policy for the subdelegated records, such as <host address>.in-addr.<zone name> (actually it's pretty arbitrary, as long as it avoids collisions. The RFC describes that in Section 5.2.

You would then have the CNAME for, for instance, 2.0.168.192.in-addr.arpa- point at 2.in-addr.arpa.example.com. (that could even be the zone the A record itself is located in), and the resolution in that zone:

$ORIGIN 0.168.192.in-addr.arpa

2                 IN    CNAME    2.in-addr.example.com.

$ORIGIN example.com

host2             IN    A        192.168.0.2
2.in-addr         IN    PTR      host2.example.com

The good news is that the current implementation I have is flexible enough to be extended to support that as well. The bad news is that it adds more complexity to the code and more exotic ways of introducing errors that must be detected and caught (and, even more difficult, anticipated before anyone else's creativity in creating such situations is higher than my own :-)).

So for the time being I restrained myself to implement the 'canonical' way of doing it as outlined in Section 4, the one you summarised as well.

[wolfspyre]

oof.
man your head is good at juggling complexity.

That's a well described summary.... ;)

In thinking about this, I have a feeling that its still gonna be worth creating the cname resources..
PARTIALLY because of the concern you elucidate wrt deleting the parent reverse zone...
and partially because we're assigning resources to a sub-authority... if netbox dns is supposed to be the source of truth, knowingly creating a scenario where the "record of authority" isn't accurate feels... well ... icky ;)

I think it's reasonable to consider the CNAME/PTRs associated with the parent arpazone managed records, similar to NS/SOA records... which I'd think the existence of would be sufficient to prevent the removal of a parent container resource when there are related records which depend on that parent.

an so we could potentially defer creating them at declaration time... however if we don't reserve all the addresses in an unmistakable way, we run the risk of permiting dual-allocation... say tommy's /25 and billy's /29... or just forgetting that the first half of this /24 is allocated to tommy

off topic sorta, but is there a plan to associate the dns records with the ipam IP address constructs within netbox?

I ask, because that may be a viable reservation option as well... if its possible to associate these N ips with this sub-subnet resource allocation, that could be another mechanism to gate dual allocation, or removal of resoruces which define the delegation (ie the parent arpa zone.)

course, that could be a can of worms so large that its a meal in and of itself ;)

regardless, thank you for working on this. I believe it'll help people.
🐺W

[peteeckel]

Hi @wolfspyre,

oof.
man your head is good at juggling complexity.

thanks, though sometimes I'm not so sure myself :-)

In thinking about this, I have a feeling that its still gonna be worth creating the cname resources..

I'm not sure you're wrong either ...

It all boils down to what the more likely scenario is, and where the expense in creating stuff ends up. At some point CNAME records must be created, it's just a question of when and how many. Let's look at the two extreme scenarios:

1. You create a new RFC2317 zone for a /25 subnet that is initially empty, i.e there are no address record that migrate to that zone as a result of zone creation. That is the case that is very cheap as it is implemented now (essentially it's like any other zone creation, except that a CNAME update for the parent zone is necessary at the end - which will, however, find nothing to create and that's it, while pre-creating all CNAMEs is pretty expensive as 126 essentially unnecessary CNAMEs have to be created.
2. You create a new RFC2317 zone whose parent is a fully populated existing reverse zone. That is one of the most expensive things that can happen in the context of this change. It will result in the PTR records being moved from the parent to the child zone, then for each PTR a CNAME will be created in the parent zone. To make it worse, for each PTR being created a CNAME update for the parent zone is run (I'll talks about deferring that later), which in itself is a moderately expensive operation. In this scenario pre-creating all the CNAMEs is clearly at advantage.

What it all boils down to now is the question how much difference in terms of performance it makes to pre-create the CNAMEs vs. creating them on demand. That's something I can measure, and I will do that at some point in the development cycle, probably rather soon. I have a gut feeling that pre-creating the CNAMEs is quite a bit faster, I just don't have an idea by what factor. That's something to find out.

What we can't measure is how likely it is that this extreme scenario happens in daily use. We need to be able to handle it for sure, even in scenario 2 - if that becomes unwieldy, the decision is simple. But if the difference in runtimes is not too large, the current implementation will be smoother in the total average.

an so we could potentially defer creating them at declaration time... however if we don't reserve all the addresses in an unmistakable way, we run the risk of permiting dual-allocation... say tommy's /25 and billy's /29... or just forgetting that the first half of this /24 is allocated to tommy

Allocation conflicts for RFC2317 zones as you described will not happen, so reserving the CNAMEs is not necessary for this purpose. NetBox DNS won't even let you get as far as creating the zone in the first place - There's a check in place that tests all RFC2317 zones for overlapping IP ranges and declines to create an overlapping zone.

Generally, deferring operations would IMHO only be sensible if there was full asyncio support for Django, which isn't something we can assume. Django itself is largely asyncio-capable, but in order to use asncio you need to run the whole NetBox environment under ASGI. I don't expect that to become a requirement any time soon, so we can't build on it. Using 'poor man's async' by running tasks in scheduled jobs is definitely not something I would want to do.

This discussion is really interesting, and thank you very much for your ideas - testing the runtimes of the two variants is the least that will result from it, and depending on the result it may be sensible to drop one or the other idea, or - always a possibility - make it a configurable option in some kind of expert setting.

The other thing that just came out of it is that subdelegating existing zones needs another check: When you have PTR records like, for instance

$ORIGIN 0.0.10.in-addr.arpa

126            IN   CNAME    host126.example.com.
127            IN   CNAME    host127.example.com.
128            IN   CNAME    host128.example.com.
129            IN   CNAME    host126.example.com.

and then try to sub-delegate 0.0.10.in-addr.arpa into 0/23.0.0.10.in-addr.arpa´ and 128/23.0.0.10.in-addr.arpa´ NetBox DNS should decline the request, as it would probably create PTR records that are not what the user intends to do: 10.0.0.127 is the broadcast address of the subnet 10.0.0.0/27', while 10.0.0.128is the network address of10.0.0.128/27'. And there's even an exception to that, as in /31 networks both addresses actually are used ... otherwise there would not be such a thing as a /31 network.

It's still possible that the user actually wants the entries for the network and broadcast address, so it is not a good idea to flatly refuse subnetting the zone if there is one. In case of /31 subnets, using the addresses for nodes is even totally legit. So I guess I'll resolve this with issuing a warning and asking for confirmation if there would be a network or broadcast address in any RFC2317 zone with a /25 to /30 mask, and silently allow it for /31 and /32.

course, that could be a can of worms so large that its a meal in and of itself ;)

The whole RFC2317 stuff is a can of worms, so it's doesn't matter too much anymore :-)

[wolfspyre]

oof.
man your head is good at juggling complexity.

thanks, though sometimes I'm not so sure myself :-)

Yup! Sure sign of intelligence is to question one’s own sanity/abilities first IMO :)

In thinking about this, I have a feeling that its still gonna be worth creating the cname resources..

I'm not sure you're wrong either ...

It all boils down to what the more likely scenario is, and where the expense in creating stuff ends up. At some point CNAME records must be created, it's just a question of when and how many. Let's look at the two extreme scenarios:

1. You create a new RFC2317 zone for a /25 subnet that is initially empty, i.e there are no address record that migrate to that zone as a result of zone creation. That is the case that is very cheap as it is implemented now (essentially it's like any other zone creation, except that a CNAME update for the parent zone is necessary at the end - which will, however, find nothing to create and that's it, while pre-creating all CNAMEs is pretty expensive as 126 essentially unnecessary CNAMEs have to be created.

well, I’d think they’re not strictly unnecessary; in that their existence is what declares the delegation from the parent level… essentially the subdelegation of a /25 is the delegation of 128 distinct resources glommed together in a single coherent operation that should succeed or fail in unison, right?

declaring them allows for the assurance that the resource’s ownership and authority is “properly” asserted from whichever lens one chooses to look through…. or am i missing something obvious again? :)

2. You create a new RFC2317 zone whose parent is a fully populated existing reverse zone. That is one of the most expensive things that can happen in the context of this change. It will result in the PTR records being moved from the parent to the child zone, then for each PTR a CNAME will be created in the parent zone. To make it worse, for each PTR being created a CNAME update for the parent zone is run (I'll talks about deferring that later), which in itself is a moderately expensive operation. In this scenario pre-creating all the CNAMEs is clearly at advantage.

this feels like a separate problem that is likely to need user interaction.

“hey! you’re trying to perform a delegation of ownership to resources which are already in-use! What do you want me to do here?”
I think this could indicate multiple distinct error cases, and i don’t know that we’d be able to programatically determine the “right” way forward here consistently

• fat finger? cancel operation?
• deprovision and delegate blank?
• retain existing records and delegate them as well?
• relocate existing records elsewhere?

any of these might be the desired result from the monkey pushing buttons… how does the code know what the hooman wanted?

What it all boils down to now is the question how much difference in terms of performance it makes to pre-create the CNAMEs vs. creating them on demand. That's something I can measure, and I will do that at some point in the development cycle, probably rather soon. I have a gut feeling that pre-creating the CNAMEs is quite a bit faster, I just don't have an idea by what factor. That's something to find out.

Yeah, Another lens might be: what strategy provides the most assurance that the source of truth we are decorating remains coherent, accurate, and incorruptible. if we have to fail an operation, or we crash, what mechanisms leave us in the least awkward place to try to recover from?

What we can't measure is how likely it is that this extreme scenario happens in daily use. We need to be able to handle it for sure, even in scenario 2 - if that becomes unwieldy, the decision is simple. But if the difference in runtimes is not too large, the current implementation will be smoother in the total average.

is this premature optimization?
idk if this is an existing problem to worry about.

we could likely separate the concern with
a ‘pending operations’ table and stuff them there. using that as a way of remaining responsive to user input regardless… while preventing the loss of resource attribution

an so we could potentially defer creating them at declaration time... however if we don't reserve all the addresses in an unmistakable way, we run the risk of permiting dual-allocation... say tommy's /25 and billy's /29... or just forgetting that the first half of this /24 is allocated to tommy

Allocation conflicts for RFC2317 zones as you described will not happen, so reserving the CNAMEs is not necessary for this purpose. NetBox DNS won't even let you get as far as creating the zone in the first place - There's a check in place that tests all RFC2317 zones for overlapping IP ranges and declines to create an overlapping zone.

Generally, deferring operations would IMHO only be sensible if there was full asyncio support for Django, which isn't something we can assume. Django itself is largely asyncio-capable, but in order to use asncio you need to run the whole NetBox environment under ASGI. I don't expect that to become a requirement any time soon, so we can't build on it. Using 'poor man's async' by running tasks in scheduled jobs is definitely not something I would want to do.

yeah, i wasn’t sure if there was a mechanism facilitating this…

This discussion is really interesting, and thank you very much for your ideas

course!! Not all my ideas are good ones, but i try to be helpful and …. let’s go with MOSTLY sane :)

• testing the runtimes of the two variants is the least that will result from it, and depending on the result it may be sensible to drop one or the other idea, or - always a possibility - make it a configurable option in some kind of expert setting.

The other thing that just came out of it is that subdelegating existing zones needs another check: When you have PTR records like, for instance

$ORIGIN 0.0.10.in-addr.arpa

126            IN   CNAME    host126.example.com.
127            IN   CNAME    host127.example.com.
128            IN   CNAME    host128.example.com.
129            IN   CNAME    host126.example.com.

and then try to sub-delegate 0.0.10.in-addr.arpa into 0/23.0.0.10.in-addr.arpa´ and 128/23.0.0.10.in-addr.arpa´ NetBox DNS should decline the request, as it would probably create PTR records that are not what the user intends to do: 10.0.0.127 is the broadcast address of the subnet 10.0.0.0/27', while 10.0.0.128is the network address of10.0.0.128/27'. And there's even an exception to that, as in /31 networks both addresses actually are used ... otherwise there would not be such a thing as a /31 network.

It's still possible that the user actually wants the entries for the network and broadcast address, so it is not a good idea to flatly refuse subnetting the zone if there is one. In case of /31 subnets, using the addresses for nodes is even totally legit. So I guess I'll resolve this with issuing a warning and asking for confirmation if there would be a network or broadcast address in any RFC2317 zone with a /25 to /30 mask, and silently allow it for /31 and /32.

would there be value in thinking of this as two different types of address bundling?

subnet delegation / routing, versus resource ownership transfer?
in a subnet delegation, you have the network and broadcast addresses as resource constructs… as they’re likely necessary…

whereas a bubble of addresses allocated to a purpose, owner, or context may not care…

that way you could
(admittedly a contrived example, but it’s wot comes to mind off the top of my head).
carve a /25 for your dhcp pool for example and allow the dhcp server to update the reverse for the hosts in that section of IP space within the /24 without burning an ip for the mask.

course, that could be a can of worms so large that its a meal in and of itself ;)

The whole RFC2317 stuff is a can of worms, so it's doesn't matter too much anymore :-)

oh sure it does.. we can always make things MORE complicated…. but that doesn’t make it better :) just look at k8s :)

[peteeckel]

Hi @wolfspyre, again, thanks for your thoughts!

well, I’d think they’re not strictly unnecessary; in that their existence is what declares the delegation from the parent level… essentially the subdelegation of a /25 is the delegation of 128 distinct resources glommed together in a single coherent operation that should succeed or fail in unison, right?

Good point. On the other hand this is more or less a matter of documentation, and there is something else we need to consider: The CNAMEs will only be created by NetBox DNS when the parent zone is managed by NetBox DNS as well, in which case we already know what is sub-delegated. Remember, we have three cases:

1. Parent zone in NetBox DNS, RFC2317 zone in NetBox DNS
2. Parent zone in NetBox DNS, RFC2317 zone managed offline (i.e. somewhere else)
3. Parent zone managed offline, RFC2317 zone in NetBox DNS

Case 4 is obviously less interesting for us.

• In case 1, NetBox DNS manages everything - PTR record delegation, CNAMEs, NS records for deleagtion etc.
• In case 2, NetBox DNS does the same, but the RFC2317 zone is only used for reference. For this purpose there is a new zone status "Offline" that tells systems taking their inventory from NetBox DNS that this zone is not to be provisioned.
• In case 3, NetBox DNS manages PTR records in the RFC2317 zone, but nothing is done for the parent zone.

Case 3 is the case in which proactive CNAME creation makes much sense, but in this case we don't own the zone so we don't need to do it. And that's the case where the CNAME-based delegation hint really makes most sense. In Case 1 and 2 NetBox DNS is the documentation.

So I still wouldn't want to do proactive CNAME creation for documentation purposes, but possibly for performance reasons.

any of these might be the desired result from the monkey pushing buttons… how does the code know what the hooman wanted?

What you mention is a generic problem in NetBox, not (only) NetBox DNS. There is no way at all to undo anything, except by saving and restoring a database snapshot. NetBox assumes the users know what they are doing, and so does NetBox DNS. And yes, I fully agree with you that this can be a pain in the backside, but a general solution to the undo scenario is something that upstream would need to provide. I think I remember that there is a plugin somewhere ...

On the other hand, there are some operations that are more tedious to recover from than others. Creating and deletion of RFC2317 zones is not one of them - you can always delete or recreate the zone, and the internal logic will roll back the resulting changes to PTR/CNAME/NS records. What we need to keep in mind, however, is performance. Many of the operations are really complex, and some of them are already quite slow as it is. RFC2317 does not really improve on this situation, so what you call 'premature optimisation' is actually necessary - as are performance analyses.

There is another issue with pending operations management that is not easily ignroable, which is testability and stability against conflicting actions by existing users. Currently all of our tests assume that when an operation returns it is actually finished. It complicates thinks further to also have to wait for all pending operations to be completed. And, worse: When one user executes an action that causes a pending operation to be queued, an operation executed by another user may itself conflict with one of the operations in the queue, or (worse) cause a different operation to be queued that conflicts with one of the operations already in the queue ... talking "can of worms" here. Been there, done that ...

let’s go with MOSTLY sane :)

It never hurts to think about ideas that may seem crazy. Once upon a time I had a customer who wanted me to set up a DNS infrastructure for their environment and mentioned that a graphical tool for DNS management would be nice ... and here we are :-)

subnet delegation / routing, versus resource ownership transfer?
in a subnet delegation, you have the network and broadcast addresses as resource constructs… as they’re likely necessary…

That's also a good point to think about, and I was mulling this issue for a while myself (scroll up :-)). Looking at RFC2317, you'll find that it's all about subnet delegation, not about sharing of responsibility in general. Your DHCP range example is a good one. I had that in mind when I considered arbitrary ranges to delegate to "RFC2317" zones, such as 10-150.0.0.10.in-addr.arpa.. Two good reasons not to do it: 1. It's not in scope of RFC2317. 2. It's a lot more complicated to manage the address ranges.

The address range issue is currently solved using the Python modules netaddr and ipaddress (the latter I'm trying to phase out to keep the footprint small), and the PostgreSQL cidr datatype with custom field lookups in Django. All of them are pretty efficient. Manually calculating overlaps and the containment of subnets or addresses in IP address ranges decidedly is not.

The simple (and most seen) way to cater for dynamic DHCP ranges is to make the whole zone containing the IP range dynamic, reserving some address space for static addresses. Again, that's not really part of the DNS data model, and it is so specific that a custom solution will most likely be the best fit for it, taking data from NetBox DNS and from the IPAM module of NetBox, which already knows IP ranges.

So if we link the feature to RFC2317, subnet delegation is the assumption to work from. And in that case we can at least warn the uses when they try to do something inconsistent with delegation - and it's cheap as well.

that way you could (admittedly a contrived example, but it’s wot comes to mind off the top of my head). carve a /25 for your dhcp pool for example and allow the dhcp server to update the reverse for the hosts in that section of IP space within the /24 without burning an ip for the mask.

In all cases I've seen so far the admins made both the address zone(s) and the reverse zone dynamic and gave the DHCP server access to it - no need for complex DNS setups. But it can still be solved with NetBox DNS if you really want to: Create an RFC2317 zone with an unmanaged parent, create the CNAMEs in the parent zone manually or programmatically (or hope for someone(TM) to implement proactive CNAME creation for RFC2317 in NetBox DNS :-)), and make the RFC2317 zone and the forward zone that has the A records dynamic. Not a big deal either.

On the other hand, dynamic address assignment and NetBox DNS don't really have that much in common anyway. The thing about NetBox (and NetBox DNS) is to have a source of truth, and in case of DHCP/DynDNS etc. that clearly isn't NetBox but the DHCP server or DynDNS service.

oh sure it does.. we can always make things MORE complicated…. but that doesn’t make it better :) just look at k8s :)

I prefer not to look at k8s. I've already had breakfast. 😈

[wolfspyre]

teehee :)
well.

In case 2, NetBox DNS does the same, but the RFC2317 zone is only used for reference. For this purpose there is a new zone status "Offline" that tells systems taking their inventory from NetBox DNS that this zone is not to be provisioned.

does this differ, api-response-wise from a forward zone netbox dns is unaware of?

should it?

(no immediate strong thoughts here either way, but did seem like an interesting question philosophically)

Case 3 is the case in which proactive CNAME creation makes much sense, but in this case we don't own the zone so we don't need to do it. And that's the case where the CNAME-based delegation hint really makes most sense. In Case 1 and 2 NetBox DNS is the documentation.

So I still wouldn't want to do proactive CNAME creation for documentation purposes, but possibly for performance reasons.

yeah, i can see that too…

so let’s say we’ve created that /25

and we’ve created a sub-zone for hosts in it.

‘dhcp.example.com’

we declare both the forward and reverse zones as resources managed by nameservers netbox dns is aware of…. (case 1)

we run a script to emit the views and records per zone from netbox dns’ perspective

what assessment/validation logic would make sense in both directions? (fwd/reverse)

what about case 1B

where parent zone 2.0.192.in-addr.arpa is owned by org A Team A1 in netbox

rfc2317 zone 0/25.2.0.192.in-addr.arpa is delegated to OrgA Team A5 in netbox

rfc2317 zone 128/25.2.0.192.in-addr.arpa is delegated to OrgB Team B4 in netbox

the forward zone

a5kicksbutt.com belongs to orgA team A5

B4isbetter.com belongs to org B team B4

the user AA1mojo has privilege to CRUD all OrgA resources.

the user AA1minion has privilege to CRUD resources which belong to team a1 and R access to all orgA resources.

the user AA5ops has R privilege to all OrgA resources. and has CRUD access to all team A5 resources.

The user happyB4 has CRUD privilege for all Org B resources

what resources should the AA5ops user be unable to alter?

what resource privileges should be different between the AA5ops and AA1minion users?

What should the user happyB4 be unable to alter?

IDK that we want to go so far as gating privilege by default, but it feels like there are a few isolation boundary validation cases that can be programmatically verified

[peteeckel]

does this differ, api-response-wise from a forward zone netbox dns is unaware of?
Yes, it does - status is "offline", not "active". Nothing NetBox DNS cares about, but something that's relevant to the consumers of the data - 'don't create zone files for offline zones', basically. Functionally, 'offline' and 'active' are identical, it's just a flag for consumers of the data.

Offline RFC2317 zones are important as they tell NetBox DNS not to create PTR records in the parent zone, but CNAMEs, and what name servers the zone should be delegated to. That is quite exactly the same that's done for active RFC2317 zones, they are just there to simplify the logic for record handling.

IDK that we want to go so far as gating privilege by default, but it feels like there are a few isolation boundary validation cases that can be programmatically verified

Possibly a valid use case, but definitely the next can of worms. And one that I'd like to keep closed for the time being, as permissions on object level are an entirely different and much more complex cup of tea. Not sure how well it is handled in Django, much less in NetBox. I'd rather wait for someone who actually has that use case and needs support for it than open that can proactively, or this feature will never see the light.