Inconsistency with the new "Unique TTL for RRSets" functionality

[peteeckel]

The "Unique TTL for RRSets" feature unfortunately creates a conflict that cannot be resolved in a simple and consistent way.

Assume we have the following records (in a single zone zone1.example.com, although the problem isn't limited to single zones):

name1    86400    IN    A    10.0.0.1
name1    86400    IN    A    10.0.0.2
name2    43200    IN    A    10.0.0.1
name2    43200    IN    A    10.0.0.2

NetBox DNS creates PTR records automatically, and uses the TTL of the address record for the PTR record as well. Now creating the above set of A records would create the following list of PTR records in zone 0.0.10.in-addr.apra:

1    86400    IN    PTR    name1.zone1.example.com.
1    43200    IN    PTR    name2.zone1.example.com.
2    86400    IN    PTR    name1.zone1.example.com.
2    43200    IN    PTR    name2.zone1.example.com.

In this case, we get inconsistent TTLs for the two PTR RRSets, which the current implementation tries to override by adjusting the TTL of the PTR record for the address record created first to the TTL of the record - moving the inconsistency elsewhere, i.e. now the A that was created first has a PTR record with a different TTL, which is worse.

There does not seam to be a reasonable way out of this catch-22. The three options that come to my mind are:

• Allow PTR RRSets with differing TTLs in this specific situation, i.e. for PTR records
• Stop deriving the PTR TTL from that of the associated address record
• Enforce uniqueness of TTL across the whole set of records, including the address records

The first seems to be the most transparent and least intrusive to me. I would not want to implement the second because it will create inconsistencies with cached RRs, and the third variant is extremely problematic because it will also work across zones, which may be in different peoples responsibilities.

Another issue arises with the cleanup_rrset_ttls management command, which currently yields somewhat random results depending on whether the PTR record or the address record is found first.

Given the fact that the situation is somewhat constructed and probably rare, and that having different TTLs is merely deprecated, not forbidden, I think that the first variant, i.e. not enforcing TTL uniqueness for PTR records, is the most reasonable choice and will implement it for the time being, but I'm open for discussion and better ideas.

[squintfox]

@peteeckel I think the behavior should follow the setting, true or false. If uniqueness is not forced, then let it fly and every PTR can have it's own TTL. If it is, then I actually think I prefer the current, existing solution.

If I read through all the NetBox DNS documentation, I don't see anywhere it says that TTL of a PTR is derived from it's source record, nor do I see that TTL is updated in accordance with the source (only name and value are mentioned). The way I see it, is PTR is a convenience that if it doesn't exist, you have to pick something (either the value of the source or the PTR zone's default), so I'm fine with picking source. But if it does exist, uniqueness should remain required and either:

• It inherits the value of what's already there (starting behavior)
• Option 3, block the update to the source as you're already doing with unique TTL enforcement. Don't allow the change to be saved if Disable PTR is unchecked and the TTL doesn't match the other existing in the set. If I've defined I want uniqueness to be enforced, then I want it everywhere.

I also think that this is a losing battle that fades away sooner rather than later. I know of at least 3 major DNS providers that don't at all allow you to have differing TTLs at all. If I want to be able to sync my source of truth into those providers, I want to know I'm getting a predictable result (that's actually why I brought it up in the first place because I was using octodns and things were getting weird when I was testing).

[peteeckel]

If I read through all the NetBox DNS documentation, I don't see anywhere it says that TTL of a PTR is derived from it's source record,

It is.

nor do I see that TTL is updated in accordance with the source

It is as well :-)

That's obviously a documentation issue. The logic it follows is that if you have a very volatile address record (dynamic IP or whatever) the PTR shouldn't linger around in the caches for days.

I'll think about option 3 ... there are good points in favour of it, but as I pointed out there are very strong points against it as well. Especially if you are using object permissions you might not even have a chance to find the reason for an inconsistency.

Given the fact that specific TTLs are in my experience the exception rather than the rule, and that specific TTLs in combination with RRSets and that together in combination with overlapping PTRs for different records in disjunct RRSets is even more exotic I don't think it's a high priority issue.

[squintfox]

@peteeckel Sorry if I didn't make it clear about the docs. I know that it is, but my point was that as a user, up until this point, there may not be a reasonable expectation that it has to be. Therefore, making it do something different might not be unreasonable.

Regarding object permissions. If I could channel my internal Jeremy Stretch for a moment, I'd say something like "well if you don't have permissions to the PTR record, you shouldn't be updating the A record in the first place" or "edge case, that's what custom scripts are for". 😛

[peteeckel]

Your internal Jeremy is obviously missing something :-)

You may be able to see the PTR for your record, you may (because object permissions are messy stuff anyway) even be able to see the PTR for the same address that someone else created via their address record, but you can still be locked out from seeing their address record that created it ...

I'm already in object permission hell because of the IPAM coupling stuff. @jean1 really needs the fine-grained access control (which is why he put a lot of weight on it), but it's very difficult to decide where to stop evaluating object permissions. IMHO the whole object permission jungle should end where automatically doing stuff begins, but requirements vary.

[squintfox]

Well I'll go ahead and put my tail back between my legs.

Unless there's some division between zones (like views), it's not a secret to the user that someone else's record may have first claim on the TTL of the PTR. You could just nslookup and see. That makes me vote even harder for option 3. If unique is enforced, it should be enforced regardless if you have the power to change it or not (unless it's going to start being a zone by zone choice (ick).

Given the permissions argument, I'll revoke my stance on the existing option. You shouldn't be able to change something you don't have access to.

[peteeckel]

It's even better: You always have control over your PTR. You may not have control over someone elses PTR, though, and it may or may not happen to have the same name as yours (though different values, otherwise it would clash with the new default for enforce_unique_records).

Option 3 is extremely intransparent, especially under the (exotic) circumstances I outlined above. And it might not even be enforcable if you can't see the other record ... that's my main issue with it. And in that case you may not have access and you're not even allowed to know why.

[squintfox]

Maybe I'm just a bit naive to the intricacies of DNS, but is a PTR really yours? Seems to me the PTR belongs to the IP address (please forgive me if my entire basis for this conversation is flawed). You're just visiting and if someone got there before you, tough beans.

I think in option 3, the error message would just say something like "Sorry, the TTL value of this IP is xxxx, you must use this or update the existing PTR records in zone yyyy.in-addr.arpa".

[peteeckel]

but is a PTR really yours?

That's an organisational question rather than a technical one.

If you are in an organisational unit that controls DNS (network department or whatever) for the organisation, likelihood is high that you're in charge of both directions, name to address and vice versa. There will also be parts of the reverse zone space that are not, however - think about external IP addresses, where you have the delegation for the forward zone but not for the reverse zone, or only for parts of the reverse zone (think IPv4 and CIDR, leading immediately to RFC2317 ... oh the horror!).

That's exactly the reason why there is the disable_ptr setting and the option to manually create PTR records, as well as RFC2317 with the option to turn rfc2317_parent_managed on or off.

Generally, when you are in the situation of having to manage most of the DNS zones, you will want to have your PTR records in sync with your address records, even more so as manually maintaining reverse zones is an uphill race ... sooner or later you will mess up. And as long as your PTRs are generated automatically, they are most likely 'yours'.

I think in option 3, the error message would just say something like "Sorry, the TTL value of this IP is xxxx, you must use this or update the existing PTR records in zone yyyy.in-addr.arpa".

Again, you might not even be allowed to know that there is another record ... implementing this is like opening the proverbial can of worms, and you probably cannot win. I am really very reluctant to invest a ton of work in an exotic edge case. If you want to do it, your contribution will be welcome, but please include a reasonable set of tests. And please make it optional ...

[squintfox]

Lol, if I could code it myself, it would be PR central here. Unfortunately, I know enough only to be dangerous.

Is there a difference here between "what the user wants the desired state to be" and "what can actually function in the context of servers and DNS" (assuming you've selected the option of enforcing uniqueness in NetBox as the "provider" of DNS)?

(excuse any RFC2317 mistakes because I'm not at all versed on that)

• It would seem that part of the issue here is that reverse zones are limited to Class C as the smallest part, so the "you have these but not other these" is a problem that can't really be solved in this conversation.
• That said, if you're a department that wants to lay claim to a particular IP address via PTR zone, as long as it's in the same NetBox view (or no view), then I can attempt to lay claim to it. (I think there was a discussion about further overlays, but I'm thinking about view in the frame of "tenant" ish). If you can see a zone in a view, you can see "the view".
• You are always going to "know" if there's a record, even if NetBox doesn't tell you. If you're in a department that can query that reverse zone, you can just make a query and see what's there already.
• I'm thinking about NetBox as basically a server configuration as if a DNS server ran off the config solely that's defined in NetBox. If that's the case, then what the user wants that isn't currently possible shouldn't get saved.

If you're on the other end of what gets input into NetBox, how do you reconcile what you see as the users inputting bad data and making your server crash? You've defined RR sets can't have different TTLs, yet here we are?

As always I appreciate the discourse and never discount your time as an open source maintainer.

[peteeckel]

Hi @squintfox, I value your input highly ... Don't get me wrong: it's not that I don't like the idea, I just don't currently see a very high priority attached to it.

RFC2317 is only a part of the IPv4 problem: Delegation is hard and can't be done for subnets smaller than /24, but RFC2317 makes it possible for more people to keep in charge of their reverse zones, which again affects automatic PTR generation. This is all much less troublesome with IPv6.

Yes, NetBox DNS (as well as the whole NetBox ecosystem) is designed to be the source of truth, so the data contained there is the input to other systems, not vice versa. As such, the idea is generally to create data that's as correct as possible.

The point in auto-creating PTR records is not documentation, not at all - it's keeping reverse resolution up to date. Querying the reverse zones is not even remotely a solution to that problem, as reverse zones tend to have stale data unless they are managed automatically, and expecting the people in a network department to regularly query PTR records to make sure they are defined and up to date is ... well, you'd be surprised. Not in a pleasant way, though.

There are, however, much more pressing problems that a potential issue with TTLs of PTR records in an edge case, so my priority of finding a resonable solution (which is by no means obvious) isn't very high. Currently I'm porting NetBox DNS to NetBox 4, which is surprisingly much work ... look at #153 and netbox-community/netbox#15520 for an impression of what's going on there.

And don't worry - a name server won't crash if fed with invalid records. In the very worst case it will refuse to load the zone. In the case of inconsistent TTLs I would expect it to give a warning and load the zone anyway, or to fix the TTLs of its own accord - after all, 'deprecated' doesn't mean 'illegal' (as, for instance, with underscores except in some well-defined exceptions).

Anyway, I will definitely think about a solution, but don't hold your breath ... it may take some time, and it will very likely not happen before May, when NetBox 4 will be released. After all, most of the NetBox DNS development is done as a side job in addition to paid consultancy work, although an increasing part of that is related to and relying on it ...

[squintfox]

I may need to brush up on my "practical uses for PTR records for dummies", I may just be missing the forest for the trees a bit on this one. I agree that this is no great priority. Getting the right values into the PTRs all the time is happening already. The TTLs are a trivial part of the equation.

I've been keeping an eye on the work on the plugin and NB4 in general. Fantastic stuff from both you and the NetBox team.