-
-
Notifications
You must be signed in to change notification settings - Fork 3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: DNS.MaxCacheTTL for DNS-over-HTTPS resolvers #8615
Conversation
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@thibmeu mind elaborating why you want to cap the Time-To-Live suggested by the DNS response? What about being able to set the minimum, or override TTL for IPNS records as well?
FYSA I am looking into adding a proper cache-control
header to all HTTP responses under /ipns/
namespace (https://github.com/ipfs/go-ipfs/issues/1818#issuecomment-1015849462) – Gateway is not utilizing TTL from DNS at all right now. This means your PR will not have much impact on gateway responses, but maybe you have a different goal?
My understanding is we want to fix cache control in Gateway responses for /ipns/
, and need to wait for #1818. After we wire up TTLs, and have both cache-control
and etag
in Gateway responses for /ipns/
content paths, then we can pick this PR up and tweak cache-control
/ based on TTL minimum and maximum for both IPNS and DNSLink records.
I believe we need to have config option for both min and max, because there are people setting records with very small TTL such as 60 seconds, and Gateway operators should be able to override the minimum to be higher, to leverage the cache more.
If we want to control this at the record cache level, then we need four OptionalDurations:
- min/max for DNSLink (and multiaddrs which resolve DNS)
DNS.MaxCacheTTL
DNS.MinCacheTTL
- min/max for IPNS records
Ipns.MaxCacheTTL
Ipns.MinCacheTTL
Instead, we could introduce min/max specific to Gateway.
It would be applied at the very end, when cache-control
header is calculated for both DNSLink and IPNS records:
Gateway.MaxCacheTTL
Gateway.MinCacheTTL
Of course, /ipfs/
would remain immutable,
min/max would be applied only to mutable responses under /ipns/
(including DNSLink websites).
I am leaning towards adding Gateway.*
ones for now, but lmk your thoughts and use case, perhaps there is a better/simpler way for controlling this?
@lidel The goal with this PR is to allow update to be propagated faster to the gateway. At the moment, DNS requests are cached for the duration of their TTL. There is no option to customise this. As a node operator, I would like to decide how long these DNS responses are cached for. Then, when a DNSLink is updated, it is seen by the gateway within a bounded timeframe (up to Regarding having |
- Clarified it is applied only to DoH - Replaced broken example of DoH resolver - Switched to released version of libp2p/go-doh-resolver
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you @thibmeu, in this context it makes sense to move forward with this PR: it is not directly related to https://github.com/ipfs/go-ipfs/issues/1818 and brings value on its own.
Pushed some cosmetic changes:
- rebased to fix some CI issues
- switched to release version of [email protected] and go-ipfs-config from upstream master (merged feat: Add DNS.MaxCacheTTL for DNS-over-HTTPS resolvers go-ipfs-config#161)
- tweaked docs: given that we don't control cache of the default (OS-level) resolver, made it clear that this setting requires DoH
- fixed broken DoH examples while at it
LGTM and ready to merge, but parking until we tag a new release of go-ipfs-config
.
@aschmahmann are you aware of any other config changes in 0.13 that we should wait for?
If not, I'd like to tag and merge this to clear up the queue.
Yes, there will likely be some related to the next go-libp2p release
🤷. It doesn't cost us anything to cut more go-ipfs-config releases so we can go ahead and merge + release. I don't know how many people are running builds from master though/if it really makes a difference though. Certainly if you think that we'll have to worry about rebasing the PRs and dealing with conflicts here then definitely just tag + merge. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Switched to go-ipfs-config v0.19.0, this is ready for merge.
Next step: review the help text. |
|
||
Default: Respect DNS Response TTL | ||
|
||
Type: `optionalDuration` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For completeness, can we add optionalDuration
to the Types
section?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good idea, but opened #8729 to not block this PR.
Co-authored-by: Gus Eggert <[email protected]>
Expose
WithMaxCacheTTL
option of go-doh-resolver within IPFS node config.This allows to cap the Time-To-Live suggested by the DNS response. This includes resolution of DNSLink captured by the custom resolver URLs.
This change relies on go-ipfs.config change. The change of go.mod and go.sum would have to be updated accordingly at this time.
Before merging, both go-doh-resolver and go-ipfs-config have to be updated.
Tagging @vyzo and @marten-seemann , as they reviewed the go-doh-resolver change.