Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

--api should accept dns addresses #5249

Closed
anguslees opened this issue Jul 18, 2018 · 9 comments · Fixed by #5372
Closed

--api should accept dns addresses #5249

anguslees opened this issue Jul 18, 2018 · 9 comments · Fixed by #5372
Labels
help wanted Seeking public contribution on this issue kind/feature A new feature

Comments

@anguslees
Copy link

Version information:

go-ipfs version: 0.4.16-92ac43a
Repo version: 7
System version: amd64/linux
Golang version: go1.10.3

Type:

Enhancement

Description:

I am running ipfs daemon on a home kubernetes cluster. The daemon runs fine. I would like to talk to the daemon's API port from another kubernetes pod, but the ipfs --api arg seems to only accept IP address literals (afaict?). In Kubernetes (and many other containerised) environments, the ipfs daemon's IP address is allocated dynamically, and can change on restarts. Services are usually found via DNS hostnames.

TL;DR: I want something like this to just work: ipfs --api /dnsaddr/api.ipfs.svc/tcp/5001

Right now that gives "Error: /dnsaddr/api.ipfs.svc/tcp/5001 is not a 'thin waist' address"

As far as I know, my only option right now is to wrap ipfs in some shell script that does a dns lookup and builds a /ip4/... address on the fly :(

@magik6k magik6k added the help wanted Seeking public contribution on this issue label Jul 18, 2018
@Stebalien Stebalien added the kind/feature A new feature label Jul 18, 2018
@Stebalien
Copy link
Member

For anyone fixing this: You'll need to use github.com/multiformats/go-multiaddr-dns to resolve the addr in apiClientForAddr in cmd/ipfs/main.go. You'll probably also want to feed in a context from the command.

@raulk
Copy link
Member

raulk commented Aug 6, 2018

@Stebalien – taking this one, if it's ok. The instructions were useful, thanks!

@Stebalien
Copy link
Member

That would be awesome!

@raulk
Copy link
Member

raulk commented Aug 11, 2018

@Stebalien – The way I see it we have several options:

  • introduced a load-balanced form of http.Client that supports different strategies (failover, round-robin, etc. defaulting to failover such that all requests are routed to the same backing client until a failure happens).
  • use net.Dial to test routability to each address, and pick the first one that succeeds.
  • create a http.Client for each result, dispatch a version command to each sequentially, and select the first one to respond successfully.
  • fail if the DNS lookup returns multiple results (not recommended).
  • pick the first lookup result (simplest solution).

Any thoughts?

@overbool – sorry, I had already been working on this issue as stated above! Happy to hear your thoughts, and if you'd like to collaborate on this one, or how else you'd like us to proceed!

@anguslees
Copy link
Author

Just try each address in the order the DNS server gave them to you is what a "normal" Unix/C client does when it gets multiple DNS responses.

"Fancy" clients like web browsers usually do something more complex like try address families in parallel, etc.

raulk pushed a commit to raulk/go-ipfs that referenced this issue Aug 11, 2018
@raulk
Copy link
Member

raulk commented Aug 11, 2018

@anguslees agree, I was thinking along those lines too for now.

Here's an initial stab to see if I'm headed in the right direction => raulk@c833393

Tests missing. Do we want to set specific timeouts for DNS resolution and socket tests?

@raulk
Copy link
Member

raulk commented Aug 11, 2018

@anguslees – I've done a bit more thinking. Ultimately I think we want to wrap all DNS results (IP addrs) in the HTTP client, and have the latter perform the failover in the order they were received.

But given the decentralised nature of IPFS, this could introduce some inconsistency as the IPFS nodes behind each IP address could hold a different state. So there is a tradeoff:

a) Pick one IP address to build the client with. If it becomes non-reachable, client fails => consistent but non-resilient.
b) Create an HTTP client with multiple backing IP addresses in failover fashion. If one fails, move on to the next reachable one => more resilient but may see inconsistent results.

Ultimately I guess we want (b), but wanted to put these thoughts out there.

@Stebalien
Copy link
Member

I'd just go with the simple solution and pick the first (don't try it, just use it). It's up to the user to make sure that the address is stable and works.

Eventually, we can do something smarter like pass the multiaddr into the commands lib and have that figure things out but the simple solution should be fine for now.

raulk pushed a commit to raulk/go-ipfs that referenced this issue Aug 11, 2018
Resolves via multiaddr-dns, and picks the first result. Uses a fixed
timeout of 10 seconds. Adds test cases for one, multiple, and no DNS
results.
raulk pushed a commit to raulk/go-ipfs that referenced this issue Aug 11, 2018
raulk pushed a commit to raulk/go-ipfs that referenced this issue Aug 11, 2018
Resolves ipfs#5249. Calls multiaddr-dns, and picks the first result.
Uses a fixed timeout of 10 seconds. Adds test cases for one, multiple,
and no DNS results.

License: MIT
Signed-off-by: Raúl Kripalani <[email protected]>
raulk pushed a commit to raulk/go-ipfs that referenced this issue Aug 11, 2018
raulk pushed a commit to raulk/go-ipfs that referenced this issue Aug 11, 2018
Resolves ipfs#5249. Calls multiaddr-dns, and picks the first result.
Uses a fixed timeout of 10 seconds. Adds test cases for one, multiple,
and no DNS results.

License: MIT
Signed-off-by: Raúl Kripalani <[email protected]>
@raulk
Copy link
Member

raulk commented Aug 12, 2018

PR here => #5372.

Stebalien pushed a commit to raulk/go-ipfs that referenced this issue Aug 23, 2018
Stebalien pushed a commit to raulk/go-ipfs that referenced this issue Aug 23, 2018
Resolves ipfs#5249. Calls multiaddr-dns, and picks the first result.
Uses a fixed timeout of 10 seconds. Adds test cases for one, multiple,
and no DNS results.

License: MIT
Signed-off-by: Raúl Kripalani <[email protected]>
keks pushed a commit that referenced this issue Aug 27, 2018
License: MIT
Signed-off-by: Raúl Kripalani <[email protected]>
keks pushed a commit that referenced this issue Aug 27, 2018
Resolves #5249. Calls multiaddr-dns, and picks the first result.
Uses a fixed timeout of 10 seconds. Adds test cases for one, multiple,
and no DNS results.

License: MIT
Signed-off-by: Raúl Kripalani <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Seeking public contribution on this issue kind/feature A new feature
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants