Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make IPv6 Great Again #826

Closed
wants to merge 2 commits into from
Closed

Make IPv6 Great Again #826

wants to merge 2 commits into from

Conversation

tgraf
Copy link
Contributor

@tgraf tgraf commented Dec 18, 2015

This makes IPv6 a proper citizen by:

  • Allowing an endpoint to have only an IPv6 address if network.enableIPv6 is set.
  • Enabling allocation of an IPv6 address if netlabel.EnableIPv6 is set even without the presence of a network.ipamV6Conf.

Existing IPv4 behaviour is kept intact: Unless network.enableIPv6 is explicitly set by the user, an IPv4 configuration failure is considered fatal even in the presence of a working IPv6 configuration.

@@ -931,7 +931,7 @@ func (n *network) ipamAllocateVersion(ipVer int, ipam ipamapi.Ipam) error {
}

if len(*cfgList) == 0 {
if ipVer == 6 {
if ipVer == 6 && !n.enableIPv6 {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this check should instead take into account whether the ipam driver supports auto-allocation of ipv6 pools.
Now that ipamapi.Capability is in master, I'd suggest to add a SupportsAutoIPv6 bool to the ipam driver capability.

In fact the reason why this function was bailing out in case of empty ipamV6Config was just because the in-built ipam driver does not have the capability of self assigning an ipv6 pool from some predefined list.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's fine as well, in fact I would even prefer if the user does not have to explicitly request it.

@aboch
Copy link
Contributor

aboch commented Dec 18, 2015

@tgraf
Thanks for the change and help, we indeed need help to fill in some ipv6 missing parts in libnetwork.

Put few comments. In summary, I think we should not overload the enableIPv6 flag with the allocate only ipV6 meaning.

I see this PR is trying to achieve at least two distinct things:

  • Allow auto allocation of IPv6 pools
  • Allow ipv6 only networks

For the first one I'd suggest to let the ipam driver express he can support it (now we have ipam capability), for the second one I think this has to come from user first class config, maybe an --ipv6only UI flag, as I was also saying in moby/moby#18607 (comment)

WDYT ?

@tgraf
Copy link
Contributor Author

tgraf commented Dec 19, 2015

@aboch What I had in mind is to add a --ipv6[=true] option to network create which sets netlabel.EnableIPv6 which would indicate that IPv6 should be treated the same way as IPv4, i.e. not require a successful IPv4 address configuration in order for the networking setup to succeed. I think we have three scenarios:

  1. Trigger allocation of IPv6 but still fail if IPv4 failed
  2. Trigger allocation of IPv6 and succeed if either IPv6 or IPv4 have succeeded
  3. Enforce an IPv6 only configuration, no IPv4 allocation at all

The IPAM capability allows to do (1) automatically but I doubt we can go to (2) without an additional user option such as --ipv6 as this might break current expectations. Users might rely on this current failure behavior. (2) reflects what this PR reflects so far and (3) would require the --ipv6-only flag that you mentioned.

@aboch
Copy link
Contributor

aboch commented Dec 19, 2015

@tgraf
Thanks for the clarification.

I am not sure about the option 2 you mentioned. Considering the behavior would be controlled by a user specified option, how would it be useful to create a network which may get either an ipv4 or an ipv6 pool without knowing it a priori ?

In a scenario where the ipam driver and network driver are implemented by the same entity, I am guessing why this would seem to be needed: It would give freedom to the network driver to decide whether to assign ipv4 or ipv6 pool for a specific network, without libnetwork complaining about it.

But this would go against the reason for which the IPAM was added into libnetwork in first place, which is to give the docker user control over the ip address management for the containers/networks. The assigned ipam behavior should be deterministic.

Probably we should allow the ipam driver also express its capability in supporting only v4, only v6 or both during registration. This way libnetwork will handle the ipam request accordingly and, as an example, won't ask for ipv4 pool if ipam driver does not or does not want to support it. This would be more explicit and confined to the iapm driver.
At the end user chooses the ipam driver, he is expected to know its behavior/capabilities.

Regarding netlabel.EnableIPv6, its current use has it carrying the semantic enable ipv6 besides ipv4, in order to satisfy the existing docker daemon --ipv6 option.
(There is also a proposal to add it to the docker network ui as first class option, currently in there it can only be passed as driver option).

In other words, ipv6 is currently considered optional and complementary to ipv4.

@tgraf
Copy link
Contributor Author

tgraf commented Dec 22, 2015

@aboch
I've rebased the changes and picked up your suggestion to use a new IPAM capability. The new capability allows the IPAM driver to indicate whether to use IPv4, IPv6, or both. If the capability is not present, the behaviour is as before. This does not require any additional user input at all and the entire decision is delegated to the IPAM.

Let me know what you think and I'll add a couple of test cases as well.

@tgraf
Copy link
Contributor Author

tgraf commented Jan 7, 2016

Rebased. Ping @mavenugo @aboch.

@mavenugo
Copy link
Contributor

mavenugo commented Jan 7, 2016

@tgraf apologies on the delay. we will get to this shortly

@tgraf
Copy link
Contributor Author

tgraf commented Jan 26, 2016

Casual ping @mavenugo

@mavenugo
Copy link
Contributor

@tgraf we couldn't prioritize this for the 1.10 release. And there are known ipv6 limitations which we are attempting to solve in 1.11 & beyond. Please let us know If you are interested in contributing to ipv6 fixes and bring it in par with the ipv4 handling.
As far as this PR goes, we will get to this once the top priority issues for 1.10 is taken care of.

@tgraf
Copy link
Contributor Author

tgraf commented Jan 27, 2016

@mavenugo Sure. What issues are you referring to?

@aidanhs
Copy link
Contributor

aidanhs commented Jan 30, 2016

What I had in mind is to add a --ipv6[=true] option to network create which sets netlabel.EnableIPv6

FYI moby/moby#17513 will do exactly this. The exact detail of what that flag will mean is (of course) up to libnetwork.

@tgraf
Copy link
Contributor Author

tgraf commented Feb 1, 2016

@aidanhs This looked like the right thing to do first but I don't see a reason anymore why the user should be required to opt into IPv6. Whether to use IPv6 or IPv4 or both depends on the infrastructure and should be handled at the driver level. At least that possibility should be given.

@aidanhs
Copy link
Contributor

aidanhs commented Feb 1, 2016

Sure, maybe the default will change, but people may still want to opt out so the flag is still necessary.

@tgraf
Copy link
Contributor Author

tgraf commented Mar 22, 2016

@mavenugo Ping, any interest in going forward with this?

@@ -263,7 +264,10 @@ As of now libnetwork accepts the following capabilities:
It is a boolean value which tells libnetwork whether the ipam driver needs to know the interface MAC address in order to properly process the `RequestAddress()` call.
If true, on `CreateEndpoint()` request, libnetwork will generate a random MAC address for the endpoint (if an explicit MAC address was not already provided by the user) and pass it to `RequestAddress()` when requesting the IP address inside the options map. The key will be the `netlabel.MacAddress` constant: `"com.docker.network.endpoint.macaddress"`.

### SupportsAutoIPv6

This boolean tells libnetwork that the ipam driver is capable of deciding whether an IPv4 address, IPv6 address, or both should be assigned to the container by returning either an address or an empty string in response to `RequestAddress()`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would not overload this flag with two meaning. The name seems clear, it tells whether the IPAM driver can come up with a IPv6 pool if user did not configure one.

@aboch
Copy link
Contributor

aboch commented Mar 22, 2016

@tgraf Sorry for the back and forth.

Looking at your new diffs, I am wondering whether it would be better to follow the approach you are using for request address (driver to return a specific error) for pool request instead, and drop the new capability flag.
At first look, it seems we no longer need a new capability flag and it is better because I feel it is hard to come out with good and meaningful ones which will not need to change in future.

We would just need to error out in case user has specified a --subnet but ipam driver did return a service error.

Do you think it makes sense ?

Also, your changes would likely not make it for 1.11, code freeze is today, but we are committed to have them in 1.12.

Thanks

@jainvipin
Copy link

@aboch, @mavenugo - no milestone attached to it. I was wondering if this PR will be merged sometime soon?

@tgraf
Copy link
Contributor Author

tgraf commented Jun 8, 2016

@aboch I addressed your feedback and reverted the initial suggestion of the capability. The proposal is now back to the original approach. I left out the gateway opt out decision for now. Let me know what you think.

@tgraf tgraf changed the title Make IPv6 a first class citizen Make IPv6 Great Again Jun 19, 2016
@tgraf
Copy link
Contributor Author

tgraf commented Jun 27, 2016

ping @nerdalert @mrjana @mavenugo

@tgraf
Copy link
Contributor Author

tgraf commented Jul 15, 2016

ping @aboch

mikebryant added a commit to kubermesh/kubermesh that referenced this pull request Jul 29, 2016
…es/kubernetes#23090 claims support for IPv6, it turns out that as docker can't give a container an IPv6 address without an IPv4 address, we never hit the code path in kubernetes, so such support doesn't work :(
Allow IPAM plugins to return ipamapi.ErrNoIPReturned to indicate that
no address of the particular address family should be assigned.

Since this is a newly introduced error, there is no risk of an existing
plugin mistakenly switching to this behaviour.

Signed-off-by: Thomas Graf <[email protected]>
This brings IPv6 in line with IPv4 behaviour which does not require the
presence of an ipamV6Config on the network in order to trigger
allocating an address.

Since the enableIPv6 flag is an opt-in, this change does not alter the
default behaviour. The user has clearly indicated intent by explicitly
enabling the flag.

Signed-off-by: Thomas Graf <[email protected]>
@tgraf
Copy link
Contributor Author

tgraf commented Aug 2, 2016

@mrjana @aboch Any feedback on what is left to get this merged? It is blocking various people wanting to run IPv6 only containers.

@mrjana
Copy link
Contributor

mrjana commented Aug 3, 2016

@tgraf The way we have to introduce ipv6 only networks is by introducing an ipv6-only option which is accessible to the user and then we can make necessary changes to say if the network is ipv6-only then we will skip ipv4 allocations/plumbing and just do an e2e ipv6 plumbing and we can also make sure the driver(any driver) can handle an ipv6-only network (or not in which case it returns an error). This ensures that the current behavior doesn't change and the new behavior is only present when ipv6-only option is provided by the user.

@tgraf
Copy link
Contributor Author

tgraf commented Aug 3, 2016

@mrjana Isn't that option already there in the form of --enable-ipv6. It's an explicit opt-in already.

@pneumaticdeath
Copy link

pneumaticdeath commented Aug 29, 2016

Just a vote of support for this. This (and #23090 in kubernetes) are blocking us from adopting k8s/docker as a container solution.

@rsrchboy
Copy link

...and another +1

@choppsv1
Copy link

choppsv1 commented Jul 3, 2017

Really need ipv6 only and auto ipv6 allocation. Hard to do IPv6 development w/o these things.

@agowa
Copy link

agowa commented Dec 14, 2019

@mrjana @aboch @tgraf What's the current status of this pr?
We're still waiting for this in moby/moby#32850
Is this pr dead and someone should fill a new one?

@telmich
Copy link

telmich commented Dec 14, 2019

Very much interested in this, too. Especially because we need to turn IPv4 off in various network constellations nowadays

@Citrullin
Copy link

Would be also interested in this. We should all move to IPv6 only networks.

@murphy83
Copy link

Any Update on this? IPv6-only would save me a lot of hassle with exhausted Ipv4-Networks during VPN-Connections at some customers ...

@frebib
Copy link

frebib commented Nov 22, 2020

It would be great to get this into 20.10. This problem alone is making me consider replacing Docker for something more forward thinking

@Robotic-Brain
Copy link

Robotic-Brain commented Oct 16, 2022

It would be great to get this into 20.10. This problem alone is making me consider replacing Docker for something more forward thinking

Which would be?
I'm desperately searching for an alternative, which doesn't devolve into creating everything from scratch with nspawn!
Something that gives me full control over networking but understands the docker image format would be perfect.

@rpodgorny
Copy link

It would be great to get this into 20.10. This problem alone is making me consider replacing Docker for something more forward thinking

Which would be? I'm desperately searching for an alternative, which doesn't devolve into creating everything from scratch with nspawn! Something that gives me full control over networking but understands the docker image format would be perfect.

podman?

@Robotic-Brain
Copy link

podman?

As far as i can tell podman is just as broken as docker is when it comes to IPv6-only networking because CNI also relies on deprecated NAT technology. (but I'll look into it)

@tgraf tgraf closed this by deleting the head repository Apr 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.