Skip to content

Conversation

@thaJeztah
Copy link
Member

@thaJeztah thaJeztah commented Oct 8, 2025

vendor: github.com/moby/moby/api, github.com/moby/moby/client master

full diff: moby/moby@4ca8aed...0769fe7

- What I did

- How I did it

- How to verify it

- Human readable description for the release notes

- A picture of a cute animal (not mandatory but encouraged)

Comment on lines 59 to 60
NetworkID string `json:",omitempty"`
Addr netip.Prefix `json:",omitempty"`
Copy link
Member Author

@thaJeztah thaJeztah Oct 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

docker stack deploy --compose-file=./testdata/full-stack.yml --detach=false test-stack-remove
ParseAddr("10.0.1.2/24"): unexpected character (at "/24")
docker stack ls
ParseAddr("10.0.1.2/24"): unexpected character (at "/24")

It looks like the service returns this;

curl -s --unix-socket /var/run/docker.sock http://localhost/v1.51/services/bmu1cozv5cca | jq .Endpoint
{
  "Spec": {
    "Mode": "vip"
  },
  "VirtualIPs": [
    {
      "NetworkID": "yy6jjster3oudzp0ck77ai0id",
      "Addr": "10.0.1.2/24"
    }
  ]
}

So older versions of docker ... returned an address range in CIDR notation?

docker network create --scope=swarm --driver=overlay hello
0k61ecfc29ifqj5lnam9usveu

docker service create --network hello --name foo alpine top
mx0vntfyklgnikrzj59gnmxur
overall progress: 1 out of 1 tasks
1/1: running
verify: Service mx0vntfyklgnikrzj59gnmxur converged

docker service inspect --format '{{json .Endpoint}}' foo
{"Spec":{"Mode":"vip"},"VirtualIPs":[{"NetworkID":"0k61ecfc29ifqj5lnam9usveu","Addr":"10.0.1.2/24"}]}

And looks indeed that .. it does?
https://github.com/moby/moby/blob/b8a4f6534f736bc5223c9ee13d210a55a22481d7/daemon/cluster/executor/container/container.go#L573-L583

Value is set from the Swarm API?
https://github.com/moby/moby/blob/6bbb92df70fb433ce89353e41c2158db67feceb0/daemon/cluster/convert/network.go#L121-L142

@thaJeztah thaJeztah force-pushed the bump_engine_take2 branch 11 times, most recently from 8782f4b to 8f2b791 Compare October 9, 2025 01:52
@thaJeztah
Copy link
Member Author

Not there yet; task listing also fails;

docker stack ps test-stack-remove -f=desired-state=running --format json
ParseAddr("10.0.1.3/24"): unexpected character (at "/24")
curl -s --unix-socket /var/run/docker.sock 'http://localhost/v1.51/tasks' | jq .
[
  {
    "ID": "l4rynxbamvixckla83rcdk557",
    "Version": {
      "Index": 52
    },
    "CreatedAt": "2025-10-09T07:53:35.684913628Z",
    "UpdatedAt": "2025-10-09T07:53:37.761470712Z",
    "Labels": {},
    "Spec": {
      "ContainerSpec": {
        "Image": "alpine:latest@sha256:4b7ce07002c69e8f3d704a9c5d6fd3053be500b7f1c69fc0d80990c2ad8dd412",
        "Labels": {
          "com.docker.stack.namespace": "test-stack-remove"
        },
        "Args": [
          "top"
        ],
        "Privileges": {
          "CredentialSpec": null,
          "SELinuxContext": null,
          "NoNewPrivileges": false
        },
        "Isolation": "default"
      },
      "Resources": {},
      "Placement": {
        "Platforms": [
          {
            "Architecture": "amd64",
            "OS": "linux"
          },
          {
            "Architecture": "unknown",
            "OS": "unknown"
          },
          {
            "OS": "linux"
          },
          {
            "Architecture": "unknown",
            "OS": "unknown"
          },
          {
            "OS": "linux"
          },
          {
            "Architecture": "unknown",
            "OS": "unknown"
          },
          {
            "Architecture": "arm64",
            "OS": "linux"
          },
          {
            "Architecture": "unknown",
            "OS": "unknown"
          },
          {
            "Architecture": "386",
            "OS": "linux"
          },
          {
            "Architecture": "unknown",
            "OS": "unknown"
          },
          {
            "Architecture": "ppc64le",
            "OS": "linux"
          },
          {
            "Architecture": "unknown",
            "OS": "unknown"
          },
          {
            "Architecture": "riscv64",
            "OS": "linux"
          },
          {
            "Architecture": "unknown",
            "OS": "unknown"
          },
          {
            "Architecture": "s390x",
            "OS": "linux"
          },
          {
            "Architecture": "unknown",
            "OS": "unknown"
          }
        ]
      },
      "Networks": [
        {
          "Target": "c1i5g04zw3zkznjvjbbv4iizc",
          "Aliases": [
            "one"
          ]
        }
      ],
      "ForceUpdate": 0
    },
    "ServiceID": "kp75t36rdju3q9sys3ir38rd4",
    "Slot": 1,
    "NodeID": "n1rgi7t220c5lgsvwu2so3krr",
    "Status": {
      "Timestamp": "2025-10-09T07:53:37.718913879Z",
      "State": "running",
      "Message": "started",
      "ContainerStatus": {
        "ContainerID": "a1693df4c760b41d38e3c13027fcdd5a88040b0f267dfcad2ace4a66e29f04d2",
        "PID": 69448,
        "ExitCode": 0
      },
      "PortStatus": {}
    },
    "DesiredState": "running",
    "NetworksAttachments": [
      {
        "Network": {
          "ID": "c1i5g04zw3zkznjvjbbv4iizc",
          "Version": {
            "Index": 38
          },
          "CreatedAt": "2025-10-09T07:53:33.956330835Z",
          "UpdatedAt": "2025-10-09T07:53:33.957696627Z",
          "Spec": {
            "Name": "test-stack-remove_default",
            "Labels": {
              "com.docker.stack.namespace": "test-stack-remove"
            },
            "DriverConfiguration": {
              "Name": "overlay"
            },
            "Scope": "swarm"
          },
          "DriverState": {
            "Name": "overlay",
            "Options": {
              "com.docker.network.driver.overlay.vxlanid_list": "4098"
            }
          },
          "IPAMOptions": {
            "Driver": {
              "Name": "default"
            },
            "Configs": [
              {
                "Subnet": "10.0.1.0/24",
                "Gateway": "10.0.1.1"
              }
            ]
          }
        },
        "Addresses": [
          "10.0.1.3/24"
        ]
      }
    ],
    "Volumes": null
  },
  {
    "ID": "to1zvgc2flky65q092g9umvr6",
    "Version": {
      "Index": 53
    },
    "CreatedAt": "2025-10-09T07:53:37.485566337Z",
    "UpdatedAt": "2025-10-09T07:53:37.964857212Z",
    "Labels": {},
    "Spec": {
      "ContainerSpec": {
        "Image": "alpine:latest@sha256:4b7ce07002c69e8f3d704a9c5d6fd3053be500b7f1c69fc0d80990c2ad8dd412",
        "Labels": {
          "com.docker.stack.namespace": "test-stack-remove"
        },
        "Args": [
          "top"
        ],
        "Privileges": {
          "CredentialSpec": null,
          "SELinuxContext": null,
          "NoNewPrivileges": false
        },
        "Isolation": "default"
      },
      "Resources": {},
      "Placement": {
        "Platforms": [
          {
            "Architecture": "amd64",
            "OS": "linux"
          },
          {
            "Architecture": "unknown",
            "OS": "unknown"
          },
          {
            "OS": "linux"
          },
          {
            "Architecture": "unknown",
            "OS": "unknown"
          },
          {
            "OS": "linux"
          },
          {
            "Architecture": "unknown",
            "OS": "unknown"
          },
          {
            "Architecture": "arm64",
            "OS": "linux"
          },
          {
            "Architecture": "unknown",
            "OS": "unknown"
          },
          {
            "Architecture": "386",
            "OS": "linux"
          },
          {
            "Architecture": "unknown",
            "OS": "unknown"
          },
          {
            "Architecture": "ppc64le",
            "OS": "linux"
          },
          {
            "Architecture": "unknown",
            "OS": "unknown"
          },
          {
            "Architecture": "riscv64",
            "OS": "linux"
          },
          {
            "Architecture": "unknown",
            "OS": "unknown"
          },
          {
            "Architecture": "s390x",
            "OS": "linux"
          },
          {
            "Architecture": "unknown",
            "OS": "unknown"
          }
        ]
      },
      "Networks": [
        {
          "Target": "c1i5g04zw3zkznjvjbbv4iizc",
          "Aliases": [
            "two"
          ]
        }
      ],
      "ForceUpdate": 0
    },
    "ServiceID": "o5gzytctv0si0j2mm81cdh58b",
    "Slot": 1,
    "NodeID": "n1rgi7t220c5lgsvwu2so3krr",
    "Status": {
      "Timestamp": "2025-10-09T07:53:37.889173462Z",
      "State": "running",
      "Message": "started",
      "ContainerStatus": {
        "ContainerID": "1f80279860feedcd36162b757a285de87a687893a679da7d5316d07872bc434c",
        "PID": 69504,
        "ExitCode": 0
      },
      "PortStatus": {}
    },
    "DesiredState": "running",
    "NetworksAttachments": [
      {
        "Network": {
          "ID": "c1i5g04zw3zkznjvjbbv4iizc",
          "Version": {
            "Index": 38
          },
          "CreatedAt": "2025-10-09T07:53:33.956330835Z",
          "UpdatedAt": "2025-10-09T07:53:33.957696627Z",
          "Spec": {
            "Name": "test-stack-remove_default",
            "Labels": {
              "com.docker.stack.namespace": "test-stack-remove"
            },
            "DriverConfiguration": {
              "Name": "overlay"
            },
            "Scope": "swarm"
          },
          "DriverState": {
            "Name": "overlay",
            "Options": {
              "com.docker.network.driver.overlay.vxlanid_list": "4098"
            }
          },
          "IPAMOptions": {
            "Driver": {
              "Name": "default"
            },
            "Configs": [
              {
                "Subnet": "10.0.1.0/24",
                "Gateway": "10.0.1.1"
              }
            ]
          }
        },
        "Addresses": [
          "10.0.1.6/24"
        ]
      }
    ],
    "Volumes": null
  }
]

@codecov-commenter
Copy link

codecov-commenter commented Oct 9, 2025

@thaJeztah
Copy link
Member Author

Getting close now; remaining failures;

Error: something went wrong
--- FAIL: TestParseWithExpose (0.00s)
    opts_test.go:461: Expected 1 exposed port, got 0
Error: 
Error: 

Looks like --ip-range accepts / accepted some fuzzy formats;

  • CIDR ("255.255.0.0.30/24")
  • <start IP>-<end IP> ("192.168.83.1-192.168.83.254")
--- FAIL: TestNetworkCreateErrors (0.00s)
    --- FAIL: TestNetworkCreateErrors/#03 (0.00s)
        create_test.go:150: assertion failed: error is not nil: invalid argument "255.255.0.0.30/24" for "--ip-range" flag: invalid string being converted to CIDR: 255.255.0.0.30/24
    --- FAIL: TestNetworkCreateErrors/#08 (0.00s)
        create_test.go:154: assertion failed: expected error to contain "cannot configure multiple ranges (192.168.1.200/24, 192.168.1.0/24) on the same subnet (192.168.1.250/24)", got "cannot configure multiple ranges (192.168.1.0/24, 192.168.1.0/24) on the same subnet (192.168.1.250/24)"
    --- FAIL: TestNetworkCreateErrors/#12 (0.00s)
        create_test.go:154: assertion failed: expected an error, got nil
    --- FAIL: TestNetworkCreateErrors/#13 (0.00s)
        create_test.go:150: assertion failed: error is not nil: invalid argument "192.168.83.1-192.168.83.254" for "--ip-range" flag: invalid string being converted to CIDR: 192.168.83.1-192.168.83.254
FAIL

@thaJeztah
Copy link
Member Author

Looks like --ip-range accepts / accepted some fuzzy formats;

  • CIDR ("255.255.0.0.30/24")
  • <start IP>-<end IP> ("192.168.83.1-192.168.83.254")

Oh! I was wrong there; it did NOT accept that, but previously the cobra command would fail, and now the flag itself invalidates it; it expects an error for that test;

flags: map[string]string{
"ip-range": "192.168.83.1-192.168.83.254",
"gateway": "192.168.80.1",
"subnet": "192.168.80.0/20",
},
expectedError: "invalid CIDR address: 192.168.83.1-192.168.83.254",

But expected the flag parsing to not fail (as it was a []string, so no special handling);

for key, value := range tc.flags {
assert.NilError(t, cmd.Flags().Set(key, value))
}

@thaJeztah thaJeztah force-pushed the bump_engine_take2 branch 4 times, most recently from bfdd48d to 6974f4f Compare October 9, 2025 10:31
@thaJeztah thaJeztah force-pushed the bump_engine_take2 branch 3 times, most recently from d7f690c to 10c859d Compare October 9, 2025 12:57
@thaJeztah
Copy link
Member Author

thaJeztah commented Oct 9, 2025

I continued looking at this failure;

#18 63.28 === FAIL: cli/command/network TestNetworkCreateErrors/ip-range=192.168.1.0/24,192.168.1.200/24,gateway=192.168.1.1,192.168.1.4,subnet=192.168.2.0/24,192.168.1.250/24,toto (0.00s)
#18 63.28     create_test.go:153: assertion failed: expected error to contain "cannot configure multiple ranges (192.168.1.200/24, 192.168.1.0/24) on the same subnet (192.168.1.250/24)", got "cannot configure multiple ranges (192.168.1.0/24, 192.168.1.0/24) on the same subnet (192.168.1.250/24)"
#18 63.28 
#18 63.28 === FAIL: cli/command/network TestNetworkCreateErrors/gateway=255.255.0.0,subnet=255.255.0.0/24,aux-address=255.255.0.30/24,toto (0.00s)
#18 63.28     create_test.go:153: assertion failed: expected an error, got nil

And.... LOL... it's just because the test is .. bad? The flag returns the same value twice because they parse to the same 😂 https://go.dev/play/p/r7a0txKjths

package main

import (
	"fmt"
	"net"
	"strings"
)

func main() {
	for _, ipNetStr := range []string{"192.168.1.0/24", "192.168.1.200/24"} {
		_, n, err := net.ParseCIDR(strings.TrimSpace(ipNetStr))
		if err != nil {
			panic(err)
		}
		fmt.Println("parsed:", ipNetStr, "->", n.String())
	}
}

Above outputs;

parsed: 192.168.1.0/24 -> 192.168.1.0/24
parsed: 192.168.1.200/24 -> 192.168.1.0/24

@thaJeztah thaJeztah force-pushed the bump_engine_take2 branch 3 times, most recently from 57ce7eb to 00489b7 Compare October 9, 2025 15:29
@thaJeztah thaJeztah force-pushed the bump_engine_take2 branch 5 times, most recently from 559bc11 to 9818ee0 Compare October 9, 2025 21:16
| Name | Type | Default | Description |
|:---------------------------------------|:---------|:--------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [`-f`](#filter), [`--filter`](#filter) | `filter` | | Filter output based on conditions provided |
| [`-f`](#filter), [`--filter`](#filter) | `filter` | `{}` | Filter output based on conditions provided |
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wondering if we can hide the default for these (as {} isn't very informative for the user)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess one option would be to not initialize that variable:

listOpts := listOptions{filter: opts.NewFilterOpt()}

and then initialize it just before doing Add in:

o.filter.Add(name, val)

?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I could play with that idea. Definitely not the most urgent I guess; also because we need to look at some of those custom options and if they're still needed / critical or if we could use more standard options / flags.

Comment on lines +63 to +64
| `--ip` | `ip` | `<nil>` | IPv4 address (e.g., 172.30.100.104) |
| `--ip6` | `ip` | `<nil>` | IPv6 address (e.g., 2001:db8::33) |
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same for these

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few optional options just have an empty field where the default is empty (--hostname, --expose etc)... maybe that'd be best here too?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I want to look what's needed to hide this; need to look how the docs-tool gets the information from the flags (possibly the flag itself returns <nil> as a string for empty values.

Comment on lines 72 to 76
pruneFilters := options.filter.Value().Clone()
pruneFilters := maps.Clone(options.filter.Value())
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs a different approach; maps.Clone does not create a deep-copy, so mutating the filter still mutates the original.

I have a draft (will push) to add a .Clone() method to the client.Filters type, but I can start with an internal package in the CLI to do a deep clone in case we're the only one needing that.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

☝️ added some local utilities for this, and opened a draft PR in moby;


// Populate non-overlapping subnets into consolidation map
for _, s := range options.subnets {
// TODO(thaJeztah): is all this validation needed on the CLI-side?
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is ... because options like --subnet 192.0.2.0/24 --subnet 198.51.100.0/24 --subnet 2001:db8:1234::/64 --gateway 198.51.100.0 --ip-range 192.0.2.128/25 need to be collated into three network.IPAMConfig objects (one per subnet) - and the only way to work out which options belong in which IPAMConfig is to understand the subnets and addresses.

And, for example ... if there are overlapping subnets the other options may be ambiguous, if there's more than one gateway in a subnet there's no way to represent that in IPAMConfig, etc.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, yes, you mentioned that, and I guess I didn't really register that. So, yes, looks like we still need this, but perhaps we should have a utility for that in the client; at least once we have functional options, this seems like something ideal to be handled there, so that others don't have to re-invent the exact same logic WDYT?

I guess we could already do that before the functional options 🤔

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It DOES make me wonder though if the ipamOptions struct would've been the better type for the client.NetworkCreateOptions

type ipamOptions struct {
driver string
subnets []string
ipRanges []string
gateways []string
auxAddresses opts.MapOpts
driverOpts opts.MapOpts
}

Because ultimately it means that we expand the ipamOptions into a network.IPAM struct, so that single options struct into a slice of []network.IPAMConfig, but all data needed for that is already captured in ipamOptions?

// IPAM represents IP Address Management
type IPAM struct {
Driver string
Options map[string]string // Per network IPAM driver options
Config []IPAMConfig
}
// IPAMConfig represents IPAM configurations
type IPAMConfig struct {
Subnet string `json:",omitempty"`
IPRange string `json:",omitempty"`
Gateway string `json:",omitempty"`
AuxAddress map[string]string `json:"AuxiliaryAddresses,omitempty"`
}

Or would there be cases where manually crafting the []network.IPAMConfig is something that would be done? 🤔

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So the IPAM field in client.NetworkCreateOptions could have a type like ipamOptions instead of network.IPAM ... and we'd do all this validation/collation in Client.NetworkCreate, in order to construct the API request?

Yes, that could be good.

Trying to think of reasons to construct it manually, not coming up with much ... in IPv6, the gateway doesn't need to be part of the subnet. For us, that'd be most likely to be useful for macvlan/ipvlan-l2 networks (not regular bridge networks anyway). So, to support that using ipamOptions in a network with more than one IPv6 subnet, I think we'd need a way to work out which one the gateway belonged to.

As you say, functional options will make it easier.

Comment on lines +63 to +64
| `--ip` | `ip` | `<nil>` | IPv4 address (e.g., 172.30.100.104) |
| `--ip6` | `ip` | `<nil>` | IPv6 address (e.g., 2001:db8::33) |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few optional options just have an empty field where the default is empty (--hostname, --expose etc)... maybe that'd be best here too?

Copy link
Contributor

@robmry robmry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM (with the planned squash!).

@thaJeztah
Copy link
Member Author

Thank YOU!!! Gonna do a big squash now 👍

@thaJeztah thaJeztah changed the title vendor: github.com/moby/moby/api, github.com/moby/moby/client master (takę 2) vendor: github.com/moby/moby/api, client 0769fe708773 (master) Oct 10, 2025
@thaJeztah thaJeztah marked this pull request as ready for review October 10, 2025 17:36
@thaJeztah thaJeztah requested review from a team and silvin-lubecki as code owners October 10, 2025 17:36
@thaJeztah thaJeztah merged commit a3e9545 into docker:master Oct 10, 2025
147 of 148 checks passed
@thaJeztah thaJeztah deleted the bump_engine_take2 branch October 10, 2025 17:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants