Docker 4.12.0 on Windows throws "The pipe has been ended" error

[mndeveci]

Hi team,

We've started to see error messages with Docker Desktop v4.12.0 on Windows. We are attaching to the container by calling {container_instance}.attach(stream=True, logs=True, demux=True) and then iterating through the logs streams. This is working fine with earlier versions but it started to throw following error with Docker Desktop v4.12.0.

Traceback (most recent call last):
  File "c:\users\administrator\workplace\aws-sam-cli\samcli\local\docker\container.py", line 401, in _write_container_output
    for stdout_data, stderr_data in output_itr:
  File "c:\users\administrator\workplace\aws-sam-cli\venv\lib\site-packages\docker\types\daemon.py", line 32, in __next__
    return next(self._stream)
  File "c:\users\administrator\workplace\aws-sam-cli\venv\lib\site-packages\docker\api\client.py", line 406, in <genexpr>
    gen = (demux_adaptor(*frame) for frame in gen)
  File "c:\users\administrator\workplace\aws-sam-cli\venv\lib\site-packages\docker\utils\socket.py", line 94, in frames_iter_no_tty
    (stream, n) = next_frame_header(socket)
  File "c:\users\administrator\workplace\aws-sam-cli\venv\lib\site-packages\docker\utils\socket.py", line 66, in next_frame_header
    data = read_exactly(socket, 8)
  File "c:\users\administrator\workplace\aws-sam-cli\venv\lib\site-packages\docker\utils\socket.py", line 51, in read_exactly
    next_data = read(socket, n - len(data))
  File "c:\users\administrator\workplace\aws-sam-cli\venv\lib\site-packages\docker\utils\socket.py", line 35, in read
    return socket.recv(n)
  File "c:\users\administrator\workplace\aws-sam-cli\venv\lib\site-packages\docker\transport\npipesocket.py", line 23, in wrapped
    return f(self, *args, **kwargs)
  File "c:\users\administrator\workplace\aws-sam-cli\venv\lib\site-packages\docker\transport\npipesocket.py", line 117, in recv
    err, data = win32file.ReadFile(self._handle, bufsize)
pywintypes.error: (109, 'ReadFile', 'The pipe has been ended.')

A simplified version of how we are reading the logs;

# stdout and stderr are StreamWriter's that is been provided earlier
logs_itr = container.attach(stream=True, logs=True, demux=True)

for stdout_data, stderr_data in output_itr:
  if stdout_data and stdout:
    stdout.write(stdout_data)

  if stderr_data and stderr:
    stderr.write(stderr_data)

See related issue here: aws/aws-sam-cli#4222

Thanks!

[brother-z]

Exact same thing happens for container.exec_run, only downgrade to earlier version of Docker Desktop fixes it.

[nicks]

Thanks for reporting! I know what this is. I filed a longer description of the problem here:
microsoft/go-winio#257

The TLDR is - attach() uses an HTTP duplex socket. Windows' npipe doesn't implement all the semantics of duplex sockets. So every async networking library has to emulate those semantics on top of npipes, and they all ended up with subtle differences in the implementation, leading to compatibility issues.

DD 4.12 has a fix to make this work with libuv. Since docker-py appears to have its own compatibility shim, i think we just need to fix it to not treat a Close() as an error.

[nicks]

standalone repro:

import docker
client = docker.from_env()
c = client.containers.run("busybox:latest", "sleep 5", detach=True)
for out, err in c.attach(stream=True, logs=True, demux=True):
    pass

print("exit")

[milas]

Please upgrade to 6.0.1, which should fix this issue with recent versions of Docker Desktop.

If you still see this error, comment here or create a new issue.

[brother-z]

Works great, updated Docker Desktop to 4.13.1 and my scripts are happy again :)