/var/run/docker.sock->@: write: broken pipe

[azizyemloul]

Since the new docker version using overlay2 as a storage driver, one of our projects stop working randomly complaining about a docker.sock broken pipe.

Docker version 18.09.6, build 481bc77

level=error msg="Handler for GET /v1.22/containers/[id]/json returned error: write unix /var/run/docker.sock->@: write: broken pipe"

http: multiple response.WriteHeader calls

Is there anyone getting the same issue ?

Thanks

[azizyemloul]

When trying to stop odoo and postgresql containers, postgresql say :

LOG:  unexpected EOF on client connection with an open transaction

and docker show

ERROR: [***]  UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=70)

Increasing COMPOSE_HTTP_TIMEOUT does nothing.

With the verbose option to docker-compose down I have the following :

compose.config.config.find: Using configuration files: ./docker-compose.yml
docker.utils.config.find_config_file: Trying paths: ['/home/user/.docker/config.json', '/home/user/.dockercfg']
docker.utils.config.find_config_file: No config file found
docker.utils.config.find_config_file: Trying paths: ['/home/user/.docker/config.json', '/home/user/.dockercfg']
docker.utils.config.find_config_file: No config file found
urllib3.connectionpool._make_request: http://localhost:None "GET /v1.22/version HTTP/1.1" 200 554
compose.cli.command.get_client: docker-compose version 1.24.0, build 0aa5906
docker-py version: 3.7.2
CPython version: 2.7.13
OpenSSL version: OpenSSL 1.1.0j  20 Nov 2018
compose.cli.command.get_client: Docker base_url: http+docker://localhost
compose.cli.command.get_client: Docker version: KernelVersion=4.9.0-9-amd64, Components=[{u'Version': u'18.09.5', u'Name': u'Engine', u'Details': {u'KernelVersion': u'4.9.0-9-amd64', u'Os': u'linux', u'BuildTime': u'2019-04-11T04:10:53.000000000+00:00', u'ApiVersion': u'1.39', u'MinAPIVersion': u'1.12', u'GitCommit': u'e8ff056', u'Arch': u'amd64', u'Experimental': u'false', u'GoVersion': u'go1.10.8'}}], Arch=amd64, BuildTime=2019-04-11T04:10:53.000000000+00:00, ApiVersion=1.39, Platform={u'Name': u'Docker Engine - Community'}, Version=18.09.5, MinAPIVersion=1.12, GitCommit=e8ff056, Os=linux, GoVersion=go1.10.8
compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=False, filters={u'label': [u'com.docker.compose.project=myservice']})
urllib3.connectionpool._make_request: http://localhost:None "GET /v1.22/containers/json?all=0&limit=-1&filters=%7B%22label%22%3A+%5B%22com.docker.compose.project%3Dmyservice%22%5D%7D&trunc_cmd=0&size=0 HTTP/1.1" 200 None
compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 2 items)
compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- (u'4c0da71efaa41add93096fd29ed06cdd3843e49b4970249371c17b62222b27c6')
ERROR: compose.cli.errors.log_timeout_error: An HTTP request took too long to complete. Retry with --verbose to obtain debug information.
If you encounter this issue regularly because of slow network conditions, consider setting COMPOSE_HTTP_TIMEOUT to a higher value (current value: 10).

I must restart the docker systemd service to get it running again.

Help would be appreciated.

Thanks

[azizyemloul]

Switching to aufs instead of overlay2 seems to resolve the problem.

Docker service should be stopped.

All images should be deleted

in /etc/docker/daemon.json ensure the following :

{
  "storage-driver": "aufs"
}

Then the images can be loaded.

[azizyemloul]

Did it once again.

According to this post moby/moby#34954 (comment) the logging is responsible.

Deactivated it with :

• --log-level=critical for odoo
• -c log_min_messages=PANIC -c log_min_error_statement=PANIC for postgres

Will see if it works

[begemotik]

@azizyemloul, did it help when you lowered logging level?

[azizyemloul]

@begemotik not really. I ended up dropping docker-compose and using docker commands directly. That what solved the problem for me.

[d-fence]

Thanks for the explanation.
As it is solved for your use case, I close this issue.

[azizyemloul]

Here is what should resolve the problem :
docker/compose#6678 (comment)