Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix http.client.RemoteDisconnected errors #1014

Merged
merged 1 commit into from
Mar 21, 2020
Merged

Conversation

rhuddleston
Copy link

With this commit 4 years ago OSError was caught:

889c5a7

Looks like there wasn't a good reason to include it give the comment. Also now with python 3 http.client.RemoteDisconnected is getting caught by OSError.

I see @fviard tried to make a fix for RemoteDisconnected here:

b4f1424

but the problem persisted until I removed these lines. There may be other places in the code that have the same issue but this fixed my specific issues with recv_file():

  File "/usr/bin/s3cmd", line 1316, in _download
    response = s3.object_get(uri, dst_stream, dst_file, extra_label = seq_label)
            http_response = conn.c.getresponse()
  File "/usr/lib/python3/dist-packages/S3/S3.py", line 715, in object_get
    response = self.recv_file(request, stream, labels, start_position)
  File "/usr/lib/python3/dist-packages/S3/S3.py", line 1612, in recv_file
    http_response = conn.c.getresponse()
  File "/usr/lib/python3.6/http/client.py", line 1331, in getresponse
    response.begin()
  File "/usr/lib/python3/dist-packages/S3/Custom_httplib3x.py", line 53, in httpresponse_patched_begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.6/http/client.py", line 266, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response```

I deployed this fix and verified it fixed the errors across our servers.

s3tools@889c5a7

Looks like there wasn't a good reason to include it give the comment. Also now with python 3 http.client.RemoteDisconnected is getting caught by OSError.
@rhuddleston
Copy link
Author

@fviard What do you think, can we merge this? Also maybe it's time for a 2.0.3 release :)

@nadiahpk
Copy link

@rhuddleston Thank you! I only started having the problem this week for some reason, and I couldn't figure it out. But this fixed my problem.

@rogierslag
Copy link

Would be awesome to get this to a stable release :)

jkroepke added a commit to adorsys-containers/s3-tools that referenced this pull request Feb 20, 2020
@fviard fviard merged commit 7690455 into s3tools:master Mar 21, 2020
@fviard
Copy link
Contributor

fviard commented Mar 21, 2020

Finally merged, thanks for the investigation and the fix!

@jkroepke
Copy link

Nice, when we can expect a stable release?

@fviard
Copy link
Contributor

fviard commented Mar 25, 2020

@jkroepke In the coming days.

@fviard
Copy link
Contributor

fviard commented Apr 7, 2020

Fyi, the new release v2.1.0 is out and available in github and in Pypi.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants