Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SendGrid Library misses timeout via URLError #104

Closed
urda opened this issue Jun 4, 2015 · 20 comments
Closed

SendGrid Library misses timeout via URLError #104

urda opened this issue Jun 4, 2015 · 20 comments

Comments

@urda
Copy link

urda commented Jun 4, 2015

I'm working with sendgrid-python with a python 3 project. While attempting to simulate a timeout from SendGrid for our project I noticed that sendgrid-python does not deal with timeouts if urllib.error.URLError becomes involved.

  1. I set my /etc/hosts file to redirect api.sendgrid.com to an address that will trigger a timeout
  2. I ran my application using the sendgrid-python library
  3. At this point I expected a SendGridClientError(408, 'Request timeout') to appear, instead I got a URLError exception

I connected my debugger and stepped through. I noticed that sendgrid-python handles socket.timeout but that becomes covered by urllib.error.URLError: <urlopen error timed out>, causing an exception that is NOT a type of SendGridError

Suggestion: Can sendgrid-python be updated to handle urllib.error.URLError types?

Related stacktrace:

Traceback (most recent call last):
  File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 1182, in do_open
    h.request(req.get_method(), req.selector, req.data, headers)
  File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/http/client.py", line 1088, in request
    self._send_request(method, url, body, headers)
  File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/http/client.py", line 1126, in _send_request
    self.endheaders(body)
  File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/http/client.py", line 1084, in endheaders
    self._send_output(message_body)
  File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/http/client.py", line 922, in _send_output
    self.send(msg)
  File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/http/client.py", line 857, in send
    self.connect()
  File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/http/client.py", line 1223, in connect
    super().connect()
  File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/http/client.py", line 834, in connect
    self.timeout, self.source_address)
  File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/socket.py", line 512, in create_connection
    raise err
  File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/socket.py", line 503, in create_connection
    sock.connect(sa)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/PATH/TO/MY/python.py", line XXX, in send
    status_code, body = self.sendgrid_client.send(message)
  File "/PATH/TO/VIRTUAL/ENV/lib/python3.4/site-packages/sendgrid/sendgrid.py", line 114, in send
    return self._raising_send(message)
  File "/PATH/TO/VIRTUAL/ENV/lib/python3.4/site-packages/sendgrid/sendgrid.py", line 128, in _raising_send
    return self._make_request(message)
  File "/PATH/TO/VIRTUAL/ENV/lib/python3.4/site-packages/sendgrid/sendgrid.py", line 108, in _make_request
    response = urllib_request.urlopen(req, timeout=10)
  File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 161, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 463, in open
    response = self._open(req, data)
  File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 481, in _open
    '_open', req)
  File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 441, in _call_chain
    result = func(*args)
  File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 1225, in https_open
    context=self._context, check_hostname=self._check_hostname)
  File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 1184, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error timed out>
@tgwizard
Copy link

Bump, 👍 on this.

@thinkingserious
Copy link
Contributor

@urda Thanks for submitting this issue!

I'm having trouble getting the URLError exception to trigger.

Here is how I tried to reproduce:
I set self.mail_url = "http://localhost/timeout.php" where timeout.php just sleeps for 5 minutes ( getting /etc/hosts working correctly on OS X Mavericks is apparently non trivial :( )

If I have self._raise_errors = opts.get('raise_errors', True) I get:
(408, timeout('timed out',))

If I have self._raise_errors = opts.get('raise_errors', False) I get:

Traceback (most recent call last):
  File "/Users/thinkingserious/Workspace/sendgrid-python/sendgrid/sendgrid.py", line 133, in _raising_send
    return self._make_request(message)
  File "/Users/thinkingserious/Workspace/sendgrid-python/sendgrid/sendgrid.py", line 111, in _make_request
    response = urllib_request.urlopen(req, timeout=10)
  File "/usr/local/Cellar/python3/3.4.3_2/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 161, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/local/Cellar/python3/3.4.3_2/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 463, in open
    response = self._open(req, data)
  File "/usr/local/Cellar/python3/3.4.3_2/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 481, in _open
    '_open', req)
  File "/usr/local/Cellar/python3/3.4.3_2/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 441, in _call_chain
    result = func(*args)
  File "/usr/local/Cellar/python3/3.4.3_2/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 1210, in http_open
    return self.do_open(http.client.HTTPConnection, req)
  File "/usr/local/Cellar/python3/3.4.3_2/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 1185, in do_open
    r = h.getresponse()
  File "/usr/local/Cellar/python3/3.4.3_2/Frameworks/Python.framework/Versions/3.4/lib/python3.4/http/client.py", line 1171, in getresponse
    response.begin()
  File "/usr/local/Cellar/python3/3.4.3_2/Frameworks/Python.framework/Versions/3.4/lib/python3.4/http/client.py", line 351, in begin
    version, status, reason = self._read_status()
  File "/usr/local/Cellar/python3/3.4.3_2/Frameworks/Python.framework/Versions/3.4/lib/python3.4/http/client.py", line 313, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/local/Cellar/python3/3.4.3_2/Frameworks/Python.framework/Versions/3.4/lib/python3.4/socket.py", line 374, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "test.py", line 11, in <module>
    ret = sg.send(message)
  File "/Users/thinkingserious/Workspace/sendgrid-python/sendgrid/sendgrid.py", line 117, in send
    return self._raising_send(message)
  File "/Users/thinkingserious/Workspace/sendgrid-python/sendgrid/sendgrid.py", line 144, in _raising_send
    raise SendGridClientError(408, 'Request timeout')
sendgrid.exceptions.SendGridClientError: (408, 'Request timeout')

I am testing using Python 3.4.3

@thinkingserious
Copy link
Contributor

@tgwizard are you still experiencing this issue also?

@tgwizard
Copy link

@thinkingserious I'm sorry, we never had this exact issue. We are unfortunately using an older version of sendgrid-python, and we get different exceptions for the timeouts (SGServiceException: <urlopen error [Errno 8] _ssl.c:510: EOF occurred in violation of protocol>).

I will upgrade when #110 is fixed.

@thinkingserious
Copy link
Contributor

Thanks, I appreciate the follow up!

@thinkingserious
Copy link
Contributor

@urda Are you still experiencing this issue?

@urda
Copy link
Author

urda commented Sep 30, 2015

@thinkingserious I haven't had a chance to re-test it yet since this has been updated 😦 I'll try and make time for it later this week / weekend.

@thinkingserious
Copy link
Contributor

No worries, I just want to make sure we help you :)

@urda
Copy link
Author

urda commented Sep 30, 2015

Sounds good! I'll report back soon.

@urda
Copy link
Author

urda commented Oct 26, 2015

Sorry for the delay, today's pip update pushed me to update this issue.

Reproduction Steps

  1. Added an entry 10.255.255.1 api.sendgrid.com into /etc/hosts to simulate the timeout.
  2. Ran the following python:
import sendgrid

sg = sendgrid.SendGridClient('REDACTED', 'REDACTED', raise_errors=True)

message = sendgrid.Mail()
message.add_to('Urda <urda@my_domain.com>')
message.set_subject('Example')
message.set_text('Body')
message.set_from('noreply@my_domain.com')

status, msg = sg.send(message)

Observed the following exception and stack trace:

---------------------------------------------------------------------------
timeout                                   Traceback (most recent call last)
/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/urllib/request.py in do_open(self, http_class, req, **http_conn_args)
   1239             try:
-> 1240                 h.request(req.get_method(), req.selector, req.data, headers)
   1241             except OSError as err: # timeout error

/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py in request(self, method, url, body, headers)
   1082         """Send a complete request to the server."""
-> 1083         self._send_request(method, url, body, headers)
   1084 

/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py in _send_request(self, method, url, body, headers)
   1127             body = body.encode('iso-8859-1')
-> 1128         self.endheaders(body)
   1129 

/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py in endheaders(self, message_body)
   1078             raise CannotSendHeader()
-> 1079         self._send_output(message_body)
   1080 

/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py in _send_output(self, message_body)
    910 
--> 911         self.send(msg)
    912         if message_body is not None:

/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py in send(self, data)
    853             if self.auto_open:
--> 854                 self.connect()
    855             else:

/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py in connect(self)
   1228 
-> 1229             super().connect()
   1230 

/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py in connect(self)
    825         self.sock = self._create_connection(
--> 826             (self.host,self.port), self.timeout, self.source_address)
    827         self.sock.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)

/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/socket.py in create_connection(address, timeout, source_address)
    706     if err is not None:
--> 707         raise err
    708     else:

/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/socket.py in create_connection(address, timeout, source_address)
    697                 sock.bind(source_address)
--> 698             sock.connect(sa)
    699             return sock

timeout: timed out

During handling of the above exception, another exception occurred:

URLError                                  Traceback (most recent call last)
<ipython-input-10-2080c419ddf2> in <module>()
----> 1 status, msg = sg.send(message)

/Users/purda/.virtualenvs/watson/lib/python3.5/site-packages/sendgrid/sendgrid.py in send(self, message)
    113     def send(self, message):
    114         if self._raise_errors:
--> 115             return self._raising_send(message)
    116         else:
    117             return self._legacy_send(message)

/Users/purda/.virtualenvs/watson/lib/python3.5/site-packages/sendgrid/sendgrid.py in _raising_send(self, message)
    127     def _raising_send(self, message):
    128         try:
--> 129             return self._make_request(message)
    130         except HTTPError as e:
    131             if 400 <= e.code < 500:

/Users/purda/.virtualenvs/watson/lib/python3.5/site-packages/sendgrid/sendgrid.py in _make_request(self, message)
    107             req.add_header('Authorization', 'Bearer ' + self.password)
    108 
--> 109         response = urllib_request.urlopen(req, timeout=10)
    110         body = response.read()
    111         return response.getcode(), body

/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/urllib/request.py in urlopen(url, data, timeout, cafile, capath, cadefault, context)
    160     else:
    161         opener = _opener
--> 162     return opener.open(url, data, timeout)
    163 
    164 def install_opener(opener):

/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/urllib/request.py in open(self, fullurl, data, timeout)
    463             req = meth(req)
    464 
--> 465         response = self._open(req, data)
    466 
    467         # post-process response

/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/urllib/request.py in _open(self, req, data)
    481         protocol = req.type
    482         result = self._call_chain(self.handle_open, protocol, protocol +
--> 483                                   '_open', req)
    484         if result:
    485             return result

/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/urllib/request.py in _call_chain(self, chain, kind, meth_name, *args)
    441         for handler in handlers:
    442             func = getattr(handler, meth_name)
--> 443             result = func(*args)
    444             if result is not None:
    445                 return result

/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/urllib/request.py in https_open(self, req)
   1281         def https_open(self, req):
   1282             return self.do_open(http.client.HTTPSConnection, req,
-> 1283                 context=self._context, check_hostname=self._check_hostname)
   1284 
   1285         https_request = AbstractHTTPHandler.do_request_

/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/urllib/request.py in do_open(self, http_class, req, **http_conn_args)
   1240                 h.request(req.get_method(), req.selector, req.data, headers)
   1241             except OSError as err: # timeout error
-> 1242                 raise URLError(err)
   1243             r = h.getresponse()
   1244         except:

URLError: <urlopen error timed out>

Expected Result

A SendGridClientError

@thinkingserious
Copy link
Contributor

Thanks @urda! I was able to reproduce. I will make a pull request when I can find out how to fix the issue.

@urda
Copy link
Author

urda commented Oct 28, 2015

@thinkingserious 👍 sorry for the delay thanks for working on it!

@thinkingserious
Copy link
Contributor

I think I've fixed it, can you please take a look? https://github.com/sendgrid/sendgrid-python/tree/timeout

@urda
Copy link
Author

urda commented Oct 28, 2015

I'm right in front of my Mac right now so let me see.

  1. Make the change 10.255.255.1 api.sendgrid.com to /etc/hosts

  2. git clone [email protected]:sendgrid/sendgrid-python.git

  3. git checkout timeout

  4. ipython

  5. Run the following code:

import sendgrid

sg = sendgrid.SendGridClient('REDACTED', 'REDACTED', raise_errors=True)

message = sendgrid.Mail()
message.add_to('Urda <urda@my_domain.com>')
message.set_subject('Example')
message.set_text('Body')
message.set_from('noreply@my_domain.com')

status, msg = sg.send(message)
  1. Observe:
---------------------------------------------------------------------------
timeout                                   Traceback (most recent call last)
/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/urllib/request.py in do_open(self, http_class, req, **http_conn_args)
   1239             try:
-> 1240                 h.request(req.get_method(), req.selector, req.data, headers)
   1241             except OSError as err: # timeout error

/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py in request(self, method, url, body, headers)
   1082         """Send a complete request to the server."""
-> 1083         self._send_request(method, url, body, headers)
   1084 

/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py in _send_request(self, method, url, body, headers)
   1127             body = body.encode('iso-8859-1')
-> 1128         self.endheaders(body)
   1129 

/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py in endheaders(self, message_body)
   1078             raise CannotSendHeader()
-> 1079         self._send_output(message_body)
   1080 

/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py in _send_output(self, message_body)
    910 
--> 911         self.send(msg)
    912         if message_body is not None:

/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py in send(self, data)
    853             if self.auto_open:
--> 854                 self.connect()
    855             else:

/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py in connect(self)
   1228 
-> 1229             super().connect()
   1230 

/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py in connect(self)
    825         self.sock = self._create_connection(
--> 826             (self.host,self.port), self.timeout, self.source_address)
    827         self.sock.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)

/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/socket.py in create_connection(address, timeout, source_address)
    706     if err is not None:
--> 707         raise err
    708     else:

/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/socket.py in create_connection(address, timeout, source_address)
    697                 sock.bind(source_address)
--> 698             sock.connect(sa)
    699             return sock

timeout: timed out

During handling of the above exception, another exception occurred:

URLError                                  Traceback (most recent call last)
/Users/purda/temp/sendgrid-python/sendgrid/sendgrid.py in _raising_send(self, message)
    132         try:
--> 133             return self._make_request(message)
    134         except HTTPError as e:

/Users/purda/temp/sendgrid-python/sendgrid/sendgrid.py in _make_request(self, message)
    110 
--> 111         response = urllib_request.urlopen(req, timeout=10)
    112         body = response.read()

/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/urllib/request.py in urlopen(url, data, timeout, cafile, capath, cadefault, context)
    161         opener = _opener
--> 162     return opener.open(url, data, timeout)
    163 

/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/urllib/request.py in open(self, fullurl, data, timeout)
    464 
--> 465         response = self._open(req, data)
    466 

/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/urllib/request.py in _open(self, req, data)
    482         result = self._call_chain(self.handle_open, protocol, protocol +
--> 483                                   '_open', req)
    484         if result:

/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/urllib/request.py in _call_chain(self, chain, kind, meth_name, *args)
    442             func = getattr(handler, meth_name)
--> 443             result = func(*args)
    444             if result is not None:

/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/urllib/request.py in https_open(self, req)
   1282             return self.do_open(http.client.HTTPSConnection, req,
-> 1283                 context=self._context, check_hostname=self._check_hostname)
   1284 

/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/urllib/request.py in do_open(self, http_class, req, **http_conn_args)
   1241             except OSError as err: # timeout error
-> 1242                 raise URLError(err)
   1243             r = h.getresponse()

URLError: <urlopen error timed out>

During handling of the above exception, another exception occurred:

SendGridClientError                       Traceback (most recent call last)
<ipython-input-8-2080c419ddf2> in <module>()
----> 1 status, msg = sg.send(message)

/Users/purda/temp/sendgrid-python/sendgrid/sendgrid.py in send(self, message)
    115     def send(self, message):
    116         if self._raise_errors:
--> 117             return self._raising_send(message)
    118         else:
    119             return self._legacy_send(message)

/Users/purda/temp/sendgrid-python/sendgrid/sendgrid.py in _raising_send(self, message)
    140                 assert False
    141         except URLError as e:
--> 142             raise SendGridClientError(408, 'Request timeout')
    143         except timeout as e:
    144             raise SendGridClientError(408, 'Request timeout')

SendGridClientError: (408, 'Request timeout')
  1. Confirm with:
try:
    status, msg = sg.send(msesage)
except sendgrid.SendGridClientError:
    print("Caught SendGridClientError")
  1. Observe: Caught SendGridClientError

@thinkingserious
Copy link
Contributor

Thank you for the detailed review :)

The changes have been released and the new version (v1.5.13) is released.

@thinkingserious
Copy link
Contributor

@urda Our team is so impressed with your bug report and follow up that we want to hook you up with some swag :)

Please take a moment to email us at [email protected] so we can hook you up!

@urda
Copy link
Author

urda commented Oct 29, 2015

@thinkingserious I'll fire off an email now!

@thinkingserious
Copy link
Contributor

Thanks! Just replied.

@ahalyakumari95
Copy link

I am facing this issue again. Got URLError for perfectly working email set for limited time and started working fine again.

@thinkingserious
Copy link
Contributor

Hello @ahalyakumari95,

Could you please open a new issue with the details of your particular case? Thank you!

With Best Regards,

Elmer

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants