Error: string index out of range error

[CXOldStar]

I tried to use socketIO_client to recv data from my server.And it raise "string index out of range error" when the opcode is 8.The opcode is definded in websocket protocol.
Error message:

Traceback(most recent call last):
File "/usr/lib64/python2.7/threading.py", line 804, in __bootstrap_inner
self.run()
File "/home/study/socketio/socketio_client.py", line 115, in run
socketIO.wait()
File "/usr/lib/python2.7/site-packages/socketIO_client/init.py", line 251, in wait
self._process_packets()
File "/usr/lib/python2.7/site-packages/socketIO_client/init.py", line 276, in _process_packets
for engineIO_packet in self._transport.recv_packet():
File "/usr/lib/python2.7/site-packages/socketIO_client/transports.py", line 157, in recv_packet
packet_text)
File "/usr/lib/python2.7/site-packages/socketIO_client/parsers.py", line 96, in parse_packet_text
packet_type = int(get_character(packet_text, 0))
File "/usr/lib/python2.7/site-packages/socketIO_client/symmetries.py", line 26, in get_character
return chr(get_byte(x, index))
File "/usr/lib/python2.7/site-packages/socketIO_client/symmetries.py", line 30, in get_byte
return six.indexbytes(x, index)
File "/usr/lib/python2.7/site-packages/six.py", line 655, in indexbytes
return ord(buf[i])
IndexError: string index out of range

[markw63]

The fix to this I think is for zero byte packets and the same issue is also in SocketIo in
ufora/ufora#203
and is to add at line 142 of transports.py, two lines and an indent

def recv_packet(self):
        try:
            packet_text = None
            while packet_text is None:
                packet_text = self._connection.recv()
        except WebSocketTimeoutException as e:

I have added the extra line in the following in my own version as this approach is terribly CPU dragging and I have other threads to run

packet_text = None
            while packet_text is None:
                packet_text = self._connection.recv()
                if packet_text is None: time.sleep(.05) # MW hack 50ms

I actually discovered my problem was sending boolean values from my socket client in node.js which are not happily dealt with.

[CXOldStar]

@markw63 When the opcode sent by server is 8, self._connection.recv() will return '' to packet_text.So the extra lines can not avoid the error of "string index out of range error" .
And I have another question.Why the server sent 8(opcode) to client?

[markw63]

Agreed . If you force it to loop on zero length then eventually it gracefully disconnects and reconnects. Trying to see why my node.js server is socketing this back. It happens just after one of my other clients sends its first message back to server via server and then this client receives it. Then fails. But only in certain circumstances around reconnecting. Something to do with Websocket not reconnecting gracefully? Is websocket 0x8 a close?

[CXOldStar]

@markw63 WebSocket Protocol :https://tools.ietf.org/html/rfc6455#page-36

[huguesdk]

I hit the same problem. Why was this issue closed if a close frame (0x8) is a valid WebSocket message? Shouldn’t this be handled properly by the library? It could either handle the empty string (feels more like a hack to me), or use .recv_data() instead of .recv(), which additionally returns the opcode. .recv() actually calls .recv_data() and returns an empty string if the opcode doesn’t correspond to text. Besides text, there are opcodes for close, ping, pong,… so maybe handling close specifically would be the way to go?

[markw63]

In agreement with the previous comments. Currently I have code where I end up with a disconnect and reconnect - this only occurs where a client disconnects from a room, then reconnects to a room where another client is sending socket messages. The first message back to the reconnecting client tells it to close.

[huguesdk]

It happens to me when my Node.js server is closed (during a deployment). It calls .close() on the server, which sends a close frame (0x8) to all clients and closes the socket. I would expect this client library to handle this properly, and just reconnect to the server when it is available again. This is how the Socket.IO JS client works. How would you handle this case?

[markw63]

from .exceptions import ConnectionError, TimeoutError, PacketError

 def recv_packet(self):
        try:
                packet_text = self._connection.recv()
                if type(packet_text)!=str: packet_text=str(packet_text) #python 2.7 fix to str
                if len(packet_text)==0: 
                    print "zero packet"
                    raise PacketError('zero size packet')

At line 20 - add PacketError to the import
at line 156 of transports.py adjust the recv_packet() function. This is python 2.7 only , but this at least disconnects and reconnects without a crash in transports. It is a hack! For reconnect automatically, Depends on what you set in your initial call to the library I think. In my application, I monitor the 'on_disconnect' event and have a separate monitoring process to reconnect. I have other events (eg 4G going through a tunnel, network dropout) which also cause disconnects, so I have to monitor anyway and automatic re-connect is not necessarily what I required.

[huguesdk]

Thanks for the code. The questions I have now are:

• Is receiving a close frame (0x8, resulting in a empty packet_text) a normal use case? I think yes.
• If it is a normal use case, why isn’t this fixed in the library instead of everyone making their own hack in their local copy?
• If the library should be fixed, could this issue please be reopened?

[markw63]

Agree - the underlying websockets library does a self.send.close() on receipt of that 0x8. Then returns a null string "" as data to the calling function. Which automatically will crash the current code. So it needs fixing

[khanhvu161188]

Do we have any luck with this issue?

[lukewitmer]

Bump. Can someone please fix this issue or expand on why it can't be fixed as described above? Thanks!!

[khanhvu161188]

I dont know is our repo still alive or not

[adityak368]

+1

[mihindupaul]

Payload section of engine.io protocol specify packet format when XHR2 is not available. if the server communicating under this protocol _read_packet_length of pharsers.py does not correctly detect the length.

i was able to fix the issue by adding extra length detection method and,

# extra function to support XHR1 style protocol
def _read_packet_length2(content, content_index):
    packet_length_string = ''
    #print content,content_index
    while get_byte(content, content_index) != ord(':'):
        #print content,content_index
        byte = get_byte(content, content_index)
        packet_length_string += chr(byte)
        #print packet_length_string
        content_index += 1
    content_index += 1
    return content_index, int(packet_length_string) 

calling it from phasers.py (note _read_packet_length2)

def decode_engineIO_content(content):
    content_index = 0
    content_length = len(content)
    while content_index < content_length:
        try:
            content_index, packet_length = _read_packet_length2(
                content, content_index)
        except IndexError:
            break
        content_index, packet_text = _read_packet_text(
            content, content_index, packet_length)
        engineIO_packet_type, engineIO_packet_data = parse_packet_text(
            packet_text)
        yield engineIO_packet_type, engineIO_packet_data

and it works with my node.js socket.io now.
https://github.com/mihindupaul/socketIO-client

[jacobgarcia]

You sir @mihindupaul are my hero. It solved all my issues while connecting to my node.js server. Truly, thanks a lot!