Decode bytes to str with Pandas reader

[nirvana-msu]

It would be nice if there was a setting which allowed to get data as str, rather than bytes, in Python 3 when used with pandas=True. I'm aware of the discussion in #35 and related example of a custom reader here, so it's definitely possible to achieve this. It just seems like there are quite a few things you need to overwrite to handle all the cases, which makes it very easy to do it wrong. If this is not going to be officially supported out of the box, then at least a recipe with QReader implementation which handles all the possible cases would be extremely helpful.

I feel like Python 3 users would prefer to get str over bytes in vast majority of cases, and using pandas DataFrame is essential in most scientific applications. If anything, at first I was naively expecting that passing encoding to QConnection would make it do exactly that. So if you don't pass any encoding it would give you bytes, but if you do pass the encoding it would give you decoded strings.

Thanks!

[seizetheday]

This is what I exactly need!!

[zjc5415]

This is what I exactly need!!

an easy way to do is:
in the file qreader.py, function _read_list:
change
if qtype == QSYMBOL_LIST:
symbols = self.buffer.get_symbols(length)
data = numpy.array(symbols, dtype = numpy.string)
return qlist(data, qtype = qtype, adjust_dtype = False)
to:
if qtype == QSYMBOL_LIST:
symbols = self._buffer.get_symbols(length)
data = numpy.array(symbols, dtype = str)
return qlist(data, qtype = qtype, adjust_dtype = False)

it works but i don't if there are side effects

[wscsgh59]

Hello zjc5415,

This hasn't solved the issue for me. I gert an error saying that 'ascii' codec can't decode byte”. Is there another way to do it ?

THnaks

[zjc5415]

Hello zjc5415,

This hasn't solved the issue for me. I gert an error saying that 'ascii' codec can't decode byte”. Is there another way to do it ?

THnaks

in my case it is python3.6。

you may debug it line by line and check the reason。