disk_io_counters() fails on Linux kernel 4.19

[gozdal]

When running on 4.19.0-1.el7.elrepo.x86_6, calling disk_io_counters() raises ValueError:

(psutil_venv) -bash-4.2$ python
Python 3.6.2 (default, Apr 24 2018, 04:27:15) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-16)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import psutil
>>> psutil.__version__
'5.4.7'
>>> psutil.disk_io_counters()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/9ld/psutil_venv/lib/python3.6/site-packages/psutil/__init__.py", line 2017, in disk_io_counters
    rawdict = _psplatform.disk_io_counters(**kwargs)
  File "/home/9ld/psutil_venv/lib/python3.6/site-packages/psutil/_pslinux.py", line 1117, in disk_io_counters
    for entry in gen:
  File "/home/9ld/psutil_venv/lib/python3.6/site-packages/psutil/_pslinux.py", line 1090, in read_procfs
    raise ValueError("not sure how to interpret line %r" % line)
ValueError: not sure how to interpret line '   8      16 sdb 147 0 9545 368 3 0 4096 17 0 352 385 0 0 0 0\n'

[apexliu]

Yes, I got the same ValueError.

[giampaolo]

Just for reference, last time we had a similar issue was #767.
In this specific case it appears 4 new fields were added in Linux 4.18+, and that breaks the parser.
From:
https://www.kernel.org/doc/Documentation/iostats.txt
Sample line:

   4.18+ diskstats:
      3    0   hda 446216 784926 9550688 4382310 424847 312726 5922052 19310380 0 3376340 23705160 0 0 0 0

[weedy]

Can we get a point release with this fix? Or will there be a new release soon?

[giampaolo]

Latest version on PYPI already includes this fix. ;)

[gozdal]

@giampaolo I've tried latest on PyPI and unfortunately I still get the same error:

$ python
Python 3.6.7 (default, Dec 10 2018, 15:09:50) 
[GCC 7.3.1 20180303 (Red Hat 7.3.1-5)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import psutil
>>> psutil.__version__
'5.4.8'
>>> psutil.disk_io_counters() 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/9ld/psutil/lib/python3.6/site-packages/psutil/__init__.py", line 2026, in disk_io_counters
    rawdict = _psplatform.disk_io_counters(**kwargs)
  File "/home/9ld/psutil/lib/python3.6/site-packages/psutil/_pslinux.py", line 1115, in disk_io_counters
    for entry in gen:
  File "/home/9ld/psutil/lib/python3.6/site-packages/psutil/_pslinux.py", line 1088, in read_procfs
    raise ValueError("not sure how to interpret line %r" % line)
ValueError: not sure how to interpret line '   8       0 sda 15274462 699151 625537577 12252901 552274484 127823702 23782984176 1750210434 0 151391907 1762284501 0 0 0 0\n'

[giampaolo]

Sorry, this should have been fixed in 8f99f37 which is still not released. If you try latest GIT version it should work.

[gozdal]

@giampaolo yes, latest git version works. Do you have any plans on releasing a new version to PyPI?

[qxo]

temporary fix for python 2.7:

sed -i  's#flen == 14:#flen == 14 or flen == 18:#g'  /usr/local/lib/python2.7/dist-packages/psutil/_pslinux.py

[rmoriz]

@giampaolo when do you cut the next release?

[giampaolo]

I am traveling now. Please be patient. :)

On Fri, 18 Jan 2019 at 15:58, rmoriz ***@***.***> wrote: @giampaolo <https://github.com/giampaolo> we do you plan to cut the next release? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1354 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAplLLDQFYHtaVjodQTzHQfmfWyWjmchks5vEeD9gaJpZM4X-0d1> .

-- Giampaolo - http://grodola.blogspot.com

[mossroy]

On debian stretch (using psutil 5.0.1 with python3), the following workaround works on kernel 5.0.8 :

sudo sed -i 's#fields_len == 14:#fields_len == 14 or fields_len == 18:#g'  /usr/lib/python3/dist-packages/psutil/_pslinux.py