Add RsT linter (#2292)

.github/workflows/build.yml

@@ -115,12 +115,8 @@ jobs:
         python-version: 3.x
     - name: 'Run linters'
       run: |
-        # py3
-        python3 -m pip install flake8 isort
-        python3 -m flake8 .
-        python3 -m isort .
-        # clinter
-        find . -type f \( -iname "*.c" -o -iname "*.h" \) | xargs python3 scripts/internal/clinter.py
+        python3 -m pip install isort rstcheck flake8 flake8-blind-except flake8-bugbear flake8-debugger flake8-print flake8-quotes sphinx
+        make lint-all
 
   # Check sanity of .tar.gz + wheel files
   check-dist:

HISTORY.rst

@@ -15,8 +15,8 @@ XXXX-XX-XX
 - 2266_: if `Process`_ class is passed a very high PID, raise `NoSuchProcess`_
   instead of OverflowError.  (patch by Xuehai Pan)
 - 2246_: drop python 3.4 & 3.5 support.  (patch by Matthieu Darbois)
-- 2290_: PID reuse is now pre-emptively checked for `Process.ppid`_  and
-  `Process.parents`_.
+- 2290_: PID reuse is now pre-emptively checked for `Process.ppid()`_  and
+  `Process.parents()`_.
 
 **Bug fixes**
 
@@ -26,15 +26,16 @@ XXXX-XX-XX
   (patch by student_2333)
 - 2268_: ``bytes2human()`` utility function was unable to properly represent
   negative values.
-- 2252_, [Windows]: `disk_usage`_ fails on Python 3.12+.  (patch by
+- 2252_, [Windows]: `disk_usage()`_ fails on Python 3.12+.  (patch by
   Matthieu Darbois)
-- 2284_, [Linux]: `Process.memory_full_info`_ may incorrectly raise
+- 2284_, [Linux]: `Process.memory_full_info()`_ may incorrectly raise
   `ZombieProcess`_ if it's determined via ``/proc/pid/smaps_rollup``. Instead
   we now fallback on reading ``/proc/pid/smaps``.
 - 2287_, [OpenBSD], [NetBSD]: `Process.is_running()`_ erroneously return
   ``False`` for zombie processes, because creation time cannot be determined.
-- 2288_, [Linux]: correctly raise `ZombieProcess`_ on `exe`_, `cmdline`_ and
-  `memory_maps`_ instead of returning a "null" value.
+- 2288_, [Linux]: correctly raise `ZombieProcess`_ on `Process.exe()`_,
+  `Process.cmdline()`_ and `Process.memory_maps()`_ instead of returning a
+  "null" value.
 - 2290_: differently from what stated in the doc, PID reuse is not
   pre-emptively checked for `Process.nice()`_ (set), `Process.ionice()`_,
   (set), `Process.cpu_affinity()`_ (set), `Process.rlimit()`_
@@ -51,10 +52,10 @@ XXXX-XX-XX
   `KeyError` bit deriving from a missed cache hit.
 - 2217_: print the full traceback when a `DeprecationWarning` or `UserWarning`
   is raised.
-- 2230_, [OpenBSD]: `psutil.net_connections`_ implementation was rewritten from
-  scratch:
+- 2230_, [OpenBSD]: `net_connections()`_ implementation was rewritten
+  from scratch:
   - We're now able to retrieve the path of AF_UNIX sockets (before it was an
-    empty string)
+  empty string)
   - The function is faster since it no longer iterates over all processes.
   - No longer produces duplicate connection entries.
 - 2238_: there are cases where `Process.cwd()`_ cannot be determined
@@ -70,7 +71,7 @@ XXXX-XX-XX
 
 **Bug fixes**
 
-- 1043_, [OpenBSD] `psutil.net_connections`_ returns duplicate entries.
+- 1043_, [OpenBSD] `net_connections()`_ returns duplicate entries.
 - 1915_, [Linux]: on certain kernels, ``"MemAvailable"`` field from
   ``/proc/meminfo`` returns ``0`` (possibly a kernel bug), in which case we
   calculate an approximation for ``available`` memory which matches "free"
@@ -128,7 +129,7 @@ XXXX-XX-XX
 
 **Bug fixes**
 
-- 2116_, [macOS], [critical]: `psutil.net_connections`_ fails with RuntimeError.
+- 2116_, [macOS], [critical]: `net_connections()`_ fails with RuntimeError.
 - 2135_, [macOS]: `Process.environ()`_ may contain garbage data. Fix
   out-of-bounds read around ``sysctl_procargs``.  (patch by Bernhard Urban-Forster)
 - 2138_, [Linux], **[critical]**: can't compile psutil on Android due to
@@ -1575,7 +1576,7 @@ XXXX-XX-XX
 - 564_: C extension version mismatch in case the user messed up with psutil
   installation or with sys.path is now detected at import time.
 - 568_: new `pidof.py`_ script.
-- 569_, [FreeBSD]: add support for `Process.cpu_affinity`_ on FreeBSD.
+- 569_, [FreeBSD]: add support for `Process.cpu_affinity()`_ on FreeBSD.
 
 **Bug fixes**
 
@@ -1587,7 +1588,7 @@ XXXX-XX-XX
   (patch by spacewander)
 - 565_, [Windows]: use proper encoding for `Process.username()`_ and `users()`_.
   (patch by Sylvain Mouquet)
-- 567_, [Linux]: in the alternative implementation of `Process.cpu_affinity`_
+- 567_, [Linux]: in the alternative implementation of `Process.cpu_affinity()`_
   ``PyList_Append`` and ``Py_BuildValue`` return values are not checked.
 - 569_, [FreeBSD]: fix memory leak in `cpu_count()`_ with ``logical=False``.
 - 571_, [Linux]: `Process.open_files()`_ might swallow `AccessDenied`_
@@ -2169,7 +2170,8 @@ In most cases accessing the old names will work but it will cause a
   representation.
 - 283_: speedup `Process.is_running()`_ by caching its return value in case the
   process is terminated.
-- 284_, [POSIX]: per-process number of opened file descriptors (`Process.num_fds`_).
+- 284_, [POSIX]: per-process number of opened file descriptors
+  (`Process.num_fds()`_).
 - 287_: `process_iter()`_ now caches `Process`_ instances between calls.
 - 290_: `Process.nice()`_ property is deprecated in favor of new ``get_nice()``
   and ``set_nice()`` methods.
@@ -2527,8 +2529,6 @@ In most cases accessing the old names will work but it will cause a
 
 .. _`Process`: https://psutil.readthedocs.io/en/latest/#psutil.Process
 .. _`psutil.Popen`: https://psutil.readthedocs.io/en/latest/#psutil.Popen
-.. _`psutil.Process`: https://psutil.readthedocs.io/en/latest/#psutil.Process
-
 
 .. _`AccessDenied`: https://psutil.readthedocs.io/en/latest/#psutil.AccessDenied
 .. _`NoSuchProcess`: https://psutil.readthedocs.io/en/latest/#psutil.NoSuchProcess
@@ -2586,7 +2586,6 @@ In most cases accessing the old names will work but it will cause a
 .. _`cpu_distribution.py`: https://github.com/giampaolo/psutil/blob/master/scripts/cpu_distribution.py
 .. _`disk_usage.py`: https://github.com/giampaolo/psutil/blob/master/scripts/disk_usage.py
 .. _`free.py`: https://github.com/giampaolo/psutil/blob/master/scripts/free.py
-.. _`ifconfig.py`: https://github.com/giampaolo/psutil/blob/master/scripts/ifconfig.py
 .. _`iotop.py`: https://github.com/giampaolo/psutil/blob/master/scripts/iotop.py
 .. _`meminfo.py`: https://github.com/giampaolo/psutil/blob/master/scripts/meminfo.py
 .. _`netstat.py`: https://github.com/giampaolo/psutil/blob/master/scripts/netstat.py

Makefile

@@ -26,6 +26,7 @@ PY3_DEPS = \
 	pyperf \
 	pypinfo \
 	requests \
+	rstcheck \
 	setuptools \
 	sphinx_rtd_theme \
 	teyit \
@@ -207,13 +208,17 @@ isort:  ## Run isort linter.
 pylint:  ## Python pylint (not mandatory, just run it from time to time)
 	@git ls-files '*.py' | xargs $(PYTHON) -m pylint --rcfile=pyproject.toml --jobs=${NUM_WORKERS}
 
-c-linter:  ## Run C linter.
+lint-c:  ## Run C linter.
 	@git ls-files '*.c' '*.h' | xargs $(PYTHON) scripts/internal/clinter.py
 
+lint-rst:  ## Run C linter.
+	@git ls-files '*.rst' | xargs rstcheck --config=pyproject.toml
+
 lint-all:  ## Run all linters
 	${MAKE} flake8
 	${MAKE} isort
-	${MAKE} c-linter
+	${MAKE} lint-c
+	${MAKE} lint-rst
 
 # ===================================================================
 # Fixers

docs/DEVGUIDE.rst

@@ -121,9 +121,6 @@ Documentation
 - doc can be built with ``make setup-dev-env; cd docs; make html``.
 - public doc is hosted at https://psutil.readthedocs.io.
 
-.. _`appveyor.yml`: https://github.com/giampaolo/psutil/blob/master/appveyor.yml
-.. _`Appveyor`: https://ci.appveyor.com/project/giampaolo/psuti
-.. _`coveralls.io`: https://coveralls.io/github/giampaolo/psuti
 .. _`CREDITS`: https://github.com/giampaolo/psutil/blob/master/CREDITS
 .. _`CONTRIBUTING.md`: https://github.com/giampaolo/psutil/blob/master/CONTRIBUTING.md
 .. _`HISTORY.rst`: https://github.com/giampaolo/psutil/blob/master/HISTORY.rst
@@ -137,5 +134,3 @@ Documentation
 .. _`psutil/tests/test_linux.py`: https://github.com/giampaolo/psutil/blob/master/psutil/tests/test_linux.py
 .. _`psutil/tests/test_process.py`: https://github.com/giampaolo/psutil/blob/master/psutil/tests/test_process.py
 .. _`psutil/tests/test_system.py`: https://github.com/giampaolo/psutil/blob/master/psutil/tests/test_system.py
-.. _`RsT syntax`: http://docutils.sourceforge.net/docs/user/rst/quickref.htm
-.. _`sphinx`: http://sphinx-doc.org

docs/index.rst

@@ -2581,7 +2581,7 @@ FAQs
   the Python script as a Windows service (ProcessHacker does this).
 
 * Q: is MinGW supported on Windows?
-* A: no, you should Visual Studio (see `development guide`_).
+* A: no, you should Visual Studio (see `development guide <https://github.com/giampaolo/psutil/blob/master/docs/DEVGUIDE.rst>`_).
 
 Running tests
 =============
@@ -2623,7 +2623,7 @@ contact`_.  Tidelift will coordinate the fix and disclosure.
 Development guide
 =================
 
-If you want to develop psutil take a look at the `development guide`_.
+If you want to develop psutil take a look at the `DEVGUIDE.rst`_.
 
 Platforms support history
 =========================
@@ -3010,7 +3010,7 @@ Timeline
 .. _`BPO-6973`: https://bugs.python.org/issue6973
 .. _`CPU affinity`: https://www.linuxjournal.com/article/6799?page=0,0
 .. _`cpu_distribution.py`: https://github.com/giampaolo/psutil/blob/master/scripts/cpu_distribution.py
-.. _`development guide`: https://github.com/giampaolo/psutil/blob/master/docs/DEVGUIDE.rst
+.. _`DEVGUIDE.rst`: https://github.com/giampaolo/psutil/blob/master/docs/DEVGUIDE.rst
 .. _`disk_usage.py`: https://github.com/giampaolo/psutil/blob/master/scripts/disk_usage.py
 .. _`enum`: https://docs.python.org/3/library/enum.html#module-enum
 .. _`fans.py`: https://github.com/giampaolo/psutil/blob/master/scripts/fans.py
@@ -3033,7 +3033,6 @@ Timeline
 .. _`nettop.py`: https://github.com/giampaolo/psutil/blob/master/scripts/nettop.py
 .. _`open`: https://docs.python.org/3/library/functions.html#open
 .. _`os.cpu_count`: https://docs.python.org/3/library/os.html#os.cpu_count
-.. _`os.getloadavg`: https://docs.python.org/3/library/os.html#os.getloadavg
 .. _`os.getpid`: https://docs.python.org/3/library/os.html#os.getpid
 .. _`os.getpriority`: https://docs.python.org/3/library/os.html#os.getpriority
 .. _`os.getresgid`: https://docs.python.org//library/os.html#os.getresgid
@@ -3064,5 +3063,4 @@ Timeline
 .. _`TerminateProcess`: https://docs.microsoft.com/en-us/windows/desktop/api/processthreadsapi/nf-processthreadsapi-terminateprocess
 .. _`threading.get_ident`: https://docs.python.org/3/library/threading.html#threading.get_ident
 .. _`threading.Thread`: https://docs.python.org/3/library/threading.html#threading.Thread
-.. _Tidelift security contact: https://tidelift.com/security
-.. _Tidelift Subscription: https://tidelift.com/subscription/pkg/pypi-psutil?utm_source=pypi-psutil&utm_medium=referral&utm_campaign=readme
+.. _`Tidelift security contact`: https://tidelift.com/security

pyproject.toml

@@ -72,6 +72,13 @@ disable = [
     "wrong-import-position",
 ]
 
+[tool.rstcheck]
+ignore_messages = [
+    "Duplicate explicit target name",
+    "Duplicate implicit target name",
+    "Hyperlink target \".*?\" is not referenced",
+]
+
 [build-system]
 requires = ["setuptools>=43", "wheel"]
 build-backend = "setuptools.build_meta"

scripts/internal/git_pre_commit.py

@@ -15,6 +15,7 @@
 - assert "flake8" checks pass
 - assert "isort" checks pass
 - assert C linter checks pass
+- assert RsT checks pass
 - abort if files were added/renamed/removed and MANIFEST.in was not updated
 
 Install this with "make install-git-hooks".
@@ -66,8 +67,12 @@ def exit(msg):
 
 
 def sh(cmd):
-    p = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE,
-                         stderr=subprocess.PIPE, universal_newlines=True)
+    if isinstance(cmd, str):
+        cmd = shlex.split(cmd)
+    p = subprocess.Popen(
+        cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE,
+        universal_newlines=True
+    )
     stdout, stderr = p.communicate()
     if p.returncode != 0:
         raise RuntimeError(stderr)
@@ -84,19 +89,23 @@ def open_text(path):
 
 
 def git_commit_files():
-    out = sh("git diff --cached --name-only")
+    out = sh(["git", "diff", "--cached", "--name-only"])
     py_files = [x for x in out.split('\n') if x.endswith('.py') and
                 os.path.exists(x)]
     c_files = [x for x in out.split('\n') if x.endswith(('.c', '.h')) and
                os.path.exists(x)]
-    new_rm_mv = sh("git diff --name-only --diff-filter=ADR --cached")
+    rst_files = [x for x in out.split('\n') if x.endswith('.rst') and
+                 os.path.exists(x)]
+    new_rm_mv = sh(
+        ["git", "diff", "--name-only", "--diff-filter=ADR", "--cached"]
+    )
     # XXX: we should escape spaces and possibly other amenities here
     new_rm_mv = new_rm_mv.split()
-    return (py_files, c_files, new_rm_mv)
+    return (py_files, c_files, rst_files, new_rm_mv)
 
 
 def main():
-    py_files, c_files, new_rm_mv = git_commit_files()
+    py_files, c_files, rst_files, new_rm_mv = git_commit_files()
     # Check file content.
     for path in py_files:
         if os.path.realpath(path) == THIS_SCRIPT:
@@ -122,27 +131,38 @@ def main():
     if py_files:
         # flake8
         assert os.path.exists('.flake8')
-        cmd = "%s -m flake8 --config=.flake8 %s" % (PYTHON, " ".join(py_files))
-        ret = subprocess.call(shlex.split(cmd))
+        print("running flake8 (%s files)" % len(py_files))
+        cmd = [PYTHON, "-m", "flake8", "--config=.flake8"] + py_files
+        ret = subprocess.call(cmd)
         if ret != 0:
             return sys.exit("python code didn't pass 'flake8' style check; "
                             "try running 'make fix-flake8'")
         # isort
-        cmd = "%s -m isort --check-only %s" % (
-            PYTHON, " ".join(py_files))
-        ret = subprocess.call(shlex.split(cmd))
+        print("running isort (%s files)" % len(py_files))
+        cmd = [PYTHON, "-m", "isort", "--check-only"] + py_files
+        ret = subprocess.call(cmd)
         if ret != 0:
             return sys.exit("python code didn't pass 'isort' style check; "
                             "try running 'make fix-imports'")
     # C linter
     if c_files:
+        print("running clinter (%s files)" % len(c_files))
         # XXX: we should escape spaces and possibly other amenities here
-        cmd = "%s scripts/internal/clinter.py %s" % (PYTHON, " ".join(c_files))
-        ret = subprocess.call(cmd, shell=True)
+        cmd = [PYTHON, "scripts/internal/clinter.py"] + c_files
+        ret = subprocess.call(cmd)
         if ret != 0:
             return sys.exit("C code didn't pass style check")
+
+    # RST linter
+    if rst_files:
+        print("running rst linter (%s)" % len(rst_files))
+        cmd = ["rstcheck", "--config=pyproject.toml"] + rst_files
+        ret = subprocess.call(cmd)
+        if ret != 0:
+            return sys.exit("RST code didn't pass style check")
+
     if new_rm_mv:
-        out = sh("%s scripts/internal/generate_manifest.py" % PYTHON)
+        out = sh([PYTHON, "scripts/internal/generate_manifest.py"])
         with open_text('MANIFEST.in') as f:
             if out.strip() != f.read().strip():
                 sys.exit("some files were added, deleted or renamed; "