Merge pull request #9822 from jakobandersen/intersphinx_role

Intersphinx role (2)

CHANGES

@@ -47,6 +47,8 @@ Features added
 * #9391: texinfo: improve variable in ``samp`` role
 * #9578: texinfo: Add :confval:`texinfo_cross_references` to disable cross
   references for readability with standalone readers
+* #9822 (and #9062), add new Intersphinx role :rst:role:`external` for explict
+  lookup in the external projects, without resolving to the local project.
 
 Bugs fixed
 ----------

doc/usage/extensions/intersphinx.rst

@@ -8,20 +8,25 @@
 
 .. versionadded:: 0.5
 
-This extension can generate automatic links to the documentation of objects in
-other projects.
-
-Usage is simple: whenever Sphinx encounters a cross-reference that has no
-matching target in the current documentation set, it looks for targets in the
-documentation sets configured in :confval:`intersphinx_mapping`.  A reference
-like ``:py:class:`zipfile.ZipFile``` can then link to the Python documentation
+This extension can generate links to the documentation of objects in external
+projects, either explicitly through the :rst:role:`external` role, or as a
+fallback resolution for any other cross-reference.
+
+Usage for fallback resolution is simple: whenever Sphinx encounters a
+cross-reference that has no matching target in the current documentation set,
+it looks for targets in the external documentation sets configured in
+:confval:`intersphinx_mapping`.  A reference like
+``:py:class:`zipfile.ZipFile``` can then link to the Python documentation
 for the ZipFile class, without you having to specify where it is located
 exactly.
 
-When using the "new" format (see below), you can even force lookup in a foreign
-set by prefixing the link target appropriately.  A link like ``:ref:`comparison
-manual <python:comparisons>``` will then link to the label "comparisons" in the
-doc set "python", if it exists.
+When using the :rst:role:`external` role, you can force lookup to any external
+projects, and optionally to a specific external project.
+A link like ``:external:ref:`comparison manual <comparisons>``` will then link
+to the label "comparisons" in whichever configured external project, if it
+exists,
+and a link like ``:external+python:ref:`comparison manual <comparisons>``` will
+link to the label "comparisons" only in the doc set "python", if it exists.
 
 Behind the scenes, this works as follows:
 
@@ -30,8 +35,8 @@ Behind the scenes, this works as follows:
 
 * Projects using the Intersphinx extension can specify the location of such
   mapping files in the :confval:`intersphinx_mapping` config value.  The mapping
-  will then be used to resolve otherwise missing references to objects into
-  links to the other documentation.
+  will then be used to resolve both :rst:role:`external` references, and also
+  otherwise missing references to objects into links to the other documentation.
 
 * By default, the mapping file is assumed to be at the same location as the rest
   of the documentation; however, the location of the mapping file can also be
@@ -79,10 +84,10 @@ linking:
    at the same location as the base URI) or another local file path or a full
    HTTP URI to an inventory file.
 
-   The unique identifier can be used to prefix cross-reference targets, so that
+   The unique identifier can be used in the :rst:role:`external` role, so that
    it is clear which intersphinx set the target belongs to.  A link like
-   ``:ref:`comparison manual <python:comparisons>``` will link to the label
-   "comparisons" in the doc set "python", if it exists.
+   ``external:python+ref:`comparison manual <comparisons>``` will link to the
+   label "comparisons" in the doc set "python", if it exists.
 
    **Example**
 
@@ -162,21 +167,50 @@ linking:
 
    The default value is an empty list.
 
-   When a cross-reference without an explicit inventory specification is being
-   resolved by intersphinx, skip resolution if it matches one of the
-   specifications in this list.
+   When a non-:rst:role:`external` cross-reference is being resolved by
+   intersphinx, skip resolution if it matches one of the specifications in this
+   list.
 
    For example, with ``intersphinx_disabled_reftypes = ['std:doc']``
    a cross-reference ``:doc:`installation``` will not be attempted to be
-   resolved by intersphinx, but ``:doc:`otherbook:installation``` will be
-   attempted to be resolved in the inventory named ``otherbook`` in
+   resolved by intersphinx, but ``:external+otherbook:doc:`installation``` will
+   be attempted to be resolved in the inventory named ``otherbook`` in
    :confval:`intersphinx_mapping`.
    At the same time, all cross-references generated in, e.g., Python,
    declarations will still be attempted to be resolved by intersphinx.
 
-   If ``*`` is in the list of domains, then no references without an explicit
-   inventory will be resolved by intersphinx.
+   If ``*`` is in the list of domains, then no non-:rst:role:`external`
+   references will be resolved by intersphinx.
+
+Explicitly Reference External Objects
+-------------------------------------
+
+The Intersphinx extension provides the following role.
+
+.. rst:role:: external
+
+   .. versionadded:: 4.4
+
+   Use Intersphinx to perform lookup only in external projects, and not the
+   current project. Intersphinx still needs to know the type of object you
+   would like to find, so the general form of this role is to write the
+   cross-refererence as if the object is in the current project, but then prefix
+   it with ``:external``.
+   The two forms are then
+
+   - ``:external:domain:reftype:`target```,
+     e.g., ``:external:py:class:`zipfile.ZipFile```, or
+   - ``:external:reftype:`target```,
+     e.g., ``:external:doc:`installation```.
+
+   If you would like to constrain the lookup to a specific external project,
+   then the key of the project, as specified in :confval:`intersphinx_mapping`,
+   is added as well to get the two forms 
 
+   - ``:external+invname:domain:reftype:`target```,
+     e.g., ``:external+python:py:class:`zipfile.ZipFile```, or
+   - ``:external+invname:reftype:`target```,
+     e.g., ``:external+python:doc:`installation```.
 
 Showing all links of an Intersphinx mapping file
 ------------------------------------------------

sphinx/ext/intersphinx.py

@@ -26,15 +26,17 @@
 import concurrent.futures
 import functools
 import posixpath
+import re
 import sys
 import time
 from os import path
-from typing import IO, Any, Dict, List, Optional, Tuple
+from types import ModuleType
+from typing import IO, Any, Dict, List, Optional, Tuple, cast
 from urllib.parse import urlsplit, urlunsplit
 
 from docutils import nodes
-from docutils.nodes import Element, TextElement
-from docutils.utils import relative_path
+from docutils.nodes import Element, Node, TextElement, system_message
+from docutils.utils import Reporter, relative_path
 
 import sphinx
 from sphinx.addnodes import pending_xref
@@ -43,10 +45,13 @@
 from sphinx.config import Config
 from sphinx.domains import Domain
 from sphinx.environment import BuildEnvironment
+from sphinx.errors import ExtensionError
 from sphinx.locale import _, __
+from sphinx.transforms.post_transforms import ReferencesResolver
 from sphinx.util import logging, requests
+from sphinx.util.docutils import CustomReSTDispatcher, SphinxRole
 from sphinx.util.inventory import InventoryFile
-from sphinx.util.typing import Inventory, InventoryItem
+from sphinx.util.typing import Inventory, InventoryItem, RoleFunction
 
 logger = logging.getLogger(__name__)
 
@@ -466,6 +471,144 @@ def missing_reference(app: Sphinx, env: BuildEnvironment, node: pending_xref,
     return resolve_reference_detect_inventory(env, node, contnode)
 
 
+class IntersphinxDispatcher(CustomReSTDispatcher):
+    """Custom dispatcher for external role.
+
+    This enables :external:***:/:external+***: roles on parsing reST document.
+    """
+
+    def role(self, role_name: str, language_module: ModuleType, lineno: int, reporter: Reporter
+             ) -> Tuple[RoleFunction, List[system_message]]:
+        if len(role_name) > 9 and role_name.startswith(('external:', 'external+')):
+            return IntersphinxRole(role_name), []
+        else:
+            return super().role(role_name, language_module, lineno, reporter)
+
+
+class IntersphinxRole(SphinxRole):
+    # group 1: just for the optionality of the inventory name
+    # group 2: the inventory name (optional)
+    # group 3: the domain:role or role part
+    _re_inv_ref = re.compile(r"(\+([^:]+))?:(.*)")
+
+    def __init__(self, orig_name: str) -> None:
+        self.orig_name = orig_name
+
+    def run(self) -> Tuple[List[Node], List[system_message]]:
+        assert self.name == self.orig_name.lower()
+        inventory, name_suffix = self.get_inventory_and_name_suffix(self.orig_name)
+        if inventory and not inventory_exists(self.env, inventory):
+            logger.warning(__('inventory for external cross-reference not found: %s'),
+                           inventory, location=(self.env.docname, self.lineno))
+            return [], []
+
+        role_name = self.get_role_name(name_suffix)
+        if role_name is None:
+            logger.warning(__('role for external cross-reference not found: %s'), name_suffix,
+                           location=(self.env.docname, self.lineno))
+            return [], []
+
+        result, messages = self.invoke_role(role_name)
+        for node in result:
+            if isinstance(node, pending_xref):
+                node['intersphinx'] = True
+                node['inventory'] = inventory
+
+        return result, messages
+
+    def get_inventory_and_name_suffix(self, name: str) -> Tuple[Optional[str], str]:
+        assert name.startswith('external'), name
+        assert name[8] in ':+', name
+        # either we have an explicit inventory name, i.e,
+        # :external+inv:role:        or
+        # :external+inv:domain:role:
+        # or we look in all inventories, i.e.,
+        # :external:role:            or
+        # :external:domain:role:
+        inv, suffix = IntersphinxRole._re_inv_ref.fullmatch(name, 8).group(2, 3)
+        return inv, suffix
+
+    def get_role_name(self, name: str) -> Optional[Tuple[str, str]]:
+        names = name.split(':')
+        if len(names) == 1:
+            # role
+            default_domain = self.env.temp_data.get('default_domain')
+            domain = default_domain.name if default_domain else None
+            role = names[0]
+        elif len(names) == 2:
+            # domain:role:
+            domain = names[0]
+            role = names[1]
+        else:
+            return None
+
+        if domain and self.is_existent_role(domain, role):
+            return (domain, role)
+        elif self.is_existent_role('std', role):
+            return ('std', role)
+        else:
+            return None
+
+    def is_existent_role(self, domain_name: str, role_name: str) -> bool:
+        try:
+            domain = self.env.get_domain(domain_name)
+            if role_name in domain.roles:
+                return True
+            else:
+                return False
+        except ExtensionError:
+            return False
+
+    def invoke_role(self, role: Tuple[str, str]) -> Tuple[List[Node], List[system_message]]:
+        domain = self.env.get_domain(role[0])
+        if domain:
+            role_func = domain.role(role[1])
+
+            return role_func(':'.join(role), self.rawtext, self.text, self.lineno,
+                             self.inliner, self.options, self.content)
+        else:
+            return [], []
+
+
+class IntersphinxRoleResolver(ReferencesResolver):
+    """pending_xref node resolver for intersphinx role.
+
+    This resolves pending_xref nodes generated by :intersphinx:***: role.
+    """
+
+    default_priority = ReferencesResolver.default_priority - 1
+
+    def run(self, **kwargs: Any) -> None:
+        for node in self.document.traverse(pending_xref):
+            if 'intersphinx' not in node:
+                continue
+            contnode = cast(nodes.TextElement, node[0].deepcopy())
+            inv_name = node['inventory']
+            if inv_name is not None:
+                assert inventory_exists(self.env, inv_name)
+                newnode = resolve_reference_in_inventory(self.env, inv_name, node, contnode)
+            else:
+                newnode = resolve_reference_any_inventory(self.env, False, node, contnode)
+            if newnode is None:
+                typ = node['reftype']
+                msg = (__('external %s:%s reference target not found: %s') %
+                       (node['refdomain'], typ, node['reftarget']))
+                logger.warning(msg, location=node, type='ref', subtype=typ)
+                node.replace_self(contnode)
+            else:
+                node.replace_self(newnode)
+
+
+def install_dispatcher(app: Sphinx, docname: str, source: List[str]) -> None:
+    """Enable IntersphinxDispatcher.
+
+    .. note:: The installed dispatcher will uninstalled on disabling sphinx_domain
+              automatically.
+    """
+    dispatcher = IntersphinxDispatcher()
+    dispatcher.enable()
+
+
 def normalize_intersphinx_mapping(app: Sphinx, config: Config) -> None:
     for key, value in config.intersphinx_mapping.copy().items():
         try:
@@ -497,7 +640,9 @@ def setup(app: Sphinx) -> Dict[str, Any]:
     app.add_config_value('intersphinx_disabled_reftypes', [], True)
     app.connect('config-inited', normalize_intersphinx_mapping, priority=800)
     app.connect('builder-inited', load_mappings)
+    app.connect('source-read', install_dispatcher)
     app.connect('missing-reference', missing_reference)
+    app.add_post_transform(IntersphinxRoleResolver)
     return {
         'version': sphinx.__display_version__,
         'env_version': 1,