Should pytorch depend on `pybind11-abi` in `host`?

[bdice]

Comment:

I was helping @harrism with some questions about PyTorch and @carterbox pointed me to https://conda-forge.org/docs/maintainer/knowledge_base/#pybind11-abi-constraints

Should pytorch be depending on pybind11-abi in host, to ensure that packages building with pytorch downstream get a runtime dependency that enforces a compatible ABI?

[rgommers]

pybind11 is used by PyTorch for Python bindings, but not (to my knowledge) for exposing a C++ ABI. Do you have any reason to believe that pybind11-abi is relevant?

[bdice]

I think PyTorch extension modules may need a way to ensure consistent pybind11 ABI here. @harrism and @rwgk may have more to add, they are investigating some issues that we think are related.

[carterbox]

From internal discussions, I believe we are coming to consensus that the pybind11 objects torch::Tensor and torch::Device in the C++ layer are part of the public API for use in downstream projects to create pytorch extensions based on pybind11 that accept these types as parameters. Any two pybind11 modules that pass wrapped C++ classes between themselves need to be using the same pybind11 ABI.

Therefore, pybind11 should be pinned to 2.* in this recipe because this is the major version that upstream uses before pytorch=2.9 AND/OR the pybind11-abi package should be used.

Link to pytorch C++ API docs: https://docs.pytorch.org/cppdocs/#c-frontend

[rwgk]

@rwgk may have more to add,

Not a lot. Just to confirm:

Therefore, pybind11 should be pinned to 2.* in this recipe because this is the major version that upstream uses before pytorch=2.9

Yes, I believe that will probably resolve the issues seen by @harrism

Starting with pytorch=2.9, pybind11 should be pinned to 3.*. I believe it will most likely not work with 2.*

For completeness, this is the pytorch PR that migrated pytorch to pybind11 v3:

• https://github.com/pytorch/pytorch/pull/160754/files

That change is not backward compatible with pybind11 v2.

[rgommers]

Thanks, that is helpful. You learn something new every day:) @benjaminglass1 since you're handling the pybind11 v3 transition for PyTorch, could you please review the above comments and add some color?

---

Therefore, pybind11 should be pinned to 2.* in this recipe because this is the major version that upstream uses before pytorch=2.9 AND/OR the pybind11-abi package should be used.

Pinning to 2.*/3.* should solve the immediate issue then, but doesn't seem enough because that pin doesn't propagate to other packages. And the use of pybind11-abi is the opposite, that is a very blunt instrument. I thought about writing up why, but it's already exactly explained in conda-forge/conda-forge-pinning-feedstock#849 (comment). Because PyTorch is so popular, it's going to cause some odd constraints and rebuilds. There about 80 packages using pybind11-abi, none of which (at a quick browse) seem to rely on libtorch, and all of them would now become hard-coupled to pytorch versions.

It seems what would be actually needed is a libtorch-abi flavor, which all packages that depend on PyTorch can opt into and that would pin both pytorch x.x and pybind11 major versions, since I don't think the current pin_subpackage('libtorch', max_pin='x.x') can add this extra constraint. That approach isn't feasible for all those 80 packages, but pytorch is important enough that that should be justified.

[rgommers]

I very rarely use AI tools for this kind of question, and I don't trust the answers much usually, but I thought I'd ask one (DeepSeek) the not-so-simple question "Do PyTorch extension modules rely on the Pybind11 ABI?". Result:

The answer is nuanced: Yes, the vast majority of PyTorch C++ extension modules rely on the Pybind11 ABI, but they do so through the PyTorch library itself, which abstracts away the direct dependency.

Let's break this down.

1. The Role of Pybind11 in PyTorch

PyTorch's core C++ libraries (libtorch, libc10, libtorch_python) use Pybind11 extensively to create the Python bindings that you use in import torch. This means that the fundamental interface between Python and the C++ tensors, autograd functions, and neural network modules is built with Pybind11.

When you create a custom C++ extension, you use the <torch/extension.h> header. This header is a convenient wrapper that includes all the necessary PyTorch headers and the Pybind11 headers.

2. How Your Extension Uses Pybind11 (Indirectly)

You don't typically #include <pybind11/pybind11.h> directly. Instead, you #include <torch/extension.h>. This header does the following for you:

• Includes all core Torch headers (ATen, the C++ tensor library).
• Includes the Pybind11 headers that are bundled with the PyTorch source code.
• Sets up all the necessary macros and boilerplate.

Your code then uses Pybind11 syntax, but it's the same version of Pybind11 that the parent PyTorch library was compiled with.

Pybind11 is indeed vendored in the PyTorch source tree (see https://github.com/pytorch/pytorch/tree/main/third_party), so that would lead one to conclude that the use of USE_SYSTEM_PYBIND11=1 in this repo is the problem here. So the simplest fix would be to get rid of that.

[isuruf]

So the simplest fix would be to get rid of that.

It's not so simple. PyTorch is simply using pybind11/pybind11.h and not something like torch/pybind11/pybind11.h. Moving to a vendored solution should be done only if that's the only way and it is reasonably contained within the package. It's not contained and therefore not using system pybind11 is not a solution here.

[h-vetinari]

There about 80 packages using pybind11-abi, none of which (at a quick browse) seem to rely on libtorch, and all of them would now become hard-coupled to pytorch versions.

I don't follow how you arrive at this conclusion. The only thing that would happen is that the pytorch feedstock would be a participant in the pybind11 migration.

[benjaminglass1]

Just want to note that PyTorch previously overrode the PyBind11 ABI compatibility string to a custom internal format, but with the 3.x release of PyBind11 is reverting to the PyBind11 ABI string. I don't know how that plays into the pybind11-abi dependency.

[rwgk]

Your code then uses Pybind11 syntax, but it's the same version of Pybind11 that the parent PyTorch library was compiled with.

I think that's not reliable, because it depends on the build system(s). If there are multiple versions of pybind11 floating around on the filesystem, and any of the usually many pieces of the build system uses a different -I/some/path/to/pybind11/include, everything will most likely build just fine (because pybind11 has a highly stable API, even between major versions 2 and 3), but at runtime the ABI-handling incompatibility will bite (could not convert default argument or similar).

Another potential source of confusion are these manipulations of PYBIND11 macros (removed only in pytorch/pytorch#160754):

def _get_pybind11_abi_build_flags():
    # Note [Pybind11 ABI constants]
    #
    # Pybind11 before 2.4 used to build an ABI strings using the following pattern:
    # f"__pybind11_internals_v{PYBIND11_INTERNALS_VERSION}{PYBIND11_INTERNALS_KIND}{PYBIND11_BUILD_TYPE}__"
    # Since 2.4 compier type, stdlib and build abi parameters are also encoded like this:
    # f"__pybind11_internals_v{PYBIND11_INTERNALS_VERSION}{PYBIND11_INTERNALS_KIND}{PYBIND11_COMPILER_TYPE}{PYBIND11_STDLIB}{PYBIND11_BUILD_ABI}{PYBIND11_BUILD_TYPE}__"
    #
    # This was done in order to further narrow down the chances of compiler ABI incompatibility
    # that can cause a hard to debug segfaults.
    # For PyTorch extensions we want to relax those restrictions and pass compiler, stdlib and abi properties
    # captured during PyTorch native library compilation in torch/csrc/Module.cpp

    abi_cflags = []
    for pname in ["COMPILER_TYPE", "STDLIB", "BUILD_ABI"]:
        pval = getattr(torch._C, f"_PYBIND11_{pname}")
        if pval is not None and not IS_WINDOWS:
            abi_cflags.append(f'-DPYBIND11_{pname}=\\"{pval}\\"')
    return abi_cflags

If those flags are somehow not applied consistently where it matters (see below), again this will result in ABI-handling incompatibility.

I'm looking at the screen here for a long time, wondering what we can do ...

• Certainly: It's impossible to make pybind11-based extensions "binary-interoperate" (see below) if the pybind11 major versions don't match.
• That by itself dictates: All pybind11-based extensions that need to binary-interoperate need to use the same pybind11 major version.
• It's totally up to the build system(s) to guarantee this. pybind11 doesn't have anything built in to enforce this at build time ("impossible" I think), and nothing to help diagnose the issue in a clear way at runtime (possible, but nobody worked on it; and now it's too late anyway, unfortunately).

---

Backing up to explain what "pybind11-based extensions that need to binary-interoperate" means.

In the most common case it boils down to:

// extension_A.cpp
py::class_<A::SomeType>("SomeType", ...)...;

// extension_B.cpp
void b_function(const A::SomeType&);
m.def("b_function", &b_function);

Elsewhere in Python:

obj = extension_A.SomeType()
b_function(obj)

This will only work if extension_A.cpp and extension_B.cpp were compiled with

• for sure: compatible pybind11 versions (major versions must match)
• maybe also: compatible _get_pybind11_abi_build_flags()-based PYBIND11 macro manipulations

---

To repeat my conclusion above before going into the next subject:

• All pybind11-based extensions that need to binary-interoperate need to use the same pybind11 major version.

Next subject: pytorch/pytorch#160754

@benjaminglass1 Please correct me if I'm wrong, but I believe that PR introduced a new forcing function:

• Any extension built against pytorch after PR 160754 needs to use pybind11 v3.

(I think it's not a hard forcing function; some or even many things will still work with pybind11 v2, but some expectations about compiler compatibilities, like clang vs gcc, will break down.)

I'm wondering, really not sure: Would it help the community to make a backward compatible change in pytorch (before the 2.9 release IIUC), so that people still have a choice between using pybind11 v2 or v3?

Speculating this could work:

• 1. Detect what pybind11 version is being used while building pytorch
• 2.a if pybind11 v2, the keep the old behavior (pre-PR-160754)
• 2.b if pybind11 v3, apply the new behavior (post-PR-160754)

That way, people would still have a choice with any version of pytorch: consistently use pybind11 v2, or consistenly use pybind11 v3. That might help them lining up all their ducks before making the switch to pybind11 v3.

---

What does the future hold? — I really believe once everybody has migrated to pybind11 v3, and _get_pybind11_abi_build_flags() is gone for good, we'll be in a much better situation. pybind11 v3 comes with a "conduit" feature which allows the b_function(obj) example above to succeed even if the pybind11 internals versions don't match. There are still limitations if the internals versions don't match, but I expect/hope that many use cases will work just fine. I think the situation will get even better after the pending pybind/pybind11#5800 goes in. @oremanj for visibility.

[benjaminglass1]

@rwgk You are correct; extensions built against PyTorch 2.9 or greater should use PyBind11 3.x (assuming we aren't covering nightlies), and extensions built against earlier versions should use PyBind11 2.x

EDIT: also, __get_pybind11_abi_build_flags will return an empty list in PyTorch 2.9 or greater, and be removed in PyTorch 2.10. It's a vestigial method to avoid breaking downstreams that imported it.