fix: vs2022 compilation, issue #3477 by Boris-Rasin · Pull Request #3497 · pybind/pybind11

Boris-Rasin · 2021-11-22T15:03:38Z

Description

Tweaks to support Microsoft Visual Studio 2022.

Suggested changelog entry:

Tweaks to support Microsoft Visual Studio 2022.

include/pybind11/detail/common.h

include/pybind11/embed.h

include/pybind11/detail/common.h

henryiii · 2021-11-24T02:22:15Z

Would you mind turning on a 2022 build? See #3480.

henryiii · 2021-11-24T05:28:12Z

We can disable the setuptools_helper test on 3.5 - unless it's our fault (I don't think so), then this is just an incompatibility between setuptools & Python 3.5 & MSVC 2022 that I don't really care about.

henryiii · 2021-11-24T05:29:18Z

I still like gating the include <crtdefs.h> for MSVC 2022+. Other than those two things, then I think this is good to go.

Boris-Rasin · 2021-11-24T13:02:31Z

How do I "disable setuptools_helper test on 3.5"?

Boris-Rasin · 2021-11-24T16:33:10Z

This problem is not with vs2022 C++ compiler, it is with vs2022 C++ standard library, which could be used by different compilers, for example clang bundled with vs2022 (or clang for windows installed separately).

Therefore, if we want to limit workaround for specific version, it should be version of msvc stdlib (_MSVC_STL_VERSION >= 143), not compiler version (_MSC_VER), which is in fact old on clang for windows.

The problem in this case is, we need to include system header (yvals.h) anyway to get _MSVC_STL_VERSION value.

henryiii · 2021-11-24T16:35:34Z

When did they start bundling clang? Nice. Maybe lfortran will be more readily available?

Anyway, at least this is much more controlled, and lets the reader know more about the issue. Hopefully yvals.h doesn't include much else.

rwgk · 2021-11-24T19:05:44Z

.github/workflows/ci.yml

      fail-fast: false
      matrix:
-        runs-on: [ubuntu-latest, windows-latest, macos-latest]
+        runs-on: [ubuntu-latest, windows-2022, macos-latest]


IIUC we're implicitly dropping windows-latest. Mostly a question for @henryiii: could we simply add windows-2022 but keep windows-latest (until it becomes the same)?

@henryiii I simply tried it out: #3517 is the same as this PR but with windows-latest added back. Here we have 73 checks, on the other PR 79. I verified that we actually have distinct builds (below). — Resource wise it's definitely not a problem to have both. I'm thinking: Why not?

$ grep ' -- The CXX compiler identification is' *.txt | grep windows-latest.*MSVC | sed 's/2021-11-30[^ ]*//' | sed 's/^[^_]*//' | uniq | sort ______2.7_____windows-latest_____x64.txt: -- The CXX compiler identification is MSVC 19.29.30137.0 ______3.10_____windows-latest_____x64.txt: -- The CXX compiler identification is MSVC 19.29.30137.0 ______3.5_____windows-latest_____x64.txt: -- The CXX compiler identification is MSVC 19.29.30137.0 ______3.6_____windows-latest_____x64_-DPYBIND11_FINDPYTHON=ON.txt: -- The CXX compiler identification is MSVC 19.29.30137.0 ______3.9_____windows-latest_____x64.txt: -- The CXX compiler identification is MSVC 19.29.30137.0 ______pypy-3.7-v7.3.7_____windows-latest_____x64.txt: -- The CXX compiler identification is MSVC 19.29.30137.0 ______pypy-3.8-v7.3.7_____windows-latest_____x64.txt: -- The CXX compiler identification is MSVC 19.29.30137.0

$ grep ' -- The CXX compiler identification is' *.txt | grep windows-2022.*MSVC | sed 's/2021-11-30[^ ]*//' | sed 's/^[^_]*//' | uniq | sort ______2.7_____windows-2022_____x64.txt: -- The CXX compiler identification is MSVC 19.30.30705.0 ______3.10_____windows-2022_____x64.txt: -- The CXX compiler identification is MSVC 19.30.30705.0 ______3.5_____windows-2022_____x64.txt: -- The CXX compiler identification is MSVC 19.30.30705.0 ______3.6_____windows-2022_____x64.txt: -- The CXX compiler identification is MSVC 19.30.30705.0 ______3.9_____windows-2022_____x64.txt: -- The CXX compiler identification is MSVC 19.30.30705.0 ______pypy-3.7-v7.3.7_____windows-2022_____x64.txt: -- The CXX compiler identification is MSVC 19.30.30705.0 ______pypy-3.8-v7.3.7_____windows-2022_____x64.txt: -- The CXX compiler identification is MSVC 19.30.30705.0

Let's just add a few windows-2019 builds, but don't think we need the full set. Also, I think the difference is that 2022 has the MSVC 2022 compiler also, but we could select the 2019 or 2017 or even 2015 compilers as well on the newer images (not sure how far back they go, used to be 2015, 2017, and 2019). If we force the older compiler, we don't need older Windows images (the windows-2016 image is being removed in March).

What should we do on this PR?

My feeling: add windows-2022 here, then trim down in a separate PR.

We can inject a 2022 build and leave the bulk of them 2019, or we can inject a 2019 build and leave the bulk of them 2022. No need to explode the build matrix.

include: ... - runs-on: windows-2019 python: 3.9

Something like that.

Sounds good to me either way. The simpler the better (without losing coverage). When trading of human time (for more complicated config) vs. 3 or 4 extra jobs, I'd come down on the side of minimizing human effort.

include/pybind11/embed.h

include/pybind11/detail/common.h

Boris-Rasin · 2021-11-24T22:13:07Z

@henryiii the single failed test seems to be completely unrelated. Can you take a look? Otherwise, I think this is ready.

rwgk · 2021-11-24T22:19:46Z

Yes, that's a known flake, safe to ignore.

Did you see my review from 3 hours ago?

rwgk

Looks good to me, thanks!

I already tested with the smart_holder branch (good; just one flake).

Leaving this for @henryiii to merge, because of my windows-latest vs windows-2022 question.

rwgk · 2021-12-02T23:19:16Z

.github/workflows/ci.yml

            python: pypy-2.7
+          # Inject a Windows 2019 run
+          - runs-on: windows-2019
+            python: 3.9


Nice! I didn't know it's that easy.
While we're still advertising Python 2.7 support, could we add that in here, too?

rwgk

Looks perfect now, thanks Henry!

rwgk · 2021-12-03T18:26:47Z

Re-testing after git rebase master.

rwgk · 2021-12-03T19:10:23Z

pypy-2.7 • macos-latest • x64 seems to be stuck on Post Cache wheels
Ignoring.
I'll merge this now, to get the lengthy process going -> smart_holder -> Google.

This commit adds comprehensive solutions for the pybind11 ssize_t error that occurs when building with Visual Studio 2022 (or VS 2022 with v142 toolset). The error manifests as: error C2065: 'ssize_t': undeclared identifier error C2338: ssize_t != Py_intptr_t Root cause: Older pybind11 versions (pre-2.9.1) don't properly support VS 2022's C++ standard library (_MSVC_STL_VERSION >= 143). Added files: - QUICK_FIX.md: Quick reference guide for immediate solutions - VS2022_BUILD_FIX.md: Comprehensive documentation with 4 solution options - update_pybind11.sh/bat: Scripts to update pybind11 to v2.11.1+ - mitsuba/cmake/vs2022_fix.cmake: CMake workaround for the issue - mitsuba/cmake/apply_vs2022_fix.cmake: Auto-apply fix when detecting VS 2022 - mitsuba/include/mitsuba/python/python_compat.h: Header workaround Solutions provided (in order of preference): 1. Update pybind11 submodule to v2.11.1+ (permanent fix) 2. Apply CMake workaround (temporary fix) 3. Use python_compat.h header (alternative workaround) 4. Switch to native VS 2022 toolset (v143) Reference: pybind/pybind11#3497

Skylion007 reviewed Nov 22, 2021

View reviewed changes

include/pybind11/detail/common.h Outdated Show resolved Hide resolved

rwgk reviewed Nov 23, 2021

View reviewed changes

include/pybind11/detail/common.h Outdated Show resolved Hide resolved

henryiii reviewed Nov 24, 2021

View reviewed changes

include/pybind11/embed.h Show resolved Hide resolved

henryiii reviewed Nov 24, 2021

View reviewed changes

include/pybind11/detail/common.h Outdated Show resolved Hide resolved

henryiii approved these changes Nov 24, 2021

View reviewed changes

rwgk reviewed Nov 24, 2021

View reviewed changes

rwgk mentioned this pull request Nov 24, 2021

VS2022 _invalid_parameter #3477

Closed

rwgk added a commit to rwgk/pybind11 that referenced this pull request Nov 28, 2021

snapshot of current PR pybind#3497

b9cbd41

rwgk mentioned this pull request Nov 28, 2021

[smart_holder] snapshot of current PR #3497 #3513

Closed

rwgk approved these changes Nov 28, 2021

View reviewed changes

rwgk added a commit to rwgk/pybind11 that referenced this pull request Nov 30, 2021

snapshot of current PR pybind#3497

05ea6ae

rwgk mentioned this pull request Nov 30, 2021

Clone of #3497 with windows-latest added back. #3517

Closed

rwgk reviewed Dec 2, 2021

View reviewed changes

rwgk approved these changes Dec 3, 2021

View reviewed changes

Boris-Rasin and others added 8 commits December 3, 2021 10:22

fix: vs2022 compilation, issue #3477

fa8ee11

silence warning for python 2.7

4aaac01

disable warning around mbstowcs call

8741ee8

move disable warning code closer to call site

261419a

turn on vs2022 ci test

5041340

ci: don't run helpers on Windows 2022 & Python 3.5

622bcee

limit workaround for stdlib shipped with vs2022 or later

9a0b1de

fix for: limit workaround for stdlib shipped with vs2022 or later

384d112

Boris-Rasin and others added 4 commits December 3, 2021 10:22

fix 2 for: limit workaround for stdlib shipped with vs2022 or later

ba6a33b

comment

865cfd2

ci: add a Windows 2019 run

2c676de

ci: add Python 2.7 check too

e295f57

rwgk merged commit a224d0c into pybind:master Dec 3, 2021

github-actions bot added the needs changelog Possibly needs a changelog entry label Dec 3, 2021

henryiii removed the needs changelog Possibly needs a changelog entry label Dec 21, 2021

SunBlack mentioned this pull request Jan 10, 2022

[pybind11] Update to 2.9.0 microsoft/vcpkg#22460

Merged

roigcarlo mentioned this pull request Jan 17, 2022

Ported fix for vs2022 problem with crtdefs KratosMultiphysics/Kratos#9543

Merged

petersteneteg mentioned this pull request Feb 28, 2022

[BUG]: undefining _DEBUG breaks build on VS 2022 #3459

Open

3 tasks

DannyHindson mentioned this pull request Mar 17, 2022

Fix problem compiling Mantid in Debug on VS2022 mantidproject/thirdparty-msvc2015#70

Merged

YannickJadoul added a commit to YannickJadoul/Parselmouth that referenced this pull request Apr 3, 2022

Fix VS2022 build failures by adopting pybind/pybind11#3497 (a224d0c)

c19d0cb

YannickJadoul added a commit to YannickJadoul/Parselmouth that referenced this pull request Apr 3, 2022

Fix VS2022 build failures by adopting pybind/pybind11#3497 (a224d0c)

34f5938

doug-walker mentioned this pull request Apr 22, 2022

OCIO won't compile with Visual Studio 2022 AcademySoftwareFoundation/OpenColorIO#1634

Closed

YannickJadoul added a commit to YannickJadoul/Parselmouth that referenced this pull request Jul 10, 2022

Fix VS2022 build failures by adopting pybind/pybind11#3497 (a224d0c)

e2cf52b

KiruyaMomochi mentioned this pull request Aug 22, 2022

Fix VS2022 and Python 3.10 build failure on Windows KiruyaMomochi/clingo#1

Merged

wsfulton mentioned this pull request Sep 9, 2022

python_wrap.cxx build failure with VS2022 Preview swig/swig#2090

Closed

KiruyaMomochi mentioned this pull request Oct 3, 2022

Windows: fix bootstrap and package install failure spack/spack#32942

Merged

VemundH mentioned this pull request Oct 23, 2022

[BUG] char8_t/u8string support broken since 2.9 #4277

Closed

rwgk mentioned this pull request Feb 10, 2023

FWD pybind11 google/pybind11clif#3497

Closed

Conversation

Boris-Rasin commented Nov 22, 2021 • edited by rwgk Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Suggested changelog entry:

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

henryiii commented Nov 24, 2021

Uh oh!

henryiii commented Nov 24, 2021

Uh oh!

henryiii commented Nov 24, 2021

Uh oh!

Boris-Rasin commented Nov 24, 2021

Uh oh!

Boris-Rasin commented Nov 24, 2021

Uh oh!

henryiii commented Nov 24, 2021

Uh oh!

rwgk Nov 24, 2021

Choose a reason for hiding this comment

Uh oh!

rwgk Nov 30, 2021

Choose a reason for hiding this comment

Uh oh!

henryiii Nov 30, 2021

Choose a reason for hiding this comment

Uh oh!

rwgk Nov 30, 2021

Choose a reason for hiding this comment

Uh oh!

henryiii Nov 30, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rwgk Nov 30, 2021

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Boris-Rasin commented Nov 24, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rwgk commented Nov 24, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rwgk left a comment

Choose a reason for hiding this comment

Uh oh!

rwgk Dec 2, 2021

Choose a reason for hiding this comment

Uh oh!

rwgk left a comment

Choose a reason for hiding this comment

Uh oh!

rwgk commented Dec 3, 2021

Uh oh!

rwgk commented Dec 3, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Boris-Rasin commented Nov 22, 2021 •

edited by rwgk

Loading

henryiii Nov 30, 2021 •

edited

Loading

Boris-Rasin commented Nov 24, 2021 •

edited

Loading

rwgk commented Nov 24, 2021 •

edited

Loading