Interpreter discovery bug wrt. Microsoft Store shortcut

[axel-kah]

Issue
hatch is using virtualenvs interpreter discovery during creation of its virtual envs. The discovery also finds the Microsoft Store python shortcut. Even though the interpreter was not installed using the MS Store, this executable is used during discovery to run virtualenvs py_info.py script. In this setting, hatch is able to successfully create its venv (read: exit code 0), but the discovery returns a bunch of UnicodeDecodeErrors and spills them on the terminal ☹️

hatch env create
Exception in thread Thread-6 (_readerthread):
Traceback (most recent call last):
  File "C:\Users\axel-kah\AppData\Local\pyapp\data\hatch\5730184961401994386\1.13.0\python\Lib\threading.py", line 1073, in _bootstrap_inner
    self.run()
  File "C:\Users\axel-kah\AppData\Local\pyapp\data\hatch\5730184961401994386\1.13.0\python\Lib\threading.py", line 1010, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Users\axel-kah\AppData\Local\pyapp\data\hatch\5730184961401994386\1.13.0\python\Lib\subprocess.py", line 1599, in _readerthread
    buffer.append(fh.read())
                  ^^^^^^^^^
  File "<frozen codecs>", line 322, in decode
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfc in position 38: invalid start byte
Exception in thread Thread-8 (_readerthread):
Traceback (most recent call last):
  File "C:\Users\axel-kah\AppData\Local\pyapp\data\hatch\5730184961401994386\1.13.0\python\Lib\threading.py", line 1073, in _bootstrap_inner
    self.run()
  File "C:\Users\axel-kah\AppData\Local\pyapp\data\hatch\5730184961401994386\1.13.0\python\Lib\threading.py", line 1010, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Users\axel-kah\AppData\Local\pyapp\data\hatch\5730184961401994386\1.13.0\python\Lib\subprocess.py", line 1599, in _readerthread
    buffer.append(fh.read())
                  ^^^^^^^^^
  File "<frozen codecs>", line 322, in decode
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfc in position 38: invalid start byte

The root cause seems to be that virtualenv is spawning a subprocess for each interpreter it finds and has it execute the py_info.py script. On Windows this will also try the same with the "mysterious" C:\Users\axel-kah\AppData\Local\Microsoft\WindowsApps\python.exe. If python was not installed using MS Store, then this executable will return an error message using the infamous cp1252 encoding. When the OS is set to using a language like german, then this error message will contain german umlauts like ü which result in the UnicodeDecodeErrors.

Proposed Fix
Change the encoding to cp1252 when on windows when launching the subprocesses during discovery, instead of using utf-8 for all platforms.

I have verified the fix by locally patching a dev install of hatch and could submit a PR.

Environment

Provide at least:

• OS: win11 (german language(!))
• hatch 1.13.0
• virtualenv 20.28.0
• pip list of the host python where virtualenv is installed:

"C:\Users\axel-kah\AppData\Local\pyapp\data\hatch\5730184961401994386\1.13.0\python\python.exe" -m pip list
Package           Version
----------------- ----------
anyio             4.6.2
certifi           2024.8.30
click             8.1.7
colorama          0.4.6
distlib           0.3.9
filelock          3.16.1
h11               0.14.0
hatch             1.13.0
hatchling         1.25.0
httpcore          1.0.6
httpx             0.27.2
hyperlink         21.0.0
idna              3.10
jaraco.classes    3.4.0
jaraco.context    6.0.1
jaraco.functools  4.1.0
keyring           25.4.1
markdown-it-py    3.0.0
mdurl             0.1.2
more-itertools    10.5.0
packaging         24.1
pathspec          0.12.1
pexpect           4.9.0
pip               24.0
platformdirs      4.3.6
pluggy            1.5.0
ptyprocess        0.7.0
Pygments          2.18.0
pywin32-ctypes    0.2.3
rich              13.9.2
setuptools        69.1.0
shellingham       1.5.4
sniffio           1.3.1
tomli_w           1.1.0
tomlkit           0.13.2
trove-classifiers 2024.10.13
userpath          1.9.2
uv                0.4.20
virtualenv        20.26.6
zstandard         0.23.0

Output of the virtual environment creation

Not applicable because venv is created implicitly by hatch.

[Darsstar]

I came here from pyenv-win. Mind trying a different patch?
Keep the encoding as utf-8, but also pass errors="backslashreplace". (see https://docs.python.org/3/library/codecs.html#error-handlers and https://docs.python.org/3/library/subprocess.html#popen-constructor)

[axel-kah]

Keep the encoding as utf-8, but also pass errors="backslashreplace"

Seems to work just as well. Maybe Bernát can make a call on how he would like to have this handled, once he's back.