Crash when trying to load an Assembly in a hostfxr environment after calling FreeConsole()

[WerWolv]

Description

Hi
I'm currently experiencing a crash in one of my applications which uses the hostfxr environment to load .NET Assemblies. The application is generally a GUI application, however we use the CONSOLE subsystem to have support for a command line interface. When run from the regular explorer, the application detects this and calls FreeConsole() to hide the console window.

This works fine for the most part, however we noticed that FreeConsole() apparently leaves the stdout, stderr and stdin handles in an invalid state which later on causes a crash here when the JIT tries to duplicate the stdout handle. The code there seems to do some checks already but doesn't seem to catch this problem.

Reproduction Steps

The latest nightly release of ImHex triggers this issue at launch on Windows 10 sometimes. It's not consistent but it happens often.
It can be downloaded from https://imhex.download/#nightly

Alternatively, a simple application which uses the CONSOLE subsystem, calls FreeConsole() and then loads an Assembly through the hostfxr functions should reproduce the issue on Windows 10.

Expected behavior

stdout should simply not be used in this case instead of causing a crash

Actual behavior

The call to _dup() at

runtime/src/coreclr/jit/ee_il_dll.cpp

Line 92 in de0ab15

int jitstdoutFd = _dup(_fileno(procstdout()));

causes a abort to happen.

Regression?

No response

Known Workarounds

Interestingly enough, this issue seems to only affect Windows 10 (and potentially even older versions of Windows). It does not happen on Windows 11.

Configuration

• .NET 7.0.400
• Windows 10
• x64
• It's specific to Windows 10 (and potentially before), not Windows 11

Other information

• My application:
  https://github.com/WerWolv/ImHex
• The code that calls FreeConsole(): https://github.com/WerWolv/ImHex/blob/master/main/source/window/win_window.cpp#L238-L251
• The code that initializes the .NET runtime and loads an Assembly: https://github.com/WerWolv/ImHex/blob/master/plugins/script_loader/source/loaders/dotnet/dotnet_loader.cpp#L156
• Stacktrace:
  

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Description

Hi
I'm currently experiencing a crash in one of my applications which uses the hostfxr environment to load .NET Assemblies. The application is generally a GUI application, however we use the CONSOLE subsystem to have support for a command line interface. When run from the regular explorer, the application detects this and calls FreeConsole() to hide the console window.

This works fine for the most part, however we noticed that FreeConsole() apparently leaves the stdout, stderr and stdin handles in an invalid state which later on causes a crash here when the JIT tries to duplicate the stdout handle. The code there seems to do some checks already but doesn't seem to catch this problem.

Reproduction Steps

The latest nightly release of ImHex triggers this issue at launch on Windows 10 sometimes. It's not consistent but it happens often.
It can be downloaded from https://imhex.download/#nightly

Alternatively, a simple application which uses the CONSOLE subsystem, calls FreeConsole() and then loads an Assembly through the hostfxr functions should reproduce the issue on Windows 10.

Expected behavior

stdout should simply not be used in this case instead of causing a crash

Actual behavior

The call to _dup() at

runtime/src/coreclr/jit/ee_il_dll.cpp

Line 92 in de0ab15

int jitstdoutFd = _dup(_fileno(procstdout()));

causes a abort to happen.

Regression?

No response

Known Workarounds

Interestingly enough, this issue seems to only affect Windows 10 (and potentially even older versions of Windows). It does not happen on Windows 11.

Configuration

• .NET 7.0.400
• Windows 10
• x64
• It's specific to Windows 10 (and potentially before), not Windows 11

Other information

• My application:
  https://github.com/WerWolv/ImHex
• The code that calls FreeConsole(): https://github.com/WerWolv/ImHex/blob/master/main/source/window/win_window.cpp#L238-L251
• The code that initializes the .NET runtime and loads an Assembly: https://github.com/WerWolv/ImHex/blob/master/plugins/script_loader/source/loaders/dotnet/dotnet_loader.cpp#L156
• Stacktrace:
  

Author:	WerWolv
Assignees:	-
Labels:	area-CodeGen-coreclr, untriaged, needs-area-label
Milestone:	-

[jkotas]

Does the release build of the JIT need to do this I/O initialization unconditionally during startup? It is unnecessary work unless there needs to be something printed to the console.

[jakobbotsch]

Can you see what the value of _fileno(procstdout()) is when the crash happens? It looks like there is already some error detection of the value above, so I suppose the value is not one of the detected cases (or there is some race condition, likely the code should reuse stdoutFd).

Does the release build of the JIT need to do this I/O initialization unconditionally during startup? It is unnecessary work unless there needs to be something printed to the console.

It would presumably be possible to refactor it to be initialized lazily, though there are a number of places we make use of jitstdout. I don't know how expensive this initialization is compared to the rest of the initialization.

This issue has been marked needs-author-action and may be missing some important information.

[BruceForstall]

I marked it .NET 9 for now, but we can adjust that when we get a repro and/or fix.

[PerikiyoXD]

Can you see what the value of _fileno(procstdout()) is when the crash happens?

Invalid bytes. It can be any value. Actually, I can't debug it as the variables are optimized away, says VS.
Although, stdoutFd equals 1, thus entering the if ((stdoutFd != -1) && (stdoutFd != -2) && (errno != EINVAL)) condition and trying to duplicate the file descriptor (Which is INVALID).

I guessed that there's a race condition too. Must be noted that we launch plugins on paralell to the execution.

Quick summary of the situation

In this case we're under a IMAGE_SUBSYSTEM_WINDOWS_CUI executable process.

This crash is probably caused by FreeConsole.

FreeConsole call does close the standard streams.

We're getting a crash trying to load hostfxr.

hostfxr tries to duplicate a file descriptor, through a closed handle.

There are times (just like this) where a CUI/CLI application wants to detach from the console by using FreeConsole and that invalidates the streams that were used on hostfxr initialization.

Note

This is kind of a python and pythonw dilemma in the end, but definitely there should be a safe-check of the std handles when accessing and duplicating handles.

[PerikiyoXD]

It would definitely need a safe check on procstdout()

[jakobbotsch]

So are you saying procstdout() is returning nullptr or some other failure value?

This code runs just once at start up, so a race condition seems unlikely if you can readily reproduce it.