Unexpected Behavior: DLL loading error when recursively calling a mix of 32-bit and 64-bit subprocesses

[kevintsq]

Bug Description

I used Detours to write a tool. When I used it to time itself, recursively calling a mix of 32-bit and 64-bit subprocesses, some internal calls of DetourCreateProcessWithDllExA to LoadNtHeaderFromProcess failed with error code 299 (0x12B), which is Only part of a ReadProcessMemory or WriteProcessMemory request was completed., causing failure of outer functions. When running as administrator, the problem persists.

Command-Line Test Case

Assuming that you have a 32-bit and 64-bit program named test32.exe and test64.exe, respectively, which calls another CPU-intensive program (python test.py) that can run for a few seconds. test.zip

git clone https://github.com/kevintsq/GetProcessCPUTimeOnWindows.git
cd GetProcessCPUTimeOnWindows/bin.X64
./time ./time test32.exe
./time ../bin.X86/time test32.exe
./time ../bin.X86/time test64.exe
cd ../bin.X86
./time ./time test64.exe
./time ../bin.X64/time test64.exe
./time ../bin.X64/time test32.exe

Expected Behavior

No matter how I mix calls to 32-bit and 64-bit subprocesses, the output of the call sequence in the above example should show almost the same timing results for the subprocess. That is, one run should output the time twice, and these two times should be almost the same. This is because, according to my timing logic, in the above recursive call example, the DLL for timing should be injected into test32.exe or test64.exe twice in succession.

But the result is that only one time is correct in some cases, and the other CPU time is almost zero, indicating that the DLL for timing is injected into test32.exe or test64.exe only once. I found that some internal calls of DetourCreateProcessWithDllExA to LoadNtHeaderFromProcess failed with error code 299 (0x12B), which is Only part of a ReadProcessMemory or WriteProcessMemory request was completed., causing failure of outer functions. So the logic of my timing should not be a problem.

Detours Version

I cloned it yesterday from the main branch.

Additional Context

Some of the above examples may experience more severe failure (time.exe may fail to start from time.exe because DetourCreateProcessWithDllExA will return an error in this case) if I build it using nmake DETOURS_CONFIG=Debug.
I'm using VS 2022 v17.5.4, cl 19.35.32217.1, link 14.35.32217.1.