Do you have any interest in the mcf thread model?

[lhmouse]

Greetings, and thank you for maintaining winlibs.

Following this discussion ¹ in MSYS2, it has been acknowledged that winpthreads, which is required by the posix thread model, is slow.

There used to be deadlocks around condition variables, which got fixed by ². Although the commit could have evaded deadlocks, I think it is a serious mistake in the design of the condition variable of winpthreads.

Basing on these facts, I think it is a good time to migrate to the mcf thread model. This requires the mcfgthread library ³ notwithstanding, we may gain considerable advantages over the posix and win32 thread models:

1. mcfgthread provides native implementation for std::mutex, std::condition_variable, std::once_flag, and finally, std::thread.
2. The mcf mutex and condition variable can be initialized statically with a constexpr initializer, and require no cleanup function, just like on Linux.
3. They are much faster, outperforming CRITICAL_SECTION, and being competitive with even the native SRWLOCK and CONDITION_VARIABLE.
4. With patching the CRT further, some more __cxa_* APIs that conform to the Itanium ABI can be utilized to get proper per-thread cleanup ⁴.

Footnotes

1. https://github.com/msys2/MINGW-packages/discussions/13259 ↩

2. https://github.com/mingw-w64/mingw-w64/commit/330025c54b85512d54b6960fad07498365c8fee3 ↩

3. https://github.com/lhmouse/mcfgthread ↩

4. https://github.com/msys2/MINGW-packages/issues/2519 ↩

[brechtsanders]

I am familiar with the mcfgthread library, and if GCC supports this via configure flags I am willing to try this.

If it requires major changes to GCC's source code it's best to take it up with its developers.

When supported and given the right instructions I al willing to try such built.

However https://winlibs.com/ aims to distribute a stable version, do to make it into my releases I will need some confirmation that it is indeed stable.

[lhmouse]

I am familiar with the mcfgthread library, and if GCC supports this via configure flags I am willing to try this.

If it requires major changes to GCC's source code it's best to take it up with its developers.

I did send a patch years ago but it seemed that there wasn't much interest in Windows targets. I can try sending it again though.

When supported and given the right instructions I al willing to try such built.

Patches are available in the mcfgthread repository ¹. There is one patch for GCC itself and another for the CRT.

In order to replace the CRT, it is necessary to build GCC in a normal procedure for cross-compiling:

1. Install mingw-w64 headers and libraries.
2. Build mcfgthread with the working GCC, and install it. This is possible because it doesn't depend on the CRT.
3. Build GCC with --disable-bootstrap. and install it. This creates a 'cross' compiler that is aware of the mcf thread model.
4. Build mingw-w64 CRT with the new GCC, and install it.
5. Perform the usual 3-stage bootstrap of GCC, and install it.
6. Build and install everything else.

However https://winlibs.com/ aims to distribute a stable version, do to make it into my releases I will need some confirmation that it is indeed stable.

Yes that's my concern too. Can't say much at the moment; I maintain a number of unit tests ² but I need some information about integration with larger projects.

Footnotes

1. https://github.com/lhmouse/mcfgthread/tree/master/patches ↩

2. https://github.com/lhmouse/mcfgthread/tree/master/test ↩

[brechtsanders]

winlibs will be sticking to what GCC officially supports, so I'm closing this ticket.

[lhmouse]

I contacted Jonathan Yong and he said he will commit my patch for the mcf thread model in a few days if there are no further comments. The mcf thread model will probably be available since GCC 13.

Thanks for you work all the same.

[brechtsanders]

I will keep an eye out for that in GCC 13, thanks!

[lhmouse]

GCC 13 bootstrapped and uploaded to https://gcc-mcf.lhmouse.com/.

The MCF thread model is part of GCC 13 and does not require a separate patch any longer.

[brechtsanders]

I will start building GCC snapshot 13-20221030 now. Any new parameters I should pay attention to regarding the MCF thread model?

[brechtsanders]

It may take a while, my first attempts failed with errors like ld.exe: error: export ordinal too large: 67260

[lhmouse]

I will start building GCC snapshot 13-20221030 now. Any new parameters I should pay attention to regarding the MCF thread model?

--enable-threads=mcf should suffice. Before building GCC, it is necessary to build and install mcfgthread first. This can be done with a working GCC, as it does not depend on GCC runtime libraries.

[lhmouse]

export ordinal too large

It's because there are too many symbols to be exported from a DLL.

If you get such errors when building GCC, try disabling plugins: msys2/MINGW-packages@9501ee2#diff-cf683a120abf18dd9f774ea5df77137d01e19f359f7cb40c1fc152355cde7b80R205

[brechtsanders]

No, it was actually symbols exported from an EXE. Got past it for now with:

sed -i.bak "s/--export-all-symbols/--discard-all/" gcc/configure

[brechtsanders]

Hmm, that didn't work. Apparently there really are files called cc1.exe.a and cc1plus.exe.a.

I will try without those and see if the compiler is functional...

[brechtsanders]

Now I tried to build with --enable-threads=mcf and got:

libgcc/gthr-default.h:29:10: fatal error: mcfgthread/gthr_libobjc.h: No such file or directory

[lhmouse]

Now I tried to build with --enable-threads=mcf and got:

libgcc/gthr-default.h:29:10: fatal error: mcfgthread/gthr_libobjc.h: No such file or directory

Which version are you using? 'gthr_libobjc.h' only got added recently. Maybe you can try build and install https://github.com/lhmouse/mcfgthread/releases/tag/v1.3-ga.1 first.

[brechtsanders]

Sorry, was on 1.2-ga.1, didn't realize there was a newer release. I will try 1.3-ga.1 now...

[brechtsanders]

Build in progress now. Took me some time to figure out I also need to distribute libmcfgthread.a/libmcfgthread.dll.a...

[lhmouse]

Build in progress now. Took me some time to figure out I also need to distribute libmcfgthread.a/libmcfgthread.dll.a...

Of course. BTW how do you distribute other libraries, such as winpthreads?

[brechtsanders]

winpthreads comes with mingw-w64, the only special library I had to add manually was libdl.

In the meantime I also noticed I need to distribute the entire include/mcfgthread folder too...

[brechtsanders]

Compilation feels slower, is that possible?

Next issue:

d:/prog/winlibs64_stage/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/13.0.0/../../../../x86_64-w64-mingw32/bin/ld.exe: jit/libgccjit.o:libgccjit.cc:(.text$_ZN16jit_version_infoC1Ev[_ZN16jit_version_infoC1Ev]+0x17): undefined reference to `pthread_mutex_lock'
d:/prog/winlibs64_stage/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/13.0.0/../../../../x86_64-w64-mingw32/bin/ld.exe: jit/libgccjit.o:libgccjit.cc:(.text$_ZN16jit_version_infoC1Ev[_ZN16jit_version_infoC1Ev]+0x45): undefined reference to `pthread_mutex_unlock'
d:/prog/winlibs64_stage/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/13.0.0/../../../../x86_64-w64-mingw32/bin/ld.exe: jit/jit-playback.o:jit-playback.c:(.text+0x5b38): undefined reference to `pthread_mutex_lock'
d:/prog/winlibs64_stage/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/13.0.0/../../../../x86_64-w64-mingw32/bin/ld.exe: jit/jit-playback.o:jit-playback.c:(.text+0x5bfe): undefined reference to `pthread_mutex_unlock'
collect2.exe: error: ld returned 1 exit status
make[2]: *** [../../gcc/jit/Make-lang.in:173: libgccjit-0.dll] Error 1
make[2]: Leaving directory '/R/winlibs64_stage/gcc-13-20221030/build_mingw/gcc'
make[1]: *** [Makefile:4604: all-gcc] Error 2
make[1]: Leaving directory '/R/winlibs64_stage/gcc-13-20221030/build_mingw'
make: *** [Makefile:1038: all] Error 2
make: Leaving directory '/R/winlibs64_stage/gcc-13-20221030/build_mingw'

Will try again without jit support for now...

[lhmouse]

libgccjit doesn't build as it references pthread directly. I think it would be feasible to replace those pthread_* APIs with __gthread_* ones, but haven't tried so far.

[brechtsanders]

Here you go: https://github.com/brechtsanders/winlibs_mingw/releases/tag/13.0.0-snapshot20221030-10.0.0-msvcrt-r1mcf

Can you test if it does what you expect (e.g. speed/performance)?

[lhmouse]

Thank you. I thought I had composited some tests about std::condition_variable but couldn't find them. I can take a look tomorrow.

[lhmouse]

This program creates 4 threads, then sends a ping request (with a mutex and condvar) to a random one of them, waits for its response, and repeats this 1,000,000 times.

perf.cc.txt

I tried compiling and running this on my Core i7-11700 machine with Windows 10 21H2. With the MSYS2 GCC 12 with posix thread model, it completed in ~5.3 seconds. With your GCC 13 build with mcf thread model, it completed in ~5.1 seconds. This may not be a good comparison, but you can try it yourself, with different parameters.

[lhmouse]

I have started the discussion about use of pthread_mutex_t in libgccjit. There are two possible solutions, one is to use std::mutex and the other is to use __gthread_mutex_t. Either way, we may hopefully get a conclusion before GCC 13 release.

[xtemp09]

@lhmouse, what is your opinion of Eric Botcazou solution? You might want to take a look at this topic.

[lhmouse]

@lhmouse, what is your opinion of Eric Botcazou solution? You might want to take a look at this topic.

I have no objection on improving the win32 thread model. The mcf thread model is not only an attempt to make std::thread etc. available, but to address more issues. Please read https://sourceforge.net/p/mingw-w64/mailman/message/37725254/ for details.

[lhmouse]

Finally.