[BUG] flex package installs include files to /usr/include, not /mingw64/include

[xobs]

Most include files appear to go into /mingw64/include or /mingw32/include, however the flex package installs its include files to /usr/include/. This path is not part of the gcc default include path, as evidenced by echo | gcc -E -Wp,-v -

flex should install its include files to /mingw64/include to avoid having to add -I/usr/include to the build.

smcro@Cuboid MINGW64 ~
$ gcc -v
Using built-in specs.
COLLECT_GCC=D:\Software\msys64\mingw64\bin\gcc.exe
COLLECT_LTO_WRAPPER=D:/Software/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/9.3.0/lto-wrapper.exe
Target: x86_64-w64-mingw32
Configured with: ../gcc-9.3.0/configure --prefix=/mingw64 --with-local-prefix=/mingw64/local --build=x86_64-w64-mingw32 --host=x86_64-w64-mingw32 --target=x86_64-w64-mingw32 --with-native-system-header-dir=/mingw64/x86_64-w64-mingw32/include --libexecdir=/mingw64/lib --enable-bootstrap --with-arch=x86-64 --with-tune=generic --enable-languages=c,lto,c++,fortran,ada,objc,obj-c++ --enable-shared --enable-static --enable-libatomic --enable-threads=posix --enable-graphite --enable-fully-dynamic-string --enable-libstdcxx-filesystem-ts=yes --enable-libstdcxx-time=yes --disable-libstdcxx-pch --disable-libstdcxx-debug --disable-isl-version-check --enable-lto --enable-libgomp --disable-multilib --enable-checking=release --disable-rpath --disable-win32-registry --disable-nls --disable-werror --disable-symvers --enable-plugin --with-libiconv --with-system-zlib --with-gmp=/mingw64 --with-mpfr=/mingw64 --with-mpc=/mingw64 --with-isl=/mingw64 --with-pkgversion='Rev1, Built by MSYS2 project' --with-bugurl=https://sourceforge.net/projects/msys2 --with-gnu-as --with-gnu-ld
Thread model: posix
gcc version 9.3.0 (Rev1, Built by MSYS2 project)

smcro@Cuboid MINGW64 ~
$ echo | gcc -E -Wp,-v -
# 1 "<stdin>"
# 1 "<built-in>"
# 1 "<command-line>"
# 1 "<stdin>"
ignoring duplicate directory "D:/Software/msys64/mingw64/lib/gcc/../../lib/gcc/x86_64-w64-mingw32/9.3.0/include"
ignoring nonexistent directory "D:/msys64/mingw64/include"
ignoring nonexistent directory "/mingw64/include"
ignoring duplicate directory "D:/Software/msys64/mingw64/lib/gcc/../../lib/gcc/x86_64-w64-mingw32/9.3.0/include-fixed"
ignoring duplicate directory "D:/Software/msys64/mingw64/lib/gcc/../../lib/gcc/x86_64-w64-mingw32/9.3.0/../../../../x86_64-w64-mingw32/include"
ignoring nonexistent directory "D:/msys64/mingw64/x86_64-w64-mingw32/include"
#include "..." search starts here:
#include <...> search starts here:
 D:/Software/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/9.3.0/include
 D:/Software/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/9.3.0/../../../../include
 D:/Software/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/9.3.0/include-fixed
 D:/Software/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/9.3.0/../../../../x86_64-w64-mingw32/include
End of search list.

smcro@Cuboid MINGW64 /d/Code/verilator
$ pacman -Qs flex
local/flex 2.6.4-1 (base base-devel)
    A tool for generating text-scanning programs

smcro@Cuboid MINGW64 ~
$

[Alexpux]

@xobs flex package is MSYS package and it install everything to /usr. MINGW packages installs everything to /mingw*. So there are no bug

[xobs]

Should /usr/include be part of the default include path? I filed a bug against mingw, and they said this is an msys problem.

The issue is that I've installed flex inside msys (or is it inside mingw64? I'm so confused now.) I have a project that includes FlexLexer.h, however it says it cannot find that file unless I add -I/usr/include to CFLAGS. That seems like a bug, and I'm trying to figure out where the bug is.

[Alexpux]

@xobs you can’t add /use/include to include path for MINGW GCC as it would break build because this path contains headers for MSYS GCC

[xobs]

I think I understand. I'm still very confused as to what I'm using, whether it's msys2, msys, or mingw64. I'm running D:\Softwre\msys64\mingw64.exe, which pops up a terminal that has a very similar logo to https://github.com/msys2, except it's blue instead of purple. Nowhere is msys2 mentioned. Am I using msys64, msys2, mingw64, mingw, or msys?

Furthermore, /mingw64/bin/gcc.exe -v indicates it's actually Rev1, Built by MSYS2 project.

Inside that terminal, somehow /mingw64/bin has been placed first in the PATH, which causes it to take precedence. I've deleted /mingw64, which causes /usr/bin/ to take prededence, allowing me to use gcc from there.

Did the mingw64 project create /mingw64/bin/gcc.exe, or did the msys2 project create that and the directory is improperly named? And is the flex package maintained by the msys project, or msys2?

[Alexpux]

@xobs this is MSYS2 project so there are nothing about MSYS and MINGW from mingw.org.
Read wiki https://www.msys2.org/wiki/Home/ to understand things

[smcpeak]

@Alexpux If there is no bug here, then how does one successfully use flex with MSYS2 and MINGW? I have followed all of the instructions at https://www.msys2.org/ , culminating in running "MSYS MinGW 64-bit" with flex available in /usr/bin. But it isn't possible to actually use flex because FlexLexer.h is in /usr/include, which is not searched by gcc (/mingw64/bin/gcc).

[ZachBacon]

@Alexpux If there is no bug here, then how does one successfully use flex with MSYS2 and MINGW? I have followed all of the instructions at https://www.msys2.org/ , culminating in running "MSYS MinGW 64-bit" with flex available in /usr/bin. But it isn't possible to actually use flex because FlexLexer.h is in /usr/include, which is not searched by gcc (/mingw64/bin/gcc).

Let me ask you this, how did you install flex? What was the command you used?

Edit:

Never mind there is only one flex package and it's compiled using msys2 gcc not mingw gcc and thus wouldn't be usable in mingw gcc at all, it would need to be natively compiled to windows using mingw gcc/clang for what you want to use it for.

[smcpeak]

Just for completeness, the command that installed flex was pacman -S --needed base-devel mingw-w64-x86_64-toolchain, which is step 7 on the MSYS2 home page.

I later tried building flex from source with mingw64 in the MSYS2 shell, but it fails during configure:

~/bld/flex-2.6.4$ ./configure --prefix=$HOME/opt/flex-2.6.4-msys2 --disable-shared
[...]
checking for sys/wait.h... no
configure: error: required header not found on your system
Exit 1

[ZachBacon]

Just for completeness, the command that installed flex was pacman -S --needed base-devel mingw-w64-x86_64-toolchain, which is step 7 on the MSYS2 home page.

I later tried building flex from source with mingw64 in the MSYS2 shell, but it fails during configure:

~/bld/flex-2.6.4$ ./configure --prefix=$HOME/opt/flex-2.6.4-msys2 --disable-shared
[...]
checking for sys/wait.h... no
configure: error: required header not found on your system
Exit 1

That's because it's trying to use POSIX functions that are not available for Windows and you'd have to supplement the code to make it work within windows unfortunately.

[smcpeak]

So is the conclusion that flex has never been successfully ported to MSYS2? If so, why does https://packages.msys2.org/package/flex exist?

[jeremyd2019]

flex has been ported to MSYS2. It has not been ported to mingw-w64.

[1480c1]

So is the conclusion that flex has never been successfully ported to MSYS2? If so, why does https://packages.msys2.org/package/flex exist?

that is build using the cygwin based msys toolchain and not mingw-w64 (meaning it has access to all the posix features cygwin provides)

[jeremyd2019]

several mingw-w64 packages successfully makedepend on flex, so there must be some way to use it without library/headers.
https://packages.msys2.org/package/flex?repo=msys&variant=x86_64

[smcpeak]

Having spent some time working with flex, I think I can explain why it works for some packages. The primary purpose of flex is to generate code, and the generated code can then be compiled with (in theory) any compiler, including the mingw-w64 compiler. The only complications arise because of libfl.a and FlexLexer.h.

It turns out that libfl.a only contains two functions, main() and yywrap(), and neither is needed--main for obvious reasons, and yywrap because you can use %option noyywrap. Thus, there is no need to link with the library, so it does not matter that it hasn't been compiled with mingw-w64.

As for FlexLexer.h, that is only needed when using %option c++, which I think is uncommon (it is not needed merely because one compiles with a C++ compiler; rather, it changes the interface of the generated scanner), so isn't an issue for most users. For those it does affect, it is in the wrong location for use with mingw-w64, but its contents are the same for all configurations, so one can just copy it elsewhere.

So what does that mean for this bug? I think there is a bug here, namely that flex does not work under MSYS2 mingw-w64 when using %option c++. The workaround of copying FlexLexer.h is available, but ideally a copy of that file would also be put into a directory that is searched by mingw-w64.

[ZachBacon]

Having spent some time working with flex, I think I can explain why it works for some packages. The primary purpose of flex is to generate code, and the generated code can then be compiled with (in theory) any compiler, including the mingw-w64 compiler. The only complications arise because of libfl.a and FlexLexer.h.

It turns out that libfl.a only contains two functions, main() and yywrap(), and neither is needed--main for obvious reasons, and yywrap because you can use %option noyywrap. Thus, there is no need to link with the library, so it does not matter that it hasn't been compiled with mingw-w64.

As for FlexLexer.h, that is only needed when using %option c++, which I think is uncommon (it is not needed merely because one compiles with a C++ compiler; rather, it changes the interface of the generated scanner), so isn't an issue for most users. For those it does affect, it is in the wrong location for use with mingw-w64, but its contents are the same for all configurations, so one can just copy it elsewhere.

So what does that mean for this bug? I think there is a bug here, namely that flex does not work under MSYS2 mingw-w64 when using %option c++. The workaround of copying FlexLexer.h is available, but ideally a copy of that file would also be put into a directory that is searched by mingw-w64.

Copying a file to be searchable by a mingw compiler is not really an option, ideally the best option would be to make the package native that way it can be made available to applicable toolchains in the repositories

[Nemoumbra]

So... What is the current state of the problem? I've got this installed:

To prepare I did pacman -S flex, pacman -S bison from this terminal.
I've just cloned my Linux compiler project that uses flex, bison and LLVM (it's a custom simple lang -> LLVM IR compiler) from Github and tried to build it with CLion. The symptoms are the same, I think: the file FlexLexer.h ended up in k:\msys64\usr\include. The toolchain that CLion calls "MinGW" is located in k:\msys64\mingw64.
Can someone answer me how to fix the problem?

[Biswa96]

Can someone answer me how to fix the problem?

It was already answer in previous comments.

flex package is MSYS package and it install everything to /usr. MINGW packages installs everything to /mingw*. So there are no bug

[Nemoumbra]

Can someone answer me how to fix the problem?

It was already answer in previous comments.

flex package is MSYS package and it install everything to /usr. MINGW packages installs everything to /mingw*. So there are no bug

This doesn't tell me how to compile my project.

[Biswa96]

This doesn't tell me how to compile my project.

Please create a new issue or discussion with your problem and steps to reproduce it.

[andrew-cogni]

I know this is an old issue, but if you run into this and need a workaround that wasn't directly stated in the previous comments:

• copy /usr/include/FlexLexer.h to whatever directory has source that you're compiling

If you're trying to make Verilator in MSYS2:

• pacman -S flex
• cp /usr/include/FlexLexer.h /some/path/to/verilator/src