fix external_project failing due to the drive letter in the path on Windows #13916

na-trium-144 · 2024-11-17T17:19:33Z

On Windows, the windows-style configure command path (like C:/path/to/configure) seems to break many configure scripts because they treat the colon in the drive letter as path separator.
Converting the configure command to unix-style path (like /c/path/to/configure) using cygpath command fixes the issue.
cygpath should be available on cygwin, msys2 and gitbash.

Edit: This conversion is done only if cygpath command is available. Otherwise it is left as is.

Edit: this PR also includes fix for enabling test cases/common/230 external project on msys2. (related to #13895)
To enable this, I had to add windows and !windows as platform value in test configuration.

Edit: Also includes fix for a bug where the leading / in the prefix is missing on MinGW, which appears to be introduced in #13886.

Edit: It turned out that with the previous approach which just removes the drive letter from prefix the generated pkg-config file has wrong prefix in it.
So this PR also convert the prefix to unix-style path when cygpath is available.

I'm new to contributing to meson. Please let me know if there are something I need to change.

eli-schwartz · 2024-11-17T17:45:00Z

cygpath should be available on cygwin, msys2 and gitbash.

How do we know that all scripts for configuring an external project via --prefix etc. are necessarily running from cygwin? In comparison, we have existing functions for determining if meson is being run from a cygwin or msys2 environment.

I suppose that it may be fairly rare for homebrewed build systems to exist and actual autoconf scripts won't run without some runtime environment similar to cygwin / msys2. How does a bare mingw install handle this?

na-trium-144 · 2024-11-17T18:42:54Z

It does not necessarily need to be running on cygwin, as long as there is sh and cygpath. msys2 and gitbash also provides cygpath (in spite of its name).

Initially I didn't think about other environment that provide sh command without cygpath command on Windows.
However now I searched a little and found at least two: BusyBox and UnxUtils.
Although I have not tested whether these can run autoconf scripts, it seems both of these have sh and do not have cygpath, and both of these uses windows style path.

So if we need to support such environments, maybe the path conversion should be done only when cygpath exists?
Then this PR would at least fix the behavior on cygwin, msys2, and gitbash, while keeping the behavior in other environment as-is, I think.

Edit: I've updated the code for this.
(But then it may fail when there are multiple sh environment installed and pick wrong cygpath. Should I care about such edge cases?)

na-trium-144 · 2024-11-20T16:10:49Z

I've pushed some additional fixes (see edited description), and now CI passed.
Could someone review this please?
maybe @eli-schwartz ?

eli-schwartz

Some of these commits are missing explanations for why they are needed.

Using "added" for some commits and "fix" for others is inconsistent style ;) consistency would be "add / fix" or "added / fixed". Git style is to use imperative mood: https://cbea.ms/git-commit/#imperative

mesonbuild/modules/external_project.py

na-trium-144 · 2024-11-23T16:37:52Z

@eli-schwartz Thanks for review! I fixed them all, so please check it again.

mesonbuild/modules/external_project.py

bruchar1

This LGTM. I think the 3 last commits could be squashed in one.

eli-schwartz · 2025-07-04T02:06:20Z

I object to using ctypes at all. :) It's fragile and not meaningfully faster than running a single external command right before running a configure script.

Additionally we currently use ctypes only twice, and both times are guarded by Windows conditionals to avoid meson being broken and failing to run on Unix systems where python was manually compiled and ctypes isn't available. This isn't an uncommon occurrence. So please keep that in mind one way or the other.

moi15moi · 2025-07-04T12:44:10Z

In this case, is there anything else to be done?

The branch correspond to mesonbuild/meson#13916 but up to date

jpakkane · 2025-07-28T22:39:34Z

This could use a release note snippet explaining how and when the path conversion happens.

moi15moi · 2025-08-21T21:49:32Z

Is there any reason why this is delayed to 1.9.1 or 1.10.0?
If there is anything that needs to be done, I can help na-trium-144.

bonzini · 2025-08-22T05:09:53Z

I don't know why it wasn't reviewed so far, but after 1.9-rc3 only a really bad regression would warrant another release candidate.

bonzini · 2025-08-26T09:24:52Z

@na-trium-144 can you rebase and push?

On Cygwin, MSYS2 and GitBash, the configure command and the prefix should be converted to unix style path by cygpath command, because the colon in the drive letter breaks many configure scripts. This also fixes bugs of previous approach where leading slash was missing from prefix and it sometimes use wrong directory as base path. In addition removing drive letter should be done on rel_prefix only.

Implemented 'windows' and '!windows' as platform value in test configuration for this. In MSVC environment this test is ignored anyway since make command is not available.

…project module with ffms2 gstreamer ffmpeg isn't always up to date. For example, ffmpeg 8.0 have been released 2025-08-22, but as of 2025-10-20, gstreamer still only use the version 7.1. Also, I don't like the fact that gstreamer doesn't use the official build system. There might be difference, so let's not take any chance and use the official build system. I wanted to use the ffmpeg build system with meson External Project module https://mesonbuild.com/External-Project-module.html, but it just doesn't work. It seems that subproject (in this case bestsource) can't depend on a external project module because they don't use -uninstalled.pc, but i'm not sure. Also, the external project module doesn't use the .pc file to retrieve the dependencies, so I need to maintain it manually which sucks: mesonbuild/meson#14767 Finally, the external project module doesn't work on msys2. There is a PR that have been opened almost a year ago and work fine, but for a reason, it's not getting merge: mesonbuild/meson#13916 So, for all these reason, let's directly use autotools for ffms2 and the official ffmpeg build system by building them "manually".

xclaesse · 2025-11-04T16:03:21Z

mesonbuild/modules/external_project.py

-        # Work around that issue by removing the drive from prefix.
-        if self.prefix.drive:
-            self.prefix = Path(relpath(self.prefix, self.prefix.drive))
+        self.prefix = self._cygpath_convert(self.prefix)


In the case cygpath is not found we don't remove the drive from self.prefix anymore. That means we now run configure --prefix=c:/foo instead of configure --prefix=/foo. Is that intentional?

I'm wondering if you also need to convert self.install_dir as that's used for DESTDIR?

That means we now run configure --prefix=c:/foo instead of configure --prefix=/foo.

I don't know which is appropriate in that case, but the latter has the problem of not working with another drive. If anyone has an environment with sh but no cygpath, they may submit an issue or PR.

I'm wondering if you also need to convert self.install_dir as that's used for DESTDIR?

Unlike the prefix used by configure, DESTDIR is used by make, right? So far I haven't had any issues with the drive letter for install_dir.

xclaesse · 2025-11-04T16:07:48Z

mesonbuild/modules/external_project.py

            configure_prog = state.find_program(configure_path.as_posix())
            configure_cmd = configure_prog.get_command()
+            if len(configure_cmd) >= 2 and configure_cmd[-1] == configure_path.as_posix():
+                configure_cmd = configure_cmd[:-1] + [self._cygpath_convert(configure_path).as_posix()]


Is this needed? run_command('configure') would not do this and I assume it's working.

This is the main point of this PR. see #13699.

xclaesse · 2025-11-04T16:09:59Z

test cases/common/230 external project/libfoo/configure

+    echo "failed to extract $srcdir using path separator $PATH_SEPARATOR"
+    exit 1
+fi
+done


I don't understand what this is doing.

This is a test for the fix related to the comment above.
I wrote it to resemble the processing done by the existing configure script, but it might be the same if I simply checked whether srcdir contains a colon.

na-trium-144 requested a review from jpakkane as a code owner November 17, 2024 17:19

na-trium-144 force-pushed the configure_path_windows branch 2 times, most recently from fdc35d9 to 20a9be7 Compare November 17, 2024 17:39

na-trium-144 force-pushed the configure_path_windows branch from 482c609 to 699562d Compare November 19, 2024 13:55

na-trium-144 mentioned this pull request Nov 19, 2024

Inconsistent shared library name prefix on MinGW and Cygwin between shared_library() and test cases #13927

Open

na-trium-144 force-pushed the configure_path_windows branch 2 times, most recently from bd1f2ea to 2675d06 Compare November 20, 2024 13:25

na-trium-144 marked this pull request as draft November 20, 2024 13:25

na-trium-144 force-pushed the configure_path_windows branch from 2675d06 to d413795 Compare November 20, 2024 14:30

na-trium-144 marked this pull request as ready for review November 20, 2024 16:01

eli-schwartz requested changes Nov 20, 2024

View reviewed changes

mesonbuild/modules/external_project.py Outdated Show resolved Hide resolved

mesonbuild/modules/external_project.py Outdated Show resolved Hide resolved

This comment was marked as resolved.

Sign in to view

na-trium-144 marked this pull request as draft November 21, 2024 15:40

na-trium-144 force-pushed the configure_path_windows branch 2 times, most recently from c579566 to e5eeb77 Compare November 23, 2024 14:52

na-trium-144 marked this pull request as ready for review November 23, 2024 16:35

na-trium-144 requested a review from eli-schwartz November 25, 2024 14:26

na-trium-144 force-pushed the configure_path_windows branch from e5eeb77 to 1f89511 Compare December 16, 2024 18:19

bonzini added this to the 1.8 milestone Jan 30, 2025

bruchar1 reviewed Jan 30, 2025

View reviewed changes

mesonbuild/modules/external_project.py Outdated Show resolved Hide resolved

na-trium-144 force-pushed the configure_path_windows branch 3 times, most recently from 41d56e3 to 6e61077 Compare February 3, 2025 06:41

bruchar1 reviewed Feb 4, 2025

View reviewed changes

na-trium-144 force-pushed the configure_path_windows branch from 6e61077 to cdbe956 Compare February 5, 2025 16:58

bruchar1 approved these changes Feb 5, 2025

View reviewed changes

moi15moi added a commit to moi15moi/VideoTimestamps that referenced this pull request Jul 10, 2025

Add a patched meson version to fix compilation on msys2

f5548f8

The branch correspond to mesonbuild/meson#13916 but up to date

moi15moi mentioned this pull request Jul 13, 2025

Use ffms2/bestsource instead of ffprobe moi15moi/VideoTimestamps#14

Merged

2 tasks

na-trium-144 force-pushed the configure_path_windows branch from ff17017 to 9c4843e Compare July 31, 2025 07:10

jpakkane force-pushed the master branch from 00d6953 to 6447832 Compare August 19, 2025 20:51

bonzini modified the milestones: 1.9, 1.10, 1.9.1 Aug 20, 2025

na-trium-144 added 2 commits August 26, 2025 18:45

Enable test case common/230 on msys2

83aad41

Implemented 'windows' and '!windows' as platform value in test configuration for this. In MSVC environment this test is ignored anyway since make command is not available.

na-trium-144 force-pushed the configure_path_windows branch from 9c4843e to 83aad41 Compare August 26, 2025 09:45

bonzini modified the milestones: 1.9.1, 1.9.2 Sep 14, 2025

xclaesse reviewed Nov 4, 2025

View reviewed changes

bonzini modified the milestones: 1.9.2, 1.11 Nov 7, 2025

Uh oh!

fix external_project failing due to the drive letter in the path on Windows #13916

Are you sure you want to change the base?

fix external_project failing due to the drive letter in the path on Windows #13916

Conversation

na-trium-144 commented Nov 17, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

eli-schwartz commented Nov 17, 2024

Uh oh!

na-trium-144 commented Nov 17, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

na-trium-144 commented Nov 20, 2024

Uh oh!

eli-schwartz left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

This comment was marked as resolved.

na-trium-144 commented Nov 23, 2024

Uh oh!

Uh oh!

bruchar1 left a comment

Choose a reason for hiding this comment

Uh oh!

eli-schwartz commented Jul 4, 2025

Uh oh!

moi15moi commented Jul 4, 2025

Uh oh!

jpakkane commented Jul 28, 2025

Uh oh!

moi15moi commented Aug 21, 2025

Uh oh!

bonzini commented Aug 22, 2025

Uh oh!

bonzini commented Aug 26, 2025

Uh oh!

xclaesse Nov 4, 2025

Choose a reason for hiding this comment

Uh oh!

xclaesse Nov 4, 2025

Choose a reason for hiding this comment

Uh oh!

na-trium-144 Nov 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

xclaesse Nov 4, 2025

Choose a reason for hiding this comment

Uh oh!

na-trium-144 Nov 7, 2025

Choose a reason for hiding this comment

Uh oh!

xclaesse Nov 4, 2025

Choose a reason for hiding this comment

Uh oh!

na-trium-144 Nov 7, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

na-trium-144 commented Nov 17, 2024 •

edited

Loading

na-trium-144 commented Nov 17, 2024 •

edited

Loading

na-trium-144 Nov 7, 2025 •

edited

Loading