Skip to content

[gmp] Switch sources on windows away from SMP fork. #23466

Merged
dan-shaw merged 18 commits intomicrosoft:masterfrom
Neumann-A:native_build_gmp
Mar 23, 2022
Merged

[gmp] Switch sources on windows away from SMP fork. #23466
dan-shaw merged 18 commits intomicrosoft:masterfrom
Neumann-A:native_build_gmp

Conversation

@Neumann-A
Copy link
Contributor

No description provided.

Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You have modified or added at least one vcpkg.json where a "license" field is missing.

Details

If you feel able to do so, please consider adding a "license" field to the following files:

  • ports/gmp/vcpkg.json

Valid values for the license field can be found in the documentation

Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You have modified or added at least one vcpkg.json where a "license" field is missing.

Details

If you feel able to do so, please consider adding a "license" field to the following files:

  • ports/gmp/vcpkg.json

Valid values for the license field can be found in the documentation

Copy link
Contributor

@ras0219-msft ras0219-msft left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an awesome improvement.

@LilyWangLL
Copy link
Contributor

@Neumann-A There are regression error on CI pipeline:

if (target_option=k; case ${target_option-} in ?) ;; *) echo "am__make_running_with_option: internal error: invalid" "target option '${target_option-}' specified" >&2; exit 1;; esac; has_opt=no; sane_makeflags=$MAKEFLAGS; if { if test -z '1'; then false; elif test -n 'x86_64-pc-msys'; then true; elif test -n '4.3' && test -n '/d/buildtrees/gmp/x64-windows-dbg'; then true; else false; fi; }; then sane_makeflags=$MFLAGS; else case $MAKEFLAGS in *\\[\ \	]*) bs=\\; sane_makeflags=`printf '%s\n' "$MAKEFLAGS" | sed "s/$bs$bs[$bs $bs	]*//g"`;; esac; fi; skip_next=no; strip_trailopt () { flg=`printf '%s\n' "$flg" | sed "s/$1.*$//"`; }; for flg in $sane_makeflags; do test $skip_next = yes && { skip_next=no; continue; }; case $flg in *=*|--*) continue;; -*I) strip_trailopt 'I'; skip_next=yes;; -*I?*) strip_trailopt 'I';; -*O) strip_trailopt 'O'; skip_next=yes;; -*O?*) strip_trailopt 'O';; -*l) strip_trailopt 'l'; skip_next=yes;; -*l?*) strip_trailopt 'l';; -[dEDm]) skip_next=yes;; -[JT]) skip_next=yes;; esac; case $flg in *$target_option*) has_opt=yes; break;; esac; done; test $has_opt = yes); then \
get_ui.obj : error LNK2005: __gmpz_get_ui already defined in bin_ui.obj
neg.obj : error LNK2005: __gmpz_neg already defined in bin_ui.obj
stronglucas.obj : error LNK2005: __gmpz_perfect_square_p already defined in perfsqr.obj
add.obj : error LNK2005: __gmpn_add already defined in add.obj
add_1.obj : error LNK2005: __gmpn_add_1 already defined in add_ui.obj
sub.obj : error LNK2005: __gmpn_sub already defined in sub.obj
sub_1.obj : error LNK2005: __gmpn_sub_1 already defined in sub.obj
neg.obj : error LNK2005: __gmpn_neg already defined in sub.obj
cmp.obj : error LNK2005: __gmpn_cmp already defined in cmp.obj
zero_p.obj : error LNK2005: __gmpn_zero_p already defined in combit.obj
LINK : fatal error LNK1104: cannot open file '.libs/libgmp-3.dll.def'

@LilyWangLL LilyWangLL added category:port-update The issue is with a library, which is requesting update new revision requires:author-response labels Mar 10, 2022
@Neumann-A
Copy link
Contributor Author

  • x64_uwp: fails because gmp builds some code generators which won't work on uwp. Needs host dep on gmp itself and manual copying of the generators.

  • x64_windows/x86_windows: Strange duplicated symbols which I can only reproduce on x86 (for me x64 build successfully). I couldn't find an obvious reason in the sources so somebody with more knowledge than me needs to look at it.

@Neumann-A
Copy link
Contributor Author

Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a new experimental fast check for PR issues. Please let us know if this bot is helpful!

After committing all other changes, the version database must be updated
git add -u && git commit
git checkout 5a8a1dfff8e2dffd4c8cc2d1fc0da379ffd18b25 -- versions
./vcpkg x-add-version --all
Diff
diff --git a/versions/g-/gmp.json b/versions/g-/gmp.json
index f6ad9ad..6e7b3ca 100644
--- a/versions/g-/gmp.json
+++ b/versions/g-/gmp.json
@@ -1,7 +1,7 @@
 {
   "versions": [
     {
-      "git-tree": "66917e3a6421c994cf676763cd48c30f8117c8ec",
+      "git-tree": "d252d4fbdebdfeff6c7d17087e9f66cdc3a0c07e",
       "version": "6.2.1",
       "port-version": 9
     },

You have modified or added at least one vcpkg.json where a "license" field is missing.

Details

If you feel able to do so, please consider adding a "license" field to the following files:

  • ports/gmp/vcpkg.json

Valid values for the license field can be found in the documentation

Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You have modified or added at least one vcpkg.json where a "license" field is missing.

Details

If you feel able to do so, please consider adding a "license" field to the following files:

  • ports/gmp/vcpkg.json
  • ports/mpc/vcpkg.json
  • ports/nettle/vcpkg.json

Valid values for the license field can be found in the documentation

Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You have modified or added at least one vcpkg.json where a "license" field is missing.

Details

If you feel able to do so, please consider adding a "license" field to the following files:

  • ports/gmp/vcpkg.json
  • ports/mpc/vcpkg.json
  • ports/nettle/vcpkg.json

Valid values for the license field can be found in the documentation

@JackBoosY JackBoosY added info:needs-maintainer-attention Lets the current 'on rotation' vcpkg maintainer know they need to look at this. and removed requires:author-response labels Mar 22, 2022
Copy link
Contributor

@ras0219-msft ras0219-msft left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM once the manual-tools directory is renamed to tools. If the concern is that users might be confused, I'm fine with adding more subdirectories underneath tools/${PORT}/... to clarify

--enable-cxx
)
if(VCPKG_CROSSCOMPILING)
list(TRANSFORM tool_names PREPEND "${CURRENT_HOST_INSTALLED_DIR}/manual-tools/${PORT}/")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
list(TRANSFORM tool_names PREPEND "${CURRENT_HOST_INSTALLED_DIR}/manual-tools/${PORT}/")
list(TRANSFORM tool_names PREPEND "${CURRENT_HOST_INSTALLED_DIR}/tools/${PORT}/")

I don't think this is special enough to need a new top-level directory "manual-tools"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Manual-tools already exists in e.g. the vcpkg-tool-python2 port and it is intended that those tools dont get automatically added to CMAKE_PROGRAM_PATH.

@dan-shaw dan-shaw merged commit 2aa0d83 into microsoft:master Mar 23, 2022
@Neumann-A Neumann-A deleted the native_build_gmp branch March 23, 2022 19:34
BillyONeal added a commit to BillyONeal/vcpkg that referenced this pull request Mar 28, 2022
* [Most recent nightly build failed](https://dev.azure.com/vcpkg/public/_build/results?buildId=69427)
* [Validation of this tool update failed](https://dev.azure.com/vcpkg/public/_build/results?buildId=69417)

## Common to both:

PASSING, REMOVE FROM FAIL LIST: chartdir:x64-windows (.\scripts\ci.baseline.txt)
PASSING, REMOVE FROM FAIL LIST: chartdir:x64-windows-static-md (.\scripts\ci.baseline.txt)
PASSING, REMOVE FROM FAIL LIST: chartdir:x86-windows (.\scripts\ci.baseline.txt)

Probably fixed by microsoft#23701

PASSING, REMOVE FROM FAIL LIST: gmp:x64-uwp (.\scripts\ci.baseline.txt)
PASSING, REMOVE FROM FAIL LIST: gmp:x64-windows-static-md (.\scripts\ci.baseline.txt)

Probably fixed by microsoft#23466 ?

REGRESSION: colmap:x64-windows-static-md failed with BUILD_FAILED. If expected, add colmap:x64-windows-static-md=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.

I don't know exactly what changed. I observe that
* this thing depends on a *lot* of stuff
* on March 14 we didn't even attempt to build this
* the x64-windows ones are already in the baseline

so I skipped it.

REGRESSION: qtdeclarative:x64-windows. If expected, add qtdeclarative:x64-windows=fail to .\scripts\ci.baseline.txt.
REGRESSION: qtdeclarative:x64-windows failed with BUILD_FAILED. If expected, add qtdeclarative:x64-windows=fail to C:\a\1\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: qtdeclarative:x64-windows failed with BUILD_FAILED. If expected, add qtdeclarative:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: qtdeclarative:x64-windows failed with BUILD_FAILED. If expected, add qtdeclarative:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: qtdeclarative:x64-windows failed with BUILD_FAILED. If expected, add qtdeclarative:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: qtdeclarative:x64-windows failed with BUILD_FAILED. If expected, add qtdeclarative:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: qtdeclarative:x64-windows failed with BUILD_FAILED. If expected, add qtdeclarative:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: qtdeclarative:x64-windows failed with BUILD_FAILED. If expected, add qtdeclarative:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.

This is a reporting change: The new world order also includes host build failures which is why it's duplicated.

See also microsoft#23714
See also microsoft#23490

I'm nervous about baslining this because it seems most of the qt world is built on top of this port

I filed microsoft#23824 about this and @Neumann-A indicated this should be fixed by microsoft#23755

REGRESSION: nettle:x64-uwp. If expected, add nettle:x64-uwp=fail to .\scripts\ci.baseline.txt.
REGRESSION: nettle:x64-windows-static-md. If expected, add nettle:x64-windows-static-md=fail to .\scripts\ci.baseline.txt.
REGRESSION: nettle:x64-uwp failed with BUILD_FAILED. If expected, add nettle:x64-uwp=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: nettle:x64-windows-static-md failed with POST_BUILD_CHECKS_FAILED. If expected, add nettle:x64-windows-static-md=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.

Didn't analyze, probably fixed by microsoft#23519 ?

REGRESSION: libgpg-error:x64-uwp. If expected, add libgpg-error:x64-uwp=fail to .\scripts\ci.baseline.txt.
REGRESSION: libgpg-error:x64-uwp failed with BUILD_FAILED. If expected, add libgpg-error:x64-uwp=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.

This was broken by VS2022 update:
```
C:\Program Files\Microsoft Visual Studio\2022\Enterprise\MSBuild\Microsoft\VisualStudio\v17.0\AppxPackage\Microsoft.AppXPackage.Targets(892,25): error MSB4086: A numeric comparison was attempted on "$(TargetPlatformMinVersion)" that evaluates to "" instead of a number, in condition "'$(TargetPlatformMinVersion)' >= '10.0.17200.0'". [C:\Dev\vcpkg\buildtrees\libgpg-error\x64-uwp-rel\error-1.42-2324ddbc71.clean\SMP\libgpg-error_winrt.vcxproj]
```

REGRESSION: libmikmod:x64-osx. If expected, add libmikmod:x64-osx=fail to .\scripts\ci.baseline.txt.
REGRESSION: libmikmod:x64-osx failed with BUILD_FAILED. If expected, add libmikmod:x64-osx=fail to /Users/vagrant/Data/work/2/s/scripts/azure-pipelines/../ci.baseline.txt.

Broken between [2022-03-16](https://dev.azure.com/vcpkg/public/_build/results?buildId=68947) and [2022-03-18](https://dev.azure.com/vcpkg/public/_build/results?buildId=69051). Unfortunately I don't see obvious reasons why. Nothing else depends on this and nobody has noticed in 2 weeks, so I'm baslining it for now. (Will investigate shortly...)

## Only broken in tool update:

REGRESSION: mesa:x64-windows failed with BUILD_FAILED. If expected, add mesa:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.

```
-- Downloading https://gitlab.freedesktop.org/mesa/mesa/-/archive/mesa-21.2.5/mesa-mesa-21.2.5.tar.gz -> mesa-mesa-mesa-21.2.5-1.tar.gz...
-- Extracting source /Users/vagrant/Data/downloads/mesa-mesa-mesa-21.2.5-1.tar.gz
-- Applying patch swravx512-post-static-link.patch
-- Applying patch swr-msvc-2.patch
-- Applying patch swr-llvm13.patch
-- Applying patch radv-msvc-llvm13-2.patch
-- Applying patch d3d10sw.patch
-- Using source at /Users/vagrant/Data/buildtrees/mesa/src/esa-21.2.5-2df234d2b1.clean
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'mako'
CMake Error at ports/mesa/portfile.cmake:85 (message):
  Python package 'mako' needs to be installed for port 'mesa'.

  Complete list of required python packages: setuptools;mako
Call Stack (most recent call first):
  ports/mesa/portfile.cmake:91 (vcpkg_get_python_package)
  scripts/ports.cmake:145 (include)
```

Looks like this is being tracked by microsoft#23089 ; perhaps that we don't have as aggressive a recycling strategy for macos boxes as we do for the others has let different machines give different results?

## Only broken without tool update:

REGRESSION: chromium-base:x64-osx. If expected, add chromium-base:x64-osx=fail to .\scripts\ci.baseline.txt.

This one has been constantly flaky; I baselined it.

REGRESSION: libxml2:x64-osx. If expected, add libxml2:x64-osx=fail to .\scripts\ci.baseline.txt.

This port uses vcpkg_from_git and the upstream server was down during the build.
BillyONeal added a commit that referenced this pull request Mar 28, 2022
* Update vcpkg-tool to 2022-03-24

* Hook up microsoft/vcpkg-tool#345

* Hook up microsoft/vcpkg-tool#442

* Update vcpkg-tool to 2022-03-25

* Analysis of failures.

* [Most recent nightly build failed](https://dev.azure.com/vcpkg/public/_build/results?buildId=69427)
* [Validation of this tool update failed](https://dev.azure.com/vcpkg/public/_build/results?buildId=69417)

## Common to both:

PASSING, REMOVE FROM FAIL LIST: chartdir:x64-windows (.\scripts\ci.baseline.txt)
PASSING, REMOVE FROM FAIL LIST: chartdir:x64-windows-static-md (.\scripts\ci.baseline.txt)
PASSING, REMOVE FROM FAIL LIST: chartdir:x86-windows (.\scripts\ci.baseline.txt)

Probably fixed by #23701

PASSING, REMOVE FROM FAIL LIST: gmp:x64-uwp (.\scripts\ci.baseline.txt)
PASSING, REMOVE FROM FAIL LIST: gmp:x64-windows-static-md (.\scripts\ci.baseline.txt)

Probably fixed by #23466 ?

REGRESSION: colmap:x64-windows-static-md failed with BUILD_FAILED. If expected, add colmap:x64-windows-static-md=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.

I don't know exactly what changed. I observe that
* this thing depends on a *lot* of stuff
* on March 14 we didn't even attempt to build this
* the x64-windows ones are already in the baseline

so I skipped it.

REGRESSION: qtdeclarative:x64-windows. If expected, add qtdeclarative:x64-windows=fail to .\scripts\ci.baseline.txt.
REGRESSION: qtdeclarative:x64-windows failed with BUILD_FAILED. If expected, add qtdeclarative:x64-windows=fail to C:\a\1\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: qtdeclarative:x64-windows failed with BUILD_FAILED. If expected, add qtdeclarative:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: qtdeclarative:x64-windows failed with BUILD_FAILED. If expected, add qtdeclarative:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: qtdeclarative:x64-windows failed with BUILD_FAILED. If expected, add qtdeclarative:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: qtdeclarative:x64-windows failed with BUILD_FAILED. If expected, add qtdeclarative:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: qtdeclarative:x64-windows failed with BUILD_FAILED. If expected, add qtdeclarative:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: qtdeclarative:x64-windows failed with BUILD_FAILED. If expected, add qtdeclarative:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.

This is a reporting change: The new world order also includes host build failures which is why it's duplicated.

See also #23714
See also #23490

I'm nervous about baslining this because it seems most of the qt world is built on top of this port

I filed #23824 about this and @Neumann-A indicated this should be fixed by #23755

REGRESSION: nettle:x64-uwp. If expected, add nettle:x64-uwp=fail to .\scripts\ci.baseline.txt.
REGRESSION: nettle:x64-windows-static-md. If expected, add nettle:x64-windows-static-md=fail to .\scripts\ci.baseline.txt.
REGRESSION: nettle:x64-uwp failed with BUILD_FAILED. If expected, add nettle:x64-uwp=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: nettle:x64-windows-static-md failed with POST_BUILD_CHECKS_FAILED. If expected, add nettle:x64-windows-static-md=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.

Didn't analyze, probably fixed by #23519 ?

REGRESSION: libgpg-error:x64-uwp. If expected, add libgpg-error:x64-uwp=fail to .\scripts\ci.baseline.txt.
REGRESSION: libgpg-error:x64-uwp failed with BUILD_FAILED. If expected, add libgpg-error:x64-uwp=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.

This was broken by VS2022 update:
```
C:\Program Files\Microsoft Visual Studio\2022\Enterprise\MSBuild\Microsoft\VisualStudio\v17.0\AppxPackage\Microsoft.AppXPackage.Targets(892,25): error MSB4086: A numeric comparison was attempted on "$(TargetPlatformMinVersion)" that evaluates to "" instead of a number, in condition "'$(TargetPlatformMinVersion)' >= '10.0.17200.0'". [C:\Dev\vcpkg\buildtrees\libgpg-error\x64-uwp-rel\error-1.42-2324ddbc71.clean\SMP\libgpg-error_winrt.vcxproj]
```

REGRESSION: libmikmod:x64-osx. If expected, add libmikmod:x64-osx=fail to .\scripts\ci.baseline.txt.
REGRESSION: libmikmod:x64-osx failed with BUILD_FAILED. If expected, add libmikmod:x64-osx=fail to /Users/vagrant/Data/work/2/s/scripts/azure-pipelines/../ci.baseline.txt.

Broken between [2022-03-16](https://dev.azure.com/vcpkg/public/_build/results?buildId=68947) and [2022-03-18](https://dev.azure.com/vcpkg/public/_build/results?buildId=69051). Unfortunately I don't see obvious reasons why. Nothing else depends on this and nobody has noticed in 2 weeks, so I'm baslining it for now. (Will investigate shortly...)

## Only broken in tool update:

REGRESSION: mesa:x64-windows failed with BUILD_FAILED. If expected, add mesa:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.

```
-- Downloading https://gitlab.freedesktop.org/mesa/mesa/-/archive/mesa-21.2.5/mesa-mesa-21.2.5.tar.gz -> mesa-mesa-mesa-21.2.5-1.tar.gz...
-- Extracting source /Users/vagrant/Data/downloads/mesa-mesa-mesa-21.2.5-1.tar.gz
-- Applying patch swravx512-post-static-link.patch
-- Applying patch swr-msvc-2.patch
-- Applying patch swr-llvm13.patch
-- Applying patch radv-msvc-llvm13-2.patch
-- Applying patch d3d10sw.patch
-- Using source at /Users/vagrant/Data/buildtrees/mesa/src/esa-21.2.5-2df234d2b1.clean
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'mako'
CMake Error at ports/mesa/portfile.cmake:85 (message):
  Python package 'mako' needs to be installed for port 'mesa'.

  Complete list of required python packages: setuptools;mako
Call Stack (most recent call first):
  ports/mesa/portfile.cmake:91 (vcpkg_get_python_package)
  scripts/ports.cmake:145 (include)
```

Looks like this is being tracked by #23089 ; perhaps that we don't have as aggressive a recycling strategy for macos boxes as we do for the others has let different machines give different results?

## Only broken without tool update:

REGRESSION: chromium-base:x64-osx. If expected, add chromium-base:x64-osx=fail to .\scripts\ci.baseline.txt.

This one has been constantly flaky; I baselined it.

REGRESSION: libxml2:x64-osx. If expected, add libxml2:x64-osx=fail to .\scripts\ci.baseline.txt.

This port uses vcpkg_from_git and the upstream server was down during the build.

* Restore chartdir to the baseline, I thought #23732 had been merged.
@wrobelda
Copy link
Contributor

FYI, you seem to have left remove_gmpd.patch behind.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

category:port-update The issue is with a library, which is requesting update new revision info:needs-maintainer-attention Lets the current 'on rotation' vcpkg maintainer know they need to look at this.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants