[vcpkg] Miscellaneous internal improvements extracted from #15424 by ras0219 · Pull Request #15677 · microsoft/vcpkg

ras0219 · 2021-01-15T21:46:18Z

While waiting on #15424, I've extracted a few improvements:

Now that we have end-to-end tests, it's no longer needed to abuse our unit tests to perform a build
Move control of g_debugging in unit tests to an environment variable VCPKG_DEBUG which is enabled in CI. This makes it reasonable to run unit tests as part of every build when developing by cutting out noise (ninja && ./vcpkg-test).
ExportPlanAction::dependencies had a totally unused argument
Added Optional::map() + tests
Include the actual triplet name in the binary abi -- we record this name into the BinaryParagraph and it causes database corruption if we add this to the status database unchanged. There's an open question about whether differently-named triplets with identical contents should hash the same, but with the current implementation they are not substitutable.
Use the new and improved CI plan algorithm. Instead of hacking up a second plan via mutating the control files in memory to add new defaults, the new algorithm directly modifies the first plan to keep the subset of actions actually needed.

…15424

toolsrc/include/vcpkg/base/optional.h

toolsrc/src/vcpkg/remove.cpp

JackBoosY · 2021-01-18T02:44:38Z

LGTM but there is a regression about simage:x64-osx:

[8/42] /Library/Developer/CommandLineTools/usr/bin/cc -DHAVE_CONFIG_H -DSIMAGE_DEBUG=1 -DSIMAGE_INTERNAL -I. -I/Users/vagrant/Data/buildtrees/simage/src/mage-1.8.0-17f988a113.clean/include -I/Users/vagrant/Data/installed/x64-osx/include -fPIC -g -arch x86_64 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -MD -MT CMakeFiles/simage.dir/src/simage_jasper.c.o -MF CMakeFiles/simage.dir/src/simage_jasper.c.o.d -o CMakeFiles/simage.dir/src/simage_jasper.c.o -c /Users/vagrant/Data/buildtrees/simage/src/mage-1.8.0-17f988a113.clean/src/simage_jasper.c
FAILED: CMakeFiles/simage.dir/src/simage_jasper.c.o 
/Library/Developer/CommandLineTools/usr/bin/cc -DHAVE_CONFIG_H -DSIMAGE_DEBUG=1 -DSIMAGE_INTERNAL -I. -I/Users/vagrant/Data/buildtrees/simage/src/mage-1.8.0-17f988a113.clean/include -I/Users/vagrant/Data/installed/x64-osx/include -fPIC -g -arch x86_64 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -MD -MT CMakeFiles/simage.dir/src/simage_jasper.c.o -MF CMakeFiles/simage.dir/src/simage_jasper.c.o.d -o CMakeFiles/simage.dir/src/simage_jasper.c.o -c /Users/vagrant/Data/buildtrees/simage/src/mage-1.8.0-17f988a113.clean/src/simage_jasper.c
/Users/vagrant/Data/buildtrees/simage/src/mage-1.8.0-17f988a113.clean/src/simage_jasper.c:231:14: error: implicitly declaring library function 'malloc' with type 'void *(unsigned long)' [-Werror,-Wimplicit-function-declaration]
    buffer = malloc(width * height * realnumcomp);
             ^
/Users/vagrant/Data/buildtrees/simage/src/mage-1.8.0-17f988a113.clean/src/simage_jasper.c:231:14: note: include the header <stdlib.h> or explicitly provide a declaration for 'malloc'
/Users/vagrant/Data/buildtrees/simage/src/mage-1.8.0-17f988a113.clean/src/simage_jasper.c:278:17: error: implicit declaration of function 'free' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
    if (buffer) free(buffer);
                ^
2 errors generated.

ras0219 · 2021-01-21T18:44:19Z

From the cartographer:x86-windows failure:

-- Found required Ceres dependency: Eigen version 3.3.9 in D:/installed/x86-windows/include/eigen3
CMake Error at D:/installed/x86-windows/share/ceres/CeresConfig.cmake:85 (message):
Failed to find Ceres - Missing requested Ceres components: [SuiteSparse]
(components requested: [SuiteSparse]). Detected Ceres version: 2.0.0
installed in: D:/installed/x86-windows/share with components:
[SchurSpecializations, Multithreading].
Call Stack (most recent call first):
D:/installed/x86-windows/share/ceres/CeresConfig.cmake:271 (ceres_report_not_found)
C:/a/1/s/scripts/buildsystems/vcpkg.cmake:644 (_find_package)
CMakeLists.txt:32 (find_package)

CMake Error at C:/a/1/s/scripts/buildsystems/vcpkg.cmake:644 (_find_package):
Found package configuration file:

D:/installed/x86-windows/share/ceres/CeresConfig.cmake

but it set Ceres_FOUND to FALSE so package "Ceres" is considered to be NOT
FOUND.
Call Stack (most recent call first):
CMakeLists.txt:32 (find_package)

-- Configuring incomplete, errors occurred!
See also "D:/buildtrees/cartographer/x86-windows-dbg/CMakeFiles/CMakeOutput.log".
See also "D:/buildtrees/cartographer/x86-windows-dbg/CMakeFiles/CMakeError.log".

However, ceres[core,cxsparse,lapack,suitesparse]:x86-windows was installed.

ras0219 · 2021-01-21T18:52:41Z

From the gdal:x64-windows failure:

stdout:

link /nologo /dll /INCLUDE:OSRValidate /INCLUDE:OGR_G_GetPointCount /INCLUDE:OGRRegisterAll /INCLUDE:GDALSimpleImageWarp /INCLUDE:GDALReprojectImage /INCLUDE:GDALComputeMedianCutPCT /INCLUDE:GDALDitherRGB2PCT /INCLUDE:OCTNewCoordinateTransformation port*.obj gcore*.obj alg*.obj frmts\o*.obj ogr\ogrsf_frmts\ogrsf_frmts.lib ogr\ogr.lib gnm*.obj gnm\gnm_frmts\o*.obj third_party\o*.obj apps\commonutils.obj apps\gdalinfo_lib.obj apps\gdal_translate_lib.obj apps\gdalwarp_lib.obj apps\ogr2ogr_lib.obj apps\gdaldem_lib.obj apps\nearblack_lib.obj apps\gdal_grid_lib.obj apps\gdal_rasterize_lib.obj apps\gdalbuildvrt_lib.obj apps\gdalmdiminfo_lib.obj apps\gdalmdimtranslate_lib.obj D:/installed/x64-windows/lib/libexpat.lib D:/installed/x64-windows/lib/libpq.lib D:/installed/x64-windows/lib/libpgcommon.lib D:/installed/x64-windows/lib/libpgport.lib Secur32.lib Shell32.lib Advapi32.lib Crypt32.lib Gdi32.lib D:/installed/x64-windows/lib/libcrypto.lib D:/installed/x64-windows/lib/libssl.lib D:/installed/x64-windows/lib/proj.lib D:/installed/x64-windows/lib/sqlite3.lib D:/installed/x64-windows/lib/geos_c.lib D:/installed/x64-windows/lib/libcurl.lib wsock32.lib wldap32.lib winmm.lib legacy_stdio_definitions.lib odbc32.lib odbccp32.lib user32.lib D:/installed/x64-windows/lib/libpng16.lib D:/installed/x64-windows/lib/zlib.lib D:/installed/x64-windows/lib/openjp2.lib D:/installed/x64-windows/lib/webp.lib D:/installed/x64-windows/lib/libxml2.lib D:/installed/x64-windows/lib/iconv.lib D:/installed/x64-windows/lib/charset.lib D:/installed/x64-windows/lib/lzma.lib ws2_32.lib kernel32.lib psapi.lib gcore\Version.res /out:gdal301.dll /implib:gdal_i.lib /debug /opt:ref /opt:icf

stderr:

NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29333\bin\HostX64\x64\link.EXE"' : return code '0xc0000005'
Stop.

Looks like a linker crash. GDAL doesn't appear to add bin/ to the path, so it seems unlikely that the linker is accidentally loading the wrong DLLs or some such.

ras0219 · 2021-01-21T19:14:39Z

Depends on #15802

…roschuma/tool-ports-infra

ras0219 · 2021-01-25T22:38:23Z

Somehow libconfuse:x86-windows failed to find windows.h in debug mode, but it succeeded in release mode? Doesn't repro locally.

ras0219 · 2021-01-26T21:49:06Z

From plplot:x86-windows:

FAILED: bindings/c++/CMakeFiles/plplotcxx.dir/plstream.cc.obj
"C:\PROGRA~2\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29333\bin\Hostx64\x86\cl.exe" /TP -DPLPLOT_HAVE_CONFIG_H -DUSINGDLL -D_CRT_SECURE_NO_DEPRECATE -Dplplotcxx_EXPORTS -ID:\buildtrees\plplot\src\1d5cb5da17-94e7666746.clean\include -I. -Iinclude /nologo /DWIN32 /D_WINDOWS /W3 /utf-8 /GR /EHsc /MP /MD /O2 /Oi /Gy /DNDEBUG /Z7 /showIncludes /Fobindings\c++\CMakeFiles\plplotcxx.dir\plstream.cc.obj /Fdbindings\c++\CMakeFiles\plplotcxx.dir\ /FS -c D:\buildtrees\plplot\src\1d5cb5da17-94e7666746.clean\bindings\c++\plstream.cc
D:\buildtrees\plplot\src\1d5cb5da17-94e7666746.clean\bindings\c++\plstream.cc(102): error C2065: 'cerr': undeclared identifier
D:\buildtrees\plplot\src\1d5cb5da17-94e7666746.clean\bindings\c++\plstream.cc(516): error C2065: 'cerr': undeclared identifier
D:\buildtrees\plplot\src\1d5cb5da17-94e7666746.clean\bindings\c++\plstream.cc(518): error C2065: 'endl': undeclared identifier

And in the config log:

// debug section
-- Check for using namespace support
-- Check for using namespace - found
// release section
-- Check for using namespace support
-- Check for using namespace - not found

So another config variation between release and debug that doesn't repro locally.

…roschuma/tool-ports-infra

ras0219 · 2021-01-29T18:37:09Z

From opencascade:x86-windows:

FAILED: win32/vc14/bind/TKXDEIGES.dll win32/vc14/libd/TKXDEIGES.lib 
cmd.exe /C "cd . && D:\downloads\tools\cmake-3.19.2-windows\cmake-3.19.2-win32-x86\bin\cmake.exe -E vs_link_dll --intdir=src\TKXDEIGES\CMakeFiles\TKXDEIGES.dir --rc="C:\PROGRA~2\Windows Kits\10\bin\10.0.19041.0\x64\rc.exe" --mt="C:\PROGRA~2\Windows Kits\10\bin\10.0.19041.0\x64\mt.exe" --manifests  -- "C:\PROGRA~2\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29333\bin\Hostx64\x86\link.exe"  src\TKXDEIGES\CMakeFiles\TKXDEIGES.dir\__\IGESCAFControl\IGESCAFControl.cxx.obj src\TKXDEIGES\CMakeFiles\TKXDEIGES.dir\__\IGESCAFControl\IGESCAFControl_Reader.cxx.obj src\TKXDEIGES\CMakeFiles\TKXDEIGES.dir\__\IGESCAFControl\IGESCAFControl_Writer.cxx.obj src\TKXDEIGES\CMakeFiles\TKXDEIGES.dir\__\__\resources\TKXDEIGES.rc.res  /out:win32\vc14\bind\TKXDEIGES.dll /implib:win32\vc14\libd\TKXDEIGES.lib /pdb:win32\vc14\bind\TKXDEIGES.pdb /dll /version:7.4 /machine:X86 /nologo    /debug /INCREMENTAL -LIBPATH:D:\installed\x86-windows\debug\lib win32\vc14\libd\TKXCAF.lib  win32\vc14\libd\TKIGES.lib  win32\vc14\libd\TKVCAF.lib  win32\vc14\libd\TKV3d.lib  win32\vc14\libd\TKService.lib  windowscodecs.lib  winmm.lib  opengl32.lib  D:\installed\x86-windows\debug\lib\freetyped.lib  win32\vc14\libd\TKMesh.lib  win32\vc14\libd\TKHLR.lib  win32\vc14\libd\TKCAF.lib  win32\vc14\libd\TKLCAF.lib  win32\vc14\libd\TKCDF.lib  win32\vc14\libd\TKXSBase.lib  win32\vc14\libd\TKBool.lib  win32\vc14\libd\TKBO.lib  win32\vc14\libd\TKShHealing.lib  win32\vc14\libd\TKPrim.lib  win32\vc14\libd\TKTopAlgo.lib  win32\vc14\libd\TKGeomAlgo.lib  win32\vc14\libd\TKBRep.lib  win32\vc14\libd\TKGeomBase.lib  win32\vc14\libd\TKG3d.lib  win32\vc14\libd\TKG2d.lib  win32\vc14\libd\TKMath.lib  win32\vc14\libd\TKernel.lib  advapi32.lib  gdi32.lib  user32.lib  wsock32.lib  psapi.lib  kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib  && cd ."
LINK Pass 1: command "C:\PROGRA~2\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29333\bin\Hostx64\x86\link.exe src\TKXDEIGES\CMakeFiles\TKXDEIGES.dir\__\IGESCAFControl\IGESCAFControl.cxx.obj src\TKXDEIGES\CMakeFiles\TKXDEIGES.dir\__\IGESCAFControl\IGESCAFControl_Reader.cxx.obj src\TKXDEIGES\CMakeFiles\TKXDEIGES.dir\__\IGESCAFControl\IGESCAFControl_Writer.cxx.obj src\TKXDEIGES\CMakeFiles\TKXDEIGES.dir\__\__\resources\TKXDEIGES.rc.res /out:win32\vc14\bind\TKXDEIGES.dll /implib:win32\vc14\libd\TKXDEIGES.lib /pdb:win32\vc14\bind\TKXDEIGES.pdb /dll /version:7.4 /machine:X86 /nologo /debug /INCREMENTAL -LIBPATH:D:\installed\x86-windows\debug\lib win32\vc14\libd\TKXCAF.lib win32\vc14\libd\TKIGES.lib win32\vc14\libd\TKVCAF.lib win32\vc14\libd\TKV3d.lib win32\vc14\libd\TKService.lib windowscodecs.lib winmm.lib opengl32.lib D:\installed\x86-windows\debug\lib\freetyped.lib win32\vc14\libd\TKMesh.lib win32\vc14\libd\TKHLR.lib win32\vc14\libd\TKCAF.lib win32\vc14\libd\TKLCAF.lib win32\vc14\libd\TKCDF.lib win32\vc14\libd\TKXSBase.lib win32\vc14\libd\TKBool.lib win32\vc14\libd\TKBO.lib win32\vc14\libd\TKShHealing.lib win32\vc14\libd\TKPrim.lib win32\vc14\libd\TKTopAlgo.lib win32\vc14\libd\TKGeomAlgo.lib win32\vc14\libd\TKBRep.lib win32\vc14\libd\TKGeomBase.lib win32\vc14\libd\TKG3d.lib win32\vc14\libd\TKG2d.lib win32\vc14\libd\TKMath.lib win32\vc14\libd\TKernel.lib advapi32.lib gdi32.lib user32.lib wsock32.lib psapi.lib kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib /MANIFEST /MANIFESTFILE:src\TKXDEIGES\CMakeFiles\TKXDEIGES.dir/intermediate.manifest src\TKXDEIGES\CMakeFiles\TKXDEIGES.dir/manifest.res" failed (exit code 1201) with the following output:
LINK : fatal error LNK1201: error writing to program database 'D:\buildtrees\opencascade\x86-windows-dbg\win32\vc14\bind\TKXDEIGES.pdb'; check for insufficient disk space, invalid path, or insufficient privilege
ninja: build stopped: subcommand failed.

…roschuma/tool-ports-infra

ras0219 · 2021-01-30T01:59:51Z

From armadillo:x64-uwp:

Downloading https://gitlab.com/conradsnicta/armadillo-code/-/archive/24b4762cbfbd3ad14c99a4854acd3560559a3195/armadillo-code-24b4762cbfbd3ad14c99a4854acd3560559a3195.tar.gz... Failed. Status: 22;"HTTP response code said error"

I have a fix prepared in https://github.com/microsoft/vcpkg/pull/15808/files#diff-20fb8471fdc2c697e520789ae25d430898df4b1a99ea13f97d571d22e12d4732 which changes armadillo to use vcpkg_from_git(). Another possible solution would be to reference a release tag; gitlab might be happier to serve that up.

…15424 (microsoft#15677) * [vcpkg] Miscellaneous internal improvements extracted from microsoft#15424 * [vcpkg] CR comments * [armadillo] Use vcpkg_from_git() to workaround gitlab missing archive Co-authored-by: Robert Schumacher <roschuma@microsoft.com> Co-authored-by: Billy Robert O'Neal III <bion@microsoft.com>

[vcpkg] Miscellaneous internal improvements extracted from microsoft#…

a6e59a3

…15424

BillyONeal approved these changes Jan 15, 2021

View reviewed changes

toolsrc/include/vcpkg/base/optional.h Outdated Show resolved Hide resolved

toolsrc/include/vcpkg/base/optional.h Show resolved Hide resolved

toolsrc/src/vcpkg/remove.cpp Outdated Show resolved Hide resolved

[vcpkg] CR comments

b997e2c

BillyONeal approved these changes Jan 15, 2021

View reviewed changes

strega-nil approved these changes Jan 16, 2021

View reviewed changes

JackBoosY self-assigned this Jan 18, 2021

JackBoosY added category:vcpkg-feature The issue is a new capability of the tool that doesn’t already exist and we haven’t committed info:internal labels Jan 18, 2021

JackBoosY added the requires:author-response label Jan 18, 2021

ras0219-msft and others added 2 commits January 19, 2021 11:49

Merge branch 'master' of https://github.com/microsoft/vcpkg into HEAD

83ad00e

Merge remote-tracking branch 'origin/master' into HEAD

664f34d

ras0219 mentioned this pull request Jan 21, 2021

[ceres] Mark all dependencies REQUIRED #15802

Merged

JackBoosY added the depends:different-pr This PR or Issue depends on a PR which has been filed label Jan 22, 2021

Merge branch 'master' of https://github.com/Microsoft/vcpkg into dev/…

2515127

…roschuma/tool-ports-infra

JackBoosY removed the depends:different-pr This PR or Issue depends on a PR which has been filed label Jan 25, 2021

Merge branch 'master' of https://github.com/Microsoft/vcpkg into dev/…

79bacee

…roschuma/tool-ports-infra

JackBoosY removed the requires:author-response label Jan 26, 2021

Merge branch 'master' of https://github.com/Microsoft/vcpkg into dev/…

3120a7e

…roschuma/tool-ports-infra

Merge branch 'master' of https://github.com/Microsoft/vcpkg into dev/…

9bc9b17

…roschuma/tool-ports-infra

[armadillo] Use vcpkg_from_git() to workaround gitlab missing archive

a6cdaa4

ras0219 force-pushed the dev/roschuma/tool-ports-infra branch from 7d53e10 to a6cdaa4 Compare January 30, 2021 10:43

ras0219-msft merged commit 5793c4b into microsoft:master Feb 2, 2021

ras0219-msft mentioned this pull request Feb 2, 2021

[vcpkg baseline][armadillo] Update to 10.2.0 and also update the repository #15973

Merged

Be-ing added a commit to Be-ing/mixxx that referenced this pull request Feb 8, 2021

revert microsoft/vcpkg#15677

c8a1718

Be-ing mentioned this pull request Feb 8, 2021

application built with libebur128 1.2.5 on x64 Windows crashing with exit code 0xc0000135 #16078

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[vcpkg] Miscellaneous internal improvements extracted from #15424#15677

[vcpkg] Miscellaneous internal improvements extracted from #15424#15677
ras0219-msft merged 9 commits intomicrosoft:masterfrom
ras0219:dev/roschuma/tool-ports-infra

ras0219 commented Jan 15, 2021

Uh oh!

Uh oh!

Uh oh!

Uh oh!

JackBoosY commented Jan 18, 2021

Uh oh!

ras0219 commented Jan 21, 2021

Uh oh!

ras0219 commented Jan 21, 2021

Uh oh!

ras0219 commented Jan 21, 2021

Uh oh!

ras0219 commented Jan 25, 2021 •

edited

Loading

Uh oh!

ras0219 commented Jan 26, 2021 •

edited

Loading

Uh oh!

ras0219 commented Jan 29, 2021

Uh oh!

ras0219 commented Jan 30, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

ras0219 commented Jan 15, 2021

Uh oh!

Uh oh!

Uh oh!

Uh oh!

JackBoosY commented Jan 18, 2021

Uh oh!

ras0219 commented Jan 21, 2021

Uh oh!

ras0219 commented Jan 21, 2021

Uh oh!

ras0219 commented Jan 21, 2021

Uh oh!

ras0219 commented Jan 25, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ras0219 commented Jan 26, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ras0219 commented Jan 29, 2021

Uh oh!

ras0219 commented Jan 30, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

ras0219 commented Jan 25, 2021 •

edited

Loading

ras0219 commented Jan 26, 2021 •

edited

Loading