Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[wasm] Clean emsdk setup #106403

Draft
wants to merge 40 commits into
base: main
Choose a base branch
from
+93 −127
Draft

[wasm] Clean emsdk setup #106403

Show file tree
Hide file tree
Changes from 30 commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
e064ec6
Remove embedded scripts, fix build on Linux.
ilonatommy Aug 14, 2024
7bc1637
Merge branch 'main' into fix-105439-changing-runtime-only - fix froze…
ilonatommy Aug 14, 2024
60bfafc
Windows - almost fixed, missing native build access to EMSDK_PATH.
ilonatommy Aug 16, 2024
07ce65d
Fix `$(EMSDK_PATH)` is missing is missing some paths.
ilonatommy Aug 16, 2024
5636d09
Fix wasi.
ilonatommy Aug 16, 2024
ea2d900
Merge branch 'main' into fix-105439-changing-runtime-only
ilonatommy Aug 16, 2024
4a3e122
Cleanup - move the pre-env script logic to emscripten provisioning.
ilonatommy Aug 16, 2024
bf2c1bb
Move remaining emsdk-provision bits to provision target.
ilonatommy Aug 19, 2024
97a5513
Fix: it's an Item, not a property.
ilonatommy Aug 19, 2024
c9852a5
More cleanup on Linux.
ilonatommy Aug 19, 2024
5f96265
Fix rebuild.
ilonatommy Aug 19, 2024
0335335
Fix "NODE_JS is set to empty value in D:\a\_work\1\s\src\mono\browser…
ilonatommy Aug 19, 2024
89148e8
Fix "BINARYEN_ROOT is set to empty value in \src\mono\browser\emsdk\e…
ilonatommy Aug 19, 2024
bd9179b
Workaround - set `EMSKD_PATH` for native build separately.
ilonatommy Aug 19, 2024
7da3a6c
Cleanup.
ilonatommy Aug 19, 2024
1a94c15
Fix "Specified Emscripten sdk at $(EMSDK_PATH) is missing some paths:…
ilonatommy Aug 19, 2024
d8b8a7c
Fix: path to python "is not recognized as an internal or external com…
ilonatommy Aug 19, 2024
6246fed
Missing change for the fix in the previous commit.
ilonatommy Aug 19, 2024
6fd5e6f
Fix wbt: add missing env vars
ilonatommy Aug 20, 2024
fbb71be
Missing "bin" part of node path.
ilonatommy Aug 21, 2024
caf5379
Lib tests have "bin" in the end of node path but wbt do not, satisfy …
ilonatommy Aug 21, 2024
2a34403
Wrong condition, c&p error.
ilonatommy Aug 21, 2024
fbd9744
Fix WBT: "binaryen executable not found (%HostBinaryenDir%\bin\wasm-o…
ilonatommy Aug 21, 2024
8ed999c
"EndsWith" did not detect the need to append "bin" correctly.
ilonatommy Aug 21, 2024
383e5cd
Revert redundant changes to user-app targets.
ilonatommy Aug 22, 2024
2dd0adb
Missing change revert
ilonatommy Aug 22, 2024
65e574e
Feedback.
ilonatommy Aug 22, 2024
7d70eb6
Merge branch 'main' into fix-105439-changing-runtime-only
ilonatommy Aug 23, 2024
1b7befc
Import after `SetupPackagePaths` finishes.
ilonatommy Aug 23, 2024
5830355
Fix
ilonatommy Aug 23, 2024
fb1738b
Remove redundant comments.
ilonatommy Aug 27, 2024
5f8908b
`WasmApp.InTree.props` are loaded too late. Move the props back to de…
ilonatommy Aug 27, 2024
8c9a3f0
Fix missing slash in the end of the path.
ilonatommy Aug 27, 2024
b14d7a9
Remove whitespace change.
ilonatommy Aug 28, 2024
755c343
Merge branch 'main' into fix-105439-changing-runtime-only
ilonatommy Aug 28, 2024
6daf79e
Make sure properties to build nuget paths on helix are not empty.
ilonatommy Sep 3, 2024
08f86f3
Debugging commit, see where are packages.
ilonatommy Sep 3, 2024
4744846
More debugging.
ilonatommy Sep 3, 2024
574548b
Fallback for helix.
ilonatommy Sep 5, 2024
f9acf9f
Cleanup needed when we have the fallback.
ilonatommy Sep 5, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion eng/native/gen-buildsys.cmd
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,7 @@ if not "%__ConfigureOnly%" == "1" (
)

if /i "%__UseEmcmake%" == "1" (
call "!EMSDK_PATH!/emsdk_env.cmd" > nul 2>&1 && emcmake "%CMakePath%" %__ExtraCmakeParams% --no-warn-unused-cli -G "%__CmakeGenerator%" -B %__IntermediatesDir% -S %__SourceDir%
call "!EMSDK_PATH!/pre_emsdk_env.cmd" > nul 2>&1 && call "!EMSDK_PATH!/emsdk_env.cmd" > nul 2>&1 && emcmake "%CMakePath%" %__ExtraCmakeParams% --no-warn-unused-cli -G "%__CmakeGenerator%" -B %__IntermediatesDir% -S %__SourceDir%
) else (
"%CMakePath%" %__ExtraCmakeParams% --no-warn-unused-cli -G "%__CmakeGenerator%" -B %__IntermediatesDir% -S %__SourceDir%
)
Expand Down
8 changes: 4 additions & 4 deletions src/mono/browser/browser.proj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -415,13 +415,13 @@
<CMakeBuildRuntimeConfigureCmd Condition="'$(WasmEnableJsInteropByValue)' == 'true'">$(CMakeBuildRuntimeConfigureCmd) -DENABLE_JS_INTEROP_BY_VALUE=1</CMakeBuildRuntimeConfigureCmd>
<CMakeBuildRuntimeConfigureCmd>$(CMakeBuildRuntimeConfigureCmd) $(CMakeConfigurationEmsdkPath)</CMakeBuildRuntimeConfigureCmd>

<CMakeBuildRuntimeConfigureCmd Condition="'$(OS)' == 'Windows_NT'">call &quot;$(RepositoryEngineeringDir)native\init-vs-env.cmd&quot; &amp;&amp; call &quot;$([MSBuild]::NormalizePath('$(EMSDK_PATH)', 'emsdk_env.cmd'))&quot; &amp;&amp; $(CMakeBuildRuntimeConfigureCmd)</CMakeBuildRuntimeConfigureCmd>
<CMakeBuildRuntimeConfigureCmd Condition="'$(OS)' != 'Windows_NT'">bash -c 'source $(EMSDK_PATH)/emsdk_env.sh 2>&amp;1 &amp;&amp; $(CMakeBuildRuntimeConfigureCmd)'</CMakeBuildRuntimeConfigureCmd>
<CMakeBuildRuntimeConfigureCmd Condition="'$(OS)' == 'Windows_NT'">call &quot;$(RepositoryEngineeringDir)native\init-vs-env.cmd&quot; &amp;&amp; call &quot;$([MSBuild]::NormalizePath('$(EMSDK_PATH)', 'pre_emsdk_env.cmd'))&quot; &amp;&amp; call &quot;$([MSBuild]::NormalizePath('$(EMSDK_PATH)', 'emsdk_env.cmd'))&quot; &amp;&amp; $(CMakeBuildRuntimeConfigureCmd)</CMakeBuildRuntimeConfigureCmd>
<CMakeBuildRuntimeConfigureCmd Condition="'$(OS)' != 'Windows_NT'">bash -c 'source $(EMSDK_PATH)/pre_emsdk_env.sh &amp;&amp; source $(EMSDK_PATH)/emsdk_env.sh 2>&amp;1 &amp;&amp; $(CMakeBuildRuntimeConfigureCmd)'</CMakeBuildRuntimeConfigureCmd>

<CMakeOptions Condition="'$(MonoVerboseBuild)' != ''">-v</CMakeOptions>
<CMakeBuildRuntimeCmd>cmake --build . --config $(Configuration) $(CmakeOptions)</CMakeBuildRuntimeCmd>
<CMakeBuildRuntimeCmd Condition="'$(OS)' == 'Windows_NT'">call &quot;$(RepositoryEngineeringDir)native\init-vs-env.cmd&quot; &amp;&amp; call &quot;$([MSBuild]::NormalizePath('$(EMSDK_PATH)', 'emsdk_env.cmd'))&quot; &amp;&amp; $(CMakeBuildRuntimeCmd)</CMakeBuildRuntimeCmd>
<CMakeBuildRuntimeCmd Condition="'$(OS)' != 'Windows_NT'">bash -c 'source $(EMSDK_PATH)/emsdk_env.sh 2>&amp;1 &amp;&amp; $(CMakeBuildRuntimeCmd)'</CMakeBuildRuntimeCmd>
<CMakeBuildRuntimeCmd Condition="'$(OS)' == 'Windows_NT'">call &quot;$(RepositoryEngineeringDir)native\init-vs-env.cmd&quot; &amp;&amp; call &quot;$([MSBuild]::NormalizePath('$(EMSDK_PATH)', 'pre_emsdk_env.cmd'))&quot; &amp;&amp; call &quot;$([MSBuild]::NormalizePath('$(EMSDK_PATH)', 'emsdk_env.cmd'))&quot; &amp;&amp; call &quot;$([MSBuild]::NormalizePath('$(EMSDK_PATH)', 'emsdk_env.cmd'))&quot; &amp;&amp; $(CMakeBuildRuntimeCmd)</CMakeBuildRuntimeCmd>
<CMakeBuildRuntimeCmd Condition="'$(OS)' != 'Windows_NT'">bash -c 'source $(EMSDK_PATH)/pre_emsdk_env.sh &amp;&amp; source $(EMSDK_PATH)/emsdk_env.sh 2>&amp;1 &amp;&amp; $(CMakeBuildRuntimeCmd)'</CMakeBuildRuntimeCmd>
</PropertyGroup>
<ItemGroup>
<_CmakeEnvironmentVariable Include="ENABLE_JS_INTEROP_BY_VALUE=1" Condition="'$(WasmEnableJsInteropByValue)' != 'false'"/>
Expand Down
8 changes: 7 additions & 1 deletion src/mono/browser/build/BrowserWasmApp.targets
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -195,13 +195,17 @@
<EmscriptenSdkToolsPath Condition="'$(EmscriptenSdkToolsPath)' != ''" >$([MSBuild]::NormalizeDirectory($(EmscriptenSdkToolsPath)))</EmscriptenSdkToolsPath>
<EmscriptenNodeToolsPath Condition="'$(EmscriptenNodeToolsPath)' != ''" >$([MSBuild]::NormalizeDirectory($(EmscriptenNodeToolsPath)))</EmscriptenNodeToolsPath>
<EmscriptenUpstreamBinPath Condition="'$(EmscriptenUpstreamBinPath)' != ''">$([MSBuild]::NormalizeDirectory($(EmscriptenUpstreamBinPath)))</EmscriptenUpstreamBinPath>
<_EmscriptenNodePath>$([MSBuild]::NormalizePath($(EmscriptenNodeToolsPath), 'bin', 'node$(_ExeExt)'))</_EmscriptenNodePath>
</PropertyGroup>

<!-- Environment variables required for running emsdk commands like `emcc` -->
<ItemGroup Condition="'$(EmscriptenSdkToolsPath)' != ''">
<EmscriptenEnvVars Include="DOTNET_EMSCRIPTEN_LLVM_ROOT=$(EmscriptenSdkToolsPath)bin" />
<EmscriptenEnvVars Include="EM_LLVM_ROOT=$(EmscriptenSdkToolsPath)bin" />
<EmscriptenEnvVars Include="DOTNET_EMSCRIPTEN_BINARYEN_ROOT=$(EmscriptenSdkToolsPath)" />
<EmscriptenEnvVars Include="DOTNET_EMSCRIPTEN_NODE_JS=$([MSBuild]::NormalizePath($(EmscriptenNodeToolsPath), 'bin', 'node$(_ExeExt)'))" />
<EmscriptenEnvVars Condition="'$(OS)' == 'Windows_NT'" Include="EM_BINARYEN_ROOT=$(EmscriptenSdkToolsPath)" />
<EmscriptenEnvVars Include="DOTNET_EMSCRIPTEN_NODE_JS=$(_EmscriptenNodePath)" />
<EmscriptenEnvVars Include="EM_NODE_JS=$(_EmscriptenNodePath)" />
</ItemGroup>

<ItemGroup>
Expand Down Expand Up @@ -285,6 +289,7 @@
<WasmAllowUndefinedSymbols Condition="'$(WasmAllowUndefinedSymbols)' == ''">false</WasmAllowUndefinedSymbols>

<WasmClang>$(EmscriptenUpstreamEmscriptenPath)emcc</WasmClang>
<!-- EmscriptenUpstreamEmscriptenPath can stop working here -->
</PropertyGroup>

<ItemGroup>
Expand Down Expand Up @@ -356,6 +361,7 @@

<EmscriptenEnvVars Include="EMSDK_PYTHON=$(EmscriptenPythonToolsPath)python.exe" Condition="'$(OS)' == 'Windows_NT'" />
<EmscriptenEnvVars Include="PYTHONPATH=$(EmscriptenPythonToolsPath)" Condition="'$(OS)' == 'Windows_NT'" />
<!-- EmscriptenPythonToolsPath can stop working here -->
<EmscriptenEnvVars Include="PYTHONHOME=" Condition="'$(OS)' == 'Windows_NT'" />
<EmscriptenEnvVars Include="EM_CACHE=$(WasmCachePath)" Condition="'$(WasmCachePath)' != ''" />
<EmscriptenEnvVars Include="EM_FROZEN_CACHE=1" Condition="'$(WasmCachePath)' == '$(EmscriptenCacheSdkCacheDir)'" />
Expand Down Expand Up @@ -510,7 +516,7 @@
</Target>

<Target Name="_CheckEmccIsExpectedVersion" BeforeTargets="_CheckToolchainIsExpectedVersion" Condition="'$(_WasmStrictVersionMatch)' == 'true'">
<Error Condition="'$(_IsToolchainMissing)' == 'true'"

Check failure on line 519 in src/mono/browser/build/BrowserWasmApp.targets

View check run for this annotation

Azure Pipelines / runtime (Build browser-wasm linux Release LibraryTests_Threading)

src/mono/browser/build/BrowserWasmApp.targets#L519 

src/mono/browser/build/BrowserWasmApp.targets(519,5): error : (NETCORE_ENGINEERING_TELEMETRY=Build) Specified Emscripten sdk at $(EMSDK_PATH)=/__w/1/s/src/mono/browser/emsdk/ is missing some paths: $(EmscriptenNodeToolsPath)=/__w/1/s/src/mono/browser/emsdk/node/bin/ . Emscripten SDK is required for building native files.

Check failure on line 519 in src/mono/browser/build/BrowserWasmApp.targets

View check run for this annotation

Azure Pipelines / runtime (Build browser-wasm linux Release LibraryTests)

src/mono/browser/build/BrowserWasmApp.targets#L519 

src/mono/browser/build/BrowserWasmApp.targets(519,5): error : (NETCORE_ENGINEERING_TELEMETRY=Build) Specified Emscripten sdk at $(EMSDK_PATH)=/__w/1/s/src/mono/browser/emsdk/ is missing some paths: $(EmscriptenNodeToolsPath)=/__w/1/s/src/mono/browser/emsdk/node/bin/ . Emscripten SDK is required for building native files.

Check failure on line 519 in src/mono/browser/build/BrowserWasmApp.targets

View check run for this annotation

Azure Pipelines / runtime

src/mono/browser/build/BrowserWasmApp.targets#L519 

src/mono/browser/build/BrowserWasmApp.targets(519,5): error : (NETCORE_ENGINEERING_TELEMETRY=Build) Specified Emscripten sdk at $(EMSDK_PATH)=/__w/1/s/src/mono/browser/emsdk/ is missing some paths: $(EmscriptenNodeToolsPath)=/__w/1/s/src/mono/browser/emsdk/node/bin/ . Emscripten SDK is required for building native files.

Check failure on line 519 in src/mono/browser/build/BrowserWasmApp.targets

View check run for this annotation

Azure Pipelines / runtime

src/mono/browser/build/BrowserWasmApp.targets#L519 

src/mono/browser/build/BrowserWasmApp.targets(519,5): error : (NETCORE_ENGINEERING_TELEMETRY=Build) Specified Emscripten sdk at $(EMSDK_PATH)=/__w/1/s/src/mono/browser/emsdk/ is missing some paths: $(EmscriptenNodeToolsPath)=/__w/1/s/src/mono/browser/emsdk/node/bin/ . Emscripten SDK is required for building native files.
Text="$(_ToolchainMissingErrorMessage) Emscripten SDK is required for building native files." />
<Error Condition="'$(RuntimeEmccVersionRaw)' == ''"
Text="%24(RuntimeEmccVersionRaw) is not set. '$(_EmccPropsPath)' should have set that."/>
Expand Down
2 changes: 2 additions & 0 deletions src/mono/browser/build/WasmApp.InTree.props
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@
<EnableTargetingPackDownload>false</EnableTargetingPackDownload>
<TargetFramework>$(NetCoreAppCurrent)</TargetFramework>
<EMSDK_PATH Condition="'$(EMSDK_PATH)' == '' and '$(MonoProjectRoot)' != ''">$([MSBuild]::NormalizeDirectory($(MonoProjectRoot), 'browser', 'emsdk'))</EMSDK_PATH>
<EmscriptenNodeToolsPath>$([MSBuild]::NormalizeDirectory( $([System.IO.Path]::Combine($(NuGetPackageRoot), 'runtime.$(_portableHostOS)-$(BuildArchitecture).microsoft.netcore.runtime.wasm.node.transport', '$(NodePackageVersion)', 'tools', '$(_portableHostOS)-$(BuildArchitecture)')) ))</EmscriptenNodeToolsPath>
ilonatommy marked this conversation as resolved.
Show resolved Hide resolved
<EmscriptenPythonToolsPath>$([MSBuild]::NormalizeDirectory( $([System.IO.Path]::Combine($(NuGetPackageRoot), 'Microsoft.NET.Runtime.Emscripten.$(EmsdkVersion).Python.win-$(BuildArchitecture)', '$(EmsdkPackageVersion)', 'tools')) ))</EmscriptenPythonToolsPath>
<RunAOTCompilation Condition="'$(RunAOTCompilation)' == ''">false</RunAOTCompilation>
<PublishTrimmed>true</PublishTrimmed>
<RunAnalyzers>false</RunAnalyzers>
Expand Down
1 change: 1 addition & 0 deletions src/mono/browser/build/WasmApp.InTree.targets
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,4 +51,5 @@
<ZipDirectory SourceDirectory="$(WasmAppDir)" DestinationFile="$(ZippedApp)" />
<Copy SourceFiles="$(ZippedApp)" DestinationFolder="$(HelixArchiveRunOnlyAppsDir)" />
</Target>

</Project>
Loading
Loading