Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Xamarin.Android.Build.Tasks] fix Inputs for _Generate*Java* targets #9174

Merged
merged 1 commit into from
Aug 6, 2024

Conversation

jonathanpeppers
Copy link
Member

Fixes: #8967
Context: #9001

You can cause a build error in .NET 8 by doing:

  • Start an x86 or x86_64 emulator
  • dotnet build -t:Run
  • Close the emulator, attach an arm or arm64 device
  • dotnet build -t:Run

Emits the error:

C:\Program Files\dotnet\packs\Microsoft.Android.Sdk.Windows\34.0.113\tools\Xamarin.Android.Common.targets(2063,3): error XA3006: Could not compile native assembly file: typemaps.x86_64.ll
stderr | C:\Program Files\dotnet\packs\Microsoft.Android.Sdk.Windows\34.0.113\tools\binutils\bin\llc.exe: error: C:\Program Files\dotnet\packs\Microsoft.Android.Sdk.Windows\34.0.113\tools\binutils\bin\llc.exe: typemaps.x86_64.ll: error: Could not open input file: no such file or directory

This works fine in Visual Studio, but not at the command-line.

The underlying problem is due to _GeneratePackageManagerJava target running, while the _GenerateJavaStubs target did not:

Skipping target "_GenerateJavaStubs" because all output files are up-to-date with respect to the input files.
...
Input file "obj\Debug\net8.0-android\resolvedassemblies.hash" is newer than output file "obj\Debug\net8.0-android\stamp\_GeneratePackageManagerJava.stamp".

These two targets should almost always run together, so we should ensure that they do.

This problem also doesn't happen .NET 9, as both targets are invalidated for a different reason:

Target Name=_LinkAssembliesNoShrink Project=UnnamedProject.csproj
Building target "_LinkAssembliesNoShrink" completely.
Output file "obj\Debug\android\assets\x86_64\UnnamedProject.dll" does not exist.

Since, we build per-RID in .NET 9, obj\Debug\android\assets\x86_64\UnnamedProject.dll has an x86_64 in the path, which is not present in the .NET 8 build.

Reviewing the two sets of Inputs:

<Target Name="_GenerateJavaStubs"
    ...
    Inputs="@(_AndroidMSBuildAllProjects);@(_ResolvedUserMonoAndroidAssemblies);$(_AndroidManifestAbs);$(_AndroidBuildPropertiesCache);@(AndroidEnvironment);@(LibraryEnvironments)"
...
<Target Name="_GeneratePackageManagerJava"
    ...
    Inputs="@(_AndroidMSBuildAllProjects);$(_ResolvedUserAssembliesHashFile);$(MSBuildProjectFile);$(_AndroidBuildPropertiesCache);@(AndroidEnvironment);@(LibraryEnvironments)"

Let's align them more closely by:

  • Track assembly changes in the exact same way:
$(_ResolvedUserAssembliesHashFile);@(_ResolvedUserMonoAndroidAssemblies);

This way if an assembly is added, removed, or timestamp changed: both targets are in sync.

  • Don't track $(MSBuildProjectFile), this should already be in @(_AndroidMSBuildAllProjects).

With this change in place manually, I'm not able to reproduce the problem any longer.

PR #9001 may also have fixed this issue, but it could cause targets to run on every incremental build -- a performance issue.

Fixes: dotnet#8967
Context: dotnet#9001

You can cause a build error in .NET 8 by doing:

* Start an x86 or x86_64 emulator
* `dotnet build -t:Run`
* Close the emulator, attach an arm or arm64 device
* `dotnet build -t:Run`

Emits the error:

    C:\Program Files\dotnet\packs\Microsoft.Android.Sdk.Windows\34.0.113\tools\Xamarin.Android.Common.targets(2063,3): error XA3006: Could not compile native assembly file: typemaps.x86_64.ll
    stderr | C:\Program Files\dotnet\packs\Microsoft.Android.Sdk.Windows\34.0.113\tools\binutils\bin\llc.exe: error: C:\Program Files\dotnet\packs\Microsoft.Android.Sdk.Windows\34.0.113\tools\binutils\bin\llc.exe: typemaps.x86_64.ll: error: Could not open input file: no such file or directory

This works fine in Visual Studio, but not at the command-line.

The underlying problem is due to `_GeneratePackageManagerJava` target
running, while the `_GenerateJavaStubs` target *did not*:

    Skipping target "_GenerateJavaStubs" because all output files are up-to-date with respect to the input files.
    ...
    Input file "obj\Debug\net8.0-android\resolvedassemblies.hash" is newer than output file "obj\Debug\net8.0-android\stamp\_GeneratePackageManagerJava.stamp".

These two targets should almost always run together, so we should
ensure that they do.

This problem also doesn't happen .NET 9, as both targets are
invalidated for a different reason:

    Target Name=_LinkAssembliesNoShrink Project=UnnamedProject.csproj
    Building target "_LinkAssembliesNoShrink" completely.
    Output file "obj\Debug\android\assets\x86_64\UnnamedProject.dll" does not exist.

Since, we build per-RID in .NET 9, `obj\Debug\android\assets\x86_64\UnnamedProject.dll`
has an `x86_64` in the path, which is not present in the .NET 8 build.

Reviewing the two sets of `Inputs`:

    <Target Name="_GenerateJavaStubs"
        ...
        Inputs="@(_AndroidMSBuildAllProjects);@(_ResolvedUserMonoAndroidAssemblies);$(_AndroidManifestAbs);$(_AndroidBuildPropertiesCache);@(AndroidEnvironment);@(LibraryEnvironments)"
    ...
    <Target Name="_GeneratePackageManagerJava"
        ...
        Inputs="@(_AndroidMSBuildAllProjects);$(_ResolvedUserAssembliesHashFile);$(MSBuildProjectFile);$(_AndroidBuildPropertiesCache);@(AndroidEnvironment);@(LibraryEnvironments)"

Let's align them more closely by:

* Track assembly changes in the exact same way:

    $(_ResolvedUserAssembliesHashFile);@(_ResolvedUserMonoAndroidAssemblies);

This way if an assembly is added, removed, or timestamp changed: both
targets are in sync.

* Don't track `$(MSBuildProjectFile)`, this should already be in
  `@(_AndroidMSBuildAllProjects)`.

With this change in place manually, I'm not able to reproduce the
problem any longer.

PR dotnet#9001 may also have fixed this issue, but it could cause targets to
run on every incremental build -- a performance issue.
@jonathanpeppers
Copy link
Member Author

The APK tests are hopefully fixed in 7451f5b, going forward.

@jonathanpeppers jonathanpeppers merged commit 175f3a0 into dotnet:main Aug 6, 2024
55 of 57 checks passed
@jonathanpeppers jonathanpeppers deleted the FixIncrementalBuild branch August 6, 2024 17:57
jonathanpeppers added a commit that referenced this pull request Aug 6, 2024
…ets (#9174)

Fixes: #8967
Context: #9001

You can cause a build error in .NET 8 by doing:

* Start an x86 or x86_64 emulator
* `dotnet build -t:Run`
* Close the emulator, attach an arm or arm64 device
* `dotnet build -t:Run`

Emits the error:

    Xamarin.Android.Common.targets(2063,3): error XA3006: Could not compile native assembly file: typemaps.x86_64.ll
    stderr | C:\Program Files\dotnet\packs\Microsoft.Android.Sdk.Windows\34.0.113\tools\binutils\bin\llc.exe: error: C:\Program Files\dotnet\packs\Microsoft.Android.Sdk.Windows\34.0.113\tools\binutils\bin\llc.exe: typemaps.x86_64.ll: error: Could not open input file: no such file or directory

This works fine in Visual Studio, but not at the command-line.

The underlying problem is due to `_GeneratePackageManagerJava` target
running, while the `_GenerateJavaStubs` target *did not*:

    Skipping target "_GenerateJavaStubs" because all output files are up-to-date with respect to the input files.
    ...
    Input file "obj\Debug\net8.0-android\resolvedassemblies.hash" is newer than output file "obj\Debug\net8.0-android\stamp\_GeneratePackageManagerJava.stamp".

These two targets should almost always run together, so we should
ensure that they do.

This problem also doesn't happen .NET 9, as both targets are
invalidated for a different reason:

    Target Name=_LinkAssembliesNoShrink Project=UnnamedProject.csproj
    Building target "_LinkAssembliesNoShrink" completely.
    Output file "obj\Debug\android\assets\x86_64\UnnamedProject.dll" does not exist.

Since, we build per-RID in .NET 9, `obj\Debug\android\assets\x86_64\UnnamedProject.dll`
has an `x86_64` in the path, which is not present in the .NET 8 build.

Reviewing the two sets of `Inputs`:

    <Target Name="_GenerateJavaStubs"
        ...
        Inputs="@(_AndroidMSBuildAllProjects);@(_ResolvedUserMonoAndroidAssemblies);$(_AndroidManifestAbs);$(_AndroidBuildPropertiesCache);@(AndroidEnvironment);@(LibraryEnvironments)"
    ...
    <Target Name="_GeneratePackageManagerJava"
        ...
        Inputs="@(_AndroidMSBuildAllProjects);$(_ResolvedUserAssembliesHashFile);$(MSBuildProjectFile);$(_AndroidBuildPropertiesCache);@(AndroidEnvironment);@(LibraryEnvironments)"

Let's align them more closely by:

* Track assembly changes in the exact same way:

    $(_ResolvedUserAssembliesHashFile);@(_ResolvedUserMonoAndroidAssemblies);

This way if an assembly is added, removed, or timestamp changed: both
targets are in sync.

* Don't track `$(MSBuildProjectFile)`, this should already be in
  `@(_AndroidMSBuildAllProjects)`.

With this change in place manually, I'm not able to reproduce the
problem any longer.

PR #9001 may also have fixed this issue, but it could cause targets to
run on every incremental build -- a performance issue.
jonathanpeppers added a commit that referenced this pull request Aug 22, 2024
Changes: 34.0.113...b0fd011

.NET 8 changelog:

* [Mono.Android] AndroidMessageHandler should follow HTTP-308 redirects (#8951)
* [ci] Fix android source path for MAUI test job (#9030)
* [ci] Update checkout path for nightly build (#9028)
* [Mono.Android-Tests] Fix repo URL in redirect tests (#9035)
* [Xamarin.Android.Build.Tasks] Support VS "Build Acceleration" (#9042)
* [xaprepare] Always use release mono bundle (#9106)
* [ci] Update sdk-insertions trigger to manual only (#9029)
* Bump to dotnet/runtime@4a37e7305c 8.0.8 (#9033)
* Bump to dotnet/installer@b638a84fba 8.0.205-servicing.24212.27 (#9032)
* [ci] Disable CodeQL on macOS, Linux, non-main jobs (#9111)
* [ci] Use DotNetCoreCLI to sign macOS files (#9102)
* [tests] fix `InvalidTargetPlatformVersion` for `net8.0` (#9110)
* [ci] Run "Push Internal" job on AzurePipelines-EO pool (#8991)
* Bump to dotnet/runtime@2d5c0b720c 8.0.8 (#9123)
* [Mono.Android] Data sharing and Close() overrides (#9103)
* [Xamarin.Android.Build.Tasks] fix `Inputs` for `_Generate*Java*` targets (#9174)
* Bump to xamarin/monodroid@e6a7cf474a (#9193)
* [release/8.0.4xx] Backport maestro and artifact drop infra improvements (#9195)
* Bump to dotnet/runtime@62f69f1e86 8.0.9 (#9191)
* Bump to dotnet/runtime@ed13b35174 8.0.9 (#9221)
* [build] Update package metadata (#9230)
* [Xamarin.Android.Build.Tasks] Rethink default property values (#9155) (#9224)
* [Xamarin.Android.Build.Tasks] Fix an issue with incremental builds (#9183)
* [ci] Improve push_signed_nugets job condition (#9240)
jonathanpeppers added a commit that referenced this pull request Aug 23, 2024
Changes: 34.0.113...b0fd011

.NET 8 changelog:

* [Mono.Android] AndroidMessageHandler should follow HTTP-308 redirects (#8951)
* [ci] Fix android source path for MAUI test job (#9030)
* [ci] Update checkout path for nightly build (#9028)
* [Mono.Android-Tests] Fix repo URL in redirect tests (#9035)
* [Xamarin.Android.Build.Tasks] Support VS "Build Acceleration" (#9042)
* [xaprepare] Always use release mono bundle (#9106)
* [ci] Update sdk-insertions trigger to manual only (#9029)
* Bump to dotnet/runtime@4a37e7305c 8.0.8 (#9033)
* Bump to dotnet/installer@b638a84fba 8.0.205-servicing.24212.27 (#9032)
* [ci] Disable CodeQL on macOS, Linux, non-main jobs (#9111)
* [ci] Use DotNetCoreCLI to sign macOS files (#9102)
* [tests] fix `InvalidTargetPlatformVersion` for `net8.0` (#9110)
* [ci] Run "Push Internal" job on AzurePipelines-EO pool (#8991)
* Bump to dotnet/runtime@2d5c0b720c 8.0.8 (#9123)
* [Mono.Android] Data sharing and Close() overrides (#9103)
* [Xamarin.Android.Build.Tasks] fix `Inputs` for `_Generate*Java*` targets (#9174)
* Bump to xamarin/monodroid@e6a7cf474a (#9193)
* [release/8.0.4xx] Backport maestro and artifact drop infra improvements (#9195)
* Bump to dotnet/runtime@62f69f1e86 8.0.9 (#9191)
* Bump to dotnet/runtime@ed13b35174 8.0.9 (#9221)
* [build] Update package metadata (#9230)
* [Xamarin.Android.Build.Tasks] Rethink default property values (#9155) (#9224)
* [Xamarin.Android.Build.Tasks] Fix an issue with incremental builds (#9183)
* [ci] Improve push_signed_nugets job condition (#9240)

~~ Other changes ~~

* Setup Maestro to track .NET8 builds

After setting up the appropriate changes in `Version.Details.xml`, I ran:

    > darc update-dependencies --id 235947
    Looking up build with BAR id 235947
    Updating 'Microsoft.Android.Sdk.Windows': '1' => '34.0.137' (from build '8.0.4xx-    b0fd011-1' of 'https://github.com/dotnet/android')
    Checking for coherency updates...
    Using 'Strict' coherency mode. If this fails, a second attempt utilizing 'Legacy' Coherency mode will be made.
    Local dependencies updated based on build with BAR id 235947 (8.0.4xx-b0fd0113c829edd9bd1bc7d742255a237ff19f69-1 from https://github.com/dotnet/android@release/8.0.4xx)
jonathanpeppers added a commit that referenced this pull request Aug 23, 2024
Changes: 34.0.113...b0fd011

.NET 8 changelog:

* [Mono.Android] AndroidMessageHandler should follow HTTP-308 redirects (#8951)
* [ci] Fix android source path for MAUI test job (#9030)
* [ci] Update checkout path for nightly build (#9028)
* [Mono.Android-Tests] Fix repo URL in redirect tests (#9035)
* [Xamarin.Android.Build.Tasks] Support VS "Build Acceleration" (#9042)
* [xaprepare] Always use release mono bundle (#9106)
* [ci] Update sdk-insertions trigger to manual only (#9029)
* Bump to dotnet/runtime@4a37e7305c 8.0.8 (#9033)
* Bump to dotnet/installer@b638a84fba 8.0.205-servicing.24212.27 (#9032)
* [ci] Disable CodeQL on macOS, Linux, non-main jobs (#9111)
* [ci] Use DotNetCoreCLI to sign macOS files (#9102)
* [tests] fix `InvalidTargetPlatformVersion` for `net8.0` (#9110)
* [ci] Run "Push Internal" job on AzurePipelines-EO pool (#8991)
* Bump to dotnet/runtime@2d5c0b720c 8.0.8 (#9123)
* [Mono.Android] Data sharing and Close() overrides (#9103)
* [Xamarin.Android.Build.Tasks] fix `Inputs` for `_Generate*Java*` targets (#9174)
* Bump to xamarin/monodroid@e6a7cf474a (#9193)
* [release/8.0.4xx] Backport maestro and artifact drop infra improvements (#9195)
* Bump to dotnet/runtime@62f69f1e86 8.0.9 (#9191)
* Bump to dotnet/runtime@ed13b35174 8.0.9 (#9221)
* [build] Update package metadata (#9230)
* [Xamarin.Android.Build.Tasks] Rethink default property values (#9155) (#9224)
* [Xamarin.Android.Build.Tasks] Fix an issue with incremental builds (#9183)
* [ci] Improve push_signed_nugets job condition (#9240)

~~ Other changes ~~

* Setup Maestro to track .NET8 builds

After setting up the appropriate changes in `Version.Details.xml`, I ran:

    > darc update-dependencies --id 235947
    Looking up build with BAR id 235947
    Updating 'Microsoft.Android.Sdk.Windows': '1' => '34.0.137' (from build '8.0.4xx-    b0fd011-1' of 'https://github.com/dotnet/android')
    Checking for coherency updates...
    Using 'Strict' coherency mode. If this fails, a second attempt utilizing 'Legacy' Coherency mode will be made.
    Local dependencies updated based on build with BAR id 235947 (8.0.4xx-b0fd0113c829edd9bd1bc7d742255a237ff19f69-1 from https://github.com/dotnet/android@release/8.0.4xx)
@github-actions github-actions bot locked and limited conversation to collaborators Sep 6, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

build issue typemaps.x86_64.ll: error: Could not open input file: no such file or directory
2 participants