d8/r8 can give lots of warnings about MANIFEST.MF

[jonathanpeppers]

Context: #2939 (comment)

Steps to Reproduce

1. In some apps you get multiple warnings such as:

R8 : warning : Resource 'META-INF/MANIFEST.MF' already exists.

I think you might see them with a Xamarin.Forms template even, just enable d8/r8.

Expected Behavior

R8 shouldn't produce warnings by default.

Actual Behavior

We get warnings!

Version Information

Happens with 16.2, and probably all versions.

[KennyDizi]

thank you @jonathanpeppers !

[gmck]

I also get a number of those and additionally META-INF/MSFTSIG.SF and META-INF/MSFTSIG.RSA. No Forms in my stuff though.

[Afshin1980]

Same problem!

[jbtdevgit]

I also have -keep public class *extends androidx.versionedparcelable.VersionedParcelable encountered when r8 is enabled

[cosminstirbu]

Same problem

[DevEddy]

Same here

[geseru]

Same here

[Bustillox]

Same here

[andreas-gabriel]

Same here

[SavikPavel]

Same here

[tranb3r]

@jonathanpeppers
Is this issue supposed to be fixed in xamarin.android 10.2.0.16 ?
I'm still having these warnings with vs 2019 16.5.0-pre1.0.

In your commit, I can see that the message used in the regex is "Warning: Resource.+already exists", while the one being shown during build is actually "warning : Resource.+already exists".
Notice the missing space between "warning" and ":". Is this a mistake ?

Thanks

[jonathanpeppers]

@tranb3r can you post your build log?

https://docs.microsoft.com/en-us/xamarin/android/troubleshooting/troubleshooting#diagnostic-msbuild-output

I modified a test so it got the same warning, and the warning went away. But it's possible something is different for you?

[tranb3r]

Sure. Here it is (only the r8 task):
build_r8.log

[jonathanpeppers]

@tranb3r are these showing up as warnings in the IDE's error pad? Have a screenshot? These will still get logged, just as messages though.

[tranb3r]

@jonathanpeppers
No, these lines are not showing up as warnings in the IDE. But they are showing up in the build log.
Is this what this issue is about, removing the warning from the IDE, even if they still show up in the build log ?
I was hoping that the root cause for those warnings in R8 would be fixed...

[jonathanpeppers]

I was hoping that the root cause for those warnings in R8 would be fixed...

R8 appears to warn if the same file is encountered in multiple jar files:

Resource 'META-INF/MANIFEST.MF' already exists.

I don't see the need for R8 to warn about this, but that is what it does.

I reviewed R8's source code and could not find a way for us to suppress the warning:

• Source: https://r8.googlesource.com/r8/
• We consume the package from maven here: https://mvnrepository.com/artifact/com.android.tools/r8

Feel free to review and let us know if you see something we could do differently (or open a PR!). Thanks.

[tranb3r]

I've added the "--no-data-resources" r8 argument and the "resource already exists" warnings are gone (and the output classes.dex is exactly the same).
@jonathanpeppers Do you know what is the impact of adding this argument ? Could it be the default for xamarin.android ?

[jonathanpeppers]

It doesn't seem to be documented very well as to what it does:

> java -jar .\bin\Debug\lib\xamarin.android\xbuild\Xamarin\Android\r8.jar --help
Usage: r8 [options] <input-files>
 where <input-files> are any combination of dex, class, zip, jar, or apk files
 and options are:
...
  --no-data-resources      # Ignore all data resources.

Thanks, Google!

Reading the source code, I found a comment that gives slightly more info:

/**
 * Set the output path-and-mode and control if data resources are included.
 *
 * <p>In addition to setting the output path-and-mode (see {@link #setOutput(Path, OutputMode)})
 * this can control if data resources should be included or not.
 *
 * <p>Data resources are non Java classfile items in the input.
 *
 * <p>If data resources are not included they are ignored in the input and will not produce
 * anything in the output. If data resources are included they are processed according to the
 * configuration and written to the output.
 *
 * @param outputPath Path to write the output to. Must be an archive or and existing directory.
 * @param outputMode Mode in which to write the output.
 * @param includeDataResources If data resources from the input should be included in the
 *     output.
 */

The default value of includeDataResources is true and --no-data-resources sets it to false.

R8 has an option to output java output instead of dex (Android-specific). It seems like that option would only apply to java output. I don't think you can put arbitrary files in a dex. Our build system puts these extra files in the APK later on:

https://github.com/xamarin/xamarin-android/blob/e22be4912e498c705e89acc4e2fc3af8a0a9a8c4/src/Xamarin.Android.Build.Tasks/Tasks/BuildApk.cs#L175-L201

Let me do some other research on this.

[KennyDizi]

thanks @jonathanpeppers

[brendanzagaeski]

Release status update

A new Preview version has now been published that includes the change to make this warning an informational message so that it will no longer appear in the Error List. That change is not yet included in a Release version. I will update this item again when a Release version is available that includes that change in behavior.

Separate from that first change, investigation will continue on possible further improvements.

Change included in Xamarin.Android 10.2.0.84.

Change included on Windows in Visual Studio 2019 version 16.5 Preview 2. To try the Preview version that includes the change, check for the latest updates in Visual Studio Preview.

Change included on macOS in Visual Studio 2019 for Mac version 8.5 Preview 1. To try the Preview version that includes the change, check for the latest updates on the Preview updater channel.

[brendanzagaeski]

Release status update

A new Release version has now been published on Windows that includes the change to make this warning an informational message so that it will no longer appear in the Error List. That change is not yet published in a Release version on macOS. I will update this item again when a Release version is available on macOS that change in behavior.

Separate from that first change, investigation will continue on possible further improvements.

Change included in Xamarin.Android 10.2.0.100.

Change included on Windows in Visual Studio 2019 version 16.5. To get the new version that includes the change, check for the latest updates or install the latest version from https://visualstudio.microsoft.com/downloads/.

(Change also included on macOS in Visual Studio 2019 for Mac version 8.5 Preview 1 and higher. To try the Preview version that includes the change, check for the latest updates on the Preview updater channel.)

[brendanzagaeski]

Release status update

A new Release version has now been published on macOS that includes the change to make this warning an informational message so that it will no longer appear in the Error List.

Separate from that first change, investigation will continue on possible further improvements.

Change included in Xamarin.Android 10.2.0.100.

Change included on macOS in Visual Studio 2019 for Mac version 8.5. To get the new version that includes the change, check for the latest updates on the Stable updater channel.

(Change also included on Windows in Visual Studio 2019 version 16.5 and higher. To get the new version that includes the change, check for the latest updates or install the latest version from https://visualstudio.microsoft.com/downloads/.)

[KennyDizi]

Kudos