-
Notifications
You must be signed in to change notification settings - Fork 52
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Discussion: Ways to *actually* fix trimmer warnings around Type.GetType()
?
#1165
Comments
It should be explicitly noted that at this time That said, the "core" architecture within The benefit to there being two similar-yet-distinct implementations is that we can prototype things in "core" -- e.g. rethink the |
Fixes: #1165 Context: #1157 If we more strongly rely on JNI signatures, we can remove the need for Java Callable Wrappers to contain assembly-qualified type names, thus removing the need for `ManagedPeer` to use `Type.GetType()` entirely, removing the [IL2057][0] warnings. Furthermore, if we add `[DynamicallyAccessedMembers]` to `JniRuntime.JniTypeManager.GetType()`, we can fix some [IL2075][1] warnings which appeared after fixing the IL2057 warnings. Aside: Excising assembly-qualified type names from Java Callable Wrappers had some "interesting" knock-on effects in the unit tests, requiring that more typemap information be explicitly provided. (This same information was *implicitly* provided before, via the provision of assembly-qualified type names everywhere…) [0]: https://learn.microsoft.com/en-us/dotnet/core/deploying/trimming/trim-warnings/IL2057 [1]: https://learn.microsoft.com/en-us/dotnet/core/deploying/trimming/trim-warnings/il2075
Fixes: #1165 Context: #1153 Context: #1157 When building for NativeAOT (#1153) or when building .NET Android apps with `-p:IsAotcompatible=true` (#1157), we get [IL2057][0] warnings from `ManagedPeer.cs`: ManagedPeer.cs(93,19,93,112): warning IL2057: Unrecognized value passed to the parameter 'typeName' of method 'System.Type.GetType(String, Boolean)'. It's not possible to guarantee the availability of the target type. ManagedPeer.cs(156,18,156,65): warning IL2057: Unrecognized value passed to the parameter 'typeName' of method 'System.Type.GetType(String, Boolean)'. It's not possible to guarantee the availability of the target type. ManagedPeer.cs(198,35,198,92): warning IL2057: Unrecognized value passed to the parameter 'typeName' of method 'System.Type.GetType(String, Boolean)'. It's not possible to guarantee the availability of the target type. These warnings are because `ManagedPeer.Construct()` and `ManagedPeer.RegisterNativeMembers()` use `Type.GetType()` on string values provided *from Java code*, and thus the IL trimmer does not have visibility into those strings, and thus cannot reliably determine which types need to be preserved: // Java Callable Wrapper /* partial */ class ManagedType { public static final String __md_methods; static { __md_methods = "n_GetString:()Ljava/lang/String;:__export__\n" + ""; net.dot.jni.ManagedPeer.registerNativeMembers ( /* nativeClass */ ManagedType.class, /* assemblyQualifiedName */ "Example.ManagedType, Hello-NativeAOTFromJNI", /* methods */ __md_methods); } public ManagedType (int p0) { super (); if (getClass () == ManagedType.class) { net.dot.jni.ManagedPeer.construct ( /* self */ this, /* assemblyQualifiedName */ "Example.ManagedType, Hello-NativeAOTFromJNI", /* constructorSignature */ "System.Int32, System.Runtime", /* arguments */ new java.lang.Object[] { p0 }); } } } `ManagedPeer.construct()` passes *two* sets of assembly-qualified type names: `assemblyQualifiedName` contains the type to construct, while `constructorSignature` contains a `:`-separated list of assembly-qualified type names for the constructor parameters. Each of these are passed to `Type.GetType()`. `ManagedPeer.registerNativeMembers()` passes an assembly-qualified type name to `ManagedPeer.RegisterNativeMembers()`, which passes the assembly-qualified type name to `Type.GetType()` to find the type to register native methods for. If we more strongly rely on JNI signatures, we can remove the need for Java Callable Wrappers to contain assembly-qualified type names entirely, thus removing the need for `ManagedPeer` to use `Type.GetType()`, removing the IL2057 warnings. For `ManagedPeer.construct()`, `assemblyQualifiedName` can be replaced with getting the JNI type signature from `self.getClass()`, and `constructorSignature` can be replaced with a *JNI method signature* of the calling constructor. For `ManagedPeer.registerNativeMembers()`, `assemblyQualifiedName` can be replaced with getting the JNI type signature from `nativeClass`. // Java Callable Wrapper /* partial */ class ManagedType { public static final String __md_methods; static { __md_methods = "n_GetString:()Ljava/lang/String;:__export__\n" + ""; net.dot.jni.ManagedPeer.registerNativeMembers ( /* nativeClass */ ManagedType.class, /* methods */ __md_methods); } public ManagedType (int p0) { super (); if (getClass () == ManagedType.class) { net.dot.jni.ManagedPeer.construct ( /* self */ this, /* constructorSignature */ "(I)V", /* arguments */ new java.lang.Object[] { p0 }); } } } Furthermore, if we add `[DynamicallyAccessedMembers]` to `JniRuntime.JniTypeManager.GetType()`, we can fix some [IL2075][1] warnings which appeared after fixing the IL2057 warnings. Aside: Excising assembly-qualified type names from Java Callable Wrappers had some "interesting" knock-on effects in the unit tests, requiring that more typemap information be explicitly provided. (This same information was *implicitly* provided before, via the provision of assembly-qualified type names everywhere…) [0]: https://learn.microsoft.com/en-us/dotnet/core/deploying/trimming/trim-warnings/IL2057 [1]: https://learn.microsoft.com/en-us/dotnet/core/deploying/trimming/trim-warnings/il2075
Fixes: #1165 Context: #1153 Context: #1157 Context: f60906c When building for NativeAOT (#1153) or when building .NET Android apps with `-p:IsAotcompatible=true` (#1157), we get [IL2057][0] warnings from `ManagedPeer.cs`: ManagedPeer.cs(93,19,93,112): warning IL2057: Unrecognized value passed to the parameter 'typeName' of method 'System.Type.GetType(String, Boolean)'. It's not possible to guarantee the availability of the target type. ManagedPeer.cs(156,18,156,65): warning IL2057: Unrecognized value passed to the parameter 'typeName' of method 'System.Type.GetType(String, Boolean)'. It's not possible to guarantee the availability of the target type. ManagedPeer.cs(198,35,198,92): warning IL2057: Unrecognized value passed to the parameter 'typeName' of method 'System.Type.GetType(String, Boolean)'. It's not possible to guarantee the availability of the target type. These warnings are because `ManagedPeer.Construct()` and `ManagedPeer.RegisterNativeMembers()` use `Type.GetType()` on string values provided *from Java code*, and thus the IL trimmer does not have visibility into those strings, and thus cannot reliably determine which types need to be preserved: // Java Callable Wrapper /* partial */ class ManagedType { public static final String __md_methods; static { __md_methods = "n_GetString:()Ljava/lang/String;:__export__\n" + ""; net.dot.jni.ManagedPeer.registerNativeMembers ( /* nativeClass */ ManagedType.class, /* assemblyQualifiedName */ "Example.ManagedType, Hello-NativeAOTFromJNI", /* methods */ __md_methods); } public ManagedType (int p0) { super (); if (getClass () == ManagedType.class) { net.dot.jni.ManagedPeer.construct ( /* self */ this, /* assemblyQualifiedName */ "Example.ManagedType, Hello-NativeAOTFromJNI", /* constructorSignature */ "System.Int32, System.Runtime", /* arguments */ new java.lang.Object[] { p0 }); } } } `ManagedPeer.construct()` passes *two* sets of assembly-qualified type names: `assemblyQualifiedName` contains the type to construct, while `constructorSignature` contains a `:`-separated list of assembly-qualified type names for the constructor parameters. Each of these are passed to `Type.GetType()`. `ManagedPeer.registerNativeMembers()` passes an assembly-qualified type name to `ManagedPeer.RegisterNativeMembers()`, which passes the assembly-qualified type name to `Type.GetType()` to find the type to register native methods for. If we more strongly rely on JNI signatures, we can remove the need for Java Callable Wrappers to contain assembly-qualified type names entirely, thus removing the need for `ManagedPeer` to use `Type.GetType()`, removing the IL2057 warnings. For `ManagedPeer.construct()`, `assemblyQualifiedName` can be replaced with getting the JNI type signature from `self.getClass()`, and `constructorSignature` can be replaced with a *JNI method signature* of the calling constructor. For `ManagedPeer.registerNativeMembers()`, `assemblyQualifiedName` can be replaced with getting the JNI type signature from `nativeClass`. `jcw-gen --codegen-target=JavaInterop1` output becomes: // New JavaInterop1 Java Callable Wrapper /* partial */ class ManagedType { public static final String __md_methods; static { __md_methods = "n_GetString:()Ljava/lang/String;:__export__\n" + ""; net.dot.jni.ManagedPeer.registerNativeMembers ( /* nativeClass */ ManagedType.class, /* methods */ __md_methods); } public ManagedType (int p0) { super (); if (getClass () == ManagedType.class) { net.dot.jni.ManagedPeer.construct ( /* self */ this, /* constructorSignature */ "(I)V", /* arguments */ new java.lang.Object[] { p0 }); } } } This does not alter `jcw-gen --codegen-target=XAJavaInterop1` output; .NET Android will continue to require `Type.GetType()` calls within xamarin/xamarin-android, e.g. [`AndroidTypeManager.RegisterNativeMembers()`][2]. Furthermore, if we add `[DynamicallyAccessedMembers]` to `JniRuntime.JniTypeManager.GetType()`, we can fix some [IL2075][1] warnings which appeared after fixing the IL2057 warnings. Aside: Excising assembly-qualified type names from Java Callable Wrappers had some "interesting" knock-on effects in the unit tests, requiring that more typemap information be explicitly provided. (This same information was *implicitly* provided before, via the provision of assembly-qualified type names everywhere…) One problem with the approach of using JNI signatures instead of using assembly-qualified names is *ambiguity*: there can be multiple managed types which correspond to a given JNI signature. Consider the JNI signature `[I`, which is a Java `int[]`. This is bound as: * C# `int[]` * `JavaArray<int>` * `JavaPrimitiveArray<int>` * `JavaInt32Array` How do we know which to use? Using assembly-qualified type names for constructor parameters nicely solved this issue, but if we're not using them anymore… Update `JavaCallableExample` to demonstrate this: partial class JavaCallableExample { [JavaCallableConstructor(SuperConstructorExpression="")] public JavaCallableExample (int[] a, JavaInt32Array b); } The intention is twofold: 1. This should result in a Java Callable Wrapper constructor with signature `JavaCallableExample(int[] p0, int[] p1)`, and 2. Java code should be able to invoke this constructor. Turns out, neither of these worked when `Type.GetType()` is not used for constructor argument lookup: `JavaCallableWrapperGenerator` didn't fully support e.g. `[JniTypeSignature("I", ArrayRank=1)]` (present on `JavaInt32Array`), so it didn't know what to do with the `JavaInt32Array` parameter. Once (1) was fixed, (2) would fail because `JniRuntime.JniTypeManager.GetType(JniTypeSignature.Parse("[I"))` would return `JavaPrimitiveArray<int>`, which wasn't used in `JavaCallableExample`, resulting in: System.NotSupportedException : Unable to find constructor Java.InteropTests.JavaCallableExample(Java.Interop.JavaPrimitiveArray`1[[System.Int32, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]], Java.Interop.JavaPrimitiveArray`1[[System.Int32, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]]). Please provide the missing constructor. ----> Java.Interop.JniLocationException : Exception of type 'Java.Interop.JniLocationException' was thrown. Stack Trace: at Java.Interop.ManagedPeer.GetConstructor(JniTypeManager typeManager, Type type, String signature, Type[]& parameterTypes) at Java.Interop.ManagedPeer.Construct(IntPtr jnienv, IntPtr klass, IntPtr n_self, IntPtr n_constructorSignature, IntPtr n_constructorArguments) … --- End of managed Java.Interop.JavaException stack trace --- java.lang.Throwable at net.dot.jni.ManagedPeer.construct(Native Method) at net.dot.jni.test.JavaCallableExample.<init>(JavaCallableExample.java:32) at net.dot.jni.test.UseJavaCallableExample.test(UseJavaCallableExample.java:8) The constructor couldn't be found because `JniRuntime.JniTypeManager.GetTypes()` was incomplete, which is a longstanding limitation from f60906c: for `[I`, it would only return `JavaPrimitiveArray<int>` and `int[]`, in that order. Fix both of these. `JniRuntime.JniTypeManager.GetTypes(JniTypeSignature.Parse("[I"))` will now include: * `JavaArray<int>` * `JavaPrimitiveArray<int>` * `JavaInt32Array` * `int[]` This now allows the `JavaCallableExample` constructor to be invoked from Java. Because `ManagedPeer.Construct()` is now doing so much extra work in order to find the `ConstructorInfo` to invoke, cache the lookups. (Technically this is a "memory leak," as cache entries are never removed.) Finally, update `CecilCompilerExpressionVisitor` to emit `newobj` in certain `VisitNew()` invocations. This was needed while trying: partial class JavaCallableExample { [JavaCallable ("getA")] public int[] GetA() => this.a; } in order to fix the IL error: % $HOME/.dotnet/tools/ilverify bin/TestDebug-net7.0/Java.Interop.Export-Tests.dll \ --tokens --system-module System.Private.CoreLib \ -r 'bin/TestDebug-net7.0/*.dll' \ -r '/usr/local/share/dotnet/shared/Microsoft.NETCore.App/7.0.10/*.dll' [IL]: Error [StackUnderflow]: […/bin/TestDebug-net7.0/Java.Interop.Export-Tests.dll : .__<$>_jni_marshal_methods::n_GetA(native int, native int)][offset 0x0000002F] Stack underflow. Unfortunately, even after the above fix invalid IL was generated during `jnimarshalmethod-gen` processing, which will be investigated later. [0]: https://learn.microsoft.com/en-us/dotnet/core/deploying/trimming/trim-warnings/IL2057 [1]: https://learn.microsoft.com/en-us/dotnet/core/deploying/trimming/trim-warnings/il2075 [2]: https://github.com/xamarin/xamarin-android/blob/main/src/Mono.Android/Android.Runtime/AndroidRuntime.cs#L481-L577
Context: dotnet/java-interop#1165 Context: dotnet/java-interop@005c914 Context: #8543 Context: dotnet/java-interop@07c7300 Context: #8625 Context: xamarin/monodroid@e3e4f12 Context: xamarin/monodroid@a04b73b Context: efbec22 Changes: dotnet/java-interop@8b85462...07c7300 * dotnet/java-interop@07c73009: [Java.Interop] Typemap support for JavaObject & `[JniTypeSignature]` (dotnet/java-interop#1181) * dotnet/java-interop@d529f3be: Bump to xamarin/xamarin-android-tools/main@ed102fc (dotnet/java-interop#1182) * dotnet/java-interop@def5bc0d: [ci] Add API Scan job (dotnet/java-interop#1178) * dotnet/java-interop@d5afa0af: [invocation-overhead] Add generated source files (dotnet/java-interop#1175) * dotnet/java-interop@473ef74c: Bump to xamarin/xamarin-android-tools/main@4889bf0 (dotnet/java-interop#1172) * dotnet/java-interop@005c9141: [Java.Interop] Avoid `Type.GetType()` in `ManagedPeer` (dotnet/java-interop#1168) * dotnet/java-interop@0f1efebd: [Java.Interop] Use PublicApiAnalyzers to ensure we do not break API (dotnet/java-interop#1170) (From the "infinite scream" department…) It started with a desire to remove some linker warnings (dotnet/java-interop#1165): external/Java.Interop/src/Java.Interop/Java.Interop/ManagedPeer.cs(93,19,93,112): warning IL2057: Unrecognized value passed to the parameter 'typeName' of method 'System.Type.GetType(String, Boolean)'. It's not possible to guarantee the availability of the target type. dotnet/java-interop@005c9141 attempted to fix this by requiring the use of "typemaps" mapping Java type signatures to managed types, replacing e.g.: Type type = Type.GetType ("Example.Type, AssemblyName", throwOnError: true)!; Type[] parameterTypes = GetParameterTypes ("System.Int32:System.Int32"); ConstructorInfo ctor = type.GetConstructor (ptypes); // ctor=Example.Type(int, int) constructor with (not exactly, but for expository purposes): Type type = GetTypeFromSignature("crc64…/Type"); Type[] parameterTypes = GetConstructorCandidateParameterTypes ("(II)V"); ConstructorInfo ctor = type.GetConstructor (ptypes); // ctor=Example.Type(int, int) constructor among other changes. This was a *significant* change that would alter *Java.Interop* semantics but *not* .NET Android semantics -- .NET Android uses `Java.Interop.TypeManager.n_Activate()` (in this repo) for Java-side "activation" scenarios, not `Java.Interop.ManagedPeer` -- so in an abundance of caution we did a manual integration test in #8543 to make sure nothing broke before merging it. Something was apparently "off" in that integration. (We're still not sure what was off, or why it was completely green.) Ever since dotnet/java-interop@005c9141 was merged, every attempt to bump xamarin/Java.Interop has failed, in a number of ways described below. However, instead of reverting dotnet/java-interop@005c9141 we took this as an opportunity to understand *how and why* things were failing, as apparently we had encountered some *long-standing* corner cases in How Things Work. The oversights and failures include: 1. In order to make the Java.Interop unit tests work in .NET Android, the (largely hand-written) Java.Interop test types *also* need to participate with .NET Android typemap support, so that there is a typemap entry mapping `net/dot/jni/test/GenericHolder` to `Java.InteropTests.GenericHolder<T>` and vice-versa. dotnet/java-interop@07c73009 updates `Java.Interop.Tools.JavaCallableWrappers` to support creating typemap entries for `Java.Interop.JavaObject` subclasses, introducing a new `TypeDefinition.HasJavaPeer()` extension method. 2. (1) meant that, for the first time ever, types in `Java.Interop-Tests` participated in .NET Android type mapping. This *sounds* fine, except that `Java.Interop-Tests` contains "competing bindings" for `java.lang.Object`: [JniTypeSignature ("java/lang/Object", GenerateJavaPeer=false)] partial class JavaLangRemappingTestObject : JavaObject { } 3. (2) means that, for the first time ever, we *could* have the typemap entry for `java/lang/Object` map to `Java.InteropTests.JavaLangRemappingTestObject, Java.Interop-Tests`, *not* `Java.Lang.Object, Mono.Android`. Arguably a bug, arguably "meh", but this setup triggered some never previously encountered error conditions: 4. `EmbeddedAssemblies::typemap_java_to_managed()` within `libmonodroid.so` returns a `System.Type` that corresponds to a JNI type. `typemap_java_to_managed()` has a bug/corner case wherein it will only provide `Type` instances from assemblies which have already been loaded. Early in startup, `Java.Interop-Tests` hasn't been loaded yet, so when `java/lang/Object` was mapped to `Java.InteropTests.JavaLangRemappingTestObject, Java.Interop-Tests`, `typemap_java_to_managed()` would return `null`. This is a bug/corner case, which is being investigated in #8625. 5. Calls to `Java.Lang.Object.GetObject<T>()` call `Java.Interop.TypeManager.CreateInstance()`, which loops through the type and all base types to find a known binding/wrapper. Because of (3)+(4), if (when) we try to find the wrapper for `java/lang/Object`, we would find *no* mapping. This would cause an `JNI DETECTED ERROR IN APPLICATION` *crash*. This was due to a "use after free" bug. See the "TypeManager.CreateInstance() Use After Free Bug" section. 6. Once (5) is fixed we encounter our next issue: the `Java.InteropTests.JnienvTest.NewOpenGenericTypeThrows()` unit test started failing because `crc641855b07eca6dcc03.GenericHolder_1` couldn't be found. This was caused by a bug in `acw-map.txt` parsing within `<R8/>`. See the "`<R8/>` and `acw-map.txt` parsing.`" section. 7. Once (6) was fixed, (3) caused a *new* set of failures: multiple tests started failing because `java/lang/Object` was being mapped to the wrong managed type. (3) becomes less "meh" and more "definitely a bug". See the "Correct `java/lang/Object` mappings" section. *Now* things should work reliably. ~~ TypeManager.CreateInstance() Use After Free Bug ~~ On 2011-Oct-19, xamarin/monodroid@e3e4f123d8 introduced a use-after-free bug within `TypeManager.CreateInstance()`: JNIEnv.DeleteRef (handle, transfer); throw new NotSupportedException ( FormattableString.Invariant ($"Internal error finding wrapper class for '{JNIEnv.GetClassNameFromInstance (handle)}'. (Where is the Java.Lang.Object wrapper?!)"), CreateJavaLocationException ()); `handle` *cannot be used* after `JNIEnv.DeleteRef(handle)`. Failure to do so results in a `JNI DETECTED ERROR IN APPLICATION` crash; with `adb shell setprop debug.mono.log lref+` set, we see: I monodroid-lref: +l+ lrefc 1 handle 0x71/L from thread '(null)'(1) D monodroid-gref: at Android.Runtime.AndroidObjectReferenceManager.CreatedLocalReference(JniObjectReference , Int32& ) D monodroid-gref: at Java.Interop.JniRuntime.JniObjectReferenceManager.CreatedLocalReference(JniEnvironmentInfo , JniObjectReference ) D monodroid-gref: at Java.Interop.JniEnvironment.LogCreateLocalRef(JniObjectReference ) D monodroid-gref: at Java.Interop.JniEnvironment.LogCreateLocalRef(IntPtr ) D monodroid-gref: at Java.Interop.JniEnvironment.InstanceMethods.CallObjectMethod(JniObjectReference , JniMethodInfo ) D monodroid-gref: … … I monodroid-lref: -l- lrefc 0 handle 0x71/L from thread '(null)'(1) D monodroid-gref: at Android.Runtime.AndroidObjectReferenceManager.DeleteLocalReference(JniObjectReference& , Int32& ) D monodroid-gref: at Java.Interop.JniRuntime.JniObjectReferenceManager.DeleteLocalReference(JniEnvironmentInfo , JniObjectReference& ) D monodroid-gref: at Java.Interop.JniObjectReference.Dispose(JniObjectReference& reference) D monodroid-gref: at Android.Runtime.JNIEnv.DeleteLocalRef(IntPtr ) D monodroid-gref: at Android.Runtime.JNIEnv.DeleteRef(IntPtr , JniHandleOwnership ) D monodroid-gref: at Java.Interop.TypeManager.CreateInstance(IntPtr , JniHandleOwnership , Type ) D monodroid-gref: at Java.Lang.Object.GetObject(IntPtr , JniHandleOwnership , Type ) D monodroid-gref: at Java.Lang.Object._GetObject[IIterator](IntPtr , JniHandleOwnership ) D monodroid-gref: at Java.Lang.Object.GetObject[IIterator](IntPtr handle, JniHandleOwnership transfer) D monodroid-gref: … D monodroid-gref: E droid.NET_Test: JNI ERROR (app bug): accessed stale Local 0x71 (index 7 in a table of size 7) F droid.NET_Test: java_vm_ext.cc:570] JNI DETECTED ERROR IN APPLICATION: use of deleted local reference 0x71 … F droid.NET_Test: runtime.cc:630] native: #13 pc 00000000003ce865 /apex/com.android.runtime/lib64/libart.so (art::(anonymous namespace)::CheckJNI::GetObjectClass(_JNIEnv*, _jobject*)+837) The immediate fix is Don't Do That™; use a temporary: class_name = JNIEnv.GetClassNameFromInstance (handle); JNIEnv.DeleteRef (handle, transfer); throw new NotSupportedException ( FormattableString.Invariant ($"Internal error finding wrapper class for '{class_name}'. (Where is the Java.Lang.Object wrapper?!)"), CreateJavaLocationException ()); Unfortunately, *just* fixing the "use-after-free" bug is insufficient; if we throw that `NotSupportedException`, things *will* break elsewhere. We'll just have an "elegant unhandled exception" app crash instead of a "THE WORLD IS ENDING" failed assertion crash. We could go with the simple fix for the crash, but this means that in order to integrate dotnet/java-interop@005c9141 & dotnet/java-interop@07c73009 we'd have to figure out how to *ensure* that `java/lang/Object` is bound as `Java.Lang.Object, Mono.Android`, not `Java.InteropTests.JavaLangRemappingTestObject, Java.Interop-Tests`. (We actually need to do this *anyway*; see the "Correct `java/lang/Object` mappings" section. At the time we I was trying to *avoid* special-casing `Mono.Android.dll`…) There is a*slightly* more complicated approach which fixes (5) while supporting (4) `typemap_java_to_managed()` returning null; consider the `-l-` callstack: at Android.Runtime.JNIEnv.DeleteRef(IntPtr , JniHandleOwnership ) at Java.Interop.TypeManager.CreateInstance(IntPtr , JniHandleOwnership , Type ) at Java.Lang.Object.GetObject(IntPtr , JniHandleOwnership , Type ) at Java.Lang.Object._GetObject[IIterator](IntPtr , JniHandleOwnership ) at Java.Lang.Object.GetObject[IIterator](IntPtr handle, JniHandleOwnership transfer) at Android.Runtime.JavaSet.Iterator() This is part of a generic `Object.GetObject<IIterator>()` invocation! Additionally, because `IIterator` is an interface, in *normal* use the `type` variable within `TypeManager.CreateInstance()` would be `Java.Lang.Object, Mono.Android` and then *immediately discarded* because `Java.Lang.Object` cannot be assigned to `IIterator`. Moving the type compatibility check to *before* the `type == null` check fixes *an* issue with `typemap_java_to_managed()` returning null. ~~ `<R8/>` and `acw-map.txt` parsing.` ~~ There are many ways for Android+Java code to refer to managed types. For example, consider the following View subclass: namespace Example { partial class MyCoolView : Android.Views.View { // … } } Within layout `.axml` files, you can mention an `Android.Views.View` subclass by: * Using the .NET Full Class Name as an element name. <Example.MyCoolView /> * Using the .NET Full Class Name with a *lowercased* namespace name as the element name. <example.MyCoolView /> * Use the Java-side name directly. <crc64….NiftyView /> Within Fragments, you can also use the *assembly-qualified name*: <fragment class="Example.MyCoolView, AssemblyName" /> At build time, all instances of the .NET type names will be *replaced* with the Java type names before the Android toolchain processes the files. The association between .NET type names and Java names is stored within `$(IntermediateOutputPath)acw-map.txt`, which was introduced in xamarin/monodroid@a04b73b3. *Normally* `acw-map.txt` contains three entries: 1. The fully-qualified .NET type name 2. The .NET type name, no assembly 3. (2) with a lowercased namespace name, *or* the `[Register]` value, if provided. For example: Mono.Android_Test.Library.CustomTextView, Mono.Android-Test.Library.NET;crc6456ab8145c81c4100.CustomTextView Mono.Android_Test.Library.CustomTextView;crc6456ab8145c81c4100.CustomTextView mono.android_test.library.CustomTextView;crc6456ab8145c81c4100.CustomTextView Java.InteropTests.GenericHolder`1, Java.Interop-Tests;net.dot.jni.test.tests.GenericHolder Java.InteropTests.GenericHolder`1;net.dot.jni.test.tests.GenericHolder net.dot.jni.test.tests.GenericHolder;net.dot.jni.test.tests.GenericHolder However, when warning XA4214 is emitted (efbec22), there is a "collision" on the .NET side (but *not* the Java side); (2) and (3) are potentially *ambiguous*, so one .NET type is arbitrarily chosen. (Collisions on the Java side result in XA4215 *errors*.) The first line is still possible, because of assembly qualification. Enter ``Java.InteropTests.GenericHolder`1``: this type is present in *both* `Java.Interop-Tests.dll` *and* `Mono.Android-Tests.dll`. dotnet/java-interop@07c73009, this was "fine" because the `GenericHolder<T>` within `Java.Interop-Tests.dll` did not participate in typemap generation. Now it does, resulting in the XA4214 warning. XA4214 *also* means that instead of three lines, it's *one* line: Java.InteropTests.GenericHolder`1, Mono.Android.NET-Tests;crc641855b07eca6dcc03.GenericHolder_1 Enter `<R8/>`, which parses `acw-map.txt` to create a `proguard_project_primary.cfg` file. `<R8/>` did it's *own* parsing of `acw-map.txt`, parsing only *one of every three lines*, on the assumption that *all* entries took three lines. This breaks in the presence of XA4214, because some entries only take one line, not three lines. This in turn meant that `proguard_project_primary.cfg` could *miss* types, which could mean that `r8` would *remove* the unspecified types, resulting in `ClassNotFoundException` at runtime: Java.Lang.ClassNotFoundException : crc641855b07eca6dcc03.GenericHolder_1 ----> Java.Lang.ClassNotFoundException : Didn't find class "crc641855b07eca6dcc03.GenericHolder_1" on path: DexPathList[[zip file "/data/app/Mono.Android.NET_Tests-2stBqO43ov5F6bHfYemJHQ==/base.apk", zip file "/data/app/Mono.Android.NET_Tests-2stBqO43ov5F6bHfYemJHQ==/split_config.x86_64.apk", zip file "/data/app/Mono.Android.NET_Tests-2stBqO43ov5F6bHfYemJHQ==/split_config.xxhdpi.apk"],nativeLibraryDirectories=[/data/app/Mono.Android.NET_Tests-2stBqO43ov5F6bHfYemJHQ==/lib/x86_64, /system/fake-libs64, /data/app/Mono.Android.NET_Tests-2stBqO43ov5F6bHfYemJHQ==/base.apk!/lib/x86_64, /data/app/Mono.Android.NET_Tests-2stBqO43ov5F6bHfYemJHQ==/split_config.x86_64.apk!/lib/x86_64, /data/app/Mono.Android.NET_Tests-2stBqO43ov5F6bHfYemJHQ==/split_config.xxhdpi.apk!/lib/x86_64, /system/lib64, /system/product/lib64]] at Java.Interop.JniEnvironment.StaticMethods.CallStaticObjectMethod(JniObjectReference , JniMethodInfo , JniArgumentValue* ) at Android.Runtime.JNIEnv.FindClass(String ) Update `<R8/>` to instead use `MonoAndroidHelper.LoadMapFile()`, which reads all lines within `acw-map.txt`. This results in a `proguard_project_primary.cfg` file which properly contains a `-keep` entry for XA4214-related types, such as `crc641855b07eca6dcc03.GenericHolder_1`. ~~ Correct `java/lang/Object` mappings ~~` Previous valiant efforts to allow `java/lang/Object` to be mapped to "anything", not just `Java.Lang.Object, Mono.Android`, eventually resulted in lots of unit test failures, e.g.: `Android.RuntimeTests.XmlReaderPullParserTest.ToLocalJniHandle()`: System.NotSupportedException : Unable to activate instance of type Java.InteropTests.JavaLangRemappingTestObject from native handle 0x19 (key_handle 0x2408476). ----> System.MissingMethodException : No constructor found for Java.InteropTests.JavaLangRemappingTestObject::.ctor(System.IntPtr, Android.Runtime.JniHandleOwnership) ----> Java.Interop.JavaLocationException : Exception_WasThrown, Java.Interop.JavaLocationException at Java.Interop.TypeManager.CreateInstance(IntPtr , JniHandleOwnership , Type ) at Java.Interop.TypeManager.CreateInstance(IntPtr , JniHandleOwnership ) at Android.Runtime.XmlResourceParserReader.FromNative(IntPtr , JniHandleOwnership ) at Android.Runtime.XmlResourceParserReader.FromJniHandle(IntPtr handle, JniHandleOwnership transfer) at Android.Content.Res.Resources.GetXml(Int32 ) at Android.RuntimeTests.XmlReaderPullParserTest.ToLocalJniHandle() at System.Reflection.MethodBaseInvoker.InterpretedInvoke_Method(Object obj, IntPtr* args) at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object , BindingFlags ) --MissingMethodException at Java.Interop.TypeManager.CreateProxy(Type , IntPtr , JniHandleOwnership ) at Java.Interop.TypeManager.CreateInstance(IntPtr , JniHandleOwnership , Type ) With a partially heavy heart, we need to special-case typemap entries by processing `Mono.Android.dll` *first*, so that it gets first dibs at bindings for `java/lang/Object` and other types. Update `NativeTypeMappingData` to process types from `Mono.Android` before processing any other module. Note that the special-casing needs to happen in `NativeTypeMappingData` because typemaps were formerly processed in *sorted module order*, in which the sort order is based on the *byte representation* of the module's MVID (a GUID). Additionally, *linking changes the MVID*, which means module order is *effectively random*. Consequently, trying to special case typemap ordering anywhere else is ineffective. ~~ Other ~~ Update `JavaCompileToolTask` to log the contents of its response file. Update LLVM-IR -related types within `src/Xamarin.Android.Build.Tasks/Utilities` to use `TaskLoggingHelper` for logging purposes, *not* `Action<string>`. Update related types to accept `TaskLoggingHelper`, so that we can more easily add diagnostic messages to these types in the future.
Context: #5652 Context: #8724 Context: dotnet/java-interop#1165 Context: dotnet/java-interop@b8f6f88 Context: dc3dc3ccf28cdbe9f8c0a705400b83c11a85c81a980ccf2 Fix another set of trimmer warnings found via: <IsTrimmable>true</IsTrimmable> <EnableAotAnalyzer>true</EnableAotAnalyzer> ~~ JavaObjectExtensions ~~ `Extensions.JavaCast<T>()` now requires `PublicConstructors` and `NonPublicConstructors` because `TypeManager.CreateProxy()` uses `ConstructorInfo.Invoke()`. This change bubbles up to various other types that have a `Find*ById<T>()` method: * `Activity` * `Dialog` * `FragmentManager` * `View` * `Window` `JavaObjectExtensions.GetInvokerType()` also has suppressions around `Assembly.GetType()` and `Type.MakeGenericType()`. We track this for the future at #8724. ~~ AndroidRuntime ~~ Update `[DynamicallyAccessedMembers]` based on changes to `RegisterNativeMembers` in dotnet/java-interop@b8f6f888. ~~ JNINativeWrapper ~~ `$(EnableAotAnalyzer)` found usage of `DynamicMethod`. Suppress for now, as we track this for the future at #8724. ~~ ResourceIdManager ~~ Usage of `Type.GetMethod ("UpdateIdValues")` leads to decoration of `[ResourceDesignerAttribute]` with: [DynamicallyAccessedMembers (DynamicallyAccessedMemberTypes.PublicMethods)] public string FullName { get; set; } I also had to suppress warnings around `Assembly.GetType()`. This *should* be OK, as `Resource.designer.cs` is always in the "root assembly" of Android application projects. Additionally, this code should no longer be used in .NET 8+ apps; see dc3ccf2. ~~ JavaProxyThrowable ~~ Suppress warning around `StackFrame.GetMethod()`; we already handle `null` return values and exceptions. The existing code appears to be "best effort" to provide additional stack trace information. ~~ TypeManager ~~ Suppress warning around a call to `Type.GetType()` with a string passed in from Java. There is not much we can really do yet, except rely on the `MarkJavaObjects` trimmer step. Likely also a problem for the future: * dotnet/java-interop#1165 * #8724 ~~ Impact on `.apk` size ~~ `BuildReleaseArm64XFormsDotNet.apkdesc` shows a ~33KB size increase in the `.apk`. Much of this is attributable to changes from dotnet/runtime (`System.Private.CoreLib.dll` is ~20KB larger). Some of this is due to increases in the size of `classes*.dex`. These changes are because more managed constructors are now preserved by the trimmer, which causes more constructors to be emitted into the Java Callable Wrappers.
Context: #1153
Context: #1157
From the PR #1153
README.md
:Which brings us to #1157 (comment):
https://github.com/xamarin/java.interop/blob/320636df084d377e1fdabef6fa6d28feb9ede6f2/src/Java.Interop/Java.Interop/ManagedPeer.cs#L93
https://github.com/xamarin/java.interop/blob/320636df084d377e1fdabef6fa6d28feb9ede6f2/src/Java.Interop/Java.Interop/ManagedPeer.cs#L156
https://github.com/xamarin/java.interop/blob/320636df084d377e1fdabef6fa6d28feb9ede6f2/src/Java.Interop/Java.Interop/ManagedPeer.cs#L198
How do we fix these? These are
Type.GetType()
invocations for string values which come from Java.Can they be fixed?
We can just add
UnconditionalSuppressMessageAttribute
to entirely ignore the warning. This will cause things to break on NativeAOT, though, and the whole point to fixing things is to make things linker friendly and usable on NativeAOT.PR #1153 implements a "punt" solution to the problem: instead of
Type.GetType()
, introduce aJniRuntime.JniTypeManager.GetTypeFromAssemblyQualifiedName()
method which optionally doesType.GetType()
. This would likely still produce IL2057, but we could also[UncondionalSuppressMessage]
to silence the warning or somehow require that it be overridden within a NativeAOT app. This permits a path which can work with NativeAOT, but it wouldn't work by default, and would require "extra opt-in logic" in the form of a new method override.A "proper by default" solution will require separately considering the
Type.GetType()
calls withinManagedPeer.cs
.Method Registration
Method Registration is done in
ManagedPeer.RegisterNativeMembers()
, which is invoked from Java:Note that
ManagedPeer.registerNativeMembers()
is given three values:java.lang.Class
instance of the type to register methods forWe could have
ManagedPeer.RegisterNativeMembers()
useJniRuntime.JniTypeManager.GetType(JniTypeSignature)
instead ofType.GetType()
, a'la:This also "punts" on the question to
JniRuntime.JniTypeManager.GetType(JniTypeSignature)
, but that's a pre-existing (and required) extension point:https://github.com/xamarin/java.interop/blob/320636df084d377e1fdabef6fa6d28feb9ede6f2/src/Java.Interop/Java.Interop/JniRuntime.JniTypeManager.cs#L270-L275
The problem with this approach is that, at present, performance would be "not great", given that
JniTypeManager.GetType(JniTypeSignature)
uses LINQ.Constructor Invocation
Constructor Invocation is done in
ManagedPeer.Construct()
, which is invoked from Java:ManagedPeer.construct()
is given four separate values:java.lang.Object
instance that we need to construct the .NET side for:
-separated sequence of assembly qualified type names for the constructor signature to invoke(2) and (3) both involve
Type.GetType()
invocations.As with
ManagedPeer.RegisterNativeMembers()
, (2) could be replaced withJniEnvironment.Runtime.TypeManager.GetType()
, usingnew JniTypeSignature(JniEnvironment.Types.GetJniTypeNameFromInstance())
.(3) is more problematic.
We could resolve this through one of two mechanisms:
a. Use a JNI method signature, and parse that into
System.Type
instances at runtime. We would thus invokeManagedPeer.construct(this, "…now ignored…", "(I)V", …)
, and turn(I)V
intonew Type[]{typeof(int)}
, again viaJniEnvironment.Runtime.TypeManager.GetType()
.b. Stop using
ManagedPeer.construct()
entirely, and instead use a native method declaration.(b) would result in a Java Callable Wrapper akin to:
(a) has the benefit of being easier to implement:
jcw-gen
already has the JNI method signature, and could emit it. (a) also "feels like" it would have more runtime overhead.(b) has the benefit of (probably) being faster at runtime, but is more complex. It would requires changes to:
jcw-gen
(to declare the newnative
method)JniRuntime.JniTypeManager.RegisterNativeMembers()
overrides/implementations would need to figure out how to handle these newnative
methods..NET Android relies on
generator
-emitted methods as part ofRegisterNativeMembers()
, but I don't see how that approach could actually work here. We could pull inSystem.Reflection.Emit
/DynamicMethod
, but we're trying to get away from that.JavaInterop1 could have
jnimarshalmethod-gen
implement it, which has the added benefit of increased efficiency, but that's not a solution for .NET Android.There be complications here.
jnimarshalmethod-gen
(to emit & register these new methods), but only for Desktop Java.Base usage.Java.Interop.Export
The text was updated successfully, but these errors were encountered: