Skip to content

Commit

Permalink
[typemaps] Handle duplicate java to managed type maps properly
Browse files Browse the repository at this point in the history 
Fixes: dotnet#4596
Context: xamarin/monodroid@192b1f1
Context: https://github.com/xamarin/java.interop/blob/fc18c54b2ccf14f444fadac0a1e7e81f837cc470/src/Java.Interop.Tools.JavaCallableWrappers/Java.Interop.Tools.JavaCallableWrappers/TypeNameMapGenerator.cs#L149-L186

In some environments, `BundleTest.TestBundleIntegerArrayList2()` fails
when  using the commercial shared runtime:

	Test 'Xamarin.Android.RuntimeTests.BundleTest.TestBundleIntegerArrayList2' failed: System.MemberAccessException : Cannot create an instance of Android.Runtime.JavaList`1[T] because Type.ContainsGenericParameters is true.
	  at System.Reflection.RuntimeConstructorInfo.DoInvoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture)
	  at System.Reflection.RuntimeConstructorInfo.Invoke (System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture)
	  at System.Reflection.ConstructorInfo.Invoke (System.Object[] parameters)
	  at Java.Interop.TypeManager.CreateProxy (System.Type type, System.IntPtr handle, Android.Runtime.JniHandleOwnership transfer)
	  at Java.Interop.TypeManager.CreateInstance (System.IntPtr handle, Android.Runtime.JniHandleOwnership transfer, System.Type targetType)
	  at Java.Lang.Object.GetObject (System.IntPtr handle, Android.Runtime.JniHandleOwnership transfer, System.Type type)
	  at Java.Lang.Object._GetObject[T] (System.IntPtr handle, Android.Runtime.JniHandleOwnership transfer)
	  at Java.Lang.Object.GetObject[T] (System.IntPtr handle, Android.Runtime.JniHandleOwnership transfer)
	  at Android.OS.Bundle.Get (System.String key)
	  at Xamarin.Android.RuntimeTests.BundleTest.TestBundleIntegerArrayList2 ()
	  at (wrapper managed-to-native) System.Reflection.RuntimeMethodInfo.InternalInvoke(System.Reflection.RuntimeMethodInfo,object,object[],System.Exception&)
	  at System.Reflection.RuntimeMethodInfo.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture)

The problem appears to be rooted in ce2bc68.  In a pre-ce2bc689 world,
there were two separate sets of "typemap" files that the application
could use:

  * Java-to-managed mappings, in `typemap.jm`
  * Managed-to-Java mappings, in `typemap.mj`.

These files did *not* need to have the same number of entries!

	% unzip /Library/Frameworks/Xamarin.Android.framework/Versions/10.2.0.100/lib/xamarin.android/xbuild/Xamarin/Android/Mono.Android.DebugRuntime-debug.apk typemap.jm
	% unzip /Library/Frameworks/Xamarin.Android.framework/Versions/10.2.0.100/lib/xamarin.android/xbuild/Xamarin/Android/Mono.Android.DebugRuntime-debug.apk typemap.mj
	% strings typemap.jm | wc -l
	   10318
	% strings typemap.mj | wc -l
	   11956

The reason why they have a different number of entries is *aliasing*:
there can be *multiple* bindings for a given Java type.  The
managed-to-Java mappings can contain *all* of them; the
Java-to-managed mapping *must* pick Only One.

For example, [`java.util.ArrayList`][0] contains (at least?) *three* bindings:

  * [`Android.Runtime.JavaList`][1]
  * [`Android.Runtime.JavaList<T>`][2]
  * `Java.Util.ArrayList` (generated at build time)

In a pre-ce2bc689 world, this was "fine": we'd binary search each file
*separately* to find type mappings.

This changed in ce2bc68, as the typemap information was merged into a
*single* array in order to de-duplicate information.  This reduced
`.apk` size.

An unanticipated result of this is that the Java-to-managed mappings
can now contain *duplicate* keys, "as if" the original `typemap.jm`
had the contents:

  java/util/ArrayList
  Android.Runtime.JavaList, Mono.Android
  java/util/ArrayList
  Android.Runtime.JavaList`1, Mono.Android
  java/util/ArrayList
  Java.Util.ArrayLlist, Mono.Android

Whereas pre-ce2bc689 there would have only been the first entry.

"Sometimes" this duplication is "fine": the typemap search
"Just happens" to return the first entry, or the 3rd entry, and apps
continue to work as intended.

"Sometimes" it isn't: the typemap binary search finds the 2nd entry,
which is a generic type.  This results in attempting to instantiate a
generic type *without appropriate type parameters*, which results in
the aforementioned `MemberAccessException`.

Update typemap generation code in `Xamarin.Android.Build.Tasks` so that
all the duplicate Java type names will point to the same managed type
name.  Additionally, make sure we select the managed type in the same
fashion the old typemap generator in `Java.Interop` worked - prefer base
types to the derived ones if the type is an interface or an abstract
class.  The effect of this change is that no matter which entry
`binary_search` ends up selecting it will always return the same managed
type name for all the aliased Java types.

Additionally, update `TestBundleIntegerArrayList2()` so that it asserts
the runtime *type* of `obj` returned, asserting that it is in fact a
`JavaList` and not e.g. `Java.Util.ArrayList` or something.
(The linker could otherwise "change" things, and this assertion will
ensure that `JavaList` won't be linked away…)

[0]: https://developer.android.com/reference/java/util/ArrayList
[1]: https://github.com/xamarin/xamarin-android/blob/61f09bf2ef0c8f09550e2d77eb97f417432b315b/src/Mono.Android/Android.Runtime/JavaList.cs#L11-L13
[2]: https://github.com/xamarin/xamarin-android/blob/61f09bf2ef0c8f09550e2d77eb97f417432b315b/src/Mono.Android/Android.Runtime/JavaList.cs#L667-L668
  • Loading branch information
@grendello
grendello committed May 6, 2020
1 parent 7dab4ee commit e0936b2
Show file tree
Hide file tree
Showing 3 changed files with 63 additions and 15 deletions.
1 change: 1 addition & 0 deletions src/Mono.Android/Test/Android.OS/BundleTest.cs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ public void TestBundleIntegerArrayList()
Assert.NotNull (list, "'key' doesn't refer to a list of integers");
var obj = b.Get ("key");
Assert.NotNull (obj, "Missing 'key' in bundle");
Assert.IsTrue (obj is global::Android.Runtime.JavaList, "`obj` should be a JavaList!");
try {
list = b.GetIntegerArrayList ("key");
Assert.NotNull (list, "'key' doesn't refer to a list of integers after non-generic call");
Expand Down
6 changes: 3 additions & 3 deletions src/Xamarin.Android.Build.Tasks/Tasks/GenerateJavaStubs.cs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -174,7 +174,7 @@ void Run (DirectoryAssemblyResolver res)

// Step 2 - Generate type maps
// Type mappings need to use all the assemblies, always.
WriteTypeMappings (allJavaTypes);
WriteTypeMappings (allJavaTypes, cache);

var javaTypes = new List<TypeDefinition> ();
foreach (TypeDefinition td in allJavaTypes) {
Expand Down Expand Up @@ -399,10 +399,10 @@ void SaveResource (string resource, string filename, string destDir, Func<string
MonoAndroidHelper.CopyIfStringChanged (template, Path.Combine (destDir, filename));
}

void WriteTypeMappings (List<TypeDefinition> types)
void WriteTypeMappings (List<TypeDefinition> types, TypeDefinitionCache cache)
{
var tmg = new TypeMapGenerator ((string message) => Log.LogDebugMessage (message), SupportedAbis);
if (!tmg.Generate (Debug, SkipJniAddNativeMethodRegistrationAttributeScan, types, TypemapOutputDirectory, GenerateNativeAssembly, out ApplicationConfigTaskState appConfState))
if (!tmg.Generate (Debug, SkipJniAddNativeMethodRegistrationAttributeScan, types, cache, TypemapOutputDirectory, GenerateNativeAssembly, out ApplicationConfigTaskState appConfState))
throw new XamarinAndroidException (4308, Properties.Resources.XA4308);
GeneratedBinaryTypeMaps = tmg.GeneratedBinaryTypeMaps.ToArray ();
BuildEngine4.RegisterTaskObject (ApplicationConfigTaskState.RegisterTaskObjectKey, appConfState, RegisteredTaskObjectLifetime.Build, allowEarlyCollection: false);
Expand Down
71 changes: 59 additions & 12 deletions src/Xamarin.Android.Build.Tasks/Utilities/TypeMapGenerator.cs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
using System.Linq;
using System.Text;

using Java.Interop.Tools.Cecil;
using Mono.Cecil;

namespace Xamarin.Android.Tasks
Expand Down Expand Up @@ -76,6 +77,7 @@ internal sealed class TypeMapDebugEntry
public string ManagedLabel;
public int JavaIndex;
public int ManagedIndex;
public TypeDefinition TypeDefinition;
}

// Widths include the terminating nul character but not the padding!
Expand Down Expand Up @@ -125,7 +127,7 @@ void UpdateApplicationConfig (TypeDefinition javaType, ApplicationConfigTaskStat
}
}

public bool Generate (bool debugBuild, bool skipJniAddNativeMethodRegistrationAttributeScan, List<TypeDefinition> javaTypes, string outputDirectory, bool generateNativeAssembly, out ApplicationConfigTaskState appConfState)
public bool Generate (bool debugBuild, bool skipJniAddNativeMethodRegistrationAttributeScan, List<TypeDefinition> javaTypes, TypeDefinitionCache cache, string outputDirectory, bool generateNativeAssembly, out ApplicationConfigTaskState appConfState)
{
if (String.IsNullOrEmpty (outputDirectory))
throw new ArgumentException ("must not be null or empty", nameof (outputDirectory));
Expand All @@ -140,25 +142,26 @@ public bool Generate (bool debugBuild, bool skipJniAddNativeMethodRegistrationAt
string typemapsOutputDirectory = Path.Combine (outputDirectory, "typemaps");

if (debugBuild) {
return GenerateDebug (skipJniAddNativeMethodRegistrationAttributeScan, javaTypes, typemapsOutputDirectory, generateNativeAssembly, appConfState);
return GenerateDebug (skipJniAddNativeMethodRegistrationAttributeScan, javaTypes, cache, typemapsOutputDirectory, generateNativeAssembly, appConfState);
}

return GenerateRelease (skipJniAddNativeMethodRegistrationAttributeScan, javaTypes, typemapsOutputDirectory, appConfState);
}

bool GenerateDebug (bool skipJniAddNativeMethodRegistrationAttributeScan, List<TypeDefinition> javaTypes, string outputDirectory, bool generateNativeAssembly, ApplicationConfigTaskState appConfState)
bool GenerateDebug (bool skipJniAddNativeMethodRegistrationAttributeScan, List<TypeDefinition> javaTypes, TypeDefinitionCache cache, string outputDirectory, bool generateNativeAssembly, ApplicationConfigTaskState appConfState)
{
if (generateNativeAssembly)
return GenerateDebugNativeAssembly (skipJniAddNativeMethodRegistrationAttributeScan, javaTypes, outputDirectory, appConfState);
return GenerateDebugFiles (skipJniAddNativeMethodRegistrationAttributeScan, javaTypes, outputDirectory, appConfState);
return GenerateDebugNativeAssembly (skipJniAddNativeMethodRegistrationAttributeScan, javaTypes, cache, outputDirectory, appConfState);
return GenerateDebugFiles (skipJniAddNativeMethodRegistrationAttributeScan, javaTypes, cache, outputDirectory, appConfState);
}

bool GenerateDebugFiles (bool skipJniAddNativeMethodRegistrationAttributeScan, List<TypeDefinition> javaTypes, string outputDirectory, ApplicationConfigTaskState appConfState)
bool GenerateDebugFiles (bool skipJniAddNativeMethodRegistrationAttributeScan, List<TypeDefinition> javaTypes, TypeDefinitionCache cache, string outputDirectory, ApplicationConfigTaskState appConfState)
{
var modules = new Dictionary<string, ModuleDebugData> (StringComparer.Ordinal);
int maxModuleFileNameWidth = 0;
int maxModuleNameWidth = 0;

var javaDuplicates = new Dictionary<string, List<TypeMapDebugEntry>> (StringComparer.Ordinal);
foreach (TypeDefinition td in javaTypes) {
UpdateApplicationConfig (td, appConfState);
string moduleName = td.Module.Assembly.Name.Name;
Expand Down Expand Up @@ -186,6 +189,7 @@ bool GenerateDebugFiles (bool skipJniAddNativeMethodRegistrationAttributeScan, L
}

TypeMapDebugEntry entry = GetDebugEntry (td);
HandleDebugDuplicates (javaDuplicates, entry, td, cache);
if (entry.JavaName.Length > module.JavaNameWidth)
module.JavaNameWidth = (uint)entry.JavaName.Length + 1;

Expand All @@ -195,6 +199,7 @@ bool GenerateDebugFiles (bool skipJniAddNativeMethodRegistrationAttributeScan, L
module.JavaToManagedMap.Add (entry);
module.ManagedToJavaMap.Add (entry);
}
SyncDebugDuplicates (javaDuplicates);

foreach (ModuleDebugData module in modules.Values) {
PrepareDebugMaps (module);
Expand All @@ -217,18 +222,22 @@ bool GenerateDebugFiles (bool skipJniAddNativeMethodRegistrationAttributeScan, L
return true;
}

bool GenerateDebugNativeAssembly (bool skipJniAddNativeMethodRegistrationAttributeScan, List<TypeDefinition> javaTypes, string outputDirectory, ApplicationConfigTaskState appConfState)
bool GenerateDebugNativeAssembly (bool skipJniAddNativeMethodRegistrationAttributeScan, List<TypeDefinition> javaTypes, TypeDefinitionCache cache, string outputDirectory, ApplicationConfigTaskState appConfState)
{
var javaToManaged = new List<TypeMapDebugEntry> ();
var managedToJava = new List<TypeMapDebugEntry> ();

var javaDuplicates = new Dictionary<string, List<TypeMapDebugEntry>> (StringComparer.Ordinal);
foreach (TypeDefinition td in javaTypes) {
UpdateApplicationConfig (td, appConfState);

TypeMapDebugEntry entry = GetDebugEntry (td);
HandleDebugDuplicates (javaDuplicates, entry, td, cache);

javaToManaged.Add (entry);
managedToJava.Add (entry);
}
SyncDebugDuplicates (javaDuplicates);

var data = new ModuleDebugData {
EntryCount = (uint)javaToManaged.Count,
Expand All @@ -246,6 +255,39 @@ bool GenerateDebugNativeAssembly (bool skipJniAddNativeMethodRegistrationAttribu
return true;
}

void SyncDebugDuplicates (Dictionary<string, List<TypeMapDebugEntry>> javaDuplicates)
{
foreach (List<TypeMapDebugEntry> duplicates in javaDuplicates.Values) {
if (duplicates.Count < 2) {
continue;
}

TypeMapDebugEntry template = duplicates [0];
for (int i = 1; i < duplicates.Count; i++) {
duplicates[i].TypeDefinition = template.TypeDefinition;
duplicates[i].ManagedName = template.ManagedName;
}
}
}

void HandleDebugDuplicates (Dictionary<string, List<TypeMapDebugEntry>> javaDuplicates, TypeMapDebugEntry entry, TypeDefinition td, TypeDefinitionCache cache)
{
List<TypeMapDebugEntry> duplicates;

if (!javaDuplicates.TryGetValue (entry.JavaName, out duplicates)) {
javaDuplicates.Add (entry.JavaName, new List<TypeMapDebugEntry> { entry });
} else {
duplicates.Add (entry);
TypeMapDebugEntry oldEntry = duplicates[0];
if (td.IsAbstract || td.IsInterface || oldEntry.TypeDefinition.IsAbstract || oldEntry.TypeDefinition.IsInterface) {
if (td.IsAssignableFrom (oldEntry.TypeDefinition, cache)) {
oldEntry.TypeDefinition = td;
oldEntry.ManagedName = GetManagedTypeName (td);
}
}
}
}

void PrepareDebugMaps (ModuleDebugData module)
{
module.JavaToManagedMap.Sort ((TypeMapDebugEntry a, TypeMapDebugEntry b) => String.Compare (a.JavaName, b.JavaName, StringComparison.Ordinal));
Expand All @@ -261,6 +303,15 @@ void PrepareDebugMaps (ModuleDebugData module)
}

TypeMapDebugEntry GetDebugEntry (TypeDefinition td)
{
return new TypeMapDebugEntry {
JavaName = Java.Interop.Tools.TypeNameMappings.JavaNativeTypeManager.ToJniName (td),
ManagedName = GetManagedTypeName (td),
TypeDefinition = td,
};
}

string GetManagedTypeName (TypeDefinition td)
{
// This is necessary because Mono runtime will return to us type name with a `.` for nested types (not a
// `/` or a `+`. So, for instance, a type named `DefaultRenderer` found in the
Expand All @@ -277,17 +328,13 @@ TypeMapDebugEntry GetDebugEntry (TypeDefinition td)
//
string managedTypeName = td.FullName.Replace ('/', '+');

return new TypeMapDebugEntry {
JavaName = Java.Interop.Tools.TypeNameMappings.JavaNativeTypeManager.ToJniName (td),
ManagedName = $"{managedTypeName}, {td.Module.Assembly.Name.Name}",
};
return $"{managedTypeName}, {td.Module.Assembly.Name.Name}";
}

bool GenerateRelease (bool skipJniAddNativeMethodRegistrationAttributeScan, List<TypeDefinition> javaTypes, string outputDirectory, ApplicationConfigTaskState appConfState)
{
int assemblyId = 0;
int maxJavaNameLength = 0;
int maxModuleFileNameLength = 0;
var knownAssemblies = new Dictionary<string, int> (StringComparer.Ordinal);
var tempModules = new Dictionary<byte[], ModuleReleaseData> ();
Dictionary <AssemblyDefinition, int> moduleCounter = null;
Expand Down

0 comments on commit e0936b2

Please sign in to comment.