Skip to content

Commit d92eaf7

Browse files
[Xamarin.Android.Build.Tasks] generate R.java like Android Studio
Context: https://android.googlesource.com/platform/tools/base/+/refs/heads/master/build-system/builder/ Fixes: dotnet#2680 Fixes: dotnet#2836 The current behavior in the `_GenerateJavaDesignerForComponent` MSBuild target does the following: * For each library that has Android resources... (in parallel) * Run an instance of aapt/aapt2 to generate the `R.java` file for each library. * This actually creates an `R.java` file that contains *every* resource id for *every* library. These libraries are not using most of these ids. This has a few problems: * dotnet#2680 notes a problem where a file is locked on Windows during `_GenerateJavaDesignerForComponent`. Xamarin.Android.Common.targets(1541,2): The process cannot access the file 'C:\repos\msalnet\tests\devapps\XForms\XForms.Android\obj\Debug\90\lp\26\jl\manifest\AndroidManifest.xml' because it is being used by another process. at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath) at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost) at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost) at System.IO.StreamWriter.CreateFile(String path, Boolean append, Boolean checkHost) at System.IO.StreamWriter..ctor(String path, Boolean append, Encoding encoding, Int32 bufferSize, Boolean checkHost) at System.IO.StreamWriter..ctor(String path, Boolean append, Encoding encoding) at Xamarin.Android.Tasks.ManifestDocument.Save(String filename) at Xamarin.Android.Tasks.Aapt.GenerateCommandLineCommands(String ManifestFile, String currentAbi, String currentResourceOutputFile) at Xamarin.Android.Tasks.Aapt.ProcessManifest(ITaskItem manifestFile) at System.Threading.Tasks.Parallel.<>c__DisplayClass30_0`2.<ForEachWorker>b__0(Int32 i) at System.Threading.Tasks.Parallel.<>c__DisplayClass17_0`1.<ForWorker>b__1() at System.Threading.Tasks.Task.InnerInvoke() at System.Threading.Tasks.Task.InnerInvokeWithArg(Task childTask) at System.Threading.Tasks.Task.<>c__DisplayClass176_0.<ExecuteSelfReplicating>b__0(Object ) [C:\repos\msalnet\tests\devapps\XForms\XForms.Android\XForms.Android.csproj] * We are hugely contributing to the dex limit for fields. Apps contain exponentially more fields for each library with resources. An example from @PureWeen: 1> trouble writing output: Too many field references to fit in one dex file: 70468; max is 65536. * Quite a few instances of aapt/aapt2 startup on developer's machines: this pegs the CPU. We have had a few general complaints about it. Reviewing the source code for the Android gradle plugin, here is what they do: * Build the main app's "full" `R.txt` file. * For each library, load its `R.txt` file. * Map each resource in the library's `R.txt` back to the main app * Write a small `R.java` file for each library: containing *only* the lines from the `R.txt` and updated integer values from the main app `R.txt` file. Looking into this, we can do the exact same thing? We have the `R.txt` file one directory above where we extract resources for each library. We already had code parsing `R.txt` files I could repurpose, the only thing *new* is a `R.java` writer: a pretty simple port from java. The results are great! Before: 3173 ms _GenerateJavaDesignerForComponentAapt2 1 calls After: 20 ms GenerateLibraryResources 1 calls `_GenerateJavaDesignerForComponent` is now completely gone. This is a total savings of ~3 seconds on first build and incremental builds with library changes. To compare APKs, I used: $ ~/android-toolchain/sdk/tools/bin/apkanalyzer dex packages Xamarin.Forms_Performance_Integration-Before.apk | grep ^F Which omits a line for each field such as: F d 0 0 16 xamarin.forms_performance_integration.R$color int abc_background_cache_hint_selector_material_dark So then, before these changes: $ ~/android-toolchain/sdk/tools/bin/apkanalyzer dex packages Xamarin.Forms_Performance_Integration-Before.apk | grep ^F | wc -l 29681 After: $ ~/android-toolchain/sdk/tools/bin/apkanalyzer dex packages Xamarin.Forms_Performance_Integration-After.apk | grep ^F | wc -l 17210 12K less fields in a "Hello World" Xamarin.Forms app! Comparing file sizes seems good, too: $ zipinfo Xamarin.Forms_Performance_Integration-Before.apk | grep classes.dex -rw-rw-r-- 6.3 unx 3657872 b- defX 19-Mar-28 16:37 classes.dex $ zipinfo Xamarin.Forms_Performance_Integration-After.apk | grep classes.dex -rw-rw-r-- 6.3 unx 3533120 b- defX 19-Mar-28 16:20 classes.dex Dex file in the APK is ~120KB smaller.
1 parent 7f60e1f commit d92eaf7

File tree

6 files changed

+383
-76
lines changed

6 files changed

+383
-76
lines changed

src/Xamarin.Android.Build.Tasks/MSBuild/Xamarin/Android/Xamarin.Android.Aapt2.targets

-28
Original file line numberDiff line numberDiff line change
@@ -138,32 +138,4 @@ Copyright (C) 2011-2012 Xamarin. All rights reserved.
138138
</ItemGroup>
139139
</Target>
140140

141-
<Target Name="_GenerateJavaDesignerForComponentAapt2"
142-
Condition=" '$(_AndroidUseAapt2)' == 'True' "
143-
Inputs="@(_AdditonalAndroidResourceCacheFiles);@(_LibraryResourceDirectoryStamps);$(_AndroidResgenFlagFile)"
144-
Outputs="$(_AndroidComponentResgenFlagFile)">
145-
<!-- Run aapt to generate R.java for additional Android resources-->
146-
<Aapt2Link
147-
Condition=" '$(_AndroidUseAapt2)' == 'True' "
148-
ContinueOnError="$(DesignTimeBuild)"
149-
AdditionalAndroidResourcePaths="@(_LibraryResourceHashDirectories)"
150-
AdditionalResourceArchives="@(_LibraryResourceHashDirectories->'$(_AndroidLibraryFlatArchivesDirectory)%(Hash).flata')"
151-
ApplicationName="$(_AndroidPackage)"
152-
AssemblyIdentityMapFile="$(_AndroidLibrayProjectAssemblyMapFile)"
153-
CompiledResourceFlatArchive="$(_AndroidLibraryFlatArchivesDirectory)\compiled.flata"
154-
ExtraArgs="$(AndroidAapt2LinkExtraArgs)"
155-
ImportsDirectory="$(_LibraryProjectImportsDirectoryName)"
156-
JavaDesignerOutputDirectory="$(IntermediateOutputPath)android\src"
157-
JavaPlatformJarPath="$(JavaPlatformJarPath)"
158-
ManifestFiles="@(_AdditonalAndroidResourceCachePaths->'%(Identity)\AndroidManifest.xml');@(LibraryResourceDirectories->'%(Identity)\..\AndroidManifest.xml')"
159-
OutputImportDirectory="$(_AndroidLibrayProjectIntermediatePath)"
160-
ResourceNameCaseMap="$(_AndroidResourceNameCaseMap)"
161-
ResourceDirectories="$(MonoAndroidResDirIntermediate)"
162-
ToolPath="$(Aapt2ToolPath)"
163-
ToolExe="$(Aapt2ToolExe)"
164-
UseShortFileNames="$(UseShortFileNames)"
165-
YieldDuringToolExecution="$(YieldDuringToolExecution)"
166-
/>
167-
<Touch Files="$(_AndroidComponentResgenFlagFile)" AlwaysCreate="True" />
168-
</Target>
169141
</Project>
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,200 @@
1+
using Microsoft.Build.Framework;
2+
using System;
3+
using System.Collections.Generic;
4+
using System.IO;
5+
using System.Text;
6+
using Xamarin.Android.Tools;
7+
using Xamarin.Build;
8+
using ThreadingTasks = System.Threading.Tasks;
9+
10+
namespace Xamarin.Android.Tasks
11+
{
12+
/// <summary>
13+
/// We used to invoke aapt/aapt2 per library (many times!), this task does the work to generate R.java for libraries without calling aapt/aapt2.
14+
/// </summary>
15+
public class GenerateLibraryResources : AsyncTask
16+
{
17+
/// <summary>
18+
/// The main R.txt for the app
19+
/// </summary>
20+
[Required]
21+
public string ResourceSymbolsTextFile { get; set; }
22+
23+
/// <summary>
24+
/// The output directory for Java source code, such as: $(IntermediateOutputPath)android\src
25+
/// </summary>
26+
[Required]
27+
public string OutputDirectory { get; set; }
28+
29+
/// <summary>
30+
/// The list of R.txt files for each library
31+
/// </summary>
32+
public string [] LibraryTextFiles { get; set; }
33+
34+
/// <summary>
35+
/// The accompanying manifest file for each library
36+
/// </summary>
37+
public string [] ManifestFiles { get; set; }
38+
39+
public override bool Execute ()
40+
{
41+
Yield ();
42+
try {
43+
var task = ThreadingTasks.Task.Run (() => {
44+
DoExecute ();
45+
}, CancellationToken);
46+
47+
task.ContinueWith (Complete);
48+
49+
base.Execute ();
50+
} finally {
51+
Reacquire ();
52+
}
53+
54+
return !Log.HasLoggedErrors;
55+
}
56+
57+
string output_directory;
58+
Dictionary<Tuple<string, string>, string> main_r_txt;
59+
60+
void DoExecute ()
61+
{
62+
if (LibraryTextFiles == null || LibraryTextFiles.Length == 0)
63+
return;
64+
65+
main_r_txt = new Dictionary<Tuple<string, string>, string> ();
66+
using (var reader = File.OpenText (ResourceSymbolsTextFile)) {
67+
foreach (var line in ParseFile (reader)) {
68+
var key = new Tuple<string, string> (line [Index.Class], line [Index.Name]);
69+
main_r_txt [key] = line [Index.Value];
70+
}
71+
}
72+
73+
Directory.CreateDirectory (OutputDirectory);
74+
//So we don't need to consider WorkingDirectory on background threads
75+
output_directory = Path.GetFullPath (OutputDirectory);
76+
77+
ThreadingTasks.ParallelOptions options = new ThreadingTasks.ParallelOptions {
78+
CancellationToken = CancellationToken,
79+
TaskScheduler = ThreadingTasks.TaskScheduler.Default,
80+
};
81+
82+
ThreadingTasks.Parallel.For (0, LibraryTextFiles.Length, options, ProcessDirectory);
83+
}
84+
85+
void ProcessDirectory (int index)
86+
{
87+
var r_txt = LibraryTextFiles [index];
88+
if (!File.Exists (r_txt)) {
89+
LogDebugMessage ($"Skipping, R.txt does not exist: {r_txt}");
90+
return;
91+
}
92+
93+
var manifestFile = ManifestFiles [index];
94+
if (!File.Exists (manifestFile)) {
95+
LogDebugMessage ($"Skipping, AndroidManifest.xml does not exist: {manifestFile}");
96+
return;
97+
}
98+
99+
var manifest = AndroidAppManifest.Load (manifestFile, MonoAndroidHelper.SupportedVersions);
100+
101+
using (var memory = new MemoryStream ())
102+
using (var writer = new StreamWriter (memory, Encoding)) {
103+
// This code is based on the Android gradle plugin
104+
// https://android.googlesource.com/platform/tools/base/+/908b391a9c006af569dfaff08b37f8fdd6c4da89/build-system/builder/src/main/java/com/android/builder/internal/SymbolWriter.java
105+
106+
writer.WriteLine ("/* AUTO-GENERATED FILE. DO NOT MODIFY.");
107+
writer.WriteLine (" *");
108+
writer.WriteLine (" * This class was automatically generated by");
109+
writer.WriteLine (" * Xamarin.Android from the resource data it found.");
110+
writer.WriteLine (" * It should not be modified by hand.");
111+
writer.WriteLine (" */");
112+
113+
writer.Write ("package ");
114+
writer.Write (manifest.PackageName);
115+
writer.WriteLine (';');
116+
writer.WriteLine ();
117+
writer.WriteLine ("public final class R {");
118+
119+
using (var reader = File.OpenText (r_txt)) {
120+
string currentClass = null;
121+
foreach (var line in ParseFile (reader)) {
122+
var type = line [Index.Type];
123+
var clazz = line [Index.Class];
124+
var name = line [Index.Name];
125+
var key = new Tuple<string, string> (clazz, name);
126+
if (main_r_txt.TryGetValue (key, out string value)) {
127+
if (clazz != currentClass) {
128+
// If not the first inner class
129+
if (currentClass != null) {
130+
writer.WriteLine ("\t}");
131+
}
132+
133+
currentClass = clazz;
134+
writer.Write ("\tpublic static final class ");
135+
writer.Write (currentClass);
136+
writer.WriteLine (" {");
137+
}
138+
139+
writer.Write ("\t\tpublic static final ");
140+
writer.Write (type);
141+
writer.Write (' ');
142+
writer.Write (name);
143+
writer.Write (" = ");
144+
// It may be an int[]
145+
if (value.StartsWith ("{", StringComparison.Ordinal)) {
146+
writer.Write ("new ");
147+
writer.Write (type);
148+
writer.Write (' ');
149+
}
150+
writer.Write (value);
151+
writer.WriteLine (';');
152+
} else {
153+
LogDebugMessage ($"{r_txt}: `{type} {clazz} {name}` value not found");
154+
}
155+
}
156+
157+
// If we wrote at least one inner class
158+
if (currentClass != null) {
159+
writer.WriteLine ("\t}");
160+
}
161+
writer.WriteLine ('}');
162+
}
163+
164+
writer.Flush ();
165+
var r_java = Path.Combine (output_directory, manifest.PackageName.Replace ('.', Path.DirectorySeparatorChar), "R.java");
166+
if (MonoAndroidHelper.CopyIfStreamChanged (memory, r_java)) {
167+
LogDebugMessage ($"Writing: {r_java}");
168+
} else {
169+
LogDebugMessage ($"Up to date: {r_java}");
170+
}
171+
}
172+
}
173+
174+
static readonly Encoding Encoding = new UTF8Encoding (encoderShouldEmitUTF8Identifier: false);
175+
static readonly char [] Delimiter = new [] { ' ' };
176+
177+
class Index
178+
{
179+
public const int Type = 0;
180+
public const int Class = 1;
181+
public const int Name = 2;
182+
public const int Value = 3;
183+
}
184+
185+
/// <summary>
186+
/// R.txt is of the format:
187+
/// int id icon 0x7f0c000a
188+
/// int[] styleable ViewStubCompat { 0x010100d0, 0x010100f2, 0x010100f3 }
189+
/// This returns a 4-length string[] of the parts.
190+
/// </summary>
191+
IEnumerable<string []> ParseFile (StreamReader reader)
192+
{
193+
while (!reader.EndOfStream) {
194+
var line = reader.ReadLine ();
195+
var items = line.Split (Delimiter, 4);
196+
yield return items;
197+
}
198+
}
199+
}
200+
}

0 commit comments

Comments
 (0)