[Xamarin.Android.Build.Tasks] Response file support for AOT #2062

dellis1972 · 2018-08-14T11:04:11Z

Our AOT system has issues when a user uses
non ASCII characters and spaces on Windows.
We use GetShortPath to get the old DOS
8.3 short names of paths to get around
paths having spaces and unicode characters.

However on the latest versions of windows
GetShortPath seems to be supported only
on the main system drive (C:). Many of our
developers and CI will be building on other
drives. So we really need to support paths
with spaces and unicode.

This commit reworks the way we provide the
--aot pargument to the cross compilers so
that it actually works in those senarios.

The first thing was how the arguments were
parsed. mono uses the build in system
command line parser GetCommandLineW to parse
arguments. Given the following --aot argument

--aot=outfile="c:\Sandbox\repo\bin\BuildAotApplicationAndBundle AndÜmläüts_x86_True_True\obj\Release\libaot-Mono.Android.dll.so",asmwriter,mtriple=i686-linux-android,tool-prefix=C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\toolchains\x86-4.9\prebuilt\windows\bin\i686-linux-android-,ld-flags=C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\toolchains\x86-4.9\prebuilt\windows\lib\gcc\i686-linux-android\4.9.x\libgcc.a;C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\platforms\android-16\arch-x86\usr\lib\libc.so;C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\platforms\android-16\arch-x86\usr\lib\libm.so,temp-path="c:\Sandbox\repo\BuildAotApplicationAndBundle AndÜmläüts_x86_True_True\obj\Release\temp"

This ends up as the following arguments

--aot=outfile="c:\Sandbox\repo\bin\BuildAotApplicationAndBundle
AndÜmläüts_x86_True_True\obj\Release\libaot-Mono.Android.dll.so"
,asmwriter,mtriple=i686-linux-android,tool-prefix=C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\toolchains\x86-4.9\prebuilt\windows\bin\i686-linux-android-,ld-flags=C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\toolchains\x86-4.9\prebuilt\windows\lib\gcc\i686-linux-android\4.9.x\libgcc.a;C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\platforms\android-16\arch-x86\usr\lib\libc.so;C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\platforms\android-16\arch-x86\usr\lib\libm.so,temp-path="c:\Sandbox\repo\BuildAotApplicationAndBundle
AndÜmläüts_x86_True_True\obj\Release\temp"

As you can see the parameters have been split
where there is a space. The solution to this is
to double quote the ENTIRE argument and remove
any quotes within the parameter list like so

"--aot=outfile=c:\Sandbox\repo\bin\BuildAotApplicationAndBundle AndÜmläüts_x86_True_True\obj\Release\libaot-Mono.Android.dll.so,asmwriter,mtriple=i686-linux-android,tool-prefix=C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\toolchains\x86-4.9\prebuilt\windows\bin\i686-linux-android-,ld-flags=C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\toolchains\x86-4.9\prebuilt\windows\lib\gcc\i686-linux-android\4.9.x\libgcc.a;C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\platforms\android-16\arch-x86\usr\lib\libc.so;C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\platforms\android-16\arch-x86\usr\lib\libm.so,temp-path=c:\Sandbox\repo\BuildAotApplicationAndBundle AndÜmläüts_x86_True_True\obj\Release\temp"

This allows windows (and mac) to parse the paremters
correctly as one block.

There is another issue however. With the new argument
line above if the temp-path= path has a space in it mono
still has issues with that path. The good news is that we
can use some domain knowledge to reduce not only the
paths which need spaces but also the overall length of
the argument.

Because we know the cross compiler will be executed within
WorkingDirectory we can shorten any path which is within
that directory structure. WorkingDirectory is set to the
directory of the projects csproj file. So the following

E:\Some Project\My Project\obj\Release\aot\System.dll\

will become

obj\Release\aot\System.dll\

This will fix the issue with the temp-path= argument. So we end up
with something like this

"--aot=outfile=obj\Release\libaot-Mono.Android.dll.so,asmwriter,mtriple=i686-linux-android,tool-prefix=C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\toolchains\x86-4.9\prebuilt\windows\bin\i686-linux-android-,ld-flags=C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\toolchains\x86-4.9\prebuilt\windows\lib\gcc\i686-linux-android\4.9.x\libgcc.a;C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\platforms\android-16\arch-x86\usr\lib\libc.so;C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\platforms\android-16\arch-x86\usr\lib\libm.so,temp-path=obj\Release\temp"

However we might still start hitting the command line length limit
on windows. This is currently 246 characters. So depending on where
a user creates the project we might end up with a long command line.

To work around this issue we can make use of the new --response=
argument in mono. Instead of passing all the arguments to the
cross compiler we can instead write them to a file and pass the
path to that file as the --response= argument. This will reduce
our command line length to be within an acceptable range unless
the user creates a project in a very very deep directory structure.
So the final call will be something like

cross-arm --response="c:\Sandbox\repo\bin\BuildAotApplicationAndBundle AndÜmläüts_x86_True_True\obj\Release\Mono.Android.dll\response.txt"

Which works perfectly.

This commit also updates the BuildAotApplication and
BuildAotApplicationAndBundle unit tests to use paths with
both spaces and non-ascii characters to make sure we support
both of those senarios.

dellis1972 · 2018-08-14T11:05:00Z

@jonpryor according to @akoeplinger --response is only in 2018-06 and above. So we are dependent on #1836

jonpryor · 2019-02-11T19:18:41Z

Have we tested this code with paths containing spaces -- or $DEITY forbid -- ümlauts?

src/Xamarin.Android.Build.Tasks/Tasks/Aot.cs

dellis1972 · 2019-02-12T15:01:04Z

ok it looks like the mono response file stuff has problems..

Path separators on windows. Having just \ in a path on windows is removed by mono when it parses the response file. So we need to provide \\
Looks like it doesn't like non-ascii response files. When trying to use a UTF8 response file I get the following

  AdditionalNativeLibraryReferences: (TaskId:207)
  [AOT] --llvm --aot=outfile="c:\\Sandbox\\xamarin-android\\bin\\TestDebug\\temp\\BuildAotApplication AndÜmläüts_armeabi-v7a_True_True\\obj\\Release\\aot\\armeabi-v7a\\libaot-UnnamedProject.dll.so",asmwriter,mtriple=armv7-linux-gnueabi,tool-prefix=C:\\ProgramData\\Microsoft\\AndroidNDK\\android-ndk-r13b\\toolchains\\arm-linux-androideabi-4.9\\prebuilt\\windows\\bin\\arm-linux-androideabi-,ld-flags=C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\toolchains\arm-linux-androideabi-4.9\prebuilt\windows\lib\gcc\arm-linux-androideabi\4.9.x\libgcc.a;C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\platforms\android-16\arch-arm\usr\lib\libc.so;C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\platforms\android-16\arch-arm\usr\lib\libm.so,llvm-path=c:\\Sandbox\\xamarin-android\\bin\\Debug\\lib\\xamarin.android\\xbuild\\Xamarin\\Android,temp-path="c:\\Sandbox\\xamarin-android\\bin\\TestDebug\\temp\\BuildAotApplication AndÜmläüts_armeabi-v7a_True_True\\obj\\Release\\aot\\armeabi-v7a\\UnnamedProject.dll" "c:\\Sandbox\\xamarin-android\\bin\\TestDebug\\temp\\BuildAotApplication AndÜmläüts_armeabi-v7a_True_True\\obj\\Release\\android\\assets\\UnnamedProject.dll" (TaskId:207)
  [AOT] MONO_PATH="c:\Sandbox\xamarin-android\bin\TestDebug\temp\BuildAotApplication AndÜmläüts_armeabi-v7a_True_True\obj\Release\android\assets" MONO_ENV_OPTIONS="" c:\\Sandbox\\xamarin-android\\bin\\Debug\\lib\\xamarin.android\\xbuild\\Xamarin\\Android\\cross-arm --response="C:\Users\dean\AppData\Local\Temp\tmp607D.tmp" (TaskId:207)
  [AOT] --llvm --aot=outfile="c:\\Sandbox\\xamarin-android\\bin\\TestDebug\\temp\\BuildAotApplication AndÜmläüts_armeabi-v7a_True_True\\obj\\Release\\aot\\armeabi-v7a\\libaot-mscorlib.dll.so",asmwriter,mtriple=armv7-linux-gnueabi,tool-prefix=C:\\ProgramData\\Microsoft\\AndroidNDK\\android-ndk-r13b\\toolchains\\arm-linux-androideabi-4.9\\prebuilt\\windows\\bin\\arm-linux-androideabi-,ld-flags=C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\toolchains\arm-linux-androideabi-4.9\prebuilt\windows\lib\gcc\arm-linux-androideabi\4.9.x\libgcc.a;C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\platforms\android-16\arch-arm\usr\lib\libc.so;C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\platforms\android-16\arch-arm\usr\lib\libm.so,llvm-path=c:\\Sandbox\\xamarin-android\\bin\\Debug\\lib\\xamarin.android\\xbuild\\Xamarin\\Android,temp-path="c:\\Sandbox\\xamarin-android\\bin\\TestDebug\\temp\\BuildAotApplication AndÜmläüts_armeabi-v7a_True_True\\obj\\Release\\aot\\armeabi-v7a\\mscorlib.dll" "c:\\Sandbox\\xamarin-android\\bin\\TestDebug\\temp\\BuildAotApplication AndÜmläüts_armeabi-v7a_True_True\\obj\\Release\\android\\assets\\shrunk\\mscorlib.dll" (TaskId:207)
  [AOT] MONO_PATH="c:\Sandbox\xamarin-android\bin\TestDebug\temp\BuildAotApplication AndÜmläüts_armeabi-v7a_True_True\obj\Release\android\assets\shrunk" MONO_ENV_OPTIONS="" c:\\Sandbox\\xamarin-android\\bin\\Debug\\lib\\xamarin.android\\xbuild\\Xamarin\\Android\\cross-arm --response="C:\Users\dean\AppData\Local\Temp\tmp608E.tmp" (TaskId:207)
  [AOT] --llvm --aot=outfile="c:\\Sandbox\\xamarin-android\\bin\\TestDebug\\temp\\BuildAotApplication AndÜmläüts_armeabi-v7a_True_True\\obj\\Release\\aot\\armeabi-v7a\\libaot-Mono.Android.dll.so",asmwriter,mtriple=armv7-linux-gnueabi,tool-prefix=C:\\ProgramData\\Microsoft\\AndroidNDK\\android-ndk-r13b\\toolchains\\arm-linux-androideabi-4.9\\prebuilt\\windows\\bin\\arm-linux-androideabi-,ld-flags=C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\toolchains\arm-linux-androideabi-4.9\prebuilt\windows\lib\gcc\arm-linux-androideabi\4.9.x\libgcc.a;C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\platforms\android-16\arch-arm\usr\lib\libc.so;C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\platforms\android-16\arch-arm\usr\lib\libm.so,llvm-path=c:\\Sandbox\\xamarin-android\\bin\\Debug\\lib\\xamarin.android\\xbuild\\Xamarin\\Android,temp-path="c:\\Sandbox\\xamarin-android\\bin\\TestDebug\\temp\\BuildAotApplication AndÜmläüts_armeabi-v7a_True_True\\obj\\Release\\aot\\armeabi-v7a\\Mono.Android.dll" "c:\\Sandbox\\xamarin-android\\bin\\TestDebug\\temp\\BuildAotApplication AndÜmläüts_armeabi-v7a_True_True\\obj\\Release\\android\\assets\\shrunk\\Mono.Android.dll" (TaskId:207)
  [AOT] MONO_PATH="c:\Sandbox\xamarin-android\bin\TestDebug\temp\BuildAotApplication AndÜmläüts_armeabi-v7a_True_True\obj\Release\android\assets\shrunk" MONO_ENV_OPTIONS="" c:\\Sandbox\\xamarin-android\\bin\\Debug\\lib\\xamarin.android\\xbuild\\Xamarin\\Android\\cross-arm --response="C:\Users\dean\AppData\Local\Temp\tmp607E.tmp" (TaskId:207)
  [AOT] --llvm --aot=outfile="c:\\Sandbox\\xamarin-android\\bin\\TestDebug\\temp\\BuildAotApplication AndÜmläüts_armeabi-v7a_True_True\\obj\\Release\\aot\\armeabi-v7a\\libaot-System.Core.dll.so",asmwriter,mtriple=armv7-linux-gnueabi,tool-prefix=C:\\ProgramData\\Microsoft\\AndroidNDK\\android-ndk-r13b\\toolchains\\arm-linux-androideabi-4.9\\prebuilt\\windows\\bin\\arm-linux-androideabi-,ld-flags=C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\toolchains\arm-linux-androideabi-4.9\prebuilt\windows\lib\gcc\arm-linux-androideabi\4.9.x\libgcc.a;C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\platforms\android-16\arch-arm\usr\lib\libc.so;C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\platforms\android-16\arch-arm\usr\lib\libm.so,llvm-path=c:\\Sandbox\\xamarin-android\\bin\\Debug\\lib\\xamarin.android\\xbuild\\Xamarin\\Android,temp-path="c:\\Sandbox\\xamarin-android\\bin\\TestDebug\\temp\\BuildAotApplication AndÜmläüts_armeabi-v7a_True_True\\obj\\Release\\aot\\armeabi-v7a\\System.Core.dll" "c:\\Sandbox\\xamarin-android\\bin\\TestDebug\\temp\\BuildAotApplication AndÜmläüts_armeabi-v7a_True_True\\obj\\Release\\android\\assets\\shrunk\\System.Core.dll" (TaskId:207)
  [AOT] MONO_PATH="c:\Sandbox\xamarin-android\bin\TestDebug\temp\BuildAotApplication AndÜmläüts_armeabi-v7a_True_True\obj\Release\android\assets\shrunk" MONO_ENV_OPTIONS="" c:\\Sandbox\\xamarin-android\\bin\\Debug\\lib\\xamarin.android\\xbuild\\Xamarin\\Android\\cross-arm --response="C:\Users\dean\AppData\Local\Temp\tmp60A0.tmp" (TaskId:207)
  [AOT] --llvm --aot=outfile="c:\\Sandbox\\xamarin-android\\bin\\TestDebug\\temp\\BuildAotApplication AndÜmläüts_armeabi-v7a_True_True\\obj\\Release\\aot\\armeabi-v7a\\libaot-Java.Interop.dll.so",asmwriter,mtriple=armv7-linux-gnueabi,tool-prefix=C:\\ProgramData\\Microsoft\\AndroidNDK\\android-ndk-r13b\\toolchains\\arm-linux-androideabi-4.9\\prebuilt\\windows\\bin\\arm-linux-androideabi-,ld-flags=C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\toolchains\arm-linux-androideabi-4.9\prebuilt\windows\lib\gcc\arm-linux-androideabi\4.9.x\libgcc.a;C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\platforms\android-16\arch-arm\usr\lib\libc.so;C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\platforms\android-16\arch-arm\usr\lib\libm.so,llvm-path=c:\\Sandbox\\xamarin-android\\bin\\Debug\\lib\\xamarin.android\\xbuild\\Xamarin\\Android,temp-path="c:\\Sandbox\\xamarin-android\\bin\\TestDebug\\temp\\BuildAotApplication AndÜmläüts_armeabi-v7a_True_True\\obj\\Release\\aot\\armeabi-v7a\\Java.Interop.dll" "c:\\Sandbox\\xamarin-android\\bin\\TestDebug\\temp\\BuildAotApplication AndÜmläüts_armeabi-v7a_True_True\\obj\\Release\\android\\assets\\shrunk\\Java.Interop.dll" (TaskId:207)
  [AOT] MONO_PATH="c:\Sandbox\xamarin-android\bin\TestDebug\temp\BuildAotApplication AndÜmläüts_armeabi-v7a_True_True\obj\Release\android\assets\shrunk" MONO_ENV_OPTIONS="" c:\\Sandbox\\xamarin-android\\bin\\Debug\\lib\\xamarin.android\\xbuild\\Xamarin\\Android\\cross-arm --response="C:\Users\dean\AppData\Local\Temp\tmp608F.tmp" (TaskId:207)
  [AOT] --llvm --aot=outfile="c:\\Sandbox\\xamarin-android\\bin\\TestDebug\\temp\\BuildAotApplication AndÜmläüts_armeabi-v7a_True_True\\obj\\Release\\aot\\armeabi-v7a\\libaot-System.dll.so",asmwriter,mtriple=armv7-linux-gnueabi,tool-prefix=C:\\ProgramData\\Microsoft\\AndroidNDK\\android-ndk-r13b\\toolchains\\arm-linux-androideabi-4.9\\prebuilt\\windows\\bin\\arm-linux-androideabi-,ld-flags=C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\toolchains\arm-linux-androideabi-4.9\prebuilt\windows\lib\gcc\arm-linux-androideabi\4.9.x\libgcc.a;C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\platforms\android-16\arch-arm\usr\lib\libc.so;C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\platforms\android-16\arch-arm\usr\lib\libm.so,llvm-path=c:\\Sandbox\\xamarin-android\\bin\\Debug\\lib\\xamarin.android\\xbuild\\Xamarin\\Android,temp-path="c:\\Sandbox\\xamarin-android\\bin\\TestDebug\\temp\\BuildAotApplication AndÜmläüts_armeabi-v7a_True_True\\obj\\Release\\aot\\armeabi-v7a\\System.dll" "c:\\Sandbox\\xamarin-android\\bin\\TestDebug\\temp\\BuildAotApplication AndÜmläüts_armeabi-v7a_True_True\\obj\\Release\\android\\assets\\shrunk\\System.dll" (TaskId:207)
  [AOT] MONO_PATH="c:\Sandbox\xamarin-android\bin\TestDebug\temp\BuildAotApplication AndÜmläüts_armeabi-v7a_True_True\obj\Release\android\assets\shrunk" MONO_ENV_OPTIONS="" c:\\Sandbox\\xamarin-android\\bin\\Debug\\lib\\xamarin.android\\xbuild\\Xamarin\\Android\\cross-arm --response="C:\Users\dean\AppData\Local\Temp\tmp60A1.tmp" (TaskId:207)
  [AOT] --llvm --aot=outfile="c:\\Sandbox\\xamarin-android\\bin\\TestDebug\\temp\\BuildAotApplication AndÜmläüts_armeabi-v7a_True_True\\obj\\Release\\aot\\armeabi-v7a\\libaot-System.Runtime.Serialization.dll.so",asmwriter,mtriple=armv7-linux-gnueabi,tool-prefix=C:\\ProgramData\\Microsoft\\AndroidNDK\\android-ndk-r13b\\toolchains\\arm-linux-androideabi-4.9\\prebuilt\\windows\\bin\\arm-linux-androideabi-,ld-flags=C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\toolchains\arm-linux-androideabi-4.9\prebuilt\windows\lib\gcc\arm-linux-androideabi\4.9.x\libgcc.a;C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\platforms\android-16\arch-arm\usr\lib\libc.so;C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\platforms\android-16\arch-arm\usr\lib\libm.so,llvm-path=c:\\Sandbox\\xamarin-android\\bin\\Debug\\lib\\xamarin.android\\xbuild\\Xamarin\\Android,temp-path="c:\\Sandbox\\xamarin-android\\bin\\TestDebug\\temp\\BuildAotApplication AndÜmläüts_armeabi-v7a_True_True\\obj\\Release\\aot\\armeabi-v7a\\System.Runtime.Serialization.dll" "c:\\Sandbox\\xamarin-android\\bin\\TestDebug\\temp\\BuildAotApplication AndÜmläüts_armeabi-v7a_True_True\\obj\\Release\\android\\assets\\shrunk\\System.Runtime.Serialization.dll" (TaskId:207)
  [AOT] MONO_PATH="c:\Sandbox\xamarin-android\bin\TestDebug\temp\BuildAotApplication AndÜmläüts_armeabi-v7a_True_True\obj\Release\android\assets\shrunk" MONO_ENV_OPTIONS="" c:\\Sandbox\\xamarin-android\\bin\\Debug\\lib\\xamarin.android\\xbuild\\Xamarin\\Android\\cross-arm --response="C:\Users\dean\AppData\Local\Temp\tmp60B1.tmp" (TaskId:207)
  [aot-compiler stderr] This mono runtime is compiled for cross-compiling. Only the --aot option is supported. (TaskId:207)
  [aot-compiler stderr] This mono runtime is compiled for cross-compiling. Only the --aot option is supported. (TaskId:207)
  [aot-compiler stderr] This mono runtime is compiled for cross-compiling. Only the --aot option is supported. (TaskId:207)
  [aot-compiler stderr] This mono runtime is compiled for cross-compiling. Only the --aot option is supported. (TaskId:207)
  [aot-compiler stderr] This mono runtime is compiled for cross-compiling. Only the --aot option is supported. (TaskId:207)
  [aot-compiler stderr] This mono runtime is compiled for cross-compiling. Only the --aot option is supported. (TaskId:207)
  [aot-compiler stderr] This mono runtime is compiled for cross-compiling. Only the --aot option is supported. (TaskId:207)

jonpryor · 2019-02-14T13:10:07Z

Does AOT currently support ümlauts, without the response file?

If so, we'll need to wait for a mono-side fix for encoding purposes and sanity.

If not, we could remove the AOT+ümlauts test and merge, with the restriction that AOT only works with ASCII paths.

dellis1972 · 2019-02-19T12:03:00Z

@jonpryor ok so we currently handle non-ascii characters for AOT without any problems. So if we merged this it would be a breaking change for Windows. It seems to work without any problems on Mac :/

So it looks like this is not handling the case with unicode characters (on windows). I'm not sure how to fix that.

dellis1972 · 2019-02-19T16:51:26Z

ok some hopefully useful info

[AOT] --aot=outfile="c:\\Sandbox\\xamarin-android\\bin\\TestDebug\\temp\\BuildAotApplicationAndBundle AndÜmläüts_x86_False_True\\obj\\Release\\aot\\x86\\libaot-UnnamedProject.dll.so",asmwriter,mtriple=i686-linux-android,tool-prefix=C:\\ProgramData\\Microsoft\\AndroidNDK\\android-ndk-r13b\\toolchains\\x86-4.9\\prebuilt\\windows\\bin\\i686-linux-android-,ld-flags=,llvm-path=c:\\Sandbox\\xamarin-android\\bin\\Debug\\lib\\xamarin.android\\xbuild\\Xamarin\\Android,temp-path="c:\\Sandbox\\xamarin-android\\bin\\TestDebug\\temp\\BuildAotApplicationAndBundle AndÜmläüts_x86_False_True\\obj\\Release\\aot\\x86\\UnnamedProject.dll" "c:\\Sandbox\\xamarin-android\\bin\\TestDebug\\temp\\BuildAotApplicationAndBundle AndÜmläüts_x86_False_True\\obj\\Release\\android\\assets\\UnnamedProject.dll" (TaskId:217)
  [AOT] MONO_PATH="c:\Sandbox\xamarin-android\bin\TestDebug\temp\BuildAotApplicationAndBundle AndÜmläüts_x86_False_True\obj\Release\android\assets" MONO_ENV_OPTIONS="" c:\\Sandbox\\xamarin-android\\bin\\Debug\\lib\\xamarin.android\\xbuild\\Xamarin\\Android\\cross-x86 --response="C:\Users\dean\AppData\Local\Temp\tmp887A.tmp" (TaskId:217)

This then turns into the following after it has been read from the file. This is the result of a fprintf.

[aot-compiler stdout] got the following arguments--aot=outfile="c:\\Sandbox\\xamarin-android\\bin\\TestDebug\\temp\\BuildAotApplicationAndBundle And├£ml├ñ├╝ts_x86_False_True\\obj\\Release\\aot\\x86\\libaot-Java.Interop.dll.so",asmwriter,mtriple=i686-linux-android,tool-prefix=C:\\ProgramData\\Microsoft\\AndroidNDK\\android-ndk-r13b\\toolchains\\x86-4.9\\prebuilt\\windows\\bin\\i686-linux-android-,ld-flags=,llvm-path=c:\\Sandbox\\xamarin-android\\bin\\Debug\\lib\\xamarin.android\\xbuild\\Xamarin\\Android,temp-path="c:\\Sandbox\\xamarin-android\\bin\\TestDebug\\temp\\BuildAotApplicationAndBundle And├£ml├ñ├╝ts_x86_False_True\\obj\\Release\\aot\\x86\\Java.Interop.dll" "c:\\Sandbox\\xamarin-android\\bin\\TestDebug\\temp\\BuildAotApplicationAndBundle And├£ml├ñ├╝ts_x86_False_True\\obj\\Release\\android\\assets\\shrunk\\Java.Interop.dll" (TaskId:217)

jonpryor · 2019-02-20T03:30:56Z

This is the result of a fprintf.

Is this with fprintf() or printf()?

printf() will be writing to the console, and is almost certainly not going to support UTF-8; see also: https://stackoverflow.com/a/10032586/83444

The above stackoverflow answer makes it sound like fprintf() should work, but I'm not entirely certain that the C runtime on Windows won't screw around with string encoding and attempt to convert the string into the ANSI locale...

dellis1972 · 2019-02-20T10:00:26Z

Its the result of the following code

			gchar *text;
			gsize len;
			
			if (!g_file_get_contents (&argv[i][11], &text, &len, NULL)){
				fprintf (stderr, "The specified response file can not be read\n");
				exit (1);
			}
			fprintf (stdout, "got the following arguments");
			fprintf (stdout, text);
			mono_parse_options (text, &argc, &argv, FALSE);
			g_free (text);

So its just a normal fprintf to stdout.

dellis1972 · 2019-02-20T10:33:25Z

Added mono/mono#13082 and mono/mono#13081

dellis1972 · 2019-07-03T09:00:48Z

This PR has been updated to only add the Unit test. We are backing out the AOT response file changes until such time as mono properly processed the paths within the response file.

I'll be raising an issue on mono as soon as I can get a good repo.

dellis1972 · 2019-09-02T16:26:12Z

Sigh, So Windows tests pass locally..
On CI we get

error XA0000: Unhandled exception: System.ArgumentException: Illegal characters in path.

from the following path

E:\A\_work\275\s\bin\TestRelease\temp\BuildAotApplicationAndBundle AndÜmläüts_armeabi-v7a_True_True\obj\Release\android\assets

and guess what. Its because GetShortPath is disabled on that E: drive it seems 🤦‍♂
So we can't merge this until mono can handle longer paths and the response file.

dellis1972 · 2019-09-03T21:24:04Z

src/Xamarin.Android.Build.Tasks/Tasks/Aot.cs


-					string aotOptionsStr = (EnableLLVM ? "--llvm " : "") + "--aot=" + string.Join (",", aotOptions);
+					string aotOptionsStr = (EnableLLVM ? "--llvm " : "") + "\"--aot=" + string.Join (",", aotOptions) + "\"";


This is the important line. All of the options inside the --aot= argument do not need quotes as long as the entire argument is wrapped in quotes. e.g

"--aot=foo=bar bar,doh,cat=true"

dellis1972 · 2019-09-03T21:47:16Z

src/Xamarin.Android.Build.Tasks/Tasks/Aot.cs

@@ -361,6 +357,10 @@ IEnumerable<Config> GetAotConfigs ()
 				if (!Directory.Exists (outdir))
 					Directory.CreateDirectory (outdir);

+				if (outdir.StartsWith (WorkingDirectory, StringComparison.InvariantCultureIgnoreCase)) {


This bit is needed because some of the aot options still don't like spaces. However all of the files we create are in the IntermediateOuputPath. So we can strip off the WorkingDirectory part of the path since it won't be need (as the process is started in the WorkingDirectory.

The WorkingDirectory is normally the root of the project where the csproj is, and that is the part which will generally contain spaces and non-ascii characters.

dellis1972 · 2019-09-03T21:48:05Z

src/Xamarin.Android.Build.Tasks/Tasks/Aot.cs

@@ -173,14 +174,6 @@ static string GetNdkToolchainLibraryDir (string binDir, AndroidTargetArch arch)
 			return GetNdkToolchainLibraryDir (binDir, NdkUtil.GetArchDirName (arch));
 		}

-		static string GetShortPath (string path)


dellis1972 · 2019-09-04T13:13:31Z

Test failure is XA5300.

src/Xamarin.Android.Build.Tasks/Tasks/Aot.cs

Our AOT system has issues when a user uses non ASCII characters and spaces on Windows. We use `GetShortPath` to get the old DOS 8.3 short names of paths to get around paths having spaces and unicode characters. However on the latest versions of windows `GetShortPath` seems to be supported only on the main system drive (C:). Many of our developers and CI will be building on other drives. So we really need to support paths with spaces and unicode. This commit reworks the way we provide the `--aot` pargument to the cross compilers so that it actually works in those senarios. The first thing was how the arguments were parsed. `mono` uses the build in system command line parser `GetCommandLineW` to parse arguments. Given the following `--aot` argument --aot=outfile="c:\Sandbox\repo\bin\BuildAotApplicationAndBundle AndÜmläüts_x86_True_True\obj\Release\libaot-Mono.Android.dll.so",asmwriter,mtriple=i686-linux-android,tool-prefix=C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\toolchains\x86-4.9\prebuilt\windows\bin\i686-linux-android-,ld-flags=C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\toolchains\x86-4.9\prebuilt\windows\lib\gcc\i686-linux-android\4.9.x\libgcc.a;C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\platforms\android-16\arch-x86\usr\lib\libc.so;C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\platforms\android-16\arch-x86\usr\lib\libm.so,temp-path="c:\Sandbox\repo\BuildAotApplicationAndBundle AndÜmläüts_x86_True_True\obj\Release\temp" This ends up as the following arguments --aot=outfile="c:\Sandbox\repo\bin\BuildAotApplicationAndBundle AndÜmläüts_x86_True_True\obj\Release\libaot-Mono.Android.dll.so" ,asmwriter,mtriple=i686-linux-android,tool-prefix=C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\toolchains\x86-4.9\prebuilt\windows\bin\i686-linux-android-,ld-flags=C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\toolchains\x86-4.9\prebuilt\windows\lib\gcc\i686-linux-android\4.9.x\libgcc.a;C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\platforms\android-16\arch-x86\usr\lib\libc.so;C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\platforms\android-16\arch-x86\usr\lib\libm.so,temp-path="c:\Sandbox\repo\BuildAotApplicationAndBundle AndÜmläüts_x86_True_True\obj\Release\temp" As you can see the parameters have been split where there is a space. The solution to this is to double quote the ENTIRE argument and remove any quotes within the parameter list like so "--aot=outfile=c:\Sandbox\repo\bin\BuildAotApplicationAndBundle AndÜmläüts_x86_True_True\obj\Release\libaot-Mono.Android.dll.so,asmwriter,mtriple=i686-linux-android,tool-prefix=C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\toolchains\x86-4.9\prebuilt\windows\bin\i686-linux-android-,ld-flags=C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\toolchains\x86-4.9\prebuilt\windows\lib\gcc\i686-linux-android\4.9.x\libgcc.a;C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\platforms\android-16\arch-x86\usr\lib\libc.so;C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\platforms\android-16\arch-x86\usr\lib\libm.so,temp-path=c:\Sandbox\repo\BuildAotApplicationAndBundle AndÜmläüts_x86_True_True\obj\Release\temp" This allows windows (and mac) to parse the paremters correctly as one block. There is another issue however. With the new argument line above if the `temp=` path has a space in it `mono` still has issues with that path. The good news is that we can use some domain knowledge to reduce not only the paths which need spaces but also the overall length of the argument. Because we know the cross compiler will be executed within `WorkingDirectory` we can shorten any path which is within that directory structure. `WorkingDirectory` is set to the directory of the projects csproj file. So the following E:\Some Project\My Project\obj\Release\aot\System.dll\ will become obj\Release\aot\System.dll\ This will fix the issue with the `temp=` argument. So we end up with something like this "--aot=outfile=obj\Release\libaot-Mono.Android.dll.so,asmwriter,mtriple=i686-linux-android,tool-prefix=C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\toolchains\x86-4.9\prebuilt\windows\bin\i686-linux-android-,ld-flags=C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\toolchains\x86-4.9\prebuilt\windows\lib\gcc\i686-linux-android\4.9.x\libgcc.a;C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\platforms\android-16\arch-x86\usr\lib\libc.so;C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\platforms\android-16\arch-x86\usr\lib\libm.so,temp-path=obj\Release\temp" However we might still start hitting the command line length limit on windows. This is currently 246 characters. So depending on where a user creates the project we might end up with a long command line. To work around this issue we can make use of the new `--response=` argument in `mono`. Instead of passing all the arguments to the cross compiler we can instead write them to a file and pass the path to that file as the `--response=` argument. This will reduce our command line length to be within an acceptable range unless the user creates a project in a very very deep directory structure. So the final call will be something like cross-arm --response="c:\Sandbox\repo\bin\BuildAotApplicationAndBundle AndÜmläüts_x86_True_True\obj\Release\Mono.Android.dll\response.txt" Which works perfectly. This commit also updates the `BuildAotApplication` and `BuildAotApplicationAndBundle` unit tests to use paths with both spaces and non-ascii characters to make sure we support both of those senarios.

jonpryor · 2019-09-04T14:13:19Z

Commit message for squash-and-merge:

Fixes: https://devdiv.visualstudio.com/DevDiv/_workitems/edit/609244
Fixes: https://developercommunity.visualstudio.com/content/problem/554407/vs2019-xamarinandroid-aot-command.html
Fixes: https://github.com/xamarin/xamarin-android/issues/3407

Our AOT system has issues when a user uses non-ASCII characters and
spaces on Windows.  We used [`GetShortPathName()`][0] to get the old
DOS 8.3 short names of paths to get around paths having spaces and
unicode characters.

However, short path name generation can be [disabled][1], in which
case `GetShortPathName()` will return the long path name.
Consequently, builds can fail when spaces appear in unexpected places:

	[aot-compiler stderr] Cannot open assembly 'Files': No such file or directory.	

Short path name generation can be disabled on a drive-by-drive basis,
and our Azure DevOps build machines have short path name generation
disabled on the `E:` drive used for builds.

We really need to support paths with spaces and unicode characters.

Rework the way we provide the `--aot` argument to the cross compilers
so that it actually works in those scenarios.

The first thing was how the arguments were parsed.  `mono` uses the
built-in system command line parser [`CommandLineToArgv()`][2] to
parse arguments.  Given the following `--aot` argument

	--aot=outfile="c:\Sandbox\repo\bin\BuildAotApplicationAndBundle AndÜmläüts_x86_True_True\obj\Release\libaot-Mono.Android.dll.so",asmwriter,mtriple=i686-linux-android,tool-prefix=C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\toolchains\x86-4.9\prebuilt\windows\bin\i686-linux-android-,ld-flags=C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\toolchains\x86-4.9\prebuilt\windows\lib\gcc\i686-linux-android\4.9.x\libgcc.a;C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\platforms\android-16\arch-x86\usr\lib\libc.so;C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\platforms\android-16\arch-x86\usr\lib\libm.so,temp-path="c:\Sandbox\repo\BuildAotApplicationAndBundle AndÜmläüts_x86_True_True\obj\Release\temp"

This ends up as the following arguments, one per-line

	[0]: --aot=outfile="c:\Sandbox\repo\bin\BuildAotApplicationAndBundle
	[1]: AndÜmläüts_x86_True_True\obj\Release\libaot-Mono.Android.dll.so"
	[2]: ,asmwriter,mtriple=i686-linux-android,tool-prefix=C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\toolchains\x86-4.9\prebuilt\windows\bin\i686-linux-android-,ld-flags=C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\toolchains\x86-4.9\prebuilt\windows\lib\gcc\i686-linux-android\4.9.x\libgcc.a;C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\platforms\android-16\arch-x86\usr\lib\libc.so;C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\platforms\android-16\arch-x86\usr\lib\libm.so,temp-path="c:\Sandbox\repo\BuildAotApplicationAndBundle
	[3]: AndÜmläüts_x86_True_True\obj\Release\temp"

As you can see the parameters have been split wherever there is a
space.  The solution to this is to double quote the ENTIRE argument
and remove any quotes within the parameter list like:

	"--aot=outfile=c:\Sandbox\repo\bin\BuildAotApplicationAndBundle AndÜmläüts_x86_True_True\obj\Release\libaot-Mono.Android.dll.so,asmwriter,mtriple=i686-linux-android,tool-prefix=C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\toolchains\x86-4.9\prebuilt\windows\bin\i686-linux-android-,ld-flags=C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\toolchains\x86-4.9\prebuilt\windows\lib\gcc\i686-linux-android\4.9.x\libgcc.a;C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\platforms\android-16\arch-x86\usr\lib\libc.so;C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\platforms\android-16\arch-x86\usr\lib\libm.so,temp-path=c:\Sandbox\repo\BuildAotApplicationAndBundle AndÜmläüts_x86_True_True\obj\Release\temp"

This allows Windows (and mac) to parse the parameters correctly as
one value.

There is another issue however.  With the new argument line above, if
the `temp-path=` path has a space in it then `mono` still has issues
with that path.  The good news is that we can use some domain
knowledge to reduce not only the paths which need spaces but also the
overall length of the argument.  Because we know the cross compiler
will be executed within `WorkingDirectory` we can shorten any path
which is within that directory structure.  `WorkingDirectory` is set
to the directory of the projects `.csproj` file.  So the following:

	E:\Some Project\My Project\obj\Release\aot\System.dll

will become

	obj\Release\aot\System.dll

This will fix the issue with the `temp-path=` argument.  We end up
with something like this

	"--aot=outfile=obj\Release\libaot-Mono.Android.dll.so,asmwriter,mtriple=i686-linux-android,tool-prefix=C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\toolchains\x86-4.9\prebuilt\windows\bin\i686-linux-android-,ld-flags=C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\toolchains\x86-4.9\prebuilt\windows\lib\gcc\i686-linux-android\4.9.x\libgcc.a;C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\platforms\android-16\arch-x86\usr\lib\libc.so;C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\platforms\android-16\arch-x86\usr\lib\libm.so,temp-path=obj\Release\temp"

However we might also still start hitting the
[command line length limit][3] on Windows.  This is currently 8191
characters on Windows XP and later.  So depending on where a user
creates the project we might end up with a "too long" command line.

To work around this issue we can make use of the new `mono --response=`
argument.  Instead of passing all the arguments to the cross compiler
we can instead write them to a file and pass the path to that file as
the `--response=` argument.  This will reduce our command line length
to be within an acceptable range unless the user creates a project in
a very very deep directory structure.  The final call will be like:

	...\cross-arm --response="c:\Sandbox\repo\bin\BuildAotApplicationAndBundle AndÜmläüts_x86_True_True\obj\Release\Mono.Android.dll\response.txt"

which works perfectly.

This commit also updates the `BuildTest.BuildAotApplication()` and
`BuildTest.BuildAotApplicationAndBundle()` unit tests to use paths
with both spaces and non-ASCII characters to ensure we support both
of those scenarios.

[0]: https://docs.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-getshortpathnamew
[1]: https://support.microsoft.com/en-us/help/121007/how-to-disable-8-3-file-name-creation-on-ntfs-partitions
[2]: https://docs.microsoft.com/en-us/windows/win32/api/shellapi/nf-shellapi-commandlinetoargvw
[3]: https://support.microsoft.com/en-us/help/830473/command-prompt-cmd-exe-command-line-string-limitation

Fixes: https://devdiv.visualstudio.com/DevDiv/_workitems/edit/609244 Fixes: https://developercommunity.visualstudio.com/content/problem/554407/vs2019-xamarinandroid-aot-command.html Fixes: #3407 Our AOT system has issues when a user uses non-ASCII characters and spaces on Windows. We used [`GetShortPathName()`][0] to get the old DOS 8.3 short names of paths to get around paths having spaces and unicode characters. However, short path name generation can be [disabled][1], in which case `GetShortPathName()` will return the long path name. Consequently, builds can fail when spaces appear in unexpected places: [aot-compiler stderr] Cannot open assembly 'Files': No such file or directory. Short path name generation can be disabled on a drive-by-drive basis, and our Azure DevOps build machines have short path name generation disabled on the `E:` drive used for builds. We really need to support paths with spaces and unicode characters. Rework the way we provide the `--aot` argument to the cross compilers so that it actually works in those scenarios. The first thing was how the arguments were parsed. `mono` uses the built-in system command line parser [`CommandLineToArgv()`][2] to parse arguments. Given the following `--aot` argument --aot=outfile="c:\Sandbox\repo\bin\BuildAotApplicationAndBundle AndÜmläüts_x86_True_True\obj\Release\libaot-Mono.Android.dll.so",asmwriter,mtriple=i686-linux-android,tool-prefix=C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\toolchains\x86-4.9\prebuilt\windows\bin\i686-linux-android-,ld-flags=C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\toolchains\x86-4.9\prebuilt\windows\lib\gcc\i686-linux-android\4.9.x\libgcc.a;C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\platforms\android-16\arch-x86\usr\lib\libc.so;C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\platforms\android-16\arch-x86\usr\lib\libm.so,temp-path="c:\Sandbox\repo\BuildAotApplicationAndBundle AndÜmläüts_x86_True_True\obj\Release\temp" This ends up as the following arguments, one per-line [0]: --aot=outfile="c:\Sandbox\repo\bin\BuildAotApplicationAndBundle [1]: AndÜmläüts_x86_True_True\obj\Release\libaot-Mono.Android.dll.so" [2]: ,asmwriter,mtriple=i686-linux-android,tool-prefix=C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\toolchains\x86-4.9\prebuilt\windows\bin\i686-linux-android-,ld-flags=C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\toolchains\x86-4.9\prebuilt\windows\lib\gcc\i686-linux-android\4.9.x\libgcc.a;C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\platforms\android-16\arch-x86\usr\lib\libc.so;C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\platforms\android-16\arch-x86\usr\lib\libm.so,temp-path="c:\Sandbox\repo\BuildAotApplicationAndBundle [3]: AndÜmläüts_x86_True_True\obj\Release\temp" As you can see the parameters have been split wherever there is a space. The solution to this is to double quote the ENTIRE argument and remove any quotes within the parameter list like: "--aot=outfile=c:\Sandbox\repo\bin\BuildAotApplicationAndBundle AndÜmläüts_x86_True_True\obj\Release\libaot-Mono.Android.dll.so,asmwriter,mtriple=i686-linux-android,tool-prefix=C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\toolchains\x86-4.9\prebuilt\windows\bin\i686-linux-android-,ld-flags=C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\toolchains\x86-4.9\prebuilt\windows\lib\gcc\i686-linux-android\4.9.x\libgcc.a;C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\platforms\android-16\arch-x86\usr\lib\libc.so;C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\platforms\android-16\arch-x86\usr\lib\libm.so,temp-path=c:\Sandbox\repo\BuildAotApplicationAndBundle AndÜmläüts_x86_True_True\obj\Release\temp" This allows Windows (and mac) to parse the parameters correctly as one value. There is another issue however. With the new argument line above, if the `temp-path=` path has a space in it then `mono` still has issues with that path. The good news is that we can use some domain knowledge to reduce not only the paths which need spaces but also the overall length of the argument. Because we know the cross compiler will be executed within `WorkingDirectory` we can shorten any path which is within that directory structure. `WorkingDirectory` is set to the directory of the projects `.csproj` file. So the following: E:\Some Project\My Project\obj\Release\aot\System.dll\ will become obj\Release\aot\System.dll\ This will fix the issue with the `temp-path=` argument. We end up with something like this "--aot=outfile=obj\Release\libaot-Mono.Android.dll.so,asmwriter,mtriple=i686-linux-android,tool-prefix=C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\toolchains\x86-4.9\prebuilt\windows\bin\i686-linux-android-,ld-flags=C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\toolchains\x86-4.9\prebuilt\windows\lib\gcc\i686-linux-android\4.9.x\libgcc.a;C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\platforms\android-16\arch-x86\usr\lib\libc.so;C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\platforms\android-16\arch-x86\usr\lib\libm.so,temp-path=obj\Release\temp" However we might also still start hitting the [command line length limit][3] on Windows. This is currently 8191 characters on Windows XP and later. So depending on where a user creates the project we might end up with a "too long" command line. To work around this issue we can make use of the new `mono --response=` argument. Instead of passing all the arguments to the cross compiler we can instead write them to a file and pass the path to that file as the `--response=` argument. This will reduce our command line length to be within an acceptable range unless the user creates a project in a very very deep directory structure. The final call will be like: ...\cross-arm --response="c:\Sandbox\repo\bin\BuildAotApplicationAndBundle AndÜmläüts_x86_True_True\obj\Release\Mono.Android.dll\response.txt" which works perfectly. This commit also updates the `BuildTest.BuildAotApplication()` and `BuildTest.BuildAotApplicationAndBundle()` unit tests to use paths with both spaces and non-ASCII characters to ensure we support both of those scenarios. [0]: https://docs.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-getshortpathnamew [1]: https://support.microsoft.com/en-us/help/121007/how-to-disable-8-3-file-name-creation-on-ntfs-partitions [2]: https://docs.microsoft.com/en-us/windows/win32/api/shellapi/nf-shellapi-commandlinetoargvw [3]: https://support.microsoft.com/en-us/help/830473/command-prompt-cmd-exe-command-line-string-limitation

dellis1972 added the do-not-merge PR should not be merged. label Aug 14, 2018

dellis1972 changed the title ~~[Xamarin.Android.Build.Tasks] Use a Response file for AOT~~ [Xamarin.Android.Build.Tasks] Use a Response file for AOT [WIP] Aug 14, 2018

dellis1972 force-pushed the aotresponse branch from b244a42 to b53a5d8 Compare September 14, 2018 09:17

dellis1972 force-pushed the aotresponse branch from b53a5d8 to c75de5b Compare October 15, 2018 10:50

dellis1972 force-pushed the aotresponse branch from 0e31d14 to 4363ba4 Compare January 4, 2019 14:10

dellis1972 requested a review from jonathanpeppers as a code owner January 4, 2019 14:10

dellis1972 removed the do-not-merge PR should not be merged. label Jan 4, 2019

dellis1972 changed the title ~~[Xamarin.Android.Build.Tasks] Use a Response file for AOT [WIP]~~ [Xamarin.Android.Build.Tasks] Use a Response file for AOT Jan 4, 2019

dellis1972 force-pushed the aotresponse branch 2 times, most recently from e57ae38 to be371a9 Compare January 9, 2019 14:21

dellis1972 force-pushed the aotresponse branch 8 times, most recently from 949cf56 to 85c6ce4 Compare January 31, 2019 12:12

jonpryor reviewed Feb 11, 2019

View reviewed changes

src/Xamarin.Android.Build.Tasks/Tasks/Aot.cs Show resolved Hide resolved

dellis1972 force-pushed the aotresponse branch from 42fb802 to 1a62e69 Compare February 12, 2019 12:09

dellis1972 force-pushed the aotresponse branch from 1a62e69 to 14a422c Compare February 13, 2019 10:09

dellis1972 force-pushed the aotresponse branch 5 times, most recently from 9db409c to 3454de1 Compare July 3, 2019 08:57

dellis1972 changed the title ~~[Xamarin.Android.Build.Tasks] Use a Response file for AOT~~ [Xamarin.Android.Build.Tasks] Add new Unit test for AOT Jul 3, 2019

dellis1972 force-pushed the aotresponse branch from 3454de1 to 453d8b4 Compare August 1, 2019 12:12

jonathanpeppers mentioned this pull request Aug 8, 2019

Profiled AOT not possible on Windows when there are spaces in project path #3407

Closed

dellis1972 force-pushed the aotresponse branch from 453d8b4 to bcab81d Compare August 12, 2019 11:47

dellis1972 force-pushed the aotresponse branch from bcab81d to cf191a6 Compare August 23, 2019 09:18

dellis1972 force-pushed the aotresponse branch from cf191a6 to 46c16e6 Compare September 2, 2019 09:04

dellis1972 commented Sep 3, 2019

View reviewed changes

dellis1972 force-pushed the aotresponse branch 2 times, most recently from fc048c8 to ec3d04b Compare September 4, 2019 08:06

dellis1972 changed the title ~~[Xamarin.Android.Build.Tasks] Add new Unit test for AOT~~ [Xamarin.Android.Build.Tasks] Response file support for AOT Sep 4, 2019

dellis1972 requested a review from jonpryor September 4, 2019 08:45

jonathanpeppers reviewed Sep 4, 2019

View reviewed changes

src/Xamarin.Android.Build.Tasks/Tasks/Aot.cs Outdated Show resolved Hide resolved

dellis1972 force-pushed the aotresponse branch from ec3d04b to ce5a293 Compare September 4, 2019 14:08

jonpryor merged commit 8923c11 into dotnet:master Sep 4, 2019

pjcollins mentioned this pull request Sep 11, 2019

AOT compilation now fails when invoking strip on Windows #3619

Closed

bbbbbr mentioned this pull request Jul 15, 2023

Errors compiling on Windows with non-C:\ folder names that have spaces gbdk-2020/gbdk-2020#506

Closed

github-actions bot locked and limited conversation to collaborators Feb 3, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Xamarin.Android.Build.Tasks] Response file support for AOT #2062

[Xamarin.Android.Build.Tasks] Response file support for AOT #2062

dellis1972 commented Aug 14, 2018 •

edited

Loading

dellis1972 commented Aug 14, 2018

jonpryor commented Feb 11, 2019

dellis1972 commented Feb 12, 2019

jonpryor commented Feb 14, 2019

dellis1972 commented Feb 19, 2019 •

edited

Loading

dellis1972 commented Feb 19, 2019

jonpryor commented Feb 20, 2019

dellis1972 commented Feb 20, 2019

dellis1972 commented Feb 20, 2019

dellis1972 commented Jul 3, 2019

dellis1972 commented Sep 2, 2019 •

edited

Loading

dellis1972 Sep 3, 2019

dellis1972 Sep 3, 2019

dellis1972 Sep 3, 2019

dellis1972 commented Sep 4, 2019

jonpryor commented Sep 4, 2019 •

edited

Loading


		string aotOptionsStr = (EnableLLVM ? "--llvm " : "") + "--aot=" + string.Join (",", aotOptions);
		string aotOptionsStr = (EnableLLVM ? "--llvm " : "") + "\"--aot=" + string.Join (",", aotOptions) + "\"";

[Xamarin.Android.Build.Tasks] Response file support for AOT #2062

[Xamarin.Android.Build.Tasks] Response file support for AOT #2062

Conversation

dellis1972 commented Aug 14, 2018 • edited Loading

dellis1972 commented Aug 14, 2018

jonpryor commented Feb 11, 2019

dellis1972 commented Feb 12, 2019

jonpryor commented Feb 14, 2019

dellis1972 commented Feb 19, 2019 • edited Loading

dellis1972 commented Feb 19, 2019

jonpryor commented Feb 20, 2019

dellis1972 commented Feb 20, 2019

dellis1972 commented Feb 20, 2019

dellis1972 commented Jul 3, 2019

dellis1972 commented Sep 2, 2019 • edited Loading

dellis1972 Sep 3, 2019

Choose a reason for hiding this comment

dellis1972 Sep 3, 2019

Choose a reason for hiding this comment

dellis1972 Sep 3, 2019

Choose a reason for hiding this comment

dellis1972 commented Sep 4, 2019

jonpryor commented Sep 4, 2019 • edited Loading

dellis1972 commented Aug 14, 2018 •

edited

Loading

dellis1972 commented Feb 19, 2019 •

edited

Loading

dellis1972 commented Sep 2, 2019 •

edited

Loading

jonpryor commented Sep 4, 2019 •

edited

Loading