Certain projects are failing to build with AOT and LLVM enabled against d15-6 on Windows

[pjcollins]

I'm seeing a new failure on Windows when compiling certain projects with both AOT and LLVM. The same error does not reproduce for me on macOS, and appears to be newly introduced in 15.6.

Steps to Reproduce

1. Download ApiDemo - https://github.com/xamarin/monodroid-samples/tree/master/ApiDemo
2. msbuild /p:Configuration=Release /t:SignAndroidPackage /p:AotAssemblies=true /p:EnableLlvm=true ApiDemo.csproj

Expected Behavior

The project builds successfully with AOT and LLVM enabled.

Actual Behavior

[aot-compiler stdout] Mono Ahead of Time compiler - compiling assembly C:\agent\_work\r2\a\monodroid-samples\ApiDemo\obj\Release\android\assets\shrunk\Mono.Android.Export.dll (TaskId:348)
  [aot-compiler stdout] AOTID 9B00228E-5ECD-7F32-1753-92A543C073ED (TaskId:348)
  [aot-compiler stdout] Executing opt: "C:\PROGRA~2\MICROS~2\2017\ENTERP~1\MSBuild\Xamarin\Android\opt" -f -targetlibinfo -no-aa -basicaa -notti -instcombine -simplifycfg -inline-cost -inline -sroa -domtree -early-cse -lazy-value-info -correlated-propagation -simplifycfg -instcombine -simplifycfg -reassociate -domtree -loops -loop-simplify -lcssa -loop-rotate -licm -lcssa -loop-unswitch -instcombine -scalar-evolution -loop-simplify -lcssa -indvars -loop-idiom -loop-deletion -loop-unroll -memdep -gvn -memdep -memcpyopt -sccp -instcombine -lazy-value-info -correlated-propagation -domtree -memdep -adce -simplifycfg -instcombine -strip-dead-prototypes -domtree -verify -o "C:\agent\_work\r2\a\MONODR~2\ApiDemo\obj\Release\aot\ARM64-~1\Mono.Android.Export.dll\temp.opt.bc" "C:\agent\_work\r2\a\MONODR~2\ApiDemo\obj\Release\aot\ARM64-~1\Mono.Android.Export.dll\temp.bc" (TaskId:348)
  [aot-compiler stdout] Executing llc: "C:\PROGRA~2\MICROS~2\2017\ENTERP~1\MSBuild\Xamarin\Android\llc"  -asm-verbose=false -mtriple=aarch64-linux-android -disable-gnu-eh-frame -enable-mono-eh-frame -mono-eh-frame-symbol=mono_aot_Mono_Android_Export_eh_frame -relocation-model=pic -o "C:\agent\_work\r2\a\MONODR~2\ApiDemo\obj\Release\aot\ARM64-~1\Mono.Android.Export.dll\temp-llvm.s" "C:\agent\_work\r2\a\MONODR~2\ApiDemo\obj\Release\aot\ARM64-~1\Mono.Android.Export.dll\temp.opt.bc" (TaskId:348)
  [aot-compiler stderr] Stack dump: (TaskId:348)
  [aot-compiler stderr] 0.	Program arguments: C:\PROGRA~2\MICROS~2\2017\ENTERP~1\MSBuild\Xamarin\Android\llc -asm-verbose=false -mtriple=aarch64-linux-android -disable-gnu-eh-frame -enable-mono-eh-frame -mono-eh-frame-symbol=mono_aot_MonoDroid_ApiDemo_eh_frame -relocation-model=pic -o C:\agent\_work\r2\a\MONODR~2\ApiDemo\obj\Release\aot\ARM64-~1\MonoDroid.ApiDemo.dll\temp-llvm.s C:\agent\_work\r2\a\MONODR~2\ApiDemo\obj\Release\aot\ARM64-~1\MonoDroid.ApiDemo.dll\temp.opt.bc  (TaskId:348)
  [aot-compiler stderr] 1.	Running pass 'Function Pass Manager' on module 'C:\agent\_work\r2\a\MONODR~2\ApiDemo\obj\Release\aot\ARM64-~1\MonoDroid.ApiDemo.dll\temp.opt.bc'. (TaskId:348)
  [aot-compiler stderr] 2.	Running pass 'Greedy Register Allocator' on function '@MonoDroid_ApiDemo_ApiDemo_GetDemoActivities_string' (TaskId:348)
  [aot-compiler stderr] AOT of image C:\agent\_work\r2\a\monodroid-samples\ApiDemo\obj\Release\android\assets\MonoDroid.ApiDemo.dll failed. (TaskId:348)

Version Information

https://gist.github.com/pjcollins/842d3a55d49841a4fa955cd403af1549
Android NDK 15.2.4203891

Log File

http://xqa.blob.core.windows.net/gist/report-7d4d56fee1504d14818b0cef6538694e.txt

[kumpera]

Log output for the AOT task looks incomplete.

Could you manually run the AOT commands as stated in the log files and provide the output?
You might have to change project settings to keep temp files.

[vargaz]

Could you upload the obj/Release directory somewhere ?

[JKennedy24]

also seeing this on my automated build machine after updating to VS 15.6

[pjcollins]

Another item to note is that this does not occur when the project only supports armeabi-v7a, from what I have seen so for I need to also enable arm64-v8a to reproduce.

@vargaz I've uploaded the intermediate output here.

@kumpera Is there a way for me to get more output aside from msbuild /v:diag? If not, should I file a new issue for enhancing error output for the AOT task for cases such as this? I am assuming that we are not getting any additional useful output because llc.exe is crashing after invoking the first failing command such as:

 C:\PROGRA~2\MICROS~2\2017\ENTERP~1\MSBuild\Xamarin\Android\llc -asm-verbose=false -mtriple=aarch64-linux-android -disable-gnu-eh-frame -enable-mono-eh-frame -mono-eh-frame-symbol=mono_aot_System_eh_frame -relocation-model=pic -o C:\Users\XAMARI~1\git\MONODR~1\ApiDemo\obj\Release\aot\ARM64-~1\System.dll\temp-llvm.s C:\Users\XAMARI~1\git\MONODR~1\ApiDemo\obj\Release\aot\ARM64-~1\System.dll\temp.opt.bc 

Results in llc.exe aborting. VS also hangs and then reboots if I reproduce this from within the IDE:

The thread 0x2240 has exited with code 0 (0x0).
Unhandled exception at 0x0091DF49 in llc.exe: 0xC0000005: Access violation reading location 0x00000010.
Exception thrown at 0x77935BBE (ntdll.dll) in llc.exe: 0xC0000005: Access violation writing location 0xFFFFFFFC.

[vargaz]

I can't reproduce on osx with the files from the Release directory, so it might be a problem with the llc executable itself, not with the input to it.

[vargaz]

The same file causes the llc.exe from 15-6 to fail, while the llc.exe from 15-5 works fine. The llvm version in 15-5/15-6 appears to be the same, so it might be a change in the way llc.exe is compiled.

[jonpryor]

@vargaz: The MXE version was bumped between d15-5 and d15-6.

Perhaps this is an MXE and/or compiler bug?

[jonpryor]

@dellis1972: Don't we have any tests which try building with AOT+LLVM enabled?

If we don't, please add one. :-)

[vargaz]

Thats most likely the culprit.

[jaykrell]

Is this "c000005" rendition of the problem easy to setup and run on Windows?
Under a debugger this will stop right away, and we have reverse debugging..
This is equivalent to sigsegv.

[jaykrell]

ps: can someone teach me how to run/repro it?

[dellis1972]

@jonpryor we do but under certain conditions they are ignored, See https://github.com/xamarin/xamarin-android/blob/master/src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/BuildTest.cs#L348

[jonpryor]

@dellis1972: Thank you. Running the cross-compiler tests only when the cross-compilers exist should be fine -- it means they'll be skipped on default PR builds but executed on master builds.

Now I have to wonder why ApiDemo fails but our default project doesn't fail... Must be something in the IL?

[pjcollins]

@jonpryor this is also a windows specific failure, I am not sure if the msbuild suite is running everything on Windows quite yet?

[vargaz]

Tried rebasing our mxe based on mxe upstream master, and compiling llvm using that version, but the resulting llc.exe fails the same way, both the 32 bit and the 64 bit version.

[jonpryor]

@pjcollins: That is correct: our VSTS+xamarin-android build is not yet running MSBuild Task unit tests.

[jonpryor]

@vargaz: so we either figure out what's wrong in MXE and fix it, or we revert to the last used MXE -- which means we cannot possibly build with newer Xcode.app versions.

:-/

(I think -- but have not yet had a chance to verify -- that a newer MXE is also required in order to build a newer libzip; see PR #863 and 91b3430, which reverted PR #863.)

[vargaz]

Tried compiling llvm with -O2 and -O1 too, the crash still happens.

[vargaz]

"Good news", compiling llvm on osx using the gcc from homebrew can be used to reproduce the crash, so it happens on osx too when using gcc.

[vargaz]

llvm testcase:

<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
@mono_aot_Mono_Android_Export_llvm_got = hidden global [516 x i64*] zeroinitializer

declare hidden i64* @.Lp_6_plt_wrapper_alloc_object_AllocSmall_intptr_intptr_llvm(i64, i64)

declare hidden i64* @.Lp_69_plt_System_Reflection_Emit_ModuleBuilder_DefineType_string_System_Reflection_TypeAttributes_System_Type_System_Type___llvm(i64*, i64*, i32, i64*, i64*)

; Function Attrs: uwtable
define hidden cc16 void @Mono_CodeGeneration_CodeClass__ctor_System_Reflection_Emit_ModuleBuilder_string_System_Reflection_TypeAttributes_System_Type_System_Type__(i64* %this, i64* %arg_mb, i64* %arg_name, i32 %arg_attr, i64* %arg_baseType, i64* %arg_interfaces) #3 {
BB0:
  br label %BB3

BB3:                                              ; preds = %BB0
  br label %BB2

BB2:                                              ; preds = %BB3
  %VTABLE_101 = load i64** getelementptr inbounds ([516 x i64*]* @mono_aot_Mono_Android_Export_llvm_got, i32 0, i32 101), !invariant.load !481
  %0 = ptrtoint i64* %VTABLE_101 to i64
  %1 = call cc16 i64* @.Lp_6_plt_wrapper_alloc_object_AllocSmall_intptr_intptr_llvm(i64 %0, i64 40)
  %SFLDA_102 = load i64** getelementptr inbounds ([516 x i64*]* @mono_aot_Mono_Android_Export_llvm_got, i32 0, i32 102), !invariant.load !481
  %t48 = load i64* %SFLDA_102
  %2 = getelementptr i64* %1, i32 2
  store volatile i64 %t48, i64* %2
  %3 = ptrtoint i64* %1 to i64
  %t49 = add i64 %3, 16
  %t50 = lshr i64 %t49, 9
  %t160 = and i64 %t50, 8388607
  %GC_CARD_TABLE_ADDR_2 = load i64** getelementptr inbounds ([516 x i64*]* @mono_aot_Mono_Android_Export_llvm_got, i32 0, i32 2), !invariant.load !481
  %4 = ptrtoint i64* %GC_CARD_TABLE_ADDR_2 to i64
  %t161 = add i64 %t160, %4
  %5 = inttoptr i64 %t161 to i8*
  %6 = getelementptr i8* %5, i32 0
  store i8 1, i8* %6
  %7 = getelementptr i64* %this, i32 3
  %8 = ptrtoint i64* %1 to i64
  store volatile i64 %8, i64* %7
  %9 = ptrtoint i64* %this to i64
  %t52 = add i64 %9, 24
  %t53 = lshr i64 %t52, 9
  %t162 = and i64 %t53, 8388607
  %GC_CARD_TABLE_ADDR_21 = load i64** getelementptr inbounds ([516 x i64*]* @mono_aot_Mono_Android_Export_llvm_got, i32 0, i32 2), !invariant.load !481
  %10 = ptrtoint i64* %GC_CARD_TABLE_ADDR_21 to i64
  %t163 = add i64 %t162, %10
  %11 = inttoptr i64 %t163 to i8*
  %12 = getelementptr i8* %11, i32 0
  store i8 1, i8* %12
  %VTABLE_1012 = load i64** getelementptr inbounds ([516 x i64*]* @mono_aot_Mono_Android_Export_llvm_got, i32 0, i32 101), !invariant.load !481
  %13 = ptrtoint i64* %VTABLE_1012 to i64
  %14 = call cc16 i64* @.Lp_6_plt_wrapper_alloc_object_AllocSmall_intptr_intptr_llvm(i64 %13, i64 40)
  %SFLDA_1023 = load i64** getelementptr inbounds ([516 x i64*]* @mono_aot_Mono_Android_Export_llvm_got, i32 0, i32 102), !invariant.load !481
  %t65 = load i64* %SFLDA_1023
  %15 = getelementptr i64* %14, i32 2
  store volatile i64 %t65, i64* %15
  %16 = ptrtoint i64* %14 to i64
  %t66 = add i64 %16, 16
  %t67 = lshr i64 %t66, 9
  %t164 = and i64 %t67, 8388607
  %GC_CARD_TABLE_ADDR_24 = load i64** getelementptr inbounds ([516 x i64*]* @mono_aot_Mono_Android_Export_llvm_got, i32 0, i32 2), !invariant.load !481
  %17 = ptrtoint i64* %GC_CARD_TABLE_ADDR_24 to i64
  %t165 = add i64 %t164, %17
  %18 = inttoptr i64 %t165 to i8*
  %19 = getelementptr i8* %18, i32 0
  store i8 1, i8* %19
  %20 = getelementptr i64* %this, i32 4
  %21 = ptrtoint i64* %14 to i64
  store volatile i64 %21, i64* %20
  %22 = ptrtoint i64* %this to i64
  %t69 = add i64 %22, 32
  %t70 = lshr i64 %t69, 9
  %t166 = and i64 %t70, 8388607
  %GC_CARD_TABLE_ADDR_25 = load i64** getelementptr inbounds ([516 x i64*]* @mono_aot_Mono_Android_Export_llvm_got, i32 0, i32 2), !invariant.load !481
  %23 = ptrtoint i64* %GC_CARD_TABLE_ADDR_25 to i64
  %t167 = add i64 %t166, %23
  %24 = inttoptr i64 %t167 to i8*
  %25 = getelementptr i8* %24, i32 0
  store i8 1, i8* %25
  %VTABLE_1016 = load i64** getelementptr inbounds ([516 x i64*]* @mono_aot_Mono_Android_Export_llvm_got, i32 0, i32 101), !invariant.load !481
  %26 = ptrtoint i64* %VTABLE_1016 to i64
  %27 = call cc16 i64* @.Lp_6_plt_wrapper_alloc_object_AllocSmall_intptr_intptr_llvm(i64 %26, i64 40)
  %SFLDA_1027 = load i64** getelementptr inbounds ([516 x i64*]* @mono_aot_Mono_Android_Export_llvm_got, i32 0, i32 102), !invariant.load !481
  %t82 = load i64* %SFLDA_1027
  %28 = getelementptr i64* %27, i32 2
  store volatile i64 %t82, i64* %28
  %29 = ptrtoint i64* %27 to i64
  %t83 = add i64 %29, 16
  %t84 = lshr i64 %t83, 9
  %t168 = and i64 %t84, 8388607
  %GC_CARD_TABLE_ADDR_28 = load i64** getelementptr inbounds ([516 x i64*]* @mono_aot_Mono_Android_Export_llvm_got, i32 0, i32 2), !invariant.load !481
  %30 = ptrtoint i64* %GC_CARD_TABLE_ADDR_28 to i64
  %t169 = add i64 %t168, %30
  %31 = inttoptr i64 %t169 to i8*
  %32 = getelementptr i8* %31, i32 0
  store i8 1, i8* %32
  %33 = getelementptr i64* %this, i32 5
  %34 = ptrtoint i64* %27 to i64
  store volatile i64 %34, i64* %33
  %35 = ptrtoint i64* %this to i64
  %t86 = add i64 %35, 40
  %t87 = lshr i64 %t86, 9
  %t170 = and i64 %t87, 8388607
  %GC_CARD_TABLE_ADDR_29 = load i64** getelementptr inbounds ([516 x i64*]* @mono_aot_Mono_Android_Export_llvm_got, i32 0, i32 2), !invariant.load !481
  %36 = ptrtoint i64* %GC_CARD_TABLE_ADDR_29 to i64
  %t171 = add i64 %t170, %36
  %37 = inttoptr i64 %t171 to i8*
  %38 = getelementptr i8* %37, i32 0
  store i8 1, i8* %38
  %VTABLE_10110 = load i64** getelementptr inbounds ([516 x i64*]* @mono_aot_Mono_Android_Export_llvm_got, i32 0, i32 101), !invariant.load !481
  %39 = ptrtoint i64* %VTABLE_10110 to i64
  %40 = call cc16 i64* @.Lp_6_plt_wrapper_alloc_object_AllocSmall_intptr_intptr_llvm(i64 %39, i64 40)
  %SFLDA_10211 = load i64** getelementptr inbounds ([516 x i64*]* @mono_aot_Mono_Android_Export_llvm_got, i32 0, i32 102), !invariant.load !481
  %t99 = load i64* %SFLDA_10211
  %41 = getelementptr i64* %40, i32 2
  store volatile i64 %t99, i64* %41
  %42 = ptrtoint i64* %40 to i64
  %t100 = add i64 %42, 16
  %t101 = lshr i64 %t100, 9
  %t172 = and i64 %t101, 8388607
  %GC_CARD_TABLE_ADDR_212 = load i64** getelementptr inbounds ([516 x i64*]* @mono_aot_Mono_Android_Export_llvm_got, i32 0, i32 2), !invariant.load !481
  %43 = ptrtoint i64* %GC_CARD_TABLE_ADDR_212 to i64
  %t173 = add i64 %t172, %43
  %44 = inttoptr i64 %t173 to i8*
  %45 = getelementptr i8* %44, i32 0
  store i8 1, i8* %45
  %46 = getelementptr i64* %this, i32 6
  %47 = ptrtoint i64* %40 to i64
  store volatile i64 %47, i64* %46
  %48 = ptrtoint i64* %this to i64
  %t103 = add i64 %48, 48
  %t104 = lshr i64 %t103, 9
  %t174 = and i64 %t104, 8388607
  %GC_CARD_TABLE_ADDR_213 = load i64** getelementptr inbounds ([516 x i64*]* @mono_aot_Mono_Android_Export_llvm_got, i32 0, i32 2), !invariant.load !481
  %49 = ptrtoint i64* %GC_CARD_TABLE_ADDR_213 to i64
  %t175 = add i64 %t174, %49
  %50 = inttoptr i64 %t175 to i8*
  %51 = getelementptr i8* %50, i32 0
  store i8 1, i8* %51
  %VTABLE_154 = load i64** getelementptr inbounds ([516 x i64*]* @mono_aot_Mono_Android_Export_llvm_got, i32 0, i32 154), !invariant.load !481
  %52 = ptrtoint i64* %VTABLE_154 to i64
  %53 = call cc16 i64* @.Lp_6_plt_wrapper_alloc_object_AllocSmall_intptr_intptr_llvm(i64 %52, i64 80)
  ; call cc16 void @.Lp_68_plt_System_Collections_Hashtable__ctor_llvm(i64* %53)
  %54 = getelementptr i64* %this, i32 7
  %55 = ptrtoint i64* %53 to i64
  store volatile i64 %55, i64* %54
  %56 = ptrtoint i64* %this to i64
  %t117 = add i64 %56, 56
  %t118 = lshr i64 %t117, 9
  %t176 = and i64 %t118, 8388607
  %GC_CARD_TABLE_ADDR_214 = load i64** getelementptr inbounds ([516 x i64*]* @mono_aot_Mono_Android_Export_llvm_got, i32 0, i32 2), !invariant.load !481
  %57 = ptrtoint i64* %GC_CARD_TABLE_ADDR_214 to i64
  %t177 = add i64 %t176, %57
  %58 = inttoptr i64 %t177 to i8*
  %59 = getelementptr i8* %58, i32 0
  store i8 1, i8* %59
  %60 = load volatile i64* %arg_mb
  %61 = call cc16 i64* @.Lp_69_plt_System_Reflection_Emit_ModuleBuilder_DefineType_string_System_Reflection_TypeAttributes_System_Type_System_Type___llvm(i64* %arg_mb, i64* %arg_name, i32 %arg_attr, i64* %arg_baseType, i64* %arg_interfaces)
  %62 = getelementptr i64* %this, i32 2
  %63 = ptrtoint i64* %61 to i64
  store volatile i64 %63, i64* %62
  %64 = ptrtoint i64* %this to i64
  %t147 = add i64 %64, 16
  %t148 = lshr i64 %t147, 9
  %t178 = and i64 %t148, 8388607
  %GC_CARD_TABLE_ADDR_215 = load i64** getelementptr inbounds ([516 x i64*]* @mono_aot_Mono_Android_Export_llvm_got, i32 0, i32 2), !invariant.load !481
  %65 = ptrtoint i64* %GC_CARD_TABLE_ADDR_215 to i64
  %t179 = add i64 %t178, %65
  %66 = inttoptr i64 %t179 to i8*
  %67 = getelementptr i8* %66, i32 0
  store i8 1, i8* %67
  %68 = getelementptr i64* %this, i32 8
  %69 = ptrtoint i64* %arg_baseType to i64
  store volatile i64 %69, i64* %68
  %70 = ptrtoint i64* %this to i64
  %t152 = add i64 %70, 64
  %t153 = lshr i64 %t152, 9
  %t180 = and i64 %t153, 8388607
  %GC_CARD_TABLE_ADDR_216 = load i64** getelementptr inbounds ([516 x i64*]* @mono_aot_Mono_Android_Export_llvm_got, i32 0, i32 2), !invariant.load !481
  %71 = ptrtoint i64* %GC_CARD_TABLE_ADDR_216 to i64
  %t181 = add i64 %t180, %71
  %72 = inttoptr i64 %t181 to i8*
  %73 = getelementptr i8* %72, i32 0
  store i8 1, i8* %73
  %74 = getelementptr i64* %this, i32 9
  %75 = ptrtoint i64* %arg_interfaces to i64
  store volatile i64 %75, i64* %74
  %76 = ptrtoint i64* %this to i64
  %t157 = add i64 %76, 72
  %t158 = lshr i64 %t157, 9
  %t182 = and i64 %t158, 8388607
  %GC_CARD_TABLE_ADDR_217 = load i64** getelementptr inbounds ([516 x i64*]* @mono_aot_Mono_Android_Export_llvm_got, i32 0, i32 2), !invariant.load !481
  %77 = ptrtoint i64* %GC_CARD_TABLE_ADDR_217 to i64
  %t183 = add i64 %t182, %77
  %78 = inttoptr i64 %t183 to i8*
  %79 = getelementptr i8* %78, i32 0
  store i8 1, i8* %79
  br label %BB1

BB1:                                              ; preds = %BB2
  ret void
}

!481 = metadata !{metadata !"<index>"}
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

llvm-as -o foo.bc foo.ll
llc -asm-verbose=false -mtriple=aarch64-linux-android -o foo.s foo.bc

[vargaz]

The stack trace looks like this:

frame #0: 0x00000001004eb117 llc`llvm::MachineInstr::addOperand(llvm::MachineFunction&, llvm::MachineOperand const&) + 23

• frame GREF logging is broken in multi-process apps #1: 0x000000010020feaa llc`llvm::AArch64InstrInfo::storeRegToStackSlot(llvm::MachineBasicBlock&, llvm::MachineBasicBlock::bundle_iterator<llvm::MachineInstr, llvm::ilist_iteratorllvm::MachineInstr >, unsigned int, bool, int, llvm::TargetRegisterClass const*, llvm::TargetRegisterInfo const*) const + 858

and the assembly:
0x10020fe7d: 4c 89 e2 movq %r12, %rdx
0x10020fe80: 31 f6 xorl %esi, %esi
0x10020fe82: 31 ff xorl %edi, %edi
0x10020fe84: c6 44 24 30 01 movb $0x1, 0x30(%rsp)
0x10020fe89: 80 64 24 32 f0 andb $-0x10, 0x32(%rsp)
0x10020fe8e: c6 44 24 31 00 movb $0x0, 0x31(%rsp)
0x10020fe93: 48 c7 44 24 38 00 00 00 00 movq $0x0, 0x38(%rsp)
0x10020fe9c: 48 c7 44 24 40 00 00 00 00 movq $0x0, 0x40(%rsp)
0x10020fea5: e8 56 b2 2d 00 callq 0x1004eb100 ; llvm::MachineInstr::addOperand(llvm::MachineFunction&, llvm::MachineOperand const&)

Basically rdi, which should contain a MachineInstr& is set to 0.

[vargaz]

A possible fix to this problem is to revert to the old version of mxe, and cherry-pick the neccessary changes instead of merging to mxe master.

[grendello]

@vargaz @jonpryor mxe bumped the default gcc to 5.4.0 (from 4.x), but it also has gcc 4, gcc 6 and gcc 7, so if the gcc version on osx is 5.4.x as well, it might mean that this gcc is buggy and the solution might be trying one of the other versions?

[grendello]

@vargaz also, does it crash when you build llvm with -O0? If no then perhaps the fix is to use one of the -f flags to disable some optimization used in the basic -O1 set?

[vargaz]

Added a workaround in:
mono/llvm@bdb3a11

[jonpryor]

Cherry-picked to d15-6 via llvm/d15-6/b918f466 and 73c16c0.

monodroid bump + build pending once the Jenkins xamarin-android-d15-6 job is complete.

[JKennedy24]

has this been merged into VS15.5.5 or is this going to be in VS15.6?

[pjcollins]

@JKennedy24 are you able to reproduce this issue against a VS15.5 variant? From what I have seen, this issue was only introduced in 15.6 and will be fixed before the final 15.6 stable release.

[JKennedy24]

@pjcollins In that case I may have confused this issue with a different one I am seeing. I am seeing this issue: https://developercommunity.visualstudio.com/content/problem/161766/xamarin-android-ci-build-fails-with-invalidoperati.html

And pressumed this was it, but actually I think it is more of an issue with msbuild