iOS regression - crash when parsing JSON

[SamsonCodeHeroes]

Since the recent Xamarin iOS tooling update, I am seeing a crash that appears to be related to JSON serialization and ImmutableArray (see repro attached below).

The following doesn't happen when using the iOS SDK 11.2, but does happen using 11.4. Strangely, commenting out an unrelated property prevents the error.

Steps to Reproduce

1. Build and run the app.
2. Press the button. This causes deserialization and re-serializing of a small piece of JSON data.
3. You should get a Null reference exception.
4. Comment out the Title property in ServiceDto (other properties should also work).
5. Re-run the app and hit the button.
6. JSON should be output to console, without crashing.

Expected Behavior

JSON string should be logged to output.

Actual Behavior

Null reference exception.

Environment

Error appears to be iPhone-only. Replicated on:

• iPhone5S running iOS version 11.0.3.
• iPhoneX running 11.4.

The crash doesn't seem to happen on all devices. The initial error report wasn't replicated on an old iPad2 running iOS 9.3.5, though I haven't been able to re-test this.

IDE:

Microsoft Visual Studio Community 2017 
Version 15.7.2
VisualStudio.15.Release/15.7.2+27703.2018
Microsoft .NET Framework
Version 4.7.03056

Installed Version: Community

ASP.NET and Web Tools 2017   15.0.40511.0
ASP.NET and Web Tools 2017

ASP.NET Core Razor Language Services   15.7.31476
Provides languages services for ASP.NET Core Razor.

AWS Toolkit for Visual Studio 2017   1.14.3.3
AWS Toolkit for Visual Studio 2017.
Copyright 2011-2018 Amazon.com, Inc. or its affiliates. All Rights Reserved.

Release notes: https://aws-vs-toolkit.s3.amazonaws.com/versioninfo.html

This software includes third party software subject to the following copyrights:
- Logging from log4net, Apache License 
  [http://logging.apache.org/log4net/license.html]
- Putty for PPK to PEM conversion, MIT license 
  [http://www.chiark.greenend.org.uk/~sgtatham/putty/licence.html]
- NGit for AWS Elastic Beanstalk incremental push
  [https://github.com/mono/ngit/blob/master/NGit.license.txt]
- NSch dependency for NGit
  [https://github.com/mono/ngit/blob/master/NSch.license.txt]
- Sharpen dependency for NGit
  [https://github.com/mono/ngit/blob/master/Sharpen/AssemblyInfo.cs]
- ICSharpCode.SharpZipLib dependency for NGit
  [http://www.icsharpcode.net/opensource/sharpziplib/]
- Mono.Posix.dll and Mono.Security.dll dependencies for NGit
  [http://mono-project.com/FAQ:_Licensing#Licensing]
- MPFProj for Visual Studio Project Framework
  [http://mpfproj10.codeplex.com/license]
- JSON Checker for JSON validation
  [http://www.raboof.com/projects/jsonchecker/]
- Newtonsoft Json.NET
  [http://www.newtonsoft.com/json] 
- YamlDotNet
  [http://aaubry.net/pages/yamldotnet.html] 
    

Azure App Service Tools v3.0.0   15.0.40424.0
Azure App Service Tools v3.0.0

C# Tools   2.8.2-beta6-62916-08. Commit Hash: 2ad4aabc7a9dada097e54e544ebba48ab1c05074
C# components used in the IDE. Depending on your project type and settings, a different version of the compiler may be used.

Color Theme Designer   1.0
Designer for creating new color themes

Common Azure Tools   1.10
Provides common services for use by Azure Mobile Services and Microsoft Azure Tools.

JavaScript Language Service   2.0
JavaScript Language Service

JetBrains ReSharper Ultimate 2017.2    Build 109.0.20170824.131346
JetBrains ReSharper Ultimate package for Microsoft Visual Studio. For more information about ReSharper Ultimate, visit http://www.jetbrains.com/resharper. Copyright © 2018 JetBrains, Inc.

Merq   1.1.19-rc (a4ffc1b)
Command Bus, Event Stream and Async Manager for Visual Studio extensions.

Microsoft Continuous Delivery Tools for Visual Studio   0.3
Simplifying the configuration of continuous build integration and continuous build delivery from within the Visual Studio IDE.

Microsoft JVM Debugger   1.0
Provides support for connecting the Visual Studio debugger to JDWP compatible Java Virtual Machines

Microsoft MI-Based Debugger   1.0
Provides support for connecting Visual Studio to MI compatible debuggers

Microsoft Visual Studio Tools for Containers   1.1
Develop, run, validate your ASP.NET Core applications in the target environment. F5 your application directly into a container with debugging, or CTRL + F5 to edit & refresh your app without having to rebuild the container.

Mono Debugging for Visual Studio   4.10.5-pre (ab58725)
Support for debugging Mono processes with Visual Studio.

NCrunch   
Continuous Testing Tool for .NET
Copyright © 2010-2018 Remco Software Ltd

NuGet Package Manager   4.6.0
NuGet Package Manager in Visual Studio. For more information about NuGet, visit http://docs.nuget.org/.

ProjectServicesPackage Extension   1.0
ProjectServicesPackage Visual Studio Extension Detailed Info

ResourcePackage Extension   1.0
ResourcePackage Visual Studio Extension Detailed Info

SettingsWindow Extension   1.0
SettingsWindow Visual Studio Extension Detailed Info

SQL Server Data Tools   15.1.61804.210
Microsoft SQL Server Data Tools

Trailing Whitespace Visualizer   2.5.83
Keeps your code files clean by making it easier than ever to identify and remove any trailing whitespace

TypeScript Tools   15.7.20419.2003
TypeScript Tools for Microsoft Visual Studio

Visual Basic Tools   2.8.2-beta6-62916-08. Commit Hash: 2ad4aabc7a9dada097e54e544ebba48ab1c05074
Visual Basic components used in the IDE. Depending on your project type and settings, a different version of the compiler may be used.

Visual F# Tools 10.1 for F# 4.1   15.7.0.0.  Commit Hash: 56f97a8dd4353d897336941f3e644423b87c794f.
Microsoft Visual F# Tools 10.1 for F# 4.1

Visual Studio Code Debug Adapter Host Package   1.0
Interop layer for hosting Visual Studio Code debug adapters in Visual Studio

VisualStudio.Mac   1.0
Mac Extension for Visual Studio

Xamarin   4.10.0.448 (4373404db)
Visual Studio extension to enable development for Xamarin.iOS and Xamarin.Android.

Xamarin Designer   4.12.270 (82d750d12)
Visual Studio extension to enable Xamarin Designer tools in Visual Studio.

Xamarin.Android SDK   8.3.0.19 (HEAD/342b2ce96)
Xamarin.Android Reference Assemblies and MSBuild support.

Xamarin.iOS and Xamarin.Mac SDK   11.10.1.178 (408d357)
Xamarin.iOS and Xamarin.Mac Reference Assemblies and MSBuild support.

Example Project

Repro.zip

[therealjohn]

I can reproduce this on my iPhone 6s. It works OK in the simulator.

[therealjohn]

I'm removing the regression label because I can reproduce the issue with Xamarin.iOS 11.10 and Xcode 9.3.1 shown in the build output linked as iOS 11.2. I can also reproduce this with Xamarin.iOS 11.12 and Xcode 9.4.1 which is the versions linked as iOS 11.4.

[spouliot]

This sounds like a linker issue, i.e. it's off by default for the simulator (which works) and enabled by default open device. Serialization (like JSON) is usually done using reflection and types/members must be preserved to work (if they are not directly used by the code).

To test this you can:

• enable the linker on the simulator and run the app; and
• disable the linker on the device;

It the results switch (exception on simulator & working on device) it means the linker needs to be told which extraneous types to preserve,

[therealjohn]

@spouliot, I should have mentioned that I tested that... The issue reproduces on a device with Don't Link.

The stacktrace is:

{System.NullReferenceException: Object reference not set to an instance of an object
  at System.Collections.Immutable.ImmutableArray`1+EnumeratorObject[T].Create (T[] array) [0x00000] in <6518d0e60ca84f6b915c07341a031a32>:0 
  at System.Collections.Immutable.ImmutableArray`1[T].System.Collections.IEnumerable.GetEnumerator () [0x0000e] in <6518d0e60ca84f6b915c07341a031a32>:0 
  at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeList (Newtonsoft.Json.JsonWriter writer, System.Collections.IEnumerable values, Newtonsoft.Json.Serialization.JsonArrayContract contract, Newtonsoft.Json.Serialization.JsonProperty member, Newtonsoft.Json.Serialization.JsonContainerContract collectionContract, Newtonsoft.Json.Serialization.JsonProperty containerProperty) [0x00049] in <2781d1b198634655944cdefb18b3309b>:0 
  at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeValue (Newtonsoft.Json.JsonWriter writer, System.Object value, Newtonsoft.Json.Serialization.JsonContract valueContract, Newtonsoft.Json.Serialization.JsonProperty member, Newtonsoft.Json.Serialization.JsonContainerContract containerContract, Newtonsoft.Json.Serialization.JsonProperty containerProperty) [0x000d4] in <2781d1b198634655944cdefb18b3309b>:0 
  at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject (Newtonsoft.Json.JsonWriter writer, System.Object value, Newtonsoft.Json.Serialization.JsonObjectContract contract, Newtonsoft.Json.Serialization.JsonProperty member, Newtonsoft.Json.Serialization.JsonContainerContract collectionContract, Newtonsoft.Json.Serialization.JsonProperty containerProperty) [0x0008c] in <2781d1b198634655944cdefb18b3309b>:0 
  at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeValue (Newtonsoft.Json.JsonWriter writer, System.Object value, Newtonsoft.Json.Serialization.JsonContract valueContract, Newtonsoft.Json.Serialization.JsonProperty member, Newtonsoft.Json.Serialization.JsonContainerContract containerContract, Newtonsoft.Json.Serialization.JsonProperty containerProperty) [0x000b0] in <2781d1b198634655944cdefb18b3309b>:0 
  at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.Serialize (Newtonsoft.Json.JsonWriter jsonWriter, System.Object value, System.Type objectType) [0x00079] in <2781d1b198634655944cdefb18b3309b>:0 
  at Newtonsoft.Json.JsonSerializer.SerializeInternal (Newtonsoft.Json.JsonWriter jsonWriter, System.Object value, System.Type objectType) [0x00253] in <2781d1b198634655944cdefb18b3309b>:0 
  at Newtonsoft.Json.JsonSerializer.Serialize (Newtonsoft.Json.JsonWriter jsonWriter, System.Object value, System.Type objectType) [0x00000] in <2781d1b198634655944cdefb18b3309b>:0 
  at Newtonsoft.Json.JsonConvert.SerializeObjectInternal (System.Object value, System.Type type, Newtonsoft.Json.JsonSerializer jsonSerializer) [0x00028] in <2781d1b198634655944cdefb18b3309b>:0 
  at Newtonsoft.Json.JsonConvert.SerializeObject (System.Object value, System.Type type, Newtonsoft.Json.JsonSerializerSettings settings) [0x00007] in <2781d1b198634655944cdefb18b3309b>:0 
  at Newtonsoft.Json.JsonConvert.SerializeObject (System.Object value) [0x00000] in <2781d1b198634655944cdefb18b3309b>:0 
  at Repro.MainPage.DeserializeAndSerializeJson () [0x0004d] in /Users/johnmiller/Downloads/Repro (1)/Repro/Repro/MainPage.xaml.cs:37 }

[kentcb]

Yes, I work with @SamsonCodeHeroes. We should have mentioned that we tried disabling the linker first (we've been bitten by that enough before!). We also suspected all sorts of things in terms of what the underlying cause was - immutable collections being the primary candidate. However, changing one thing would change the end result in confounding ways. At one point we even suspected a byte alignment issue with the compiler and tried injecting empty byte arrays to check whether that would cause the issue to go away (it didn't for us).

[spouliot]

hmm... weird

at System.Collections.Immutable.ImmutableArray`1+EnumeratorObject[T].Create (T[] array) [0x00000] in <6518d0e60ca84f6b915c07341a031a32>:0

does hint to some reflection code, e.g. where T might not be fully preserved.

I'll investigate this further using the attached sample. Thanks both for replying with extra information - the obvious is not always the culprit :-)

[bsheen]

@spouliot I work with @kentcb and @SamsonCodeHeroes, I was just wondering if you'd made any progress resolving this issue. It's currently blocking a release for an important client of ours and the app has a significant active user base.

[WilliamNolanch]

@spouliot @therealjohn, I'm following up on the above. Are we able to get an update? Thanks

[spouliot]

So, the issue is not related to the (managed) linker.

The issue only shows up on ARM64 (both mono and LLVM AOT backends). It works correctly when AOT'ed on ARMv7 (32bits). Very likely it's not a real NullReferenceException but a remapped SIGSEGV from the runtime.

Going back in time...

• XI 11.16 (15.9) w/mono 2018-04 fails
• XI 11.14 (15.8) w/mono 2018-02 fails
• XI 11.12 (15.7) w/mono 2017-12 fails
  but according to the [original description](url
  https://github.com/bitrise-io/bitrise.io/blob/master/system_reports/osx-vs4mac-previous-stable.log)
  mtouch 11.10.1.178 (d15-7: 408d3574) worked
  but installing something just a bit older
  mtouch 11.10.0.1 (d15-7: eadb8376) failed for me

@vargaz can you have a look at this ? thanks!

[bsheen]

@vargaz have you been able to make any progress. Our client needs a release out ASAP and this issue is blocking that.

[vargaz]

As a workaround, try to change the order of the parameters to the ServiceDto constructor:
instead of:
bool isOpenNow,
ImmutableArray relevantPeriods)
do:
ImmutableArray relevantPeriods,
bool isOpenNow)

[kentcb]

Thanks for fixing this.

Whilst we made some progress by re-ordering parameters, it seems like the crash is manifesting in other ways. For example:

2018-09-19 11:36:48.499 QUT[9897:17016688] critical: Stacktrace:
2018-09-19 11:36:48.500 QUT[9897:17016688] critical:   at <unknown> <0xffffffff>
2018-09-19 11:36:48.500 QUT[9897:17016688] critical:   at (wrapper managed-to-native) object.__icall_wrapper_mono_gc_alloc_vector (intptr,intptr,intptr) <0x00007>
2018-09-19 11:36:48.500 QUT[9897:17016688] critical:   at (wrapper alloc) object.AllocVector (intptr,intptr) <0x0016b>
2018-09-19 11:36:48.500 QUT[9897:17016688] critical:   at System.Collections.Generic.Dictionary`2<TKey_REF, TValue_REF>.Initialize (int) [0x0002e] in <3c7b99a36820490fb2cbc5a6fc6b06d8#0955DAA3-D8E5-6EB0-C74C-615A1A72EFF5>:0
2018-09-19 11:36:48.501 QUT[9897:17016688] critical:   at System.Collections.Generic.Dictionary`2<TKey_REF, TValue_REF>..ctor (int,System.Collections.Generic.IEqualityComparer`1<TKey_REF>) [0x00015] in <3c7b99a36820490fb2cbc5a6fc6b06d8#0955DAA3-D8E5-6EB0-C74C-615A1A72EFF5>:0
<snip>

Our hope is that these other crashes we're experiencing have the same underlying cause.

Is there an ETA as to when the fix will hit the alpha channel?

[vargaz]

Can someone answer that ?

[kentcb]

Here's another more complete example, showing the hard mono crash too:

2018-09-19 13:23:49.684 <<redacted>>[9925:17043038] critical: Stacktrace:
2018-09-19 13:23:49.684 <<redacted>>[9925:17043038] critical:   at <unknown> <0xffffffff>
2018-09-19 13:23:49.684 <<redacted>>[9925:17043038] critical:   at (wrapper managed-to-native) object.__icall_wrapper_mono_gc_alloc_vector (intptr,intptr,intptr) <0x00007>
2018-09-19 13:23:49.684 <<redacted>>[9925:17043038] critical:   at (wrapper alloc) object.AllocVector (intptr,intptr) <0x0016b>
2018-09-19 13:23:49.685 <<redacted>>[9925:17043038] critical:   at System.IO.Stream/<CopyToAsyncInternal>d__27.MoveNext () [0x00018] in /Library/Frameworks/Xamarin.iOS.framework/Versions/11.14.0.14/src/Xamarin.iOS/mcs/class/referencesource/mscorlib/system/io/stream.cs:169
2018-09-19 13:23:49.685 <<redacted>>[9925:17043038] critical:   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start<System.IO.Stream/<CopyToAsyncInternal>d__27> (System.IO.Stream/<CopyToAsyncInternal>d__27&) [0x0002c] in /Library/Frameworks/Xamarin.iOS.framework/Versions/11.14.0.14/src/Xamarin.iOS/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:316
20
18-09-19 13:23:49.685 <<redacted>>[9925:17043038] critical:   at System.IO.Stream.CopyToAsyncInternal (System.IO.Stream,int,System.Threading.CancellationToken) [0x0003f] in <3c7b99a36820490fb2cbc5a6fc6b06d8#4879182D-E8B3-D865-E8E4-B66DD945F01D>:0
2018-09-19 13:23:49.686 <<redacted>>[9925:17043038] critical:   at System.IO.Stream.CopyToAsync (System.IO.Stream,int,System.Threading.CancellationToken) [0x0009d] in /Library/Frameworks/Xamarin.iOS.framework/Versions/11.14.0.14/src/Xamarin.iOS/mcs/class/referencesource/mscorlib/system/io/stream.cs:159
2018-09-19 13:23:49.686 <<redacted>>[9925:17043038] critical:   at System.Net.Http.StreamContent.SerializeToStreamAsync (System.IO.Stream,System.Net.TransportContext) [0x0003c] in /Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/external/mono/mcs/class/System.Net.Http/System.Net.Http/StreamContent.cs:102
2018-09-19 13:23:49.686 <<redacted>>[9925:17043038] critical:   at System.Net.Http.HttpContent/<LoadIntoBufferAsync>d__17.MoveNext () [0x00048] in /Users/builder/jenkins/workspace/xamarin-m
acios/xamarin-macios/external/mono/mcs/class/System.Net.Http/System.Net.Http/HttpContent.cs:146
2018-09-19 13:23:49.686 <<redacted>>[9925:17043038] critical:   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start<System.Net.Http.HttpContent/<LoadIntoBufferAsync>d__17> (System.Net.Http.HttpContent/<LoadIntoBufferAsync>d__17&) [0x0002c] in /Library/Frameworks/Xamarin.iOS.framework/Versions/11.14.0.14/src/Xamarin.iOS/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:316
2018-09-19 13:23:49.687 <<redacted>>[9925:17043038] critical:   at System.Net.Http.HttpContent.LoadIntoBufferAsync (long) [0x0002f] in <d6ff9da2f46d44678957ce9fa3797199#4879182D-E8B3-D865-E8E4-B66DD945F01D>:0
2018-09-19 13:23:49.687 <<redacted>>[9925:17043038] critical:   at System.Net.Http.HttpClient/<SendAsyncWorker>d__48.MoveNext () [0x00103] in /Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/external/mono/mcs/class/System.Net.Http/System.Net.Http/HttpClient.cs:284
2018-09-19 13:23:49.687 <<redacted>>[9925:17043038
] critical:   at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.InvokeMoveNext (object) [0x00000] in /Library/Frameworks/Xamarin.iOS.framework/Versions/11.14.0.14/src/Xamarin.iOS/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1089
2018-09-19 13:23:49.688 <<redacted>>[9925:17043038] critical:   at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00071] in /Library/Frameworks/Xamarin.iOS.framework/Versions/11.14.0.14/src/Xamarin.iOS/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:957
2018-09-19 13:23:49.688 <<redacted>>[9925:17043038] critical:   at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00000] in /Library/Frameworks/Xamarin.iOS.framework/Versions/11.14.0.14/src/Xamarin.iOS/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:904
2018-09-19 13:23:49.688 <<redacted>>[9925
:17043038] critical:   at System.Runtime.CompilerServices.AsyncMethodBuilderCore/MoveNextRunner.Run () [0x00024] in /Library/Frameworks/Xamarin.iOS.framework/Versions/11.14.0.14/src/Xamarin.iOS/mcs/class/referencesource/mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs:1070
2018-09-19 13:23:49.689 <<redacted>>[9925:17043038] critical:   at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction (System.Action,bool,System.Threading.Tasks.Task&) [0x0001c] in /Library/Frameworks/Xamarin.iOS.framework/Versions/11.14.0.14/src/Xamarin.iOS/mcs/class/referencesource/mscorlib/system/threading/Tasks/TaskContinuation.cs:823
2018-09-19 13:23:49.689 <<redacted>>[9925:17043038] critical:   at System.Threading.Tasks.Task.FinishContinuations () [0x00068] in /Library/Frameworks/Xamarin.iOS.framework/Versions/11.14.0.14/src/Xamarin.iOS/mcs/class/referencesource/mscorlib/system/threading/Tasks/Task.cs:3631
2018-09-19 13:23:49.689 <<redacted>>[9925:17043038] critical:   at System.Threading.Tasks.Task.FinishStageThree () [0x0003c] in 
/Library/Frameworks/Xamarin.iOS.framework/Versions/11.14.0.14/src/Xamarin.iOS/mcs/class/referencesource/mscorlib/system/threading/Tasks/Task.cs:2373
2018-09-19 13:23:49.689 <<redacted>>[9925:17043038] critical:   at System.Threading.Tasks.Task`1<TResult_REF>.TrySetResult (TResult_REF) [0x0004f] in <3c7b99a36820490fb2cbc5a6fc6b06d8#4879182D-E8B3-D865-E8E4-B66DD945F01D>:0
2018-09-19 13:23:49.690 <<redacted>>[9925:17043038] critical:   at System.Threading.Tasks.TaskCompletionSource`1<TResult_REF>.TrySetResult (TResult_REF) [0x00000] in <3c7b99a36820490fb2cbc5a6fc6b06d8#4879182D-E8B3-D865-E8E4-B66DD945F01D>:0
2018-09-19 13:23:49.690 <<redacted>>[9925:17043038] critical:   at System.Reactive.Threading.Tasks.TaskObservableExtensions/ToTaskObserver`1<TResult_REF>.OnCompleted () [0x00008] in <e9c1ccec51844dbd92b833a0b4bc960e#4879182D-E8B3-D865-E8E4-B66DD945F01D>:0
2018-09-19 13:23:49.690 <<redacted>>[9925:17043038] critical:   at System.Reactive.Subjects.AsyncSubject`1<T_REF>.OnCompleted () [0x0007f] in <e9c1ccec51844dbd92b833a0b4bc960e#4879182D-E8B3-D
865-E8E4-B66DD945F01D>:0
2018-09-19 13:23:49.690 <<redacted>>[9925:17043038] critical:   at System.Reactive.SafeObserver`1<TSource_REF>.OnCompleted () [0x00000] in <99f8205c51c44bb480747b577b8001ff#4879182D-E8B3-D865-E8E4-B66DD945F01D>:0
2018-09-19 13:23:49.691 <<redacted>>[9925:17043038] critical:   at System.Reactive.Linq.ObservableImpl.AsObservable`1/_<TSource_REF>.OnCompleted () [0x00000] in <e9c1ccec51844dbd92b833a0b4bc960e#4879182D-E8B3-D865-E8E4-B66DD945F01D>:0
2018-09-19 13:23:49.691 <<redacted>>[9925:17043038] critical:   at System.Reactive.Subjects.AsyncSubject`1<T_REF>.OnCompleted () [0x0007f] in <e9c1ccec51844dbd92b833a0b4bc960e#4879182D-E8B3-D865-E8E4-B66DD945F01D>:0
2018-09-19 13:23:49.691 <<redacted>>[9925:17043038] critical:   at System.Reactive.Threading.Tasks.TaskObservableExtensions.ToObservableDone<TResult_REF> (System.Threading.Tasks.Task`1<TResult_REF>,System.IObserver`1<TResult_REF>) [0x00028] in <e9c1ccec51844dbd92b833a0b4bc960e#4879182D-E8B3-D865-E8E4-B66DD945F01D>:0
2018-09-19 13:23:49.691 <<redacted>>[9925:17043038] critical:   at System.Reactive.Threading.Tasks.TaskObservableExtensions/<>c__DisplayClass8_0`1<TResult_REF>.<ToObservableSlow>b__0 (System.Threading.Tasks.Task`1<TResult_REF>) [0x00000] in <e9c1ccec51844dbd92b833a0b4bc960e#4879182D-E8B3-D865-E8E4-B66DD945F01D>:0
2018-09-19 13:23:49.691 <<redacted>>[9925:17043038] critical:   at System.Threading.Tasks.ContinuationTaskFromResultTask`1<TAntecedentResult_REF>.InnerInvoke () [0x00024] in <3c7b99a36820490fb2cbc5a6fc6b06d8#4879182D-E8B3-D865-E8E4-B66DD945F01D>:0
2018-09-19 13:23:49.692 <<redacted>>[9925:17043038] critical:   at System.Threading.Tasks.Task.Execute () [0x00010] in /Library/Frameworks/Xamarin.iOS.framework/Versions/11.14.0.14/src/Xamarin.iOS/mcs/class/referencesource/mscorlib/system/threading/Tasks/Task.cs:2509
2018-09-19 13:23:49.692 <<redacted>>[9925:17043038] critical:   at System.Threading.Tasks.Task.ExecutionContextCallback (object) [0x00006] in /Library/Frameworks/Xamarin.iOS.framework/Versions/11.14.0.14/src/Xamarin.iOS/mcs/class/referencesource/mscorlib/system/threading/Tasks/Task.cs:2872
2018-09-19 13:23:49.692 <<redacted>>[9925:17043038] critical:   at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00071] in /Library/Frameworks/Xamarin.iOS.framework/Versions/11.14.0.14/src/Xamarin.iOS/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:957
2018-09-19 13:23:49.693 <<redacted>>[9925:17043038] critical:   at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00000] in /Library/Frameworks/Xamarin.iOS.framework/Versions/11.14.0.14/src/Xamarin.iOS/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:904
2018-09-19 13:23:49.694 <<redacted>>[9925:17043038] critical:   at System.Threading.Tasks.Task.ExecuteWithThreadLocal (System.Threading.Tasks.Task&) [0x00050] in /Library/Frameworks/Xamarin.iOS.framework/Versions/11.14.0.14/src/Xamarin.iOS/mcs/class/referencesource/mscorlib/system/threading/Tasks/Task.cs:2834
2018-09-19 13:23:49.694 <<redacted>>[9925:17043038] critical:   at System.Threading.Tasks.Task.ExecuteEntry (bool) [0x00058] in /Library/Frameworks/Xamarin.iOS.framework/Versions/11.14.0.14/src/Xamarin.iOS/mcs/class/referencesource/mscorlib/system/threading/Tasks/Task.cs:2767
2018-09-19 13:23:49.694 <<redacted>>[9925:17043038] critical:   at System.Threading.Tasks.Task.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem () [0x00000] in /Library/Frameworks/Xamarin.iOS.framework/Versions/11.14.0.14/src/Xamarin.iOS/mcs/class/referencesource/mscorlib/system/threading/Tasks/Task.cs:2714
2018-09-19 13:23:49.695 <<redacted>>[9925:17043038] critical:   at System.Threading.ThreadPoolWorkQueue.Dispatch () [0x00074] in /Library/Frameworks/Xamarin.iOS.framework/Versions/11.14.0.14/src/Xamarin.iOS/mcs/class/referencesource/mscorlib/system/threading/threadpool.cs:858
2018-09-19 13:23:49.695 <<redacted>>[9925:17043038] critical:   at ObjCRuntime.Runtime.ThreadPoolDispatcher (System.Func`1<bool>) [0x00006] in /Library/Frameworks/Xamarin.iOS.framework/Versions/11.14.0.14/src/Xamarin.iOS/ObjCRuntime/Runtime.cs:280
2018-09-19 13:23:49.695 <<redacted>>[9925:17043038] critical:   at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback () [0x00009] in /Library/Frameworks/Xamarin.iOS.framework/Versions/11.14.0.14/src/Xamarin.iOS/mcs/class/referencesource/mscorlib/system/threading/threadpool.cs:1210
2018-09-19 13:23:49.703 <<redacted>>[9925:17043038] critical: 	0   <<redacted>>                                 0x00000001068086e4 <<redacted>> + 100075236
2018-09-19 13:23:49.696 <<redacted>>[9925:17043038] critical:   at (wrapper runtime-invoke) object.runtime_invoke_dynamic (intptr,intptr,intptr,intptr) [0x0001e] in <3c7b99a36820490fb2cbc5a6fc6b06d8#4879182D-E8B3-D865-E8E4-B66DD945F01D>:0
2018-09-19 13:23:49.696 <<redacted>>[9925:17043038] critical: 
Native stacktrace:
2018-09-19 13:23:49.703 <<redacted>>[9925:17043038] critical: 	1   <<redacted>>                                 0x0000000106815820 mono_pmip + 13988
2018-09-19 13:23:49.704 <<redacted>>[9925:17043038] critical: 	2   libsystem_platform.dylib            0x0000000182118b58 _sigtramp + 52
2018-09-19 13:23:49.704 <<redacted>>[9925:17043038] critical: 	3   <<redacted>>                                 0x000000010690a64c mono_pmip + 1017040
2018-09-19 13:23:49.704 <<redacted>>[9925:17043038] critical: 	4   <<redacted>>                                 0x000000010690d31c mono_pmip + 1028512
2018-09-19 13:23:49.704 <<redacted>>[9925:17043038] critical: 	5   <<redacted>>                                 0x00000001069002f8 mono_pmip + 975228
2018-09-19 13:23:49.704 <<redacted>>[9925:17043038] critical: 	6   <<redacted>>                                 0x0000000106900ddc mono_pmip + 978016
2018-09-19 13:23:49.705 <<redacted>>[9925:17043038] critical: 	7   <<redacted>>                                 0x00000001068ff95c mono_pmip + 972768
2018-09-19 13:23:49.705 <<redacted>>[9925:17043038] critical: 	8   <<redacted>>                                 0x00000001068fbcbc mono_pmip + 957248
2018-09-19 13:23:49.705 <<redacted>>[9925:17043038] critical: 	9   <<redacted>>                                 0x0000000106904de0 mono_pmip + 994404
2018-09-19 13:23:49.705 <<redacted>>[9925:17043038] critical: 	10  <<redacted>>                                 0x00000001068f1934 mono_pmip + 915384
2018-09-19 13:23:49.705 <<redacted>>[9925:17043038] critical: 	11  <<redacted>>                                 0x00000001068bd090 mono_pmip + 700180
2018-09-19 13:23:49.705 <<redacted>>[9925:17043038] critical: 	12  <<redacted>>                                 0x0000000100f55450 <<redacted>> + 7066704
2018-09-19 13:23:49.705 <<redacted>>[9925:17043038] critical: 	13  <<redacted>>                                 0x0000000100f5036c <<redacted>> + 7045996
2018-09-19 13:23:49.706 <<redacted>>[9925:17043038] critical: 	14  <<redacted>>                                 0x0000000100ba50b4 <<redacted>> + 3199156
2018-09-19 13:23:49.706 <<redacted>>[9925:17043038] critical: 	15  <<redacted>>                                 0x0000000100ed8b6c <<redacted>> + 6556524
2018-09-19 13:23:49.706 <<redacted>>[9925:17043038] critical: 	16  <<redacted>>                                 0x0000000100ba0ba4 <<redacted>> + 3181476
2018-09-19 13:23:49.706 <<redacted>>[9925:17043038] critical: 	17  <<redacted>>                                 0x0000000100ba08d0 <<redacted>> + 3180752
2018-09-19 13:23:49.706 <<redacted>>[9925:17043038] critical: 	18  <<redacted>>                                 0x0000000103b98bd0 <<redacted>> + 53480400
2018-09-19 13:23:49.706 <<redacted>>[9925:17043038] critical: 	19  <<redacted>>                                 0x0000000103bbb584 <<redacted>> + 53622148
2018-09-19 13:23:49.707 <<redacted>>[9925:17043038] critical: 	20  <<redacted>>                                 0x0000000103be4530 <<redacted>> + 53790000
2018-09-19 13:23:49.707 <<redacted>>[9925:17043038] critical: 	21  <<redacted>>                                 0x0000000103b9556c <<redacted>> + 53466476
2018-09-19 13:23:49.707 <<redacted>>[9925:17043038] critical: 	22  <<redacted>>                                 0x0000000103bb4404 <<redacted>> + 53593092
2018-09-19 13:23:49.707 <<redacted>>[9925:17043038] critical: 	23  <<redacted>>                                 0x0000000100d0952c <<redacted>> + 4658476
2018-09-19 13:23:49.707 <<redacted>>[9925:17043038] critical: 	24  <<redacted>>                                 0x0000000100c41a64 <<redacted>> + 3840612
2018-09-19 13:23:49.707 <<redacted>>[9925:17043038] critical: 	25  <<redacted>>                                 0x0000000100c418bc <<redacted>> + 3840188
2018-09-19 13:23:49.708 <<redacted>>[9925:17043038] critical: 	26  <<redacted>>                                 0x0000000100d093ec <<redacted>> + 4658156
2018-09-19 13:23:49.708 <<redacted>>[9925:17043038] critical: 	27  <<redacted>>                                 0x0000000100c8001c <<redacted>> + 4096028
2018-09-19 13:23:49.708 <<redacted>>[9925:17043038] critical: 	28  <<redacted>>                                 0x0000000100c75dd0 <<redacted>> + 4054480
2018-09-19 13:23:49.708 <<redacted>>[9925:17043038] critical: 	29  <<redacted>>                                 0x0000000100c73cb8 <<redacted>> + 4046008
2018-09-19 13:23:49.708 <<redacted>>[9925:17043038] critical: 	30  <<redacted>>                                 0x0000000100c5d2f4 <<redacted>> + 3953396
2018-09-19 13:23:49.708 <<redacted>>[9925:17043038] critical: 	31  <<redacted>>                                 0x0000000100c7ddbc <<redacted>> + 4087228
2018-09-19 13:23:49.709 <<redacted>>[9925:17043038] critical: 	32  <<redacted>>                                 0x00000001048012f0 <<redacted>> + 66491120
2018-09-19 13:23:49.709 <<redacted>>[9925:17043038] critical: 	33  <<redacted>>                                 0x00000001046fd4b8 <<redacted>> + 65426616
2018-09-19 13:23:49.709 <<redacted>>[9925:17043038] critical: 	34  <<redacted>>                                 0x000000010460e10c <<redacted>> + 64446732
2018-09-19 13:23:49.709 <<redacted>>[9925:17043038] critical: 	35  <<redacted>>                                 0x0000000104834a70 <<redacted>> + 66701936
2018-09-19 13:23:49.709 <<redacted>>[9925:17043038] critical: 	36  <<redacted>>                                 0x00000001046fd4b8 <<redacted>> + 65426616
2018-09-19 13:23:49.710 <<redacted>>[9925:17043038] critical: 	37  <<redacted>>                                 0x00000001046fbef4 <<redacted>> + 65421044
2018-09-19 13:23:49.710 <<redacted>>[9925:17043038] critical: 	38  <<redacted>>                                 0x000000010480183c <<redacted>> + 66492476
2018-09-19 13:23:49.710 <<redacted>>[9925:17043038] critical: 	39  <<redacted>>                                 0x0000000100c7e5f0 <<redacted>> + 4089328
2018-09-19 13:23:49.710 <<redacted>>[9925:17043038] critical: 	40  <<redacted>>                                 0x0000000100c741f4 <<redacted>> + 4047348
2018-09-19 13:23:49.710 <<redacted>>[9925:17043038] critical: 	41  <<redacted>>                                 0x0000000100c74ad8 <<redacted>> + 4049624
2018-09-19 13:23:49.711 <<redacted>>[9925:17043038] critical: 	42  <<redacted>>                                 0x0000000100c41a64 <<redacted>> + 3840612
2018-09-19 13:23:49.711 <<redacted>>[9925:17043038] critical: 	43  <<redacted>>                                 0x0000000100c418bc <<redacted>> + 3840188
2018-09-19 13:23:49.711 <<redacted>>[9925:17043038] critical: 	44  <<redacted>>                                 0x0000000100c749f0 <<redacted>> + 4049392
2018-09-19 13:23:49.712 <<redacted>>[9925:17043038] critical: 	45  <<redacted>>                                 0x0000000100c74800 <<redacted>> + 4048896
2018-09-19 13:23:49.712 <<redacted>>[9925:17043038] critical: 	46  <<redacted>>                                 0x0000000100c74648 <<redacted>> + 4048456
2018-09-19 13:23:49.712 <<redacted>>[9925:17043038] critical: 	47  <<redacted>>                                 0x0000000100c4b278 <<redacted>> + 3879544
2018-09-19 13:23:49.712 <<redacted>>[9925:17043038] critical: 	48  <<redacted>>                                 0x0000000102cfc970 <<redacted>> + 38160752
2018-09-19 13:23:49.712 <<redacted>>[9925:17043038] critical: 	49  <<redacted>>                                 0x0000000100c4d208 <<redacted>> + 3887624
2018-09-19 13:23:49.712 <<redacted>>[9925:17043038] critical: 	50  <<redacted>>                                 0x0000000100f4fde8 <<redacted>> + 7044584
2018-09-19 13:23:49.713 <<redacted>>[9925:17043038] critical: 	51  <<redacted>>                                 0x000000010681898c mono_pmip + 26640
2018-09-19 13:23:49.713 <<redacted>>[9925:17043038] critical: 	52  <<redacted>>                                 0x000000010689fd88 mono_pmip + 580620
2018-09-19 13:23:49.713 <<redacted>>[9925:17043038] critical: 	53  <<redacted>>                                 0x00000001068cfe04 mono_pmip + 777352
2018-09-19 13:23:49.713 <<redacted>>[9925:17043038] critical: 	54  <<redacted>>                                 0x00000001068cdcc8 mono_pmip + 768844
2018-09-19 13:23:49.713 <<redacted>>[9925:17043038] critical: 	55  <<redacted>>                                 0x00000001068d806c mono_pmip + 810736
2018-09-19 13:23:49.713 <<redacted>>[9925:17043038] critical: 	56  libsystem_pthread.dylib             0x000000018211d220 <redacted> + 272
2018-09-19 13:23:49.714 <<redacted>>[9925:17043038] critical: 	57  libsystem_pthread.dylib             0x000000018211d110 <redacted> + 0
2018-09-19 13:23:49.714 <<redacted>>[9925:17043038] critical: 	58  libsystem_pthread.dylib             0x000000018211bb10 thread_start + 4
2018-09-19 13:23:49.714 <<redacted>>[9925:17043038] critical: 
=================================================================
Got a SIGSEGV while executing native code. This usually indicates
a fatal error in the mono runtime or one of the native libraries 
used by your application.
=================================================================

We really need some help getting past this issue for our client. Can we get access to early bits with the above fix? Can we get access to older bits before this bug was introduced?

[rolfbjarne]

@kentcb Can you try this package: xamarin.ios-11.17.0.92.pkg and see if it fixes the problem? It contains the fix for this bug.

You may run into other issues with this package, but if anything shows up we'll help you get through them since it would be nice to at the very least get a confirmation that the fix actually works for your app.

[kentcb]

Thanks @rolfbjarne! ❤️

My preliminary testing indicates that this fixes both issues (I rolled back our workaround to be sure). This is looking positive, but we'll conduct more in-depth testing and get back to you shortly.

[kentcb]

@rolfbjarne We've done quite extensive testing now and have been unable to reproduce the crash. We're now in discussions around the best path forward with respect to the release. We are cognizant of the risk of a compiler-induced bug being caught after the release, after which we would have no recourse (again). One thing that would help us in our decision making is to have a rough understanding of when this fix will hit the alpha channel. Are you able to provide a rough date, or even an indicative "days", "weeks" or "months"?

[rolfbjarne]

Are you able to provide a rough date, or even an indicative "days", "weeks" or "months"?

My personal guess would be that it should get into the Alpha channel within a couple of months (in an upcoming (but not the first) preview of our d16-0 release). The d16-0 release is scheduled for next Spring, which is when it would reach the Stable channel.

[cainunable]

@rolfbjarne I see that the fix has been merged a while back, but I don't know how to see which channel it has been released in. Is this fix out and available yet?

[rolfbjarne]

@cainunable the first preview release with this fix should come out in a couple of weeks.

[cainunable]

@rolfbjarne I'm getting a similar error as kentcb's so I am hoping this might be a possible solution. If I just install the .pkg above to test that theory, how difficult is it to restore my current working environment?

[rolfbjarne]

@cainunable it should be just a matter of checking for updates in Visual Studio for Mac, and install the offered Xamarin.iOS update. If that doesn't work, you can force the update by switching to a different channel and then switch back again.