[debugger] Work around a debugger issue when using the interpreter #15451

thaystg · 2022-07-11T15:05:26Z

TODO:
Only start runtime using mono_main when it's on interpreter mode and debug enabled.

CONS:
With this implementation we see in the log a message like this:

Usage is: mono [options] program [program-options]

Development:
    --aot[=<options>]      Compiles the assembly to native code
    --debug=ignore         Disable debugging support (on by default)
    --debug=[<options>]    Disable debugging support or enable debugging extras, use --help-debug for details
    --debugger-agent=options Enable the debugger agent
    --profile[=profiler]   Runs in profiling mode with the specified profiler module
    --trace[=EXPR]         Enable tracing, use --help-trace for details
    --help-devel           Shows more options available to developers

Runtime:
    --config FILE          Loads FILE as the Mono config
    --verbose, -v          Increases the verbosity level
    --help, -h             Show usage information
    --version

Maybe we could pass on args --version then we would see the mono version but wouldn't see this help message.

rolfbjarne · 2022-07-11T15:14:07Z

runtime/monovm-bridge.m

+	};
+
+	if (xamarin_init_mono_debug)
+		mono_main (2, (char**)argc);


Could you explain runtime-wise why this works around the problem?

Yes, sure.
The issue on runtime side was that we are initializing interpreter before setting the debugger attributes. So when we execute this code: https://github.com/dotnet/runtime/blob/5529dc79e9f35d70e9d01318d31844984b860dd4/src/mono/mono/mini/interp/interp.c#L8121
mdb_optimizations was false, then we keep the default optimizations of the interpreter which was breaking the debugger experience.
In this workaround I'm passing --interp=-all in the parameters, so I'm disabling the interpreter optimizations and the debugger works well.

Is this only a problem in .NET 6? Or legacy mono as well? (i.e. mono/mono's 2020-02 branch)

I'm almost sure that it's a problem only on .net 6, but I would ask someone to test to double check. It's really easy to reproduce:

public static void MethodMyMethod() { var e2 = new Exception("hi thays"); var e1 = e2; if (e1.Equals(e2)) Console.WriteLine("equals"); }

Adding a breakpoint on if statement and looking at locals, with the optimizations turned on we only see the content of variable e2, e1 is null.

Is this going to have implications for hot reload on .NET MAUI?

I don't think so. @lambdageek do you have any thoughts about this workaround?

rolfbjarne · 2022-07-11T15:14:36Z

/azp run

azure-pipelines · 2022-07-11T15:14:54Z

Azure Pipelines successfully started running 1 pipeline(s).

mandel-macaque · 2022-07-11T21:32:03Z

the xtra failures are interesting, is complaining due to a binding problem, when it should not happen. Merge with main, if issues are the same I'll fix the PR or main

runtime/monovm-bridge.m

mauroa · 2022-07-12T17:24:15Z

runtime/monovm-bridge.m

+#if DOTNET
+	bool use_mono_workaround = xamarin_init_mono_debug;
+#else
+	bool use_mono_workaround = false;


@rolfbjarne I also had a repro with Hot Restart in Xamarin Forms and after discussing with @thaystg, she confirmed that this bug might also repro in a mono runtime environment so I think we should consider the workaround for non dotnet scenarios

Just to follow up: legacy scenarios will be fixed with #15507.

rolfbjarne · 2022-07-13T20:04:46Z

Mac Catalyst apps crash with like this with this fix: https://gist.github.com/rolfbjarne/293ecffb7fcf3e4169494b81d7e4cd97

thaystg · 2022-07-14T13:51:18Z

Mac Catalyst apps crash with like this with this fix: https://gist.github.com/rolfbjarne/293ecffb7fcf3e4169494b81d7e4cd97

This is exactly the scenario that I was testing and I don't see this crash:
I'm building xamarin-macios locally and then:
cp ./_build/Microsoft.MacCatalyst.Runtime.maccatalyst-x64/runtimes/maccatalyst-x64/native/libxamarin-dotnet-debug.* /Users/thaysgrazia/testJason/bin/Debug/net6.0-maccatalyst/maccatalyst-x64/testJason.app/Contents/MonoBundle/

Then I debug the app and I can see the locals variables correctly.

rolfbjarne · 2022-07-14T13:54:34Z

Mac Catalyst apps crash with like this with this fix: gist.github.com/rolfbjarne/293ecffb7fcf3e4169494b81d7e4cd97

This is exactly the scenario that I was testing and I don't see this crash: I'm building xamarin-macios locally and then: cp ./_build/Microsoft.MacCatalyst.Runtime.maccatalyst-x64/runtimes/maccatalyst-x64/native/libxamarin-dotnet-debug.* /Users/thaysgrazia/testJason/bin/Debug/net6.0-maccatalyst/maccatalyst-x64/testJason.app/Contents/MonoBundle/

Then I debug the app and I can see the locals variables correctly.

You need this commit for it to crash reliably:

9b78d77

Otherwise it only crashes on macOS 10 and macOS 11, on newer versions you get random behavior (which happens to work).

vs-mobiletools-engineering-service2 · 2022-07-14T18:35:36Z

💻 [PR Build] Tests on macOS Mac Catalina (10.15) passed 💻

✅ All tests on macOS Mac Catalina (10.15) passed.

Pipeline on Agent
Hash: 718494f97d77e0a30ddfba9efbc5f3b24bf1e9c4 [PR build]

vs-mobiletools-engineering-service2 · 2022-07-14T18:35:41Z

✅ API diff for current PR / commit

Legacy Xamarin (No breaking changes)

~~iOS~~ (no change detected)
~~tvOS~~ (no change detected)
~~watchOS~~ (no change detected)
~~macOS~~ (no change detected)

NET (empty diffs)

~~iOS~~: (empty diff detected)
~~tvOS~~: (empty diff detected)
~~MacCatalyst~~: (empty diff detected)
~~macOS~~: (empty diff detected)

✅ API diff vs stable

Legacy Xamarin (No breaking changes)

iOS: vsdrops gist (No breaking changes)
tvOS: vsdrops gist (No breaking changes)
watchOS: vsdrops gist (No breaking changes)
macOS: vsdrops gist (No breaking changes)

.NET (No breaking changes)

iOS: vsdrops gist (No breaking changes)
tvOS: vsdrops gist (No breaking changes)
MacCatalyst: vsdrops gist (No breaking changes)
macOS: vsdrops gist (No breaking changes)
Microsoft.iOS vs Microsoft.MacCatalyst: vsdrops gist

Legacy Xamarin (stable) vs .NET

iOS: vsdrops gist
tvOS: vsdrops gist
macOS: vsdrops gist

✅ Generator diff

Generator diff is empty

Pipeline on Agent
Hash: 718494f97d77e0a30ddfba9efbc5f3b24bf1e9c4 [PR build]

vs-mobiletools-engineering-service2 · 2022-07-14T18:39:24Z

📚 [PR Build] Artifacts 📚

Packages generated

View packages

Pipeline on Agent XAMBOT-1099.Monterey'
Hash: 718494f97d77e0a30ddfba9efbc5f3b24bf1e9c4 [PR build]

vs-mobiletools-engineering-service2 · 2022-07-14T18:44:37Z

❌ [PR Build] Tests on macOS M1 - Mac Big Sur (11.5) failed ❌

Failed tests are:

introspection
monotouch-test

Pipeline on Agent
Hash: 718494f97d77e0a30ddfba9efbc5f3b24bf1e9c4 [PR build]

vs-mobiletools-engineering-service2 · 2022-07-15T06:35:45Z

🔥 [CI Build] Test results 🔥

Test results

❌ Tests failed on VSTS: simulator tests

0 tests crashed, 1 tests failed, 222 tests passed.

Failures

❌ dotnettests tests

1 tests failed, 0 tests passed.

Details

DotNet tests: Failed (Execution failed with exit code 1)

Html Report (VSDrops) Download

Successes

✅ bcl: All 69 tests passed. Html Report (VSDrops) Download
✅ cecil: All 1 tests passed. Html Report (VSDrops) Download
✅ fsharp: All 7 tests passed. Html Report (VSDrops) Download
✅ framework: All 8 tests passed. Html Report (VSDrops) Download
✅ generator: All 2 tests passed. Html Report (VSDrops) Download
✅ interdependent_binding_projects: All 7 tests passed. Html Report (VSDrops) Download
✅ install_source: All 1 tests passed. Html Report (VSDrops) Download
✅ introspection: All 8 tests passed. Html Report (VSDrops) Download
✅ linker: All 65 tests passed. Html Report (VSDrops) Download
✅ mac_binding_project: All 1 tests passed. Html Report (VSDrops) Download
✅ mmp: All 2 tests passed. Html Report (VSDrops) Download
✅ mononative: All 12 tests passed. Html Report (VSDrops) Download
✅ monotouch: All 23 tests passed. Html Report (VSDrops) Download
✅ msbuild: All 2 tests passed. Html Report (VSDrops) Download
✅ mtouch: All 1 tests passed. Html Report (VSDrops) Download
✅ xammac: All 3 tests passed. Html Report (VSDrops) Download
✅ xcframework: All 8 tests passed. Html Report (VSDrops) Download
✅ xtro: All 2 tests passed. Html Report (VSDrops) Download

Pipeline on Agent
Hash: [PR build]

rolfbjarne · 2022-07-15T11:32:45Z

Test failures are unrelated (https://github.com/xamarin/maccore/issues/2595).

…otnet#15451) There's a bug in the Mono runtime where the interpreter does not disable optimizations when the debugger is attached, which leads to the interpreter optimizing code and the debugger ending up rather confused. The bug is fixed in the Mono runtime (dotnet/runtime#71436), but there's no immediate way for the runtime to release this fix, so here we're implementing a workaround that disables interpreter optimizations if the debugging is enabled. It's somewhat clunky because the Mono external API wasn't designed for this, so we have to abuse the API a bit to accomplish the effect we want. This is somewhat risky (since we're changing the startup path in a pretty big way), but there's an escape hatch via an environment variable, and also the workaround will not be in effect for release builds. While the runtime issue exists in legacy mono/mono as well, we'll fix the Mono runtime for legacy, because we don't have to wait to consume legacy mono (dotnet#15507). This means that the workaround is for .NET scenarios only. Co-authored-by: Manuel de la Pena <[email protected]> Co-authored-by: Rolf Bjarne Kvinge <[email protected]>

There's a bug in the Mono runtime where the interpreter does not disable optimizations when the debugger is attached, which leads to the interpreter optimizing code and the debugger ending up rather confused. The bug is fixed in the Mono runtime (dotnet/runtime#71436), but there's no immediate way for the runtime to release this fix, so here we're implementing a workaround that disables interpreter optimizations if the debugging is enabled. It's somewhat clunky because the Mono external API wasn't designed for this, so we have to abuse the API a bit to accomplish the effect we want. This is somewhat risky (since we're changing the startup path in a pretty big way), but there's an escape hatch via an environment variable, and also the workaround will not be in effect for release builds. While the runtime issue exists in legacy mono/mono as well, we'll fix the Mono runtime for legacy, because we don't have to wait to consume legacy mono (dotnet#15507). This means that the workaround is for .NET scenarios only. This is a backport of #dotnet#15451.

…15510) There's a bug in the Mono runtime where the interpreter does not disable optimizations when the debugger is attached, which leads to the interpreter optimizing code and the debugger ending up rather confused. The bug is fixed in the Mono runtime (dotnet/runtime#71436), but there's no immediate way for the runtime to release this fix, so here we're implementing a workaround that disables interpreter optimizations if the debugging is enabled. It's somewhat clunky because the Mono external API wasn't designed for this, so we have to abuse the API a bit to accomplish the effect we want. This is somewhat risky (since we're changing the startup path in a pretty big way), but there's an escape hatch via an environment variable, and also the workaround will not be in effect for release builds. While the runtime issue exists in legacy mono/mono as well, we'll fix the Mono runtime for legacy, because we don't have to wait to consume legacy mono (#15507). This means that the workaround is for .NET scenarios only. This is a backport of ##15451. Co-authored-by: Thays Grazia <[email protected]>

…15509) There's a bug in the Mono runtime where the interpreter does not disable optimizations when the debugger is attached, which leads to the interpreter optimizing code and the debugger ending up rather confused. The bug is fixed in the Mono runtime (dotnet/runtime#71436), but there's no immediate way for the runtime to release this fix, so here we're implementing a workaround that disables interpreter optimizations if the debugging is enabled. It's somewhat clunky because the Mono external API wasn't designed for this, so we have to abuse the API a bit to accomplish the effect we want. This is somewhat risky (since we're changing the startup path in a pretty big way), but there's an escape hatch via an environment variable, and also the workaround will not be in effect for release builds. While the runtime issue exists in legacy mono/mono as well, we'll fix the Mono runtime for legacy, because we don't have to wait to consume legacy mono (#15507). This means that the workaround is for .NET scenarios only. This is a backport of ##15451. Co-authored-by: Thays Grazia <[email protected]>

thaystg added 2 commits July 11, 2022 11:59

fix debugger

d05066a

Merge branch 'main' into thays_debug_workaround

359ec53

rolfbjarne reviewed Jul 11, 2022

View reviewed changes

rolfbjarne added the bug If an issue is a bug or a pull request a bug fix label Jul 11, 2022

This comment has been minimized.

Sign in to view

Merge branch 'main' into thays_debug_workaround

610c6f2

thaystg requested a review from lambdageek July 11, 2022 22:08

This comment has been minimized.

Sign in to view

rolfbjarne added 2 commits July 12, 2022 13:56

Hide output, add comment

86ceac3

We only need the workaround for .NET.

88a834b

thaystg commented Jul 12, 2022

View reviewed changes

runtime/monovm-bridge.m Show resolved Hide resolved

Keep calling mono_debug_init if workaround is not in effect.

7bd541b

mauroa reviewed Jul 12, 2022

View reviewed changes

This comment has been minimized.

Sign in to view

Add an escape hatch.

2f8e7c6

NULL-terminate the argc array.

9b78d77

This comment has been minimized.

Sign in to view

Add a dummy argument.

718494f

rolfbjarne approved these changes Jul 15, 2022

View reviewed changes

rolfbjarne marked this pull request as ready for review July 15, 2022 11:35

rolfbjarne changed the title ~~[debugger] Trying to fix debugger and avoid waiting a new runtime release~~ [debugger] Work around a debugger issue when using the interpreter Jul 15, 2022

rolfbjarne merged commit 7cd34d2 into dotnet:main Jul 15, 2022

rolfbjarne mentioned this pull request Jul 15, 2022

[release/6.0.3xx] [debugger] Work around a debugger issue when using the interpreter. #15509

Merged

rolfbjarne mentioned this pull request Jul 15, 2022

[release/6.0.4xx] [debugger] Work around a debugger issue when using the interpreter. #15510

Merged

dalexsoto mentioned this pull request Jul 27, 2022

Revert [debugger] Work around a debugger issue when using the interpreter #15585

Closed

[debugger] Work around a debugger issue when using the interpreter #15451

[debugger] Work around a debugger issue when using the interpreter #15451

Uh oh!

Conversation

thaystg commented Jul 11, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rolfbjarne Jul 11, 2022

Choose a reason for hiding this comment

Uh oh!

thaystg Jul 11, 2022

Choose a reason for hiding this comment

Uh oh!

rolfbjarne Jul 11, 2022

Choose a reason for hiding this comment

Uh oh!

thaystg Jul 11, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

samhouts Jul 12, 2022

Choose a reason for hiding this comment

Uh oh!

thaystg Jul 12, 2022

Choose a reason for hiding this comment

Uh oh!

rolfbjarne commented Jul 11, 2022

Uh oh!

azure-pipelines bot commented Jul 11, 2022

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

mandel-macaque commented Jul 11, 2022

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Uh oh!

mauroa Jul 12, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rolfbjarne Jul 15, 2022

Choose a reason for hiding this comment

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

rolfbjarne commented Jul 13, 2022

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

thaystg commented Jul 14, 2022

Uh oh!

rolfbjarne commented Jul 14, 2022

Uh oh!

vs-mobiletools-engineering-service2 commented Jul 14, 2022

💻 [PR Build] Tests on macOS Mac Catalina (10.15) passed 💻

Uh oh!

vs-mobiletools-engineering-service2 commented Jul 14, 2022

✅ API diff for current PR / commit

✅ API diff vs stable

✅ Generator diff

Uh oh!

vs-mobiletools-engineering-service2 commented Jul 14, 2022

📚 [PR Build] Artifacts 📚

Packages generated

Uh oh!

vs-mobiletools-engineering-service2 commented Jul 14, 2022

❌ [PR Build] Tests on macOS M1 - Mac Big Sur (11.5) failed ❌

thaystg commented Jul 11, 2022 •

edited

Loading

thaystg Jul 11, 2022 •

edited

Loading

mauroa Jul 12, 2022 •

edited

Loading