Interop source generator testing improvements

[jkoritzinsky]

Now that the majority of the new interop features are based on source-generators, we plan on improving our testing story. Our existing tests around diagnostics we issue from the source generators is pretty bare-bones, and there's some places where we can end up in a bad state when working on new features. We also want to improve our test coverage. Below is a list of improvements we plan to make in our test suites.

Improvements to existing test infrastructure

• Change our general compilation tests to use the Source Generator testing SDK from dotnet/roslyn-sdk (without the generated sources matching)
• Provide a mechanism to conditionally reference Ancillary.Interop in the compilation tests, to ensure that we ship all of the types we need to compile the various test cases that we expect to work in the shipping product.
• Refactor out some of the incremental testing infrastructure to be more easily reused from our various different generator tests (and possibly usable from other non-interop generator testing as well)

Additional testing for the COM source generators

• Add tests around the attributes we place on the generated code and expect to be placed on the generated code.
• Add tests for incrementality to ensure that we're only re-generating code when expected.

Testing for Microsoft.Interop.SourceGeneration

We have a lot of shared code in this library that we have aspirations to ship separately in the future. Currently, all testing of this library is done through our end-to-end testing of our interop source generators. We plan to add unit tests for the various components in this library.

• Add unit tests for MarshallingInfoParser and related types.
• Add unit tests for our in-box IMarshallingGeneratorFactory implementations.
• Add unit tests for our various helper methods.
• Add unit tests for BoundGenerators
• Add unit tests for VariableDeclarations
• Add unit tests for SignatureContext and related types.

[sharwell]

Add tests for incrementality to ensure that we're only re-generating code when expected.

I'd love to add support for this somewhere in the testing library. In theory, the test would be structured like this:

• As input, the test provides the original project state (same as we have today) and a delegate to apply a transformation to that project
• The infrastructure somehow returns a GeneratorRunResult for the incremental update after applying the transformation
• The test writes assertions surrounding TrackedSteps and/or TrackedOutputSteps

In practice it's not exactly trivial so still trying to build a proof of concept.

[ghost]

Tagging subscribers to this area: @dotnet/interop-contrib
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Now that the majority of the new interop features are based on source-generators, we plan on improving our testing story. Our existing tests around diagnostics we issue from the source generators is pretty bare-bones, and there's some places where we can end up in a bad state when working on new features. We also want to improve our test coverage. Below is a list of improvements we plan to make in our test suites.

Improvements to existing test infrastructure

• Change our general compilation tests to use the Source Generator testing SDK from dotnet/roslyn-sdk (without the generated sources matching)
• Provide a mechanism to conditionally reference Ancillary.Interop in the compilation tests, to ensure that we ship all of the types we need to compile the various test cases that we expect to work in the shipping product.
• Refactor out some of the incremental testing infrastructure to be more easily reused from our various different generator tests (and possibly usable from other non-interop generator testing as well)

Additional testing for the COM source generators

• Add tests around the attributes we place on the generated code and expect to be placed on the generated code.
• Add tests for incrementality to ensure that we're only re-generating code when expected.

Testing for Microsoft.Interop.SourceGeneration

We have a lot of shared code in this library that we have aspirations to ship separately in the future. Currently, all testing of this library is done through our end-to-end testing of our interop source generators. We plan to add unit tests for the various components in this library.

• Add unit tests for MarshallingInfoParser and related types.
• Add unit tests for our in-box IMarshallingGeneratorFactory implementations.
• Add unit tests for our various helper methods.
• Add unit tests for BoundGenerators
• Add unit tests for VariableDeclarations
• Add unit tests for SignatureContext and related types.

Author:	jkoritzinsky
Assignees:	-
Labels:	area-System.Runtime.InteropServices, test-enhancement, source-generator
Milestone:	-

[Sergio0694]

+1 on some tooling support for testing incrementality across incremental runs, would love to use that in the MVVM Toolkit and ComputeSharp 😄