Initial scaffolding and partial implementation of ilasm in C# #122112

jkoritzinsky · 2025-12-02T19:18:10Z

Start an implementation of ilasm in C# in src/tools/ilasm. ANTLR4 is used to build the grammar.

I decided to structure it as a library (with a frontend to come later) to match ILVerify/ILVerification.

This implementation is far from complete. The grammar is missing any updates from the last year or two and much is untested and unimplemented.

It has one passing test for a simple case so far.

Putting out a PR now so other team members and community members can work on it.

Contributes to #121847

…l representation we can construct a metadata image from.

…concepts (and some easy cases to knock out now)

…reprocessor. Add some basic tests for the preprocessor

…iversal/easier to use

…ods that we can't implement E2E due to SRM limitations (found some!)

… the MetadataBuilder (including methods).

…efs when possible, correctly handle MethodRefSig scenarios.

…iling) unit test.

Copilot

Pull request overview

This PR introduces the initial scaffolding and partial implementation of a C# version of ilasm (IL Assembler) in src/tools/ilasm. The implementation uses ANTLR4 for grammar parsing and is structured as a library with ILAssembler as the main project and ILAssembler.Tests for unit tests. The PR includes basic compilation infrastructure, preprocessor support, metadata building, and one passing test case.

Key changes include:

Core assembler library with grammar visitor pattern for IL parsing
Entity registry system for managing metadata entities
Preprocessor token source for handling #define, #ifdef, and #include directives
Build pipeline configuration to enable testing of the new managed ilasm

Reviewed changes

Copilot reviewed 26 out of 27 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
`ILAssembler/GrammarVisitor.cs`	Main visitor implementation for traversing parsed IL grammar (~4350 lines)
`ILAssembler/EntityRegistry.cs`	Registry and entity management for metadata building (~1300 lines)
`ILAssembler/PreprocessedTokenSource.cs`	Preprocessor implementation for handling directives
`ILAssembler/DocumentCompiler.cs`	Top-level document compilation orchestration
`ILAssembler/NameHelpers.cs`	Utility for splitting dotted names into namespace and name
`ILAssembler.Tests/*`	Unit tests for preprocessor and document compiler
`eng/pipelines/coreclr/ilasm.yml`	Pipeline configuration for building and testing managed ilasm
`eng/Subsets.props`	Build subset configuration for the tools.ilasm subset

src/tools/ilasm/ILAssembler/NameHelpers.cs

src/tools/ilasm/ILAssembler/DocumentCompiler.cs

src/tools/ilasm/ILAssembler/EntityRegistry.cs

src/tools/ilasm/ILAssembler/GrammarVisitor.cs

src/tools/ilasm/ilasm.sln

Co-authored-by: Copilot <[email protected]>

jkotas · 2025-12-03T02:00:32Z

Putting out a PR now so other team members and community members can work on it.

It would be useful to add a punch list of things to work on in the issue.

jkotas · 2025-12-03T01:28:43Z

src/tools/ilasm/src/ILAssembler/ILAssembler.csproj

+    <EnableDefaultCompileItems>true</EnableDefaultCompileItems>
+    <Nullable>enable</Nullable>
+    <NoWarn>$(NoWarn);CS3021</NoWarn>
+    <AllowUnsafeBlocks>true</AllowUnsafeBlocks>


Does this really need unsafe code?

Yes. We have two places that use unsafe code:

We use the BlobBuilder constructor, which takes an unmanaged pointer.

Native ilasm has the ability to specify an ANSI string that should be interpreted as a Unicode string. The APIs on Marshal to get an ANSI string without using unmanaged memory are all internal, so I'm using the APIs that allocate unmanaged memory. To be explicit, I'm not using IntPtr, and the upcoming unsafe definition change would require the code to be marked unsafe anyway.

We use the BlobBuilder constructor, which takes an unmanaged pointer.

Hmmm, that's unfortunately. I am wondering whether it would be possible to introduce a safe S.R.M API to do this.

Native ilasm has the ability to specify an ANSI string that should be interpreted as a Unicode string.

Is it actually using ANSI code page? If I am reading the code correctly, we force the codepage to UTF8 everywhere.

I think you're right. I'll shift it to encode as UTF-8 always.

I'll add the BlobBuilder API to the list of APIs to propose for SRM.

src/tools/ilasm/src/ILAssembler/ILAssembler.csproj

src/tools/ilasm/ILAssembler/ILAssembler.csproj

src/tools/ilasm/src/ILAssembler/NameHelpers.cs

src/tools/ilasm/ILAssembler/InstructionEncoderExtensions.cs

jkotas · 2025-12-03T01:46:11Z

src/tools/ilasm/src/ILAssembler/GrammarVisitor.cs

+        public GrammarResult VisitTls(CILParser.TlsContext context)
+        {
+            // TODO-SRM: System.Reflection.Metadata doesn't provide APIs to point a data declaration at a TLS slot or into the IL stream.
+            // We have tests for the TLS case (CoreCLR only supports it on Win-x86), but not for the IL case.


I think it should be ok to drop the support for managed C++ TLS in ilasm.

into the IL stream

What does this mean?

native ilasm has the ability to point a data declaration at 3 places:

read-only data.

TLS data.

The data stream with method bodies.

We have no tests for the 3rd case, but it does exist in the grammar. We can remove it (especially if we're removing the support for managed C++ TLS).

Just for my education, what does the IL source for data stream with method bodies look like?

In IL, it looks like the following:

.data cil Foo = int32(32)

I think below is the intended use case:

.method public static int32 main() { .data cil byearray ( 00 01 02 /* etc */ ) }

I think the intention was to allow you to specify arbitrary method bodies, but I don't think the emitting actually supports that correctly.

src/tools/ilasm/src/ILAssembler/CIL.g4

jkotas · 2025-12-03T02:01:50Z

cc @dotnet/jit-contrib

hez2010 · 2025-12-03T05:21:07Z

src/tools/ilasm/src/ILAssembler/GrammarVisitor.cs

+                "noncasdemand" => new(DeclarativeSecurityActionEx.NonCasDemand),
+                "noncaslinkdemand" => new(DeclarativeSecurityActionEx.NonCasLinkDemand),
+                "noncasinheritance" => new(DeclarativeSecurityActionEx.NonCasInheritanceDemand),
+                _ => throw new UnreachableException()


It seems runtime async is not included in either GraamaVisitor.cs or CIL.g4. Did you use an old grammar definition before runtime async?

This grammar was written in December of 2023, so yeah it was pre-runtime-async. As I mentioned, it needs some updates.

src/tools/ilasm/ILAssembler/PreprocessedTokenSource.cs

huoyaoyuan · 2025-12-03T12:04:05Z

Putting out a PR now so other team members and community members can work on it.

How are we expected to collaborate on this? Apparently it needs to fulfill more functionalities. Should we merge it as scaffold and open more PRs?

hez2010 · 2025-12-03T13:29:59Z

Well, I'm not against moving ilasm/ildasm to managed, just put my two cents here.
When I was working on the const generics prototype which requires changes to the metadata, I found that as long as I could make ilasm/ildasm work, the coreclr should automatically be able to resolve the new metadata because they share the same code path. By making ilasm/ildasm work, you will end up making coreclr metadata reader barely work at the same time, which gives you the initial direction about what needs to be done to implement a new metadata-level feature. And you can check the output of ildasm to verify whether you have implemented the feature correctly, without the need to inspect all those data structures and (compressed) tokens under a debugger.
This is a great advantage of keeping ilasm/ildasm in C++, so I would like to confirm that we don't care about losing such an ability.

huoyaoyuan · 2025-12-03T14:23:31Z

This is a great advantage of keeping ilasm/ildasm in C++

I think we can continue the discussion in #121847. Since the metadata reader is expected to be replaced by DNMD, the precondition can change here.

jkotas · 2025-12-03T15:08:00Z

How are we expected to collaborate on this? Apparently it needs to fulfill more functionalities. Should we merge it as scaffold and open more PRs?

Yes, it is the idea.

I would like to confirm that we don't care about losing such an ability.

I do not mind with losing this. We actually reached a point where it tends to be easier to prototype new functionality like that in NAOT first. NAOT uses managed System.Reflection.Metadata.

Since the metadata reader is expected to be replaced by DNMD

This is not a committed plan. (If we were to do this, it needs to come with measurable user-observable improvements and it needs to replace the existing metadata reader completely.)

src/tools/ilasm/ilasm.slnx

src/tools/ilasm/src/ILAssembler/InstructionEncoderExtensions.cs

jkoritzinsky added 28 commits December 27, 2023 16:05

Initial start on managed ilasm

1846335

Add grammar in ANTLR4 format so we can get started.

a28f090

Start implementing the visitor that goes from a parse tree to a usefu…

a932050

…l representation we can construct a metadata image from.

Implement most of serInit parsing/serialization.

8b90478

Add parsing for class declaration info as well as some other related …

342f944

…concepts (and some easy cases to knock out now)

Implement looking up typeRefs and most of the support for typeSpecs

ce8753d

Implement signature parsing and the start of method parsing and the p…

d560635

…reprocessor. Add some basic tests for the preprocessor

Start work on method bodies and refactor flags handling to be more un…

d950d48

…iversal/easier to use

Implement instruction and exception handling emit. Start marking meth…

9a3861a

…ods that we can't implement E2E due to SRM limitations (found some!)

Implement all non-debugging branches of methodDecl

616fc4f

Implement property and event parsing and clean up grammar

c97a514

Add TODOs for the unimplemented nodes

4bb3db9

Get build passing in VS

fb4f372

Implement more cases

83d1884

Implement manifest resources.

809c33f

Set method entrypoint

32c4e0e

Move a little code around

508e1aa

Finish up exported types and start on emitting

874e98f

Add token resolution and add the first few tables of writing out into…

48971f6

… the MetadataBuilder (including methods).

Handle Param and InterfaceImpl tables

e6ed4d9

Write out the rest of the easy tables.

7aafa5c

Implement MethodSpec parsing correctly, resolve MemberRefs to MethodD…

b9ca5f9

…efs when possible, correctly handle MethodRefSig scenarios.

Finish up DocumentParser.Parse so we can start testing basic scenarios.

6cc9396

Depend on an LKG SRM instead of a live one. Write the first (still fa…

a21e185

…iling) unit test.

Get the first compilation test passing!

ba1b2e8

Merge branch 'main' of https://github.com/dotnet/runtime into ilasm

d7bd8a3

Hook up build infra

b2ffc81

Upgrade ilasm to NetCoreAppToolCurrent

df76130

jkoritzinsky added the area-ILTools-coreclr label Dec 2, 2025

Copilot AI review requested due to automatic review settings December 2, 2025 19:18

Copilot finished reviewing on behalf of jkoritzinsky December 2, 2025 19:20

Copilot AI reviewed Dec 2, 2025

View reviewed changes

am11 reviewed Dec 2, 2025

View reviewed changes

src/tools/ilasm/ilasm.sln Outdated Show resolved Hide resolved

AaronRobinsonMSFT marked this pull request as draft December 2, 2025 19:34

AaronRobinsonMSFT added this to the Future milestone Dec 2, 2025

jkoritzinsky and others added 2 commits December 2, 2025 14:27

Apply suggestions from code review

f128815

Co-authored-by: Copilot <[email protected]>

Migrate to slnx

4669f81

jkotas reviewed Dec 3, 2025

View reviewed changes

hez2010 reviewed Dec 3, 2025

View reviewed changes

am11 reviewed Dec 3, 2025

View reviewed changes

src/tools/ilasm/ILAssembler/PreprocessedTokenSource.cs Outdated Show resolved Hide resolved

PR feedback

6327d92

jkotas reviewed Dec 5, 2025

View reviewed changes

src/tools/ilasm/ilasm.slnx Outdated Show resolved Hide resolved

jkotas reviewed Dec 5, 2025

View reviewed changes

src/tools/ilasm/src/ILAssembler/InstructionEncoderExtensions.cs Show resolved Hide resolved

jkoritzinsky added 3 commits December 5, 2025 18:48

PR feedback

c7ba930

Reduce unsafe to the minimum locations

5fc486b

Fix typo caught by code-anlysis

fbc4aeb

build-analysis bot mentioned this pull request Dec 5, 2025

System.Security.Cryptography.Tests .IsSupported_AgreesWithPlatform failures #122228

Open

Initial scaffolding and partial implementation of ilasm in C# #122112

Are you sure you want to change the base?

Initial scaffolding and partial implementation of ilasm in C# #122112

Uh oh!

Conversation

jkoritzinsky commented Dec 2, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jkotas commented Dec 3, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

jkotas commented Dec 3, 2025

Uh oh!

hez2010 Dec 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

huoyaoyuan commented Dec 3, 2025

Uh oh!

hez2010 commented Dec 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

huoyaoyuan commented Dec 3, 2025

Uh oh!

jkotas commented Dec 3, 2025

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

hez2010 Dec 3, 2025 •

edited

Loading

hez2010 commented Dec 3, 2025 •

edited

Loading