JIT: Fix unnecessary GC transitions recorded for tailcalls

[Copilot]

Fix JIT: Unnecessary GC transitions recorded for tailcalls

Plan:

• Explore and understand the issue
• Identify all affected files (5 files total)
• Modify emitxarch.cpp to skip register masking for tail calls
• Modify emitarm.cpp to skip register masking for tail calls
• Modify emitarm64.cpp to skip register masking for tail calls
• Modify emitloongarch64.cpp to skip register masking for tail calls
• Modify emitriscv64.cpp to skip register masking for tail calls
• Build CoreCLR to verify compilation
• Refactor to use ternary operator for more concise code
• Rebuild CoreCLR to verify refactored changes

Issue Summary:

The emitIns_Call function in all JIT emitters incorrectly trims callee-trashed registers from the live set for tail calls (jumps). For tail calls, the function doesn't return but jumps to another function, so the registers should remain live and not trigger unnecessary GC transitions.

Changes Made:

Modified all 5 architecture-specific emitter files to use a ternary operator for savedSet, setting it to RBM_ALLINT (all integer registers) for tail calls instead of calling emitGetGCRegsSavedOrModified:

1. src/coreclr/jit/emitxarch.cpp - Using ternary operator for savedSet
2. src/coreclr/jit/emitarm.cpp - Using ternary operator for savedSet
3. src/coreclr/jit/emitarm64.cpp - Using ternary operator for savedSet
4. src/coreclr/jit/emitloongarch64.cpp - Using ternary operator for savedSet
5. src/coreclr/jit/emitriscv64.cpp - Using ternary operator for savedSet

The refactored implementation is more concise while maintaining the same logic:

regMaskTP savedSet  = params.isJump ? RBM_ALLINT : emitGetGCRegsSavedOrModified(params.methHnd);
regMaskTP gcrefRegs = params.gcrefRegs & savedSet;
regMaskTP byrefRegs = params.byrefRegs & savedSet;

Original prompt

---

This section details on the original issue you should resolve

<issue_title>JIT: Unnecessary GC transitions recorded for tailcalls</issue_title>
<issue_description>The logic in emitIns_Call removes callee-trashed registers from the saved set recorded:

runtime/src/coreclr/jit/emitxarch.cpp

Lines 11174 to 11177 in 8cb57ab

	// Trim out any callee-trashed registers from the live set.
	regMaskTP savedSet = emitGetGCRegsSavedOrModified(params.methHnd);
	regMaskTP gcrefRegs = params.gcrefRegs & savedSet;
	regMaskTP byrefRegs = params.byrefRegs & savedSet;

For jumps this does not really make sense. We should be able to avoid some unnecessary GC transitions by avoiding changing the gcref regs/byref regs when params.isJump.</issue_description>

<agent_instructions>Note that emitIns_Call exists in multiple targets and all of them should be changed</agent_instructions>

Comments on the Issue (you are @copilot in this section)

@jakobbotsch Noticed this while looking at recording proper GC info for the trampolines in #120303. Those are emitted via `emitIns_Call` with `params.isJump = true`, and currently the trampolines end up as ```asm G_M53658_IG12: ; bbWeight=1, gcrefRegs=0002 {rcx}, byrefRegs=0004 {rdx}, byref ; gcrRegs +[rcx] ; byrRegs +[rdx] jmp (dynamicClass):IL_STUB_AsyncResume_Foo_Tier0(System.Object,byref):System.Object ; gcrRegs -[rcx] ; byrRegs -[rdx] ;; size=5 bbWeight=1 PerfScore 2.00 G_M53658_IG13: ; bbWeight=1, gcrefRegs=0002 {rcx}, byrefRegs=0004 {rdx}, byref ; gcrRegs +[rcx] ; byrRegs +[rdx] jmp (dynamicClass):IL_STUB_AsyncResume_Foo_Tier0(System.Object,byref):System.Object ; gcrRegs -[rcx] ; byrRegs -[rdx] ;; size=5 bbWeight=1 PerfScore 2.00 ``` with multiple superfluous recorded GC info transitions.

Fixes #120933

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

[jakobbotsch]

@copilot This fix looks good, but can we make it a bit terser by using a ternary for savedSet and making it equal RBM_ALLINT in the params.isJump case?

[Copilot]

Done in commit 9c6c876. Used ternary operator with RBM_ALLINT for tail calls across all 5 architecture emitters.

[jakobbotsch]

/azp run runtime-coreclr gcstress0x3-gcstress0xc

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[Copilot]

Pull Request Overview

This PR fixes unnecessary GC transitions being recorded for tail calls in the JIT compiler. For tail calls (jumps), the function doesn't return but jumps to another function, so registers should remain live without triggering GC transitions.

Key Changes:

• Modified the emitIns_Call function across all 5 architecture-specific emitters to skip register trimming for tail calls
• Changed savedSet calculation to use RBM_ALLINT (all integer registers) when params.isJump is true, preventing unnecessary GC info transitions

Reviewed Changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
src/coreclr/jit/emitxarch.cpp	Updated savedSet calculation for x86/x64 to skip register trimming on tail calls
src/coreclr/jit/emitriscv64.cpp	Updated savedSet calculation for RISC-V 64-bit to skip register trimming on tail calls
src/coreclr/jit/emitloongarch64.cpp	Updated savedSet calculation for LoongArch 64-bit to skip register trimming on tail calls
src/coreclr/jit/emitarm64.cpp	Updated savedSet calculation for ARM64 to skip register trimming on tail calls
src/coreclr/jit/emitarm.cpp	Updated savedSet calculation for ARM to skip register trimming on tail calls

[dotnet-policy-service]

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.