[RISC-V] Enable On Stack Replacement #96558

sirntar · 2024-01-05T15:46:22Z

IMPORTANT: Before this PR could be merge, it requires changes from #96136 changes have been merged

This pull request:

redesign PROLOG/EPILOG in order to enable OSR (now they are more similar to LA64 architecture)
implements genAllocLclFrame with genStackProbe ~~to be use in order to probe stack~~ I disabled genAllocLclFrame method for now
fixes the cast of floating-point types to uint32 in volatileLocal1.dll
fixes the register conflict for large offset addresses of OSR root frames when DOTNET_JitStressRegs=2 in more_tailcalls.sh
~~fixes an assertion failure in debug mode with verbose switch while removing conditional jump block~~ moved to separate PR (Replace call to GetFalseTarget method with GetTarget inside fgRemoveConditionalJump #96560)

Part of #84834
cc @wscho77 @HJLeee @clamp03 @JongHeonChoi @t-mustafin @gbalykov @viewizard @ashaurtaev @brucehoult @tomeksowi @yurai007 @Bajtazar

ghost · 2024-01-05T15:46:35Z

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

Issue Details

IMPORTANT: Before this PR could be merge, it requires changes from #96136

This pull request:

redesign PROLOG/EPILOG in order to enable OSR (now they are more similar to LA64 architecture)
implements genAllocLclFrame with genSmallStackProbe and genStackProbe to be use in order to probe stack
fixes the cast of floating-point types to uint32 in volatileLocal1.dll
fixes the register conflict for large offset addresses of OSR root frames when DOTNET_JitStressRegs=2 in more_tailcalls.sh
fixes an assertion failure in debug mode with verbose switch while removing conditional jump block

Part of #84834
cc @wscho77 @HJLeee @clamp03 @JongHeonChoi @t-mustafin @gbalykov @viewizard @ashaurtaev @brucehoult @tomeksowi @yurai007 @Bajtazar

Author:	sirntar
Assignees:	-
Labels:	`area-CodeGen-coreclr`, `community-contribution`
Milestone:	-

sirntar · 2024-01-05T15:56:15Z

fixes an assertion failure in debug mode with verbose switch while removing conditional jump block

I'm referring here to #95773. After this PR I had a problem to get pass through mainlooptry3 test. Back then I've been using debug compilation with verbose option turned on.

Bug details

// A fragment of method from /runtime/src/coreclr/jit/fgopt.cpp
void Compiler::fgRemoveConditionalJump(BasicBlock* block)
{
    noway_assert(block->KindIs(BBJ_COND) && block->TrueTargetIs(block->GetFalseTarget()));
    assert(compRationalIRForm == block->IsLIR());

    FlowEdge* flow = fgGetPredForBlock(block->GetFalseTarget(), block);
    noway_assert(flow->getDupCount() == 2);

    // Change the BBJ_COND to BBJ_ALWAYS, and adjust the refCount and dupCount.
    --block->GetFalseTarget()->bbRefs;
    block->SetKind(BBJ_ALWAYS);
    block->SetFlags(BBF_NONE_QUIRK);
    flow->decrementDupCount();

#ifdef DEBUG
    if (verbose)
    {
      // BUG HERE:  block is now BBJ_ALWAYS, not BBJ_COND, 
     //  so assertion inside  GetFalseTarget method will fail; correct method to call should be GetTarget
        printf("Block " FMT_BB " becoming a BBJ_ALWAYS to " FMT_BB " (jump target is the same whether the condition"
               " is true or false)\n",
               block->bbNum, block->GetFalseTarget()->bbNum); 
    }
#endif
//...

jakobbotsch · 2024-01-05T16:04:34Z

               block->bbNum, block->GetFalseTarget()->bbNum);

Do you mind submitting a separate PR for this fix?

sirntar · 2024-01-05T16:07:46Z

               block->bbNum, block->GetFalseTarget()->bbNum); 
Do you mind submitting a separate PR for this fix?

Not at all, will do that in a minute

src/coreclr/jit/codegenriscv64.cpp

Note: Pinned local test is failing.

Co-authored-by: Tomasz Sowiński <[email protected]>

sirntar · 2024-01-11T11:57:22Z

Remove stack probing (genAllocLclFrame)

After discussion with @tomeksowi, I decided to run a lot of tests on both the qemu and the VisionFive2 board. I did not find any problems that could be solved with this method, nor any reason to continue using it. So I decided to disable it for now. I left its implementation in this PR just in case.

About tests

On both: the qemu and the VisionFive2 board all OSR tests are passing with success.

clamp03 · 2024-01-12T00:40:21Z

src/coreclr/jit/codegenriscv64.cpp

        emit->emitIns_R_R_R(INS_sub, EA_PTRSIZE, regCnt, REG_SPBASE, regCnt);

        // Overflow, set regCnt to lowest possible value
        emit->emitIns_R_R_I(INS_beq, EA_PTRSIZE, tempReg, REG_R0, 2 << 2);
        emit->emitIns_R_R_I(INS_addi, EA_PTRSIZE, regCnt, REG_R0, 0);

-        assert(compiler->eeGetPageSize() == ((compiler->eeGetPageSize() >> 12) << 12));
-        emit->emitIns_R_I(INS_lui, EA_PTRSIZE, regTmp, compiler->eeGetPageSize() >> 12);
+        GetEmitter()->emitIns_R_I(INS_lui, EA_PTRSIZE, rPageSize, pageSize >> 12);


Please update GetEmitter() to emit

I think using emit instead of GetEmitter() is a good practice, even if GetEmitter() is marked as inline. I discussed this with the team and I decided to replace GetEmitter() with emit in all my changes.

clamp03 · 2024-01-12T01:22:17Z

src/coreclr/jit/codegenriscv64.cpp


+    // each instr is 4 bytes
+    // if (rOffset >= rLimit) goto Loop;
+    GetEmitter()->emitIns_R_R_I(INS_bge, EA_PTRSIZE, rOffset, rLimit, -2 << 2);


IMO, rLimit should be -frameSize + REG_SPBASE and initial value of rOffset should be REG_SPBASE - rPageSize (and exchange INS_sub and INS_lw in the loop)? Could you please check?

IMO, rLimit should be -frameSize + REG_SPBASE and initial value of rOffset should be REG_SPBASE - rPageSize

Nice catch! Thanks, of course it should be -frameSize + REG_SPBASE. After the update, the assembly should look more or less like this:

; load rLimit, -frameSize add rLimit, rLimit, REG_SPBASE ; rLimit <-- (-frameSize + REG_SPBASE) lui rPageSize, pageSize >> 12 ; rPageSize <-- 0x1000 sub rOffset, REG_SPBASE, rPageSize ; rOffset <-- (REG_SPBASE - rPageSize) ; loop: lw R0, rOffset ; tickle the page sub rOffset, rOffset, rPageSize bge rOffset, rLimit, -2 << 2

src/coreclr/jit/codegenriscv64.cpp

…CalleeSavedRegisters

sirntar · 2024-01-15T08:21:06Z

@jakobbotsch I think it's ready for review. As for the test failures, I don't think they are related to the changes. Can you check it?

jakobbotsch · 2024-01-15T09:18:24Z

cc @dotnet/jit-contrib. @AndyAyersMS can you please take a look?

jakobbotsch · 2024-01-15T09:39:01Z

for the test failures, I don't think they are related to the changes. Can you check it?

Yep, they all look like known issues, #94728, #96876, dotnet/dnceng#1423, #96904

AndyAyersMS

Cool to see this. I did not look at the codegen in detail.

Have you tried running any form of OSR stress (eg setting the various config values very low (see here for details)? We do this in the jit-experimental CI run and it has been useful for finding bugs.

sirntar · 2024-01-17T14:13:53Z

@AndyAyersMS

Have you tried running any form of OSR stress (eg setting the various config values very low (see here for details)? We do this in the jit-experimental CI run and it has been useful for finding bugs.

Since morning I've been playing with stress tests by setting different values for DOTNET_OSR_HitCount, DOTNET_OSR_CounterBump and DOTNET_TC_OnStackReplacement_InitialCounter. In each configuration, the OSR test pass rate is 100% on both the QEMU and VisionFive2 board.

* [RISC-V] Implement On Stack Replacement Note: Pinned local test is failing. * [RISC-V] Apply suggestions from code review Co-authored-by: Tomasz Sowiński <[email protected]> * [RISC-V] apply jit-format * [RISC-V] Cosmetic changes after code review * [RISC-V] Changes assuming memory page is always equal 4KiB * [RISC-V] Remove stack probing * [RISC-V] Replace GetEmitter() with emit * [RISC-V] Sync frame type 1 in genPushCalleeSavedRegisters with genPopCalleeSavedRegisters * [RISC-V] Fix assembly emmited by genStackProbe * [RISC-V] Apply jit-formatter --------- Co-authored-by: Tomasz Sowiński <[email protected]>

ghost added the community-contribution Indicates that the PR has been added by a community member label Jan 5, 2024

dotnet-issue-labeler bot added the area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI label Jan 5, 2024

sirntar mentioned this pull request Jan 5, 2024

Replace call to GetFalseTarget method with GetTarget inside fgRemoveConditionalJump #96560

Merged

build-analysis bot mentioned this pull request Jan 5, 2024

Checkout failure: "Git fetch failed with exit code 128" dotnet/arcade#9009

Open

2 tasks

clamp03 assigned sirntar Jan 6, 2024

clamp03 added the arch-riscv Related to the RISC-V architecture label Jan 6, 2024

tomeksowi suggested changes Jan 8, 2024

View reviewed changes

sirntar force-pushed the rv64-osr branch from 166fd32 to 2999d82 Compare January 8, 2024 08:55

This was referenced Jan 8, 2024

Test failure - System.NullReferenceException in System.Threading.Lock.TryInitializeStatics #94728

Closed

NRE in iOS.Device.Aot.Test when starting up xharness #96403

Closed

sirntar force-pushed the rv64-osr branch from d27cf16 to 9255351 Compare January 10, 2024 13:47

tomeksowi approved these changes Jan 11, 2024

View reviewed changes

sirntar and others added 6 commits January 11, 2024 12:42

[RISC-V] Implement On Stack Replacement

bfe57d4

Note: Pinned local test is failing.

[RISC-V] Apply suggestions from code review

0ceba9b

Co-authored-by: Tomasz Sowiński <[email protected]>

[RISC-V] apply jit-format

c7d36a4

[RISC-V] Cosmetic changes after code review

ebf6edb

[RISC-V] Changes assuming memory page is always equal 4KiB

c91bdb7

[RISC-V] Remove stack probing

1ed090d

sirntar force-pushed the rv64-osr branch from 9255351 to 1ed090d Compare January 11, 2024 11:42

sirntar marked this pull request as ready for review January 11, 2024 11:44

clamp03 reviewed Jan 12, 2024

View reviewed changes

src/coreclr/jit/codegenriscv64.cpp Outdated Show resolved Hide resolved

sirntar added 2 commits January 12, 2024 11:00

[RISC-V] Replace GetEmitter() with emit

c448ff5

[RISC-V] Sync frame type 1 in genPushCalleeSavedRegisters with genPop…

900c77a

…CalleeSavedRegisters

[RISC-V] Fix assembly emmited by genStackProbe

6a0c6ca

clamp03 approved these changes Jan 12, 2024

View reviewed changes

[RISC-V] Apply jit-formatter

006061e

This was referenced Jan 12, 2024

pip fails to build wheels for cffi (gcc not found) in ubuntu.1804.armarch.open using image alpine-3.15-helix-arm32v7 dotnet/dnceng#1423

Open

CollectionsMarshal.AsSpan assertion failure in System.Text.Json tests #96876

Closed

clamp03 requested review from jakobbotsch and shushanhf January 15, 2024 08:35

jakobbotsch requested a review from AndyAyersMS January 15, 2024 09:18

AndyAyersMS approved these changes Jan 16, 2024

View reviewed changes

ashaurtaev approved these changes Jan 17, 2024

View reviewed changes

jakobbotsch approved these changes Jan 22, 2024

View reviewed changes

jakobbotsch merged commit 99b621f into dotnet:main Jan 22, 2024
123 of 129 checks passed

github-actions bot locked and limited conversation to collaborators Feb 22, 2024

sirntar deleted the rv64-osr branch October 9, 2024 10:02

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[RISC-V] Enable On Stack Replacement #96558

[RISC-V] Enable On Stack Replacement #96558

sirntar commented Jan 5, 2024 •

edited

Loading

ghost commented Jan 5, 2024

sirntar commented Jan 5, 2024

jakobbotsch commented Jan 5, 2024

sirntar commented Jan 5, 2024

sirntar commented Jan 11, 2024 •

edited

Loading

clamp03 Jan 12, 2024

sirntar Jan 12, 2024

clamp03 Jan 12, 2024 •

edited

Loading

sirntar Jan 12, 2024

sirntar commented Jan 15, 2024

jakobbotsch commented Jan 15, 2024

jakobbotsch commented Jan 15, 2024

AndyAyersMS left a comment

sirntar commented Jan 17, 2024 •

edited

Loading

[RISC-V] Enable On Stack Replacement #96558

[RISC-V] Enable On Stack Replacement #96558

Conversation

sirntar commented Jan 5, 2024 • edited Loading

ghost commented Jan 5, 2024

sirntar commented Jan 5, 2024

jakobbotsch commented Jan 5, 2024

sirntar commented Jan 5, 2024

sirntar commented Jan 11, 2024 • edited Loading

Remove stack probing (genAllocLclFrame)

About tests

clamp03 Jan 12, 2024

Choose a reason for hiding this comment

sirntar Jan 12, 2024

Choose a reason for hiding this comment

clamp03 Jan 12, 2024 • edited Loading

Choose a reason for hiding this comment

sirntar Jan 12, 2024

Choose a reason for hiding this comment

sirntar commented Jan 15, 2024

jakobbotsch commented Jan 15, 2024

jakobbotsch commented Jan 15, 2024

AndyAyersMS left a comment

Choose a reason for hiding this comment

sirntar commented Jan 17, 2024 • edited Loading

sirntar commented Jan 5, 2024 •

edited

Loading

sirntar commented Jan 11, 2024 •

edited

Loading

clamp03 Jan 12, 2024 •

edited

Loading

sirntar commented Jan 17, 2024 •

edited

Loading