[Coverage][MCDC] Prepare MCDC Builder for pattern matching and llvm 19 #126677

ZhuUx · 2024-06-19T09:33:45Z

Prepare for pattern matching decisions and adaption to llvm-19.

Changes

MCDCBuilder is refactored. The main change is the way to calculate depth of decisions and that DecisionCtx is responsible to process decisions (for now we have just BooleanDecisionCtx, while in Support mcdc analysis for pattern matching #124278 we introduce MatchingDecisionCtx). We use MCDCState to store context processing decisions. If a nested decision is to be introduced, the parent's state will be "stashed" and a new state will be created for the nested decision. Once the nested decision is finished the parent's state is "unstashed" and states for nested decisions will be removed later. The new mechanism can fix that nested decisions in top-level boolean expressions like a || func(b && c) can not be processed properly by current nightly compiler, as tests in nested_in_boolean_exprs.rs.
MCDCBranchSpan and MCDCBranch can hold multiple true markers or false markers. This is for matching patterns.
MCDC spans are grouped by decisions till instrument, in case some branches could not generate mappings but decision mappings still count them.

rustbot · 2024-06-19T09:33:53Z

rustbot has assigned @TaKO8Ki.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

rustbot · 2024-06-19T09:33:54Z

Some changes occurred to MIR optimizations

cc @rust-lang/wg-mir-opt

Some changes occurred in coverage tests.

cc @Zalathar

Some changes occurred in coverage instrumentation.

cc @Zalathar

Some changes occurred in coverage instrumentation.

cc @Zalathar

Some changes occurred in coverage instrumentation.

cc @Zalathar

ZhuUx · 2024-06-19T09:35:37Z

cc @RenjiSann

Zalathar · 2024-06-19T10:47:21Z

@rustbot label +A-code-coverage

Zalathar · 2024-06-19T10:50:06Z

By the way, I think the MC/DC limit warnings should probably be lints instead of ordinary warnings, so that they interact with the #[allow(..)] and #[deny(..)] attributes.

I don't think that concern should block this PR, but we should remember to fix it at some point.

ZhuUx · 2024-06-19T11:22:27Z

By the way, I think the MC/DC limit warnings should probably be lints instead of ordinary warnings, so that they interact with the #[allow(..)] and #[deny(..)] attributes.

I don't think that concern should block this PR, but we should remember to fix it at some point.

That's meaningful. #82448 also introduces two options to control the limit. I think we can fix that with these options once llvm-19 is released.

bors · 2024-06-30T22:26:31Z

☔ The latest upstream changes (presumably #127174) made this pull request unmergeable. Please resolve the merge conflicts.

rustbot · 2024-07-03T08:35:37Z

Some changes occurred in match lowering

cc @Nadrieril

compiler/rustc_mir_build/src/build/expr/into.rs

ZhuUx · 2024-07-04T10:35:18Z

Some commits are shared with #127234 now. I wish to merge that first.

bors · 2024-07-05T14:12:14Z

☔ The latest upstream changes (presumably #127360) made this pull request unmergeable. Please resolve the merge conflicts.

ZhuUx · 2024-07-10T03:00:41Z

Since #127234 has been merged we can work on this, which lays down the base for #126733 and #124278

ZhuUx · 2024-07-15T05:56:25Z

@Zalathar @TaKO8Ki This pr is prepared. Could we start reviewing?

compiler/rustc_mir_transform/src/coverage/mod.rs

Zalathar · 2024-07-15T06:47:04Z

compiler/rustc_mir_transform/src/coverage/mod.rs

+        &hir_info,
+        &mut extracted_mappings,
+        &basic_coverage_blocks,
+        &mut coverage_counters,


I've been thinking about this &mut coverage_counters, because I did the same thing in my draft of #124154, but now I think it's the wrong approach.

Instead, I think we should try to teach CoverageCounters::make_bcb_counters how to create the complex expressions that will be needed later, so that all the counter-creation happens together.

Then some keys may be essential to mappings so that we can get these sum expressions from coverage_counters later, sounds a bit redundant.
Currently we only need to create Expression but not Counter here. What are the drawbacks of the current approach?

After some more thought, I think it's OK to do this for now.

I still have mixed feelings about it, but I don't know what the specific alternative should look like, and I don't think this change should have to wait for me to figure that out.

compiler/rustc_mir_transform/src/coverage/mod.rs

ZhuUx · 2024-07-15T07:53:30Z

I would add commit for each fix. After all are done I will squash them into the primary commits (the target commits are hinted at head of the message)

compiler/rustc_mir_transform/src/coverage/mod.rs

compiler/rustc_middle/src/mir/coverage.rs

coverage: Restrict `ExpressionUsed` simplification to `Code` mappings In the future, branch and MC/DC mappings might have expressions that don't correspond to any single point in the control-flow graph. That makes it trickier to keep track of which expressions should expect an `ExpressionUsed` node. We therefore sidestep that complexity by only performing `ExpressionUsed` simplification for expressions associated directly with ordinary `Code` mappings. (This simplification step is inherited from the original coverage implementation, which only supported `Code` mappings anyway, so there's no particular reason to extend it to other kinds of mappings unless we specifically choose to.) Relevant to: - rust-lang#124154 - rust-lang#126677 - rust-lang#124278 `@rustbot` label +A-code-coverage

coverage: Restrict `ExpressionUsed` simplification to `Code` mappings In the future, branch and MC/DC mappings might have expressions that don't correspond to any single point in the control-flow graph. That makes it trickier to keep track of which expressions should expect an `ExpressionUsed` node. We therefore sidestep that complexity by only performing `ExpressionUsed` simplification for expressions associated directly with ordinary `Code` mappings. (This simplification step is inherited from the original coverage implementation, which only supported `Code` mappings anyway, so there's no particular reason to extend it to other kinds of mappings unless we specifically choose to.) Relevant to: - rust-lang#124154 - rust-lang#126677 - rust-lang#124278 ``@rustbot`` label +A-code-coverage

coverage: Restrict `ExpressionUsed` simplification to `Code` mappings In the future, branch and MC/DC mappings might have expressions that don't correspond to any single point in the control-flow graph. That makes it trickier to keep track of which expressions should expect an `ExpressionUsed` node. We therefore sidestep that complexity by only performing `ExpressionUsed` simplification for expressions associated directly with ordinary `Code` mappings. (This simplification step is inherited from the original coverage implementation, which only supported `Code` mappings anyway, so there's no particular reason to extend it to other kinds of mappings unless we specifically choose to.) Relevant to: - rust-lang#124154 - rust-lang#126677 - rust-lang#124278 ```@rustbot``` label +A-code-coverage

Rollup merge of rust-lang#127758 - Zalathar:expression-used, r=oli-obk coverage: Restrict `ExpressionUsed` simplification to `Code` mappings In the future, branch and MC/DC mappings might have expressions that don't correspond to any single point in the control-flow graph. That makes it trickier to keep track of which expressions should expect an `ExpressionUsed` node. We therefore sidestep that complexity by only performing `ExpressionUsed` simplification for expressions associated directly with ordinary `Code` mappings. (This simplification step is inherited from the original coverage implementation, which only supported `Code` mappings anyway, so there's no particular reason to extend it to other kinds of mappings unless we specifically choose to.) Relevant to: - rust-lang#124154 - rust-lang#126677 - rust-lang#124278 ```@rustbot``` label +A-code-coverage

…n and change the way to calculate decision depth

…gs and support multiple branch markers

Zalathar · 2024-07-25T05:39:59Z

It's not clear to me which parts of this are for LLVM 19, which parts are for pattern matching, and why they are combined in one PR.

Is there some reason why the changes needed for LLVM 19 can't proceed separately from the changes needed for pattern matching?

ZhuUx · 2024-07-25T06:50:07Z

It's not clear to me which parts of this are for LLVM 19, which parts are for pattern matching, and why they are combined in one PR.

The change for llvm 19 is group decision and its conditions to mcdc_spans so that we can calculate index of test vectors at extract_mcdc_mappings.
(At first I do it in mcdc.rs, but after I found that for loops also generate pattern matching decisions here I moved it to extract_mcdc_mappings, where decisions from for loops are removed due to span).

Is there some reason why the changes needed for LLVM 19 can't proceed separately from the changes needed for pattern matching?

No, actually they can be separated indeed as long as we separate degraded mcdc branches from mcdc spans early, as the change introduced at the last commit of this pr. So I worked on patches for llvm 19 based this.
Maybe we could make that separately for llvm 19?

ZhuUx · 2024-07-25T08:13:33Z

I have separated all work for llvm-19, this pr is no longer needed. Pattern matching will still be based on that.

rustbot assigned TaKO8Ki Jun 19, 2024

rustbot added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jun 19, 2024

rustbot added the T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. label Jun 19, 2024

rustbot added the A-code-coverage Area: Source-based code coverage (-Cinstrument-coverage) label Jun 19, 2024

ZhuUx force-pushed the prepare-pattern-matching branch from 0d23d53 to 8712975 Compare June 20, 2024 11:15

This was referenced Jun 20, 2024

[Coverage][MCDC] Adapt mcdc to llvm 19 #126733

Merged

Tracking implementation for MC/DC #124144

Open

ZhuUx force-pushed the prepare-pattern-matching branch from 8712975 to 61b608c Compare July 3, 2024 08:35

ZhuUx commented Jul 3, 2024

View reviewed changes

compiler/rustc_mir_build/src/build/expr/into.rs Outdated Show resolved Hide resolved

ZhuUx force-pushed the prepare-pattern-matching branch 2 times, most recently from a49e7b2 to becc530 Compare July 10, 2024 02:56

ZhuUx mentioned this pull request Jul 10, 2024

Support mcdc analysis for pattern matching #124278

Open

ZhuUx force-pushed the prepare-pattern-matching branch from becc530 to 1e6e3fb Compare July 15, 2024 03:29

Zalathar reviewed Jul 15, 2024

View reviewed changes

compiler/rustc_mir_transform/src/coverage/mod.rs Outdated Show resolved Hide resolved

Zalathar reviewed Jul 15, 2024

View reviewed changes

compiler/rustc_mir_transform/src/coverage/mod.rs Outdated Show resolved Hide resolved

This comment has been minimized.

Sign in to view

ZhuUx force-pushed the prepare-pattern-matching branch from 37f9ad8 to 97596dd Compare July 15, 2024 08:12

Zalathar reviewed Jul 15, 2024

View reviewed changes

compiler/rustc_mir_transform/src/coverage/mod.rs Outdated Show resolved Hide resolved

Zalathar reviewed Jul 15, 2024

View reviewed changes

compiler/rustc_middle/src/mir/coverage.rs Outdated Show resolved Hide resolved

Zalathar mentioned this pull request Jul 15, 2024

coverage: Restrict ExpressionUsed simplification to Code mappings #127758

Merged

This comment has been minimized.

Sign in to view

coverage. Warn about too much decision depth

881b0b4

ZhuUx force-pushed the prepare-pattern-matching branch from 06bc597 to c42a016 Compare July 16, 2024 02:58

ZhuUx added 2 commits July 25, 2024 09:35

coverage. Refactor MCDCInfoBuilder for pattern matching implementatio…

e160308

…n and change the way to calculate decision depth

coverage. Group MCDC decisions and conditions until instrument mappin…

bc11434

…gs and support multiple branch markers

ZhuUx force-pushed the prepare-pattern-matching branch from c42a016 to bc11434 Compare July 25, 2024 01:36

ZhuUx mentioned this pull request Jul 25, 2024

MC/DC LLVM 19 intrinsics changes #126672

Open

ZhuUx closed this Jul 25, 2024

ZhuUx deleted the prepare-pattern-matching branch October 9, 2024 02:37

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Coverage][MCDC] Prepare MCDC Builder for pattern matching and llvm 19 #126677

[Coverage][MCDC] Prepare MCDC Builder for pattern matching and llvm 19 #126677

ZhuUx commented Jun 19, 2024 •

edited

Loading

rustbot commented Jun 19, 2024

rustbot commented Jun 19, 2024

ZhuUx commented Jun 19, 2024

Zalathar commented Jun 19, 2024

Zalathar commented Jun 19, 2024

ZhuUx commented Jun 19, 2024

bors commented Jun 30, 2024

rustbot commented Jul 3, 2024

ZhuUx commented Jul 4, 2024 •

edited

Loading

bors commented Jul 5, 2024

ZhuUx commented Jul 10, 2024 •

edited

Loading

ZhuUx commented Jul 15, 2024

Zalathar Jul 15, 2024

ZhuUx Jul 15, 2024 •

edited

Loading

Zalathar Jul 25, 2024

ZhuUx commented Jul 15, 2024

This comment has been minimized.

This comment has been minimized.

Zalathar commented Jul 25, 2024

ZhuUx commented Jul 25, 2024

ZhuUx commented Jul 25, 2024

[Coverage][MCDC] Prepare MCDC Builder for pattern matching and llvm 19 #126677

[Coverage][MCDC] Prepare MCDC Builder for pattern matching and llvm 19 #126677

Conversation

ZhuUx commented Jun 19, 2024 • edited Loading

Changes

rustbot commented Jun 19, 2024

rustbot commented Jun 19, 2024

ZhuUx commented Jun 19, 2024

Zalathar commented Jun 19, 2024

Zalathar commented Jun 19, 2024

ZhuUx commented Jun 19, 2024

bors commented Jun 30, 2024

rustbot commented Jul 3, 2024

ZhuUx commented Jul 4, 2024 • edited Loading

bors commented Jul 5, 2024

ZhuUx commented Jul 10, 2024 • edited Loading

ZhuUx commented Jul 15, 2024

Zalathar Jul 15, 2024

Choose a reason for hiding this comment

ZhuUx Jul 15, 2024 • edited Loading

Choose a reason for hiding this comment

Zalathar Jul 25, 2024

Choose a reason for hiding this comment

ZhuUx commented Jul 15, 2024

This comment has been minimized.

This comment has been minimized.

Zalathar commented Jul 25, 2024

ZhuUx commented Jul 25, 2024

ZhuUx commented Jul 25, 2024

ZhuUx commented Jun 19, 2024 •

edited

Loading

ZhuUx commented Jul 4, 2024 •

edited

Loading

ZhuUx commented Jul 10, 2024 •

edited

Loading

ZhuUx Jul 15, 2024 •

edited

Loading