Probe-rs support

[xd009642]

https://github.com/probe-rs/probe-rs

Probe-rs is a debugger framework for embedded rust projects. Given it provides an interface similar to ptrace in inspecting the running of a program it should be possible to get tarpaulin to collect coverage via probe-rs like it does for ptrace. I imagine this would involve adding a new execution state machine so is also a way to see if my initial stab at abstracting the states involved in cross platform process tracing has any glaring holes.

[jamesmunns]

CC @Yatekii and the other probe-rs folks. It would be awesome to use probe-rs + tarpaulin to collect on-device coverage numbers for embedded systems!

[Yatekii]

I like this idea very much :)

[xd009642]

@Yatekii that's good to know 😄. I'm a bit overloaded right now but I hope to start this in a couple of months time and then I may ping you if I have any questions

[Yatekii]

Yeah same story here ... and the list of thing to do on probe-rs is long :/ Once I have more time I'll definitely come back here too!

[xd009642]

So one of my mentees via rust-awesome-mentors was potentially interested in working on this. To aid that I've created a basic repo to prototype things in with a simple embedded project to get coverage for and another project that uses tarpaulin as a library runs some of the initial things tarpaulin does and then leaves gaps for them to implement the next steps https://github.com/xd009642/probe-cov-sandbox so if anyone here is really interested in how this looks that repo will get the initial bleeding edge experimental implementation before it gets adopted into tarpaulin

[xd009642]

@Yatekii so not quite related to tarpaulin, slightly more standalone but I did some work and figured out the rough steps to using the llvm coverage instrumentation on embedded targets https://github.com/xd009642/llvm-embedded-coverage and the next step would be to figure out the way to transfer the reports down to the wire, collect them and serialize them to a file.

This would let embedded coverage work separate to tarpaulin just using nightly rust, though it'll need a little bit of code at the start and end of a custom test runner (which I figure probe-rs is doing anyway to get cargo test working on HIL testing). I'm still playing with it a bit but wondered if it was something to open an issue with on probe-rs once I've figured out something that could potentially be more deeply integrated: so we can just do RUSTFLAGS="-Z instrument-coverage" cargo +nightly test and see the default.profraw appear in the project directory.

[Yatekii]

Yeah, I am not entirely educated what we need to integrate cargo test and what is actually possible there. It would be cool to support this out of the box. An issue sounds great :)

[xd009642]

Oh wait the test runner is in https://crates.io/crates/defmt-test I was only lightly skimmed https://ferrous-systems.com/blog/gha-hil-tests/#hardware-testing-workflow to try and remind me who let me run cargo test to test on device

[xd009642]

Issue raised on defmt knurling-rs/defmt#631

[newAM]

I'm still playing with it a bit but wondered if it was something to open an issue with on probe-rs once I've figured out something that could potentially be more deeply integrated: so we can just do RUSTFLAGS="-Z instrument-coverage" cargo +nightly test and see the default.profraw appear in the project directory.

I tried compiling with instrument-coverage for an embedded target and got this:

$ RUSTFLAGS="-Z instrument-coverage" cargo +nightly test -p testsuite --target thumbv7em-none-eabi --bin info --no-run
   Compiling ...
error[E0463]: can't find crate for `profiler_builtins`
  |
  = note: the compiler may have been built without the profiler runtime

For more information about this error, try `rustc --explain E0463`.
error: could not compile `bare-metal` due to previous error
warning: build failed, waiting for other jobs to finish...
error: build failed

From this issue I surmise that rustc needs to be recompiled with profile=true: rust-lang/rust#81684
Were you able to compile that for an embedded target? Sorry if I missed something, I just saw the defmt issue and wanted to poke around 😅

[xd009642]

@newAM so if you look at the code I just use it to write into the buffer and then use fs::write to write the buffer contents to a file. But that's the only std part, mainly because I haven't figured out the communication channel for an embedded target

[newAM]

As a first proof of concept it should be possible to dump data from the probe using an rtt_target channel.

RTT is just a chunk of RAM on the embedded device that gets polled continuously by the probe. Existing tools such as cargo-embed can dump the binary data to a file.

[xd009642]

Awesome I'll start looking at the docs and figure it out 👍

[xd009642]

@newAM I now have a branch feat/make-no-std, and I've hit the same issue as rust-lang/rust#81684

my laptop isn't quite up to compiling rustc in any reasonable time so I'll shelve it for now but if you want to try and build for an embedded target with profile enabled feel free. Though reading some comments it sounds like mmap may be required or other posix functions. So there may be more work required 😞

[xd009642]

@newAM I compiled it for an embedded target now using my own branch of minicov. I tried to flash it to a dev board of mine but the coverage data makes the binary too large so I need to dig out a board with more flash 😢 but you can try it here https://github.com/xd009642/llvm-embedded-coverage/tree/feat/make-no-std

[newAM]

If you're looking for something new the STM32H7 is a well supported microcontroller with megabytes of flash. ST sells it on a nucleo dev board.

I'll give that a go myself later today 👍

[xd009642]

I've got an STM32F469xx around with tons of flash so I reckon that'll do. I do occasionally look at the STM32H series though and consider splashing out but I don't do enough embedded anymore to warrant it 😅