Embedded development on stable

[japaric]

Triage(2018-07-15): This is only waiting on #[panic_implementation] being stabilized. Proposal for FCP.

---

Development of no_std applications is relegated to the nightly channel because it requires opting
into several unstable features. This makes Rust hard to sell to C developers; they'd rather stick to
their stable vendor provided toolchains instead of having to deal with constant breakage /
regressions.

It may not be possible to stabilize all the features needed for application development this year,
but the following unstable features are widely used in library development and it might be possible
to stabilize them this year:

• Xargo. rust-std components for embedded targets.

Once those are in place you'll be able to use Cargo as you normally would by first doing rustup target add thumbv7m-none-eabi.

Tracking issue rust-lang/rust#49382

Xargo requires the nightly channel to build the core crate / std facade. Having that functionality
in stable Cargo would make Xargo unnecessary.

The Cargo team said they'll work on this this year.

I wrote how to land the functionality in Cargo.

• asm! We'll RFC "Stable assembly operations" instead of stabilizing asm!. See Stable assembly operations #63 for details

A pre-rfc on this topic is being discussed.

Upstream tracking issue rust-lang/rust#29722

• const fn Has been proposed for stabilization in const fn tracking issue (RFC 911) rust-lang/rust#24111 (comment)

The old const evaluator is going to be replaced with miri (cf. rust-lang/rust#46882). This fixes a
bunch of bugs and removes several restrictions the old const eval had. It's more likely this feature
will be stabilized once miri is in place.

Upstream tracking issue rust-lang/rust#24111

---

• panic_fmt #[panic_implementation]

If we want to support the use case of linking a Rust library into a C program then setting a
no_std panic handler must be possible on stable.

An RFC on this topic has been accepted (tracking issue rust-lang/rust#44489) and part of the
implementation has landed in rust-lang/rust#47687

---

If we want to be able to build pure Rust no-std applications then a lot of other unstable features
are needed. It's unclear whether we can stabilize any of them this year.

• compiler_builtins_lib #![no_std] expansion will include a stable extern crate compiler_builtins so you won't require opting into the compiler_builtins_lib feature gate

A PR implementing this has been submitted: rust-lang/rust#49503

Ideally these builtins should be shipped with core and it shouldn't be necessary to explicitly link
to the compiler-builtins crate.

• start and termination lang items Can be worked around using unmangled symbols at the cost of not supporting the standard main interface. See stable-embedded-rust for an example.

These are required to support the standard main function interface.

• linkage(weak) Can be worked around by using PROVIDE in a linker script. See stable-embedded-rust for an example.

Used to define a weak interrupt table with a default catch all handler while still allowing the user
to override any handler in the table.

• global_asm! Weak aliases can be implemented using PROVIDE in a linker script. See stable-embedded-rust for an example.

Used to implement weak aliases; a feature which is not supported in the language.

• #[used]

Used to force the compiler to preserve some "unused" symbols that are required at link time (like
the interrupt table).

Tracking issue: rust-lang/rust#40289

RFC for stabilization: rust-lang/rfcs#2386

---

There's other aspect to stability here and that's the lack of CI testing for no-std targets in
rust-lang/rust. The consequence of this is that changes unrelated to unstable features can break
these targets. This happened twice last year: changes in the core crate related to atomics broke
the thumbv6m-none-eabi target.

This issue can be addressed by having no-std targets being tested as part of CI testing. As we don't
want to block rust-lang/rust development on codegen / LLVM bugs it should be possible to easily
disable testing on these no-std targets; a mechanism for this is already in place for tooling like
clippy and miri. Here we should define how much testing we want to do: just building core (simpler),
or also testing that linking no-std applications works (more involved as it requires several
unstable features).

#52 tracks testing embedded targets in rust-lang/rust CI.

cc @pftbest

[jcsoo]

Does anyone have experience with Unicorn, which has what looks to be a fairly solid Rust binding in unicorn-rs? It wouldn't be able to test peripheral-dependent crates without someone mocking the peripherals themselves, but could it be useful for testing Rust codegen and linking for other no-std libs?

[japaric]

As I was writing the embedonomicon (cf. #59) I explored the possibility of making embedded no-std
binaries work on stable Rust. I know that's not quite possible today so I started by just drastically
reducing the number of unstable features required to build a functional (not minimal) no_std
binary. The result was this repo.

The conclusion of my exploration was that there are only three unstable features that prevent us
from developing no_std binaries on stable, and they all seem possible to fix / stabilize this year:

• Xargo. If rust-lang/rust makes a rust-std component (pre-compiled core et al.) available via
  rustup then we don't need Xargo and can use plain Cargo.

• panic_fmt. RFC 2070 proposes a stable alternative to this lang item. The RFC has been accepted
  (cf. Tracking issue for RFC 2070: stable mechanism to specify the behavior of panic! in no-std applications  rust-lang/rust#44489) but it has not been fully implemented.

• compiler_builtins (*). This is a compiler implementation detail. Ideally a user should never
  have to deal with this crate but that's not the case today. The portability WG is looking into
  this issue (compiler_builtins rust-lang-nursery/portability-wg#16).

(*) There's a stable workaround for the unstable compiler_builtins crate and that's linking to
libgcc.a and libc.a but that's not forward compatible with the resolution of the compiler_builtins
problem (namely merging it with core).

The question is whether we want to push for having this form -- take a close look to the linked
repo because the user interface looks different from what you see in e.g. cortex-m-quickstart --
of embedded Rust become stable this year.

Making this stable will not only mean dealing with these 3 unstable features but also with making
this form of embedded Rust part of rust-lang/rust test suite and making sure that compilation and
linking never ever breaks for as long as Rust 1.0 exists.

Thoughts?

cc @hannobraun @jamesmunns @jcsoo @thejpster
cc @dvc94ch @dylanmckay @pftbest could you please check how much of this repo can be
re-implemented for AVR, MSP430 and RISCV without requiring more unstable features? Check the
embededonomicon for a description of the step by step process of building that repo.

[thejpster]

Just wanted to say, I read through this and it's excellent. Great work! Can't wait to apply some of these changes to stellaris-launchpad.

[pftbest]

I have ported this repository for MSP430. (link) There are a couple of things that I learned in the process:

1. MSP430 requires one more #![feature(abi_msp430_interrupt)], which will never be on stable. The alternative is a costly wrappers for each interrupt written in assembly. See calling convention for MSP430 interrupts rust-lang/rust#38465

2. There is a semihosting implementation (called CIO) in binutils simulator for MSP430. But it doesn't work in gdb, so I created a python script to implement CIO in gdb. Works both in simulator and on a real hardware.

3. core::fmt is huge. I was not able to compile asm example in debug mode because it takes more than 16K of ROM. Even in release mode just printing a static string takes 4K, which is too much.

4. There is a buffer overflow in mem example. ptr::write_bytes is multiplying len by the size_of<T>, and in this case T is a full array not just one element.

5. Empty loop {} are still being optimized away, especially in panic_fmt. This leads to some random crashes and "happy" debugging sessions.

[therealprof]

@pftbest Yeah, fmt is a real pig. But I'm curious why write_str() is so inefficient, or do you meant print a string using a fmt?

[pftbest]

writeln!

[therealprof]

@pftbest Yeah, that's using fmt. I wouldn't do that. write_str() is a hell lot more efficient, even if you have to use it twice to add the newline...

[japaric]

Some updates after Rust All Hands:

Xargo.

We'll be producing rust-std components for ARM Cortex-M and MSP430, and eventually for AVR and RISCV once they are more mature.

Once those components are in place you'll be able to use Cargo, instead of Xargo, for embedded development by first doing rustup target add thumb7m-none-eabi once. cargo build --target thumbv7m-none-eabi will then work.

Tracking issue: rust-lang/rust#49382

PR for building rust-std for ARM Cortex-M: rust-lang/rust#49563

asm!

We'll open a RFC to stabilize some assembly operations as "Rust intrinsics" that live in core. See #63 for details.

const fn

Has been proposed for stabilization in rust-lang/rust#24111 (comment)

panic_fmt #[panic_implementation]

We discussed the binary size problem (see #41). We decided that we'll go ahead and land the accepted RFC (2070), and that we'll fix the binary size problem in the future using pure MIR rlibs (not yet implemented).

compiler_builtins_lib

We discussed merging the compiler-builtins crate into core, but ultimately opted for a much simpler solution: #![no_std] expansion will include a stable extern crate compiler_builtins so you will no longer need to write extern crate compiler_builtins (which requires the compiler_builtins_lib feature gate).

A PR implementing this has been submitted: rust-lang/rust#49503

#[used]

I have submitted an RFC proposing the stabilization of this experimental feature: rust-lang/rfcs#2386

[japaric]

Xargo. rust-std components for embedded targets.

This is done. See PSA: You no longer need Xargo to do ARM Cortex-M development.

compiler_builtins_lib #![no_std] expansion will include a stable extern crate compiler_builtins so you won't require opting into the compiler_builtins_lib feature gate

This is done. Thanks @oli-obk! See PSA: [breaking-change] extern crate compiler_builtins is now included in #![no_std] crates.

[japaric]

I had totally forgotten about unstable compiler flags.

The most used unstable flag is -Z linker-flavor. People use this flag to change the linker flavor
from the default GCC one to the LD one. Changing the flavor lets them use arm-none-eabi-ld instead
of arm-none-eabi-gcc; this way they only need to install the ARM binutils
(arm-none-eabi-{ld,objcopy,..}) instead of the complete C toolchain (arm-none-eabi-gcc).

-Z linker-flavor is not a blocker for making embedded Rust possible on stable, but it would help
with the goal of reducing the number of things one needs to install to do embedded development. If
one wants to use the LLD shipped with the Rust toolchain one needs to change the linker flavor to
ld.lld.

I think the way forward here is to ask for -Z linker-flavor to be stabilized. It's tracking issue
is rust-lang/rust#49794.

I thought it might be possible to get away with not having to stabilize that flag and still be able
to use LLD by changing the default linker flavor of the THUMB targets to ld.lld but that's risky
because LLD is still buggy. If we were to stabilize that and you hit some LLD bug then you would
have to switch to nightly to change the linker flavor to LD and be able to use GNU LD to link your
program.

Is there any other unstable compiler flag that comes often in embedded development?

[therealprof]

@japaric

If one wants to use the LLD shipped with the Rust toolchain one needs to change the linker flavor to
ld.lld.

Actually that seems to be the default. I tried changing it to ld.lld initially but that failed, simply removing -Z linker-flavor worked fine for me.

[japaric]

@therealprof

$ rustc -Z unstable-options --target thumbv7m-none-eabi --print target-spec-json | grep link
  "linker": "arm-none-eabi-gcc",
  "linker-flavor": "gcc",

You need both -C linker=lld and -Z linker-flavor=ld.lld to use the LLD in the sysroot.

[therealprof]

@japaric Interesting. At least that explains why I was seeing a difference between no -Z linker-flavor and the default because I was using

  "-C", "linker=arm-none-eabi-ld",
  "-Z", "linker-flavor=ld",

I wrongly assumed it would be using lld by default.

[japaric]

Update: We now have a preview of embedded Rust on stable at rust-embedded/cortex-m-quickstart#29.

The preview only requires a single unstable feature: lang = "panic_fmt" which already has a path towards stabilization. The current ETA for embedded Rust on stable is 1.28 but the preview already shows what the development experience will be.

Quite a few pieces need to land before you can try out this using crates.io releases but we would greatly appreciate if you could try the quickstart PR now and let us know what you think (please comment in the quickstart PR). In particular we would love your input on these unresolved questions.

[japaric]

I have open a PR to stabilize opt-level={s,z}: rust-lang/rust#50265.

[japaric]

I have open a PR to stabilize opt-level={s,z}: rust-lang/rust#50265.

This landed 2 weeks ago.

---

#[panic_implementation] has landed! 🎉

Check out the announcement! You may experience some (planned) breakage. Once that feature is stabilized no_std application development will be possible on stable.

[japaric]

Update: #[panic_implementation] has been proposed for stabilization in rust-lang/rust#44489 (comment).

[japaric]

As the only thing left to do here is rust-lang/rust#44489 I'm going to close this in favor of that upstream issue, which I have assigned to the RC1 milestone.