JIT: Emit Intel CET instructions

[riptl]

Description

Intel CET ("Control-flow Enforcement Technology") is a technology that aims to defend against exploitation techniques that rely on taking over control-flow like ROP.

It is a relatively recent extension that's only supported on recent Intel and AMD CPUs. Certain CET instructions map to the NOP opcode space to allow for backwards compatibility with older CPUs.

Since we are generating code at runtime, we can always conditionally enable new features based on CPUID. This would come at the expense of added complexity (=attack surface) however.

Introducing CET support to the SBF JIT compiler could improve the security of the Solana program sandbox.

Concepts

IBT (Indirect Branch Tracking)

endbr64 is a new instruction that encodes to f3 0f 1e fa, which is a NOP on older machines.
It marks valid indirect branch targets and return sites.

When enabled, the CPU will refuse to follow indirect branches or return addresses that do not have endbr64 as the next instruction.

Shadow Stack

Special stack that only stores return addresses to defend against stack smashing.

[riptl]

@Lichtso I would love to hear your thoughts on this. I'm interested in working on IBT support. It looks like a quick win.

[riptl]

At the very least, when Rust decides to do IBT by default the Solana JIT will brick. So we’ll have to do something about it, either disable CET, or add IBT support.

[Lichtso]

For reference this is the tracking issue in Rust I think: rust-lang/rust#73820

The problem is that at the moment we do not know where possible indirect jump targets are, or in other words: Every BPF instruction (even e.g. inside loops) could be.

It is on the TODO list for SBFv2 here: solana-labs/solana#20323

Restrict the callx jump targets to only allow known symbols (e.g. registered functions in the ELF)

[riptl]

Thanks for taking a look. Another consideration is that crafted SBF code input can probably be used to create x86 R/C/J/whatever-OP gadgets. Coupled with the ability to introduce valid branch targets in lots of places (by defining SBFv2 symbols), this would make IBT much less useful.

[riptl]

@Lichtso I'll try this week to get a CET-enabled build of Solana rbpf-cli to pass various programs. I think we have a doable way to get there.

Even if SBFv2 can't track indirect targets yet, IBT also redefines the 0x3e instruction prefix for jumps as the "notrack" hint. It removes the endbr64 requirement at the end of the branch. I guess it's the Rust unsafe equivalent of x86 in a way, so it would be good to avoid widespread usage of notrack.

3e 67 ff 20             notrack jmp QWORD PTR [eax] 

In terms of testing approach, I would propose the following.

1. Create a testbed for CET.
   • QEMU seems to support CET, https://lists.gnu.org/archive/html/qemu-devel/2020-10/msg03502.html I'm not sure if IBT is enforced though
   • If it is supported by QEMU, a sanity check of (1) CET enablement and (2) successful execution of a program would be useful additions to CI pipelines
2. Build Solana with IBT and make sure it passes in interpreter mode
3. Enforce IBT for all "JIT statically generated" indirect branches.