Direct Mapping Supercharged

[Lichtso]

Problem

Direct mapping is very complex and still buggy. So far, we have been trying to match the existing behavior exactly, including things which are undefined behavior from the programs (not the runtimes) perspective. E.g. the runtime allows reads (in the realloc padding) beyond the current length of an account. From a programs perspective this is uninitialized memory and no sane program should be reading from it before writing to it first. If we were to stop supporting this and instead throw InstructionError::AccountDataTooSmall we could significantly simplify the direct mapping feature.

Summary of Changes

Removes:

• Support for noncontiguous memops syscalls (memcpy, memmov, memset, memcmp)
• Support for noncontiguous memory mapping in SBPF
• Automatic update of MemoryRegion::host_addr and MemoryRegion::writable in SBPF
• Interior mutability of MemoryRegion (Cell<> around host_addr and writable) in SBPF, which prevented declaring proper lifetimes on the results of map / translate calls in this repo
• Hacks involving Vec::spare_capacity_mut
• Hacks which temporarily marked MemoryRegion as writable and then reverted that using scopeguard::defer
• Zeroing of 10 KiB realloc padding during serialization (serialize_parameters_aligned)
• Zeroing of truncated account length during CPI return (update_caller_account)
• Zeroing of the spare capacity during CPI return (update_caller_account) when the underlying Vec was reallocated
• Copying of the realloc padding back to the runtime in CPI call (update_callee_account)
• Copying of the realloc padding back to the caller in CPI return (update_caller_account)
• Copying of the realloc padding back to the runtime during deserialization (deserialize_parameters_aligned)
• Mapping the realloc padding as a MemoryRegion
• Unused Cargo dependencies, code and tests thereof

Adds:

• New key C37iaPi6VE4CZDueU1vL8y6pGp5i8amAbEsF31xzz723 for the feature gate bpf_account_data_direct_mapping
• Bumps SBPF to v0.12.0
• SBPF memory access violation callback parameters: Address space reserved for the region, access type, VM start address and length
• Two phase translation of mutable references (AccessType::Store) in syscalls which enables use of the borrow checker to ensure that the AccessViolationHandler can not invalidate translated references during the syscall
• Realloc and zero fill account immediately in TransactionContext::access_violation_handler() when resize padding is written to (AccessType::Store)
• EbpfError::AccessViolation error translation to InstructionError::AccountDataTooSmall and InstructionError::InvalidRealloc
• test_deny_access_beyond_current_length() for InstructionError::AccountDataTooSmall and InstructionError::InvalidRealloc
• test_access_violation_handler() for the interaction of TransactionContext::access_violation_handler() and create_memory_region_of_account()

Feature Gate Issue: https://github.com/anza-xyz/feature-gate-tracker/issues/16

[mergify]

The Firedancer team maintains a line-for-line reimplementation of the
native programs, and until native programs are moved to BPF, those
implementations must exactly match their Agave counterparts.
If this PR represents a change to a native program implementation (not
tests), please include a reviewer from the Firedancer team. And please
keep refactors to a minimum.

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 84.67742% with 76 lines in your changes missing coverage. Please review.

Project coverage is 83.2%. Comparing base (ddefc13) to head (6f5321a).
Report is 8 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff            @@
##           master    #5871     +/-   ##
=========================================
- Coverage    83.2%    83.2%   -0.1%     
=========================================
  Files         853      853             
  Lines      377582   376498   -1084     
=========================================
- Hits       314502   313544    -958     
+ Misses      63080    62954    -126     

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.