SIMD-0177: Program Runtime ABI v2

proposals/0177-program-runtime-abiv2.md

@@ -0,0 +1,364 @@
+---
+simd: '0177'
+title: Program Runtime ABI v2
+authors:
+  - Alexander Meißner
+  - Lucas Stuernagel
+category: Standard
+type: Core
+status: Idea
+created: 2025-02-23
+feature: TBD
+extends: SIMD-0219
+---
+
+## Summary
+
+Align the layout of the virtual address space to large pages in order to
+simplify the address translation logic and allow for easy direct mapping.
+
+## Motivation
+
+Direct mapping of the account payload data is enabled by SIMD-0219.
+However, there remains a big optimization potential for both programs and the
+program runtime:
+
+- Instruction data could be mapped directly as well
+- Return data could be mapped directly too
+- Account payload could be resized freely (no more 10 KiB growth limit)
+- CPI could become cheaper in terms of CU consumption
+- Most structures could be shared between programs and program runtime,
+requiring only a single serialization at the beginning of a transaction and
+only small adjustments after
+- Per instruction serialization before a program runs could be removed entriely
+- Per instruction deserialization after a program runs could be removed too
+- Deserialization inside the dApp could be reduced to a minimum
+- programs would only have to pay for what they use, not having to deserialize
+all instruction accounts which were passed in
+- Scanning sibling instructions would not require a syscall
+- Memory regions (and thus address translation) which SIMD-0219 made unaligned
+could be aligned (to 4 GiB) again
+
+All of these however do necessitate a major change in the layout how the
+program runtime and programs interface (ABI).
+
+## Alternatives Considered
+
+None.
+
+## New Terminology
+
+None.
+
+## Detailed Design
+
+Programs signal that they expect ABIv2 through their SBPF version field being
+v4 or above.
+
+### Memory Regions
+
+#### Transaction metadata area
+
+At the beginning of a transaction the program runtime must prepare a 
+readonly memory region starting at `0x400000000`. This region is shared by all 
+instructions running programs with support to new ABI. It must be updated as
+as instructions through out the transaction modify the CPI scratchpad or the 
+return data. The contents of this memory region are the following:
+
+- Key of the program which wrote to the return-data scratchpad most 
+  recently: `[u8; 32]`
+- The return-data scratchpad: `&[u8]`, which is composed of:
+  - Pointer to return-data scratchpad: `u64`
+  - Length of return-data scratchpad: `u64`
+- The CPI scratchpad: `&[u8]`, which consists of:
+  - Pointer to CPI scratchpad: `u64`
+  - Length of CPI scratchpad: `u64`
+- The CPI accounts scratchpad: `&[InstructionAccount]`, which consists of:
+    - Pointer to slice: `u64`
+    - Number of elements in slice: `u64`
+- Index of current executing instruction: `u32`
+- Total number of instructions in transaction (including CPIs and top level 
+  instructions): `u32`
+- Number of CPIs in trace (under execution and finished): `u32`
+- The number of transaction accounts: `u32`
+
+##### Payer information
+
+As accounts in this area sorted by their index in transaction, the payer 
+account must always be the account at index zero, as we surface runtime's
+internal account ordering for programs.
+
+#### Account metadata area
+
+This region starts at `0x500000000`, is readonly and holds the metadata for 
+all accounts in the transaction. It is shared by all instructions running 
+programs with support for ABIv2, and must be updated as instruction modify the 
+metadata with the provided syscalls (see the `Changes to syscalls` section).
+The contents for this region are as follow:
+
+- For each transaction account:
+  - Key: `[u8; 32]`
+  - Owner: `[u8; 32]`
+  - Lamports: `u64`
+  - Account payload: `&[u8]` which is composed of:
+    - Pointer to account payload: `u64`
+    - Account payload length: `u64`
+
+#### Instruction area
+
+For each transaction, the program runtime must also prepare two memory regions. 
+The first one is a readonly region starting at `0x600000000`. It must be 
+updated at each CPI call edge. The contents of this region are the following:
+
+- For each instruction in transaction:
+  - Reserved filed for alignment and potential future usage: `u16`
+  - Index in transaction of program account to be executed: `u16`
+  - CPI nesting level: `u16`
+  - Index of parent instruction (`u16::MAX` for top-level instructions): `u16`
+  - Reference to a slice of instruction accounts `&[InstructionAccount]`, 
+    consisting of:
+    - Pointer to slice: `u64`
+    - Number of elements in slice: `u64`
+  - Instruction data `&[u8]`, which is composed of:
+    - Pointer to data: `u64`
+    - Length of data: `u64`
+
+Let `InstructionAccount` contain the following fields:
+
+  - Index to transaction account: `u16`
+  - Signer flag: `u8` (1 for signer, 0 for non-signer)
+  - Writable flag: `u8` (1 for writable, 0 for readonly)
+
+#### Return data scratchpad
+
+A writable memory region starting at `0x700000000` must be mapped in for the
+return-data scratchpad.
+
+#### Accounts area
+
+For each unique (meaning deduplicated) instruction account the payload must
+be mapped in at `0x800000000` plus `0x100000000` times the index of the
+**transaction** account (not the index of the instruction account). Only if the
+instruction account has the writable flag set and is owned by the current
+program it is mapped in as a writable region. The writability of a region must
+be updated as programs through out the transaction modify the account metadata.
+
+The runtime must only map the payload for accounts that belong in the current
+executing instruction. The payload for accounts belonging to sibling instructions
+must NOT be mapped.
+
+#### Instruction payload area
+
+For each instruction, the runtime must map its payload at address 
+`0x10800000000` plus `0x100000000` times the index of the instruction in the 
+transaction. All instruction payload mappings are readonly.
+
+One extra writable mapping must be created after the last instruction payload 
+area to be the CPI scratch pad, i.e. at address `0x10800000000` plus 
+`0x100000000` times the number of instructions in the transaction. Its purpose  
+is for programs to write CPI instruction data directly to it and avoid copies.
+
+#### Instruction accounts area
+
+For each instruction, the runtime must map an array of `InstructionAccount`
+(as previously defined) at address `0x14800000000` plus `0x100000000` times 
+the index of the instruction in the transaction. This mapped are is readonly.
+
+Each of these memory regions contain the following for each instruction:
+
+- For each account in instruction:
+  - `InstructionAccount`, consisting of:
+    - Index to transaction account: `u16`
+    - Signer flag: `u8` (1 for signer, 0 for non-singer)
+    - Writable flag: `u8` (1 for writable, 0 for readonly)
+
+One extra writable mapping must be created after the last instruction accounts 
+area to be the CPI scratch pad, i.e. at address `0x14800000000` plus 
+`0x100000000` times the number of instructions in the transaction. Its purpose 
+is for programs to write CPI accounts directly to it and avoid copies.
+
+#### Sysvar accounts area
+
+For each existing (non deprecated) sysvar account, the runtime must map its
+payload at address `0x18800000000` plus `0x100000000` times the index of the
+sysvar in the following order:
+
+0. Clock
+1. Epoch rewards
+2. Epoch Schdule
+3. Last restart slot
+4. Rent
+5. Slot hashes
+6. Stake history
+
+### VM initialization
+
+During the initilization of the virtual machine, the runtime must load the 
+following values to registers:
+
+1. Register R1: A pointer to the metadata of the instruction under execution.
+   (see section [Instruction area](#instruction-area)).
+2. Register R2: A pointer to the instruction accounts slice for the 
+   instruction under execution (see section 
+   [Instruction accounts area](#instruction-accounts-area)).
+3. Register R3: The number of instruction accounts for the instruction under 
+   execution.
+4. Register R4: A pointer to the instruction payload of the instruction under 
+   execution (see section [Instruction payload area](#instruction-payload-area)).
+5. Register R5: The payload lenght for the instruction under execution.
+
+
+### Changes to syscalls
+
+Changes to the account metadata must now be communicated with specific 
+syscalls, as detailed below:
+
+- `sol_assign_owner(u64, *const [u8; 32])`.
+   - `u64`: Index in transaction of the account whose owner is changing,
+   - `*const [u8; 32]`: Pointer to the public key of the new owner.
+- `sol_transfer_lamports(u64, u64, u64)`: 
+   - `u64`: Index in transaction of the destination account.
+   - `u64`: Index in transaction of the source account.
+   - `u64`: Lamports amount.
+
+Changes to the account payload length and all the scratchpads sections 
+introduced in this SIMD (the return-data scratchpad and the CPI scratchpad) 
+must be communicated via a new sycall `set_buffer_length(u64, u64)`, with the 
+following parameters:
+
+- `u64`: Base address of region to be resized.
+- `u64`: New length of region.
+
+The syscall must check if the address matches the base address of either a 
+writable account payload mapping or one of the scratchpad mappings and return 
+an error otherwise. Constrains for the maximum resizable limits must also be 
+verified for each region separetely.
+
+The `set_buffer_length` must charge a base cost (to be determined) plus the 
+same CU per byte ratio as the `memset` syscall.
+
+The verifier must reject SBPFv4 programs containing the `sol_invoke_signed_c` 
+and `sol_invoke_signed_rust`, since they are not compatible with ABIv2. A new 
+syscall `sol_invoke_signed_v2` must replace them. The parameters for 
+`sol_invoke_signed_v2` are the following:
+
+- Index in transaction of program ID to be called: `u64`.
+- A pointer to the singer seeds of type `VmSlice<VmSlice<VmSlice<u8>>>`.
+- The length of the outer signer seeds slice in 
+  `VmSlice<VmSlice<VmSlice<u8>>>`.
+
+`VmSlice<T>` is a stable layout type defined to share slices between the guest 
+and the host. It consists of:
+
+- `u64`: Pointer to the data.
+- `u64`: Length of data (number of elements `T`)
+
+Programs using `sol_get_return_data` and `sol_set_return_data` must be 
+rejected by the verfier if ABI v2 is in use.
+
+### Scratchpad management
+
+This SIMD introduces two scratch pad regions: the return-data scratchpad and 
+the CPI scratchpad. At the beginning of every instruction, these scratchpads 
+must be empty and their size must be zero.
+
+Programs must set the desired length for them using the `set_buffer_length` 
+syscall. Reads and writes to a region beyond the scratchpad length must 
+trigger an access violation error.
+
+The management for the writable accounts payload must work similarly, except 
+that they must not be initialized empty, but instead with the pre-existing 
+data it holds.
+
+### CPIs
+
+#### Program side
+
+The workflow for cross program invokation on the program side will change. 
+Instead of programs themselves allocating memory on the heap or the stack for 
+CPI instruction data and CPI accounts, the program runtime must already 
+provide the pointers for programs to write to.
+
+The CPI data scratchpad is a region in the 
+[Instruction payload area](#instruction-payload-area), right after the space 
+reserved for the last intruction in the instruction trace.
+In other words, `0x10800000000` plus `0x100000000` times the number of 
+instructions in the transaction.
+
+Likewise, the CPI accounts must be written to a runtime provided region 
+residing in the [Instruction accounts area](#instruction-accounts-area) 
+at `0x14800000000` plus `0x100000000` times the number of instructions in the 
+transaction.
+
+Both the aforementioned regions must begin with size zero, and programs must 
+resize them before writing to them using the `set_buffer_length` syscall.
+
+#### Runtime side
+
+With the new `sol_invoke_signed_v2` syscall, CPIs must be managed 
+differently. At each CPI call, the runtime must perform the following actions:
+
+1. Verify that all account indexes received in the `InstructionAccount` area 
+   belong in the current executing instruction. Likewise, the prgram ID index 
+   that should be called must also undergo the same verification.
+2. Verify that accounts have the correct signer and writer flags set, avoiding 
+   privelege promotion.
+3. Append a new instruction at the end of the serialization array kept at 
+   `0x600000000`.
+4. Transform the caller CPI scratchpad into a readonly instruction payload 
+   region visible for the callee.
+5. Change the visibility and write permissions for the account payload 
+   regions, according to the CPI accounts and their flags.
+6. Update the address for the callee CPI scratchpad, the index of current 
+   executing transaction, and the number of instructions in transaction at 
+   address `0x400000000`.
+
+When the CPI returns, the runtime must do the following:
+
+1. Update the address for the CPI scratchpad, and keep the previouly used one 
+   in its exsiting address assigned during CPI call. The new CPI scratchpad 
+   address is the same as the previous one plus `0x100000000`.
+2. Change the read and write permission for the account payload regions, 
+   according to potential changes in account ownership.
+3. Update the index of current executing instruction.
+
+CPIs between ABIv0/v1 and ABIv2 program must be allowed, but costs will difer.
+A CPI from an ABIv2 to another ABIv2 program must cost less than a CPI from an 
+ABIv2 to an ABIv0/v1 program, due to the decreased work overhead from program 
+runtime.
+
+### Changes to CU metering
+
+CPI will no longer charge CUs for the length of account payloads. Instead TBD
+CUs will be charged for every instruction account. Also TBD CUs will be charged
+for the three new account metadata updating syscalls. TBD will be charged for
+resizing a scratchpad.
+
+### Lazy deserialization on the dApp side (inside the SDK)
+
+With this design a program SDK can (but no longer needs to) eagerly deserialize
+all account metadata at the entrypoint. Because this layout is strictly aligned
+and uses proper arrays, it is possible to directly calculate the offset of a
+single accounts metadata with only one indirect lookup and no need to scan all
+preceeding metadata. This allows a program SDK to offer a lazy interface which
+only interacts with the account metadata fields which are needed, only of the
+accounts which are of interest and only when necessary.
+
+## Impact
+
+This change is expected to drastically reduce the CU costs as the cost will no
+longer depend on the length of the instruction account payloads or instruction
+data.
+
+From the dApp devs perspective almost all changes are hidden inside the SDK.
+
+## Security Considerations
+
+What security implications/considerations come with implementing this feature?
+Are there any implementation-specific guidance or pitfalls?
+
+## Drawbacks
+
+This will require parallel code paths for serialization, deserialization, CPI
+call edges and CPI return edges. All of these will coexist with the exisiting
+ABI v0 and v1 for the forseeable future, until we decide to deprecate them.