From e3e3e19bcc3d371c59dd9356dbb03123a1caca21 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Alexander=20Mei=C3=9Fner?= Date: Tue, 1 Oct 2024 17:14:17 +0200 Subject: [PATCH 01/31] First draft --- proposals/0177-program-runtime-abiv2.md | 156 ++++++++++++++++++++++++ 1 file changed, 156 insertions(+) create mode 100644 proposals/0177-program-runtime-abiv2.md diff --git a/proposals/0177-program-runtime-abiv2.md b/proposals/0177-program-runtime-abiv2.md new file mode 100644 index 000000000..ce7e00a9d --- /dev/null +++ b/proposals/0177-program-runtime-abiv2.md @@ -0,0 +1,156 @@ +--- +simd: '0177' +title: Program Runtime ABI v2 +authors: + - Alexander Meißner +category: Standard +type: Core +status: Idea +created: 2025-02-23 +feature: TBD +extends: SIMD-0219 +--- + +## Summary + +Align the layout of the virtual address space to large pages in order to +simplify the address translation logic and allow for easy direct mapping. + +## Motivation + +At the moment all validator implementations have to copy (and compare) data in +and out of the virtual memory of the virtual machine. There are four possible +account data copy paths: + +- Serialization: Copy from program runtime (host) to virtual machine (guest) +- CPI call edge: Copy from virtual machine (guest) to program runtime (host) +- CPI return edge: Copy from program runtime (host) to virtual machine (guest) +- Deserialization: Copy from virtual machine (guest) to program runtime (host) + +To avoid this a feature named "direct mapping" was designed which uses the +address translation logic of the virtual machine to emulate the serialization +and deserialization without actually performing copies. + +Implementing direct mapping in the current ABI v0 and v1 is very complex +because of unaligned virtual memory regions and memory accesses overlapping +multiple virtual memory regions. Instead the layout of the virtual address +space should be adjusted so that all virtual memory regions are aligned to +4 GiB. + +## Alternatives Considered + +None. + +## New Terminology + +None. + +## Detailed Design + +Programs signal their support through their SBPF version field being v4 or +above while the program runtime signals which ABI is chosen through the +serialized magic field. + +### Per Transaction Serialization + +At the beginning of a transaction the program runtime must prepare the +following which is shared by all instructions running programs suporting the +new ABI. This memory region starts at `0x400000000` and is readonly. It must be +updated as instructions through out the transaction modify the account metadata +or the scratchpad via `sol_set_return_data`. + +- Key of the program which wrote to the scratchpad most recently: `[u8; 32]` +- The scratchpad data: `&[u8]` which is composed of: + - Pointer to scratchpad data: `u64` + - Length of scratchpad data: `u64` +- The number of transaction accounts: `u64` +- For each transaction account: + - Key: `[u8; 32]` + - Owner: `[u8; 32]` + - Lamports: `u64` + - Account payload: `&[u8]` which is composed of: + - Pointer to account payload: `u64` + - Account payload length: `u64` + +A readonly memory region starting at `0x500000000` must be mapped in for the +scratchpad data. It must be updated when `sol_set_return_data` is called. + +### Per Instruction Serialization + +For each instruction the program runtime must prepare the following. +This memory region starts at `0x600000000` and is readonly. It does not require +any updates once serialized. + +- The instruction data: `&[u8]` which is composed of: + - Pointer to instruction data: `u64` + - Length of instruction data: `u64` +- Programm account index in transaction: `u16` +- Number of instruction accounts: `u16` +- For each instruction account: + - Index to transaction account: `u16` + - Flags bitfield: `u16` (bit 0 is signer, bit 1 is writable) + +### Per Instruction Mappings + +A readonly memory region starting at `0x700000000` must be mapped +in for the instruction data. It too does not require any updates. + +For each unique (meaning deduplicated) instruction account the payload must +be mapped in at `0x800000000` plus `0x100000000` times the index of the +**transaction** account (not the index of the instruction account). Only if the +instruction account has the writable flag set and is owned by the current +program it is mapped in as a writable region. The writability of a region must +be updated as programs through out the transaction modify the account metadata. + +### Lazy deserialization on the dApp side (inside the SDK) + +With this design a program SDK can (but no longer needs to) eagerly deserialize +all account metadata at the entrypoint. Because this layout is strictly aligned +and uses proper arrays, it is possible to directly calculate the offset of a +single accounts metadata with only one indirect lookup and no need to scan all +preceeding metadata. This allows a program SDK to offer a lazy interface which +only interacts with the account metadata fields which are needed, only of the +accounts which are of interest and only when necessary. + +### Changes to syscalls + +The `AccountInfo` parameter of the CPI syscalls (`sol_invoke_signed_c` and +`sol_invoke_signed_rust`) will be ignored if ABI v2 is in use. Instead the +changes to account metadata will be communicated explicitly through separate +syscalls `sol_set_account_owner`, `sol_set_account_lamports` and +`sol_set_account_length`. Each of these must take a guest pointer to the +structure of the transaction account (see per transaction serialization) to be +updated and the new value as second parameter. In case of the pubkey parameter +the guest pointer to a 32 byte slice is taken instead. + +### Changes to CU metering + +CPI will no longer charge CUs for the length of account payloads. Instead TBD +CUs will be charged for every instruction account. + +## Impact + +This change is expected to drastically reduce the CU costs as the cost will no +longer depend on the length of the instruction account payloads or instruction +data. + +From the dApp devs perspective almost all changes are hidden inside the SDK. + +## Security Considerations + +What security implications/considerations come with implementing this feature? +Are there any implementation-specific guidance or pitfalls? + +## Drawbacks + +This will require parallel code paths for serialization, deserialization, CPI +call edges and CPI return edges. All of these will coexist with the exisiting +ABI v0 and v1 for the forseeable future, until we decide to deprecate them. + +## Backwards Compatibility + +The magic field (`u32`) and version field (`u32`) of ABI v2 are placed at the +beginning, where ABI v0 and v1 would otherwise indicate the number of +instruction accounts as an `u64`. Because the older ABIs will never serialize +more than a few hundred accounts, it is possible to differentiate the ABI +that way without breaking the older layouts. From 39486ca9edf9487349691c4273a90bad134133fc Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Alexander=20Mei=C3=9Fner?= Date: Fri, 30 May 2025 21:07:31 +0200 Subject: [PATCH 02/31] Changes to account metadata updating syscalls. --- proposals/0177-program-runtime-abiv2.md | 22 +++++++++++++--------- 1 file changed, 13 insertions(+), 9 deletions(-) diff --git a/proposals/0177-program-runtime-abiv2.md b/proposals/0177-program-runtime-abiv2.md index ce7e00a9d..9efe104f5 100644 --- a/proposals/0177-program-runtime-abiv2.md +++ b/proposals/0177-program-runtime-abiv2.md @@ -114,19 +114,23 @@ accounts which are of interest and only when necessary. ### Changes to syscalls -The `AccountInfo` parameter of the CPI syscalls (`sol_invoke_signed_c` and -`sol_invoke_signed_rust`) will be ignored if ABI v2 is in use. Instead the -changes to account metadata will be communicated explicitly through separate -syscalls `sol_set_account_owner`, `sol_set_account_lamports` and -`sol_set_account_length`. Each of these must take a guest pointer to the -structure of the transaction account (see per transaction serialization) to be -updated and the new value as second parameter. In case of the pubkey parameter -the guest pointer to a 32 byte slice is taken instead. +The `AccountInfo` parameter of the CPI syscall `sol_invoke_signed_c` must be +ignored and programs using `sol_invoke_signed_rust` must be rejected if ABI v2 +is in use. Instead the changes to account metadata will be communicated +explicitly through separate syscalls: + +- `sol_resize_account`: Dst account, new length as `u64` +- `sol_assign_owner`: Dst account, new owner as `&[u8; 32]` +- `sol_transfer_lamports`: Dst account, src account, amount as `u64` + +The account parameters are guest pointers to the structure of the transaction +accounts (see per transaction serialization). ### Changes to CU metering CPI will no longer charge CUs for the length of account payloads. Instead TBD -CUs will be charged for every instruction account. +CUs will be charged for every instruction account. Also TBD CUs will be charged +for the three new account metadata updating syscalls. ## Impact From 75cb2fd59fd127cc3c50d370918e57e1a2001874 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Alexander=20Mei=C3=9Fner?= Date: Wed, 11 Jun 2025 20:29:23 +0200 Subject: [PATCH 03/31] Removes the magic header and renames the scratchpad back to return-data. --- proposals/0177-program-runtime-abiv2.md | 27 +++++++++---------------- 1 file changed, 10 insertions(+), 17 deletions(-) diff --git a/proposals/0177-program-runtime-abiv2.md b/proposals/0177-program-runtime-abiv2.md index 9efe104f5..848bd0176 100644 --- a/proposals/0177-program-runtime-abiv2.md +++ b/proposals/0177-program-runtime-abiv2.md @@ -47,9 +47,8 @@ None. ## Detailed Design -Programs signal their support through their SBPF version field being v4 or -above while the program runtime signals which ABI is chosen through the -serialized magic field. +Programs signal that they expect ABIv2 through their SBPF version field being +v4 or above. ### Per Transaction Serialization @@ -57,12 +56,12 @@ At the beginning of a transaction the program runtime must prepare the following which is shared by all instructions running programs suporting the new ABI. This memory region starts at `0x400000000` and is readonly. It must be updated as instructions through out the transaction modify the account metadata -or the scratchpad via `sol_set_return_data`. +or the return-data via `sol_set_return_data`. -- Key of the program which wrote to the scratchpad most recently: `[u8; 32]` -- The scratchpad data: `&[u8]` which is composed of: - - Pointer to scratchpad data: `u64` - - Length of scratchpad data: `u64` +- Key of the program which wrote to the return-data most recently: `[u8; 32]` +- The return-data data: `&[u8]` which is composed of: + - Pointer to return-data data: `u64` + - Length of return-data data: `u64` - The number of transaction accounts: `u64` - For each transaction account: - Key: `[u8; 32]` @@ -73,7 +72,7 @@ or the scratchpad via `sol_set_return_data`. - Account payload length: `u64` A readonly memory region starting at `0x500000000` must be mapped in for the -scratchpad data. It must be updated when `sol_set_return_data` is called. +return-data data. It must be updated when `sol_set_return_data` is called. ### Per Instruction Serialization @@ -126,6 +125,8 @@ explicitly through separate syscalls: The account parameters are guest pointers to the structure of the transaction accounts (see per transaction serialization). +Programs using `sol_get_return_data` must be rejected if ABI v2 is in use. + ### Changes to CU metering CPI will no longer charge CUs for the length of account payloads. Instead TBD @@ -150,11 +151,3 @@ Are there any implementation-specific guidance or pitfalls? This will require parallel code paths for serialization, deserialization, CPI call edges and CPI return edges. All of these will coexist with the exisiting ABI v0 and v1 for the forseeable future, until we decide to deprecate them. - -## Backwards Compatibility - -The magic field (`u32`) and version field (`u32`) of ABI v2 are placed at the -beginning, where ABI v0 and v1 would otherwise indicate the number of -instruction accounts as an `u64`. Because the older ABIs will never serialize -more than a few hundred accounts, it is possible to differentiate the ABI -that way without breaking the older layouts. From d6d9221bd2fc0fc1a72f5ab7fd6faf54f7b26956 Mon Sep 17 00:00:00 2001 From: Lucas Date: Fri, 13 Jun 2025 16:35:23 -0300 Subject: [PATCH 04/31] Redesign memory regions --- proposals/0177-program-runtime-abiv2.md | 199 ++++++++++++++++++------ 1 file changed, 155 insertions(+), 44 deletions(-) diff --git a/proposals/0177-program-runtime-abiv2.md b/proposals/0177-program-runtime-abiv2.md index 848bd0176..34d444df2 100644 --- a/proposals/0177-program-runtime-abiv2.md +++ b/proposals/0177-program-runtime-abiv2.md @@ -3,6 +3,7 @@ simd: '0177' title: Program Runtime ABI v2 authors: - Alexander Meißner + - Lucas Stuernagel category: Standard type: Core status: Idea @@ -27,7 +28,7 @@ account data copy paths: - CPI return edge: Copy from program runtime (host) to virtual machine (guest) - Deserialization: Copy from virtual machine (guest) to program runtime (host) -To avoid this a feature named "direct mapping" was designed which uses the +To avoid this, a feature named "direct mapping" was designed which uses the address translation logic of the virtual machine to emulate the serialization and deserialization without actually performing copies. @@ -50,18 +51,27 @@ None. Programs signal that they expect ABIv2 through their SBPF version field being v4 or above. -### Per Transaction Serialization - -At the beginning of a transaction the program runtime must prepare the -following which is shared by all instructions running programs suporting the -new ABI. This memory region starts at `0x400000000` and is readonly. It must be -updated as instructions through out the transaction modify the account metadata -or the return-data via `sol_set_return_data`. - -- Key of the program which wrote to the return-data most recently: `[u8; 32]` -- The return-data data: `&[u8]` which is composed of: - - Pointer to return-data data: `u64` - - Length of return-data data: `u64` +### Memory Regions + +#### Transaction area + +At the beginning of a transaction the program runtime must prepare a +readonly memory region starting at `0x400000000`. This region is shared by all +instructions running programs with support to new ABI. It must be updated as +as instructions through out the transaction modify the account metadata, the +CPI scratchpad or the return data. The contents of this memory region are the +following: + +- Key of the program which wrote to the return-data scratchpad most + recently: `[u8; 32]` +- The return-data scratchpad: `&[u8]`, which is composed of: + - Pointer to return-data scratchpad: `u64` + - Length of return-data scratchpad: `u64` +- The CPI scratchpad: `&[u8]`, which consists of: + - Pointer to CPI scratchpad: `u64` + - Length of CPI scratchpad: `u64` +- Index of current executing instruction: `u64` +- Number of instructions in transaction: `u64` - The number of transaction accounts: `u64` - For each transaction account: - Key: `[u8; 32]` @@ -71,28 +81,43 @@ or the return-data via `sol_set_return_data`. - Pointer to account payload: `u64` - Account payload length: `u64` -A readonly memory region starting at `0x500000000` must be mapped in for the -return-data data. It must be updated when `sol_set_return_data` is called. +#### Return data scratchpad + +A writable memory region starting at `0x500000000` must be mapped in for the +return-data scratchpad. -### Per Instruction Serialization +#### Instruction area -For each instruction the program runtime must prepare the following. -This memory region starts at `0x600000000` and is readonly. It does not require -any updates once serialized. +For each transction, the program runtime must also preapre two memory regions. +The first one is a readonly region starting at `0x600000000`. It must be +updated at each CPI call edge. The contents of this region are the following: + +- For each instruction in transaction: + - Program ID: `[u8; 32]` + - Reference to a slice of instruction accounts `&[InstructionAccount]`, + consisting of: + - Pointer to beginning of slice: `u64` + - Number of elements in slice: `u64` + - Instruction data `&[u8]`, which is composed of: + - Pointer to data: `u64` + - Length of data: `u64` + +Let `InstrucionAccount` contain the following fields: -- The instruction data: `&[u8]` which is composed of: - - Pointer to instruction data: `u64` - - Length of instruction data: `u64` -- Programm account index in transaction: `u16` -- Number of instruction accounts: `u16` -- For each instruction account: - Index to transaction account: `u16` - Flags bitfield: `u16` (bit 0 is signer, bit 1 is writable) -### Per Instruction Mappings +The second region is readonly and starts at `0x700000000`. It must also be +updated at each CPI call edge. This region contains an array of all +`InstructionAccounts` for every instruction in the transction. In other words: + +- For instruction in transaction: + - For each account in instruction: + - `InstructionAccount`, consisting of: + - Index to transaction account: `u16` + - Flags bitfield: `u16` (bit 0 is signer, bit 1 is writable) -A readonly memory region starting at `0x700000000` must be mapped -in for the instruction data. It too does not require any updates. +### Accounts area For each unique (meaning deduplicated) instruction account the payload must be mapped in at `0x800000000` plus `0x100000000` times the index of the @@ -101,37 +126,123 @@ instruction account has the writable flag set and is owned by the current program it is mapped in as a writable region. The writability of a region must be updated as programs through out the transaction modify the account metadata. -### Lazy deserialization on the dApp side (inside the SDK) +The runtime must only map the payload for accounts that belong in the current +executing instruction. The payload for accounts belonging to sibling instructions +must NOT be mapped. -With this design a program SDK can (but no longer needs to) eagerly deserialize -all account metadata at the entrypoint. Because this layout is strictly aligned -and uses proper arrays, it is possible to directly calculate the offset of a -single accounts metadata with only one indirect lookup and no need to scan all -preceeding metadata. This allows a program SDK to offer a lazy interface which -only interacts with the account metadata fields which are needed, only of the -accounts which are of interest and only when necessary. +### Instruction payload area + +For each instruction, the runtime must map its payload at address +`0x10800000000` plus `0x100000000` times the index of the instruction in the +trasaction. All instruction payload mappings are readonly. + +One extra writable mapping must be created after the last instruction payload +area to be the CPI scratch pad, i.e. at address `0x10800000000` plus +`0x100000000` times the number of instructions in the transaction. Its purpose +is for programs to write CPI instruction data directly to it and avoid copies. + +### VM initialization + +During the initilization of the virtual machine, the runtime must load the +value `0x400000000` in register `R1` and value `0x600000000` in register `R2`. +These values represent addresses for programs to easily find the areas to read +information about the transaction and the instructions. ### Changes to syscalls -The `AccountInfo` parameter of the CPI syscall `sol_invoke_signed_c` must be -ignored and programs using `sol_invoke_signed_rust` must be rejected if ABI v2 -is in use. Instead the changes to account metadata will be communicated -explicitly through separate syscalls: +Changes to the account metadata must now be communicated with specific +syscalls, as detailed below: - `sol_resize_account`: Dst account, new length as `u64` - `sol_assign_owner`: Dst account, new owner as `&[u8; 32]` - `sol_transfer_lamports`: Dst account, src account, amount as `u64` -The account parameters are guest pointers to the structure of the transaction -accounts (see per transaction serialization). +The account parameters are the index of the account in the transaction. + +Changes to the account payload length and all the scratchpads sections +introduced in this SIMD (the return-data scratchpad and the CPI scratchpad) +must be communicated via a new sycall `set_buffer_length`, with the following +parameters: + +- Address of region to be resized: `u64` +- New length of region: `u64` + +The syscall must check if the address belongs to either a writable account +payload or one of the scratchpads and return and error otherwise. Constrains +for the maximum resizable limits must also be verified (10 kb). + +The verifier must reject SBPFv4 programs containing the `sol_invoke_signed_c` +and `sol_invoke_signed_rust`, since they are not compatible with ABIv2. A new +syscall `sol_invoke_signed_v2` must replace them. The parameters for +`sol_invoke_signed_v2` are the following: -Programs using `sol_get_return_data` must be rejected if ABI v2 is in use. +- Index in transaction of program ID to be called: `u64`. +- A slice `&[InstructionAccount]`, with each element `InstructionAccount` + containing, as previously mentioned: + - Index to transaction account: `u16` + - Flags bitfield: `u16` (bit 0 is signer, bit 1 is writable) +- Signer seeds: `&[&[u8]]` + +Programs using `sol_get_return_data` and `sol_set_return_data` must be +rejected by the verfier if ABI v2 is in use. + +### Scratchpads managemnts + +This SIMD introduces two scratch pad regions: the return-data scratchpad and +the CPI scratchpad. At the beginning of every instruction, these scratchpads +must be empty and their size must be zero. + +Programs must set the desired length for them using the `set_buffer_length` +syscall. Reads and writes to a region beyond the scratchpad length must +trigger an access violation error. + +The management for the writable accounts payload must work similarly, except +that they must not be initialized empty, but instead with the pre-existing +data it holds. + +### CPIs + +With ABIv2 and the new `sol_invoke_signed_v2` syscall, CPIs must be managed +differently. At each CPI call, the runtime must perform the following actions: + +1. Append the slice `&[InstructionAccount]` passed as a parameter to the + array kept at address `0x700000000`. +2. Append a new instruction at the end of the serialization array kept at + `0x600000000`. +3. Transform the caller CPI scratchpad into a readonly instruction payload + region visible for the callee. +4. Change the visibility and write permissions for the account payload + regions, according to the CPI accounts and their flags. +5. Update the address for the callee CPI scratchpad, the index of current + executing transaction, and the number of instructions in transaction at + address `0x400000000`. + +When the CPI returns, the runtime must do the following: + +1. Update the address for the CPI scratchpad, and keep the previouly used one + in its exsiting address assigned during CPI call. The new CPI scratchpad + address is the same as the previous one plus `0x100000000`. +2. Change the read and write permission for the account payload regions, + according to potential changes in account ownership. +3. Update the index of current executing instruction. +4. No changes must be done in addresses `0x600000000` and `0x700000000`. ### Changes to CU metering CPI will no longer charge CUs for the length of account payloads. Instead TBD CUs will be charged for every instruction account. Also TBD CUs will be charged -for the three new account metadata updating syscalls. +for the three new account metadata updating syscalls. TBD will be charged for +resizing a scratchpad. + +### Lazy deserialization on the dApp side (inside the SDK) + +With this design a program SDK can (but no longer needs to) eagerly deserialize +all account metadata at the entrypoint. Because this layout is strictly aligned +and uses proper arrays, it is possible to directly calculate the offset of a +single accounts metadata with only one indirect lookup and no need to scan all +preceeding metadata. This allows a program SDK to offer a lazy interface which +only interacts with the account metadata fields which are needed, only of the +accounts which are of interest and only when necessary. ## Impact From 6fdcac4a008ae082bdc1c53e43a2ce102f42f1e4 Mon Sep 17 00:00:00 2001 From: Lucas Steuernagel Date: Tue, 17 Jun 2025 11:57:37 -0300 Subject: [PATCH 05/31] Use index and update CPI steps --- proposals/0177-program-runtime-abiv2.md | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/proposals/0177-program-runtime-abiv2.md b/proposals/0177-program-runtime-abiv2.md index 34d444df2..ce0234b3b 100644 --- a/proposals/0177-program-runtime-abiv2.md +++ b/proposals/0177-program-runtime-abiv2.md @@ -93,7 +93,7 @@ The first one is a readonly region starting at `0x600000000`. It must be updated at each CPI call edge. The contents of this region are the following: - For each instruction in transaction: - - Program ID: `[u8; 32]` + - Index in transaction of program account to be executed: `u64` - Reference to a slice of instruction accounts `&[InstructionAccount]`, consisting of: - Pointer to beginning of slice: `u64` @@ -181,7 +181,6 @@ syscall `sol_invoke_signed_v2` must replace them. The parameters for containing, as previously mentioned: - Index to transaction account: `u16` - Flags bitfield: `u16` (bit 0 is signer, bit 1 is writable) -- Signer seeds: `&[&[u8]]` Programs using `sol_get_return_data` and `sol_set_return_data` must be rejected by the verfier if ABI v2 is in use. @@ -205,15 +204,18 @@ data it holds. With ABIv2 and the new `sol_invoke_signed_v2` syscall, CPIs must be managed differently. At each CPI call, the runtime must perform the following actions: -1. Append the slice `&[InstructionAccount]` passed as a parameter to the +1. Verify that all account indexes received in the `InstructionAccount` array + belong in the current executing instruction. Likewise, the prgram ID index + that should be called must also undergo the same verification. +2. Append the slice `&[InstructionAccount]` passed as a parameter to the array kept at address `0x700000000`. -2. Append a new instruction at the end of the serialization array kept at +3. Append a new instruction at the end of the serialization array kept at `0x600000000`. -3. Transform the caller CPI scratchpad into a readonly instruction payload +4. Transform the caller CPI scratchpad into a readonly instruction payload region visible for the callee. -4. Change the visibility and write permissions for the account payload +5. Change the visibility and write permissions for the account payload regions, according to the CPI accounts and their flags. -5. Update the address for the callee CPI scratchpad, the index of current +6. Update the address for the callee CPI scratchpad, the index of current executing transaction, and the number of instructions in transaction at address `0x400000000`. From 18c05b133f82462ecffbe955dc55a056efc46b54 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Alexander=20Mei=C3=9Fner?= Date: Fri, 20 Jun 2025 18:58:46 +0200 Subject: [PATCH 06/31] Updates "Motivation" section. --- proposals/0177-program-runtime-abiv2.md | 40 ++++++++++++++----------- 1 file changed, 22 insertions(+), 18 deletions(-) diff --git a/proposals/0177-program-runtime-abiv2.md b/proposals/0177-program-runtime-abiv2.md index ce0234b3b..acf5823fc 100644 --- a/proposals/0177-program-runtime-abiv2.md +++ b/proposals/0177-program-runtime-abiv2.md @@ -19,24 +19,28 @@ simplify the address translation logic and allow for easy direct mapping. ## Motivation -At the moment all validator implementations have to copy (and compare) data in -and out of the virtual memory of the virtual machine. There are four possible -account data copy paths: - -- Serialization: Copy from program runtime (host) to virtual machine (guest) -- CPI call edge: Copy from virtual machine (guest) to program runtime (host) -- CPI return edge: Copy from program runtime (host) to virtual machine (guest) -- Deserialization: Copy from virtual machine (guest) to program runtime (host) - -To avoid this, a feature named "direct mapping" was designed which uses the -address translation logic of the virtual machine to emulate the serialization -and deserialization without actually performing copies. - -Implementing direct mapping in the current ABI v0 and v1 is very complex -because of unaligned virtual memory regions and memory accesses overlapping -multiple virtual memory regions. Instead the layout of the virtual address -space should be adjusted so that all virtual memory regions are aligned to -4 GiB. +Direct mapping of the account payload data is enabled by SIMD-0219. +However, there remains a big optimization potential for both programs and the +program runtime: + +- Instruction data could be mapped directly as well +- Return data could be mapped directly too +- Account payload could be resized freely (no more 10 KiB growth limit) +- CPI could become cheaper in terms of CU consumption +- Most structures could be shared between programs and program runtime, +requiring only a single serialization at the beginning of a transaction and +only small adjustments after +- Per instruction serialization before a program runs could be removed entriely +- Per instruction deserialization after a program runs could be removed too +- Deserialization inside the dApp could be reduced to a minimum +- programs would only have to pay for what they use, not having to deserialize +all instruction accounts which were passed in +- Scanning sibling instructions would not require a syscall +- Memory regions (and thus address translation) which SIMD-0219 made unaligned +could be aligned (to 4 GiB) again + +All of these however do necessitate a major change in the layout how the +program runtime and programs interface (ABI). ## Alternatives Considered From 8c9336eac9d11c2dd1ab661ed7c65de9dd887328 Mon Sep 17 00:00:00 2001 From: Lucas Steuernagel Date: Fri, 20 Jun 2025 15:44:46 -0300 Subject: [PATCH 07/31] Add CPI nesting level and index of parent instruction --- proposals/0177-program-runtime-abiv2.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/proposals/0177-program-runtime-abiv2.md b/proposals/0177-program-runtime-abiv2.md index acf5823fc..e6c25b2da 100644 --- a/proposals/0177-program-runtime-abiv2.md +++ b/proposals/0177-program-runtime-abiv2.md @@ -98,6 +98,8 @@ updated at each CPI call edge. The contents of this region are the following: - For each instruction in transaction: - Index in transaction of program account to be executed: `u64` + - CPI nesting level: `u16` + - Index of parent instruction (`u16::MAX` for top-level instructions): `u16` - Reference to a slice of instruction accounts `&[InstructionAccount]`, consisting of: - Pointer to beginning of slice: `u64` From 0b218551ce00e556f7cd382271a31c898ef15beb Mon Sep 17 00:00:00 2001 From: Lucas Date: Wed, 25 Jun 2025 11:54:55 -0300 Subject: [PATCH 08/31] Remove unnecessary syscall --- proposals/0177-program-runtime-abiv2.md | 1 - 1 file changed, 1 deletion(-) diff --git a/proposals/0177-program-runtime-abiv2.md b/proposals/0177-program-runtime-abiv2.md index e6c25b2da..2bb277b31 100644 --- a/proposals/0177-program-runtime-abiv2.md +++ b/proposals/0177-program-runtime-abiv2.md @@ -159,7 +159,6 @@ information about the transaction and the instructions. Changes to the account metadata must now be communicated with specific syscalls, as detailed below: -- `sol_resize_account`: Dst account, new length as `u64` - `sol_assign_owner`: Dst account, new owner as `&[u8; 32]` - `sol_transfer_lamports`: Dst account, src account, amount as `u64` From aa600a7a0f181e9c8f89ed14cda43f8f7408d5cf Mon Sep 17 00:00:00 2001 From: Lucas Date: Thu, 3 Jul 2025 15:37:53 -0300 Subject: [PATCH 09/31] Use u8 instead of u16 bitfield --- proposals/0177-program-runtime-abiv2.md | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/proposals/0177-program-runtime-abiv2.md b/proposals/0177-program-runtime-abiv2.md index 2bb277b31..95a3fca8c 100644 --- a/proposals/0177-program-runtime-abiv2.md +++ b/proposals/0177-program-runtime-abiv2.md @@ -111,7 +111,8 @@ updated at each CPI call edge. The contents of this region are the following: Let `InstrucionAccount` contain the following fields: - Index to transaction account: `u16` - - Flags bitfield: `u16` (bit 0 is signer, bit 1 is writable) + - Signer flag: `u8` (1 for signer, 0 for non-singer) + - Writable flag: `u8` (1 for writable, 0 for readonly) The second region is readonly and starts at `0x700000000`. It must also be updated at each CPI call edge. This region contains an array of all @@ -121,7 +122,8 @@ updated at each CPI call edge. This region contains an array of all - For each account in instruction: - `InstructionAccount`, consisting of: - Index to transaction account: `u16` - - Flags bitfield: `u16` (bit 0 is signer, bit 1 is writable) + - Signer flag: `u8` (1 for signer, 0 for non-singer) + - Writable flag: `u8` (1 for writable, 0 for readonly) ### Accounts area @@ -185,7 +187,8 @@ syscall `sol_invoke_signed_v2` must replace them. The parameters for - A slice `&[InstructionAccount]`, with each element `InstructionAccount` containing, as previously mentioned: - Index to transaction account: `u16` - - Flags bitfield: `u16` (bit 0 is signer, bit 1 is writable) + - Signer flag: `u8` (1 for signer, 0 for non-singer) + - Writable flag: `u8` (1 for writable, 0 for readonly) Programs using `sol_get_return_data` and `sol_set_return_data` must be rejected by the verfier if ABI v2 is in use. From 5053e9a5a5e5aa30d54479dbddce12fe4e6b635b Mon Sep 17 00:00:00 2001 From: Lucas Date: Thu, 3 Jul 2025 15:49:03 -0300 Subject: [PATCH 10/31] Split account metadata and update addresses --- proposals/0177-program-runtime-abiv2.md | 43 ++++++++++++++++--------- 1 file changed, 27 insertions(+), 16 deletions(-) diff --git a/proposals/0177-program-runtime-abiv2.md b/proposals/0177-program-runtime-abiv2.md index 95a3fca8c..22c0be910 100644 --- a/proposals/0177-program-runtime-abiv2.md +++ b/proposals/0177-program-runtime-abiv2.md @@ -57,14 +57,13 @@ v4 or above. ### Memory Regions -#### Transaction area +#### Transaction metadata area At the beginning of a transaction the program runtime must prepare a readonly memory region starting at `0x400000000`. This region is shared by all instructions running programs with support to new ABI. It must be updated as -as instructions through out the transaction modify the account metadata, the -CPI scratchpad or the return data. The contents of this memory region are the -following: +as instructions through out the transaction modify the CPI scratchpad or the +return data. The contents of this memory region are the following: - Key of the program which wrote to the return-data scratchpad most recently: `[u8; 32]` @@ -77,6 +76,16 @@ following: - Index of current executing instruction: `u64` - Number of instructions in transaction: `u64` - The number of transaction accounts: `u64` + + +#### Account metadata area + +This region starts at `0x500000000`, is readonly and holds the metadata for +all accounts in the transaction. It is shared by all instructions running +programs with support for ABIv2, and must be updated as instruction modify the +metadata with the provided syscalls (see the `Changes to syscalls` section). +The contents for this region are as follow: + - For each transaction account: - Key: `[u8; 32]` - Owner: `[u8; 32]` @@ -85,11 +94,6 @@ following: - Pointer to account payload: `u64` - Account payload length: `u64` -#### Return data scratchpad - -A writable memory region starting at `0x500000000` must be mapped in for the -return-data scratchpad. - #### Instruction area For each transction, the program runtime must also preapre two memory regions. @@ -102,7 +106,8 @@ updated at each CPI call edge. The contents of this region are the following: - Index of parent instruction (`u16::MAX` for top-level instructions): `u16` - Reference to a slice of instruction accounts `&[InstructionAccount]`, consisting of: - - Pointer to beginning of slice: `u64` + - Pointer to beginning of slice: `u64` (points to the `0x700000000` + region) - Number of elements in slice: `u64` - Instruction data `&[u8]`, which is composed of: - Pointer to data: `u64` @@ -125,10 +130,15 @@ updated at each CPI call edge. This region contains an array of all - Signer flag: `u8` (1 for signer, 0 for non-singer) - Writable flag: `u8` (1 for writable, 0 for readonly) +#### Return data scratchpad + +A writable memory region starting at `0x800000000` must be mapped in for the +return-data scratchpad. + ### Accounts area For each unique (meaning deduplicated) instruction account the payload must -be mapped in at `0x800000000` plus `0x100000000` times the index of the +be mapped in at `0x900000000` plus `0x100000000` times the index of the **transaction** account (not the index of the instruction account). Only if the instruction account has the writable flag set and is owned by the current program it is mapped in as a writable region. The writability of a region must @@ -141,20 +151,21 @@ must NOT be mapped. ### Instruction payload area For each instruction, the runtime must map its payload at address -`0x10800000000` plus `0x100000000` times the index of the instruction in the +`0x10900000000` plus `0x100000000` times the index of the instruction in the trasaction. All instruction payload mappings are readonly. One extra writable mapping must be created after the last instruction payload -area to be the CPI scratch pad, i.e. at address `0x10800000000` plus +area to be the CPI scratch pad, i.e. at address `0x10900000000` plus `0x100000000` times the number of instructions in the transaction. Its purpose is for programs to write CPI instruction data directly to it and avoid copies. ### VM initialization During the initilization of the virtual machine, the runtime must load the -value `0x400000000` in register `R1` and value `0x600000000` in register `R2`. -These values represent addresses for programs to easily find the areas to read -information about the transaction and the instructions. +value `0x400000000` in register `R1`, value `0x500000000` in register `R2`, +and value `0x600000000` in register `R3`. These values represent addresses for +programs to easily find the areas to read information about the transaction +and the instructions. ### Changes to syscalls From 4d65a1889748a7785c8c2a3a99da976103d914a2 Mon Sep 17 00:00:00 2001 From: Lucas Date: Mon, 8 Sep 2025 16:10:28 -0300 Subject: [PATCH 11/31] Update parameters to sol_invoke_signed_v2 --- proposals/0177-program-runtime-abiv2.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/proposals/0177-program-runtime-abiv2.md b/proposals/0177-program-runtime-abiv2.md index 22c0be910..26e7c1eb1 100644 --- a/proposals/0177-program-runtime-abiv2.md +++ b/proposals/0177-program-runtime-abiv2.md @@ -195,11 +195,15 @@ syscall `sol_invoke_signed_v2` must replace them. The parameters for `sol_invoke_signed_v2` are the following: - Index in transaction of program ID to be called: `u64`. -- A slice `&[InstructionAccount]`, with each element `InstructionAccount` +- A pointer to a slice `&[InstructionAccount]`, with each element + `InstructionAccount` containing, as previously mentioned: - Index to transaction account: `u16` - Signer flag: `u8` (1 for signer, 0 for non-singer) - Writable flag: `u8` (1 for writable, 0 for readonly) +- The length of the `&[InstructionAccount]` slice. +- A pointer to the singer seeds of type `&[&[&[u8]]]`. +- The length of the outer signer seeds slice in `&[&[&[u8]]]`. Programs using `sol_get_return_data` and `sol_set_return_data` must be rejected by the verfier if ABI v2 is in use. From fab4724c659e037c8b30b063002054a8cba6ccad Mon Sep 17 00:00:00 2001 From: Lucas Date: Thu, 4 Dec 2025 15:49:51 -0300 Subject: [PATCH 12/31] Use u32 indexes for alignment --- proposals/0177-program-runtime-abiv2.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/0177-program-runtime-abiv2.md b/proposals/0177-program-runtime-abiv2.md index 26e7c1eb1..1aee31a35 100644 --- a/proposals/0177-program-runtime-abiv2.md +++ b/proposals/0177-program-runtime-abiv2.md @@ -102,8 +102,8 @@ updated at each CPI call edge. The contents of this region are the following: - For each instruction in transaction: - Index in transaction of program account to be executed: `u64` - - CPI nesting level: `u16` - - Index of parent instruction (`u16::MAX` for top-level instructions): `u16` + - CPI nesting level: `u32` + - Index of parent instruction (`u32::MAX` for top-level instructions): `u32` - Reference to a slice of instruction accounts `&[InstructionAccount]`, consisting of: - Pointer to beginning of slice: `u64` (points to the `0x700000000` From 0aee7400c9785232d33d5e488efa6d64c15ff8d6 Mon Sep 17 00:00:00 2001 From: Lucas Date: Thu, 4 Dec 2025 15:54:38 -0300 Subject: [PATCH 13/31] Add number of CPIs --- proposals/0177-program-runtime-abiv2.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/proposals/0177-program-runtime-abiv2.md b/proposals/0177-program-runtime-abiv2.md index 1aee31a35..d9b5a35a4 100644 --- a/proposals/0177-program-runtime-abiv2.md +++ b/proposals/0177-program-runtime-abiv2.md @@ -73,9 +73,10 @@ return data. The contents of this memory region are the following: - The CPI scratchpad: `&[u8]`, which consists of: - Pointer to CPI scratchpad: `u64` - Length of CPI scratchpad: `u64` -- Index of current executing instruction: `u64` -- Number of instructions in transaction: `u64` -- The number of transaction accounts: `u64` +- Index of current executing instruction: `u32` +- Number of instructions in transaction (including CPIs): `u32` +- Number of executed CPIs: `u32` +- The number of transaction accounts: `u32` #### Account metadata area From b849c2d97587c894a8dbda23e11e335ea0e330d4 Mon Sep 17 00:00:00 2001 From: Lucas Date: Thu, 4 Dec 2025 17:07:33 -0300 Subject: [PATCH 14/31] Update instruction accounts area --- proposals/0177-program-runtime-abiv2.md | 38 +++++++++++++------------ 1 file changed, 20 insertions(+), 18 deletions(-) diff --git a/proposals/0177-program-runtime-abiv2.md b/proposals/0177-program-runtime-abiv2.md index d9b5a35a4..9a2f32219 100644 --- a/proposals/0177-program-runtime-abiv2.md +++ b/proposals/0177-program-runtime-abiv2.md @@ -107,8 +107,7 @@ updated at each CPI call edge. The contents of this region are the following: - Index of parent instruction (`u32::MAX` for top-level instructions): `u32` - Reference to a slice of instruction accounts `&[InstructionAccount]`, consisting of: - - Pointer to beginning of slice: `u64` (points to the `0x700000000` - region) + - Pointer to slice: `u64` - Number of elements in slice: `u64` - Instruction data `&[u8]`, which is composed of: - Pointer to data: `u64` @@ -120,26 +119,15 @@ Let `InstrucionAccount` contain the following fields: - Signer flag: `u8` (1 for signer, 0 for non-singer) - Writable flag: `u8` (1 for writable, 0 for readonly) -The second region is readonly and starts at `0x700000000`. It must also be -updated at each CPI call edge. This region contains an array of all -`InstructionAccounts` for every instruction in the transction. In other words: - -- For instruction in transaction: - - For each account in instruction: - - `InstructionAccount`, consisting of: - - Index to transaction account: `u16` - - Signer flag: `u8` (1 for signer, 0 for non-singer) - - Writable flag: `u8` (1 for writable, 0 for readonly) - #### Return data scratchpad -A writable memory region starting at `0x800000000` must be mapped in for the +A writable memory region starting at `0x700000000` must be mapped in for the return-data scratchpad. ### Accounts area For each unique (meaning deduplicated) instruction account the payload must -be mapped in at `0x900000000` plus `0x100000000` times the index of the +be mapped in at `0x800000000` plus `0x100000000` times the index of the **transaction** account (not the index of the instruction account). Only if the instruction account has the writable flag set and is owned by the current program it is mapped in as a writable region. The writability of a region must @@ -152,14 +140,28 @@ must NOT be mapped. ### Instruction payload area For each instruction, the runtime must map its payload at address -`0x10900000000` plus `0x100000000` times the index of the instruction in the +`0x10800000000` plus `0x100000000` times the index of the instruction in the trasaction. All instruction payload mappings are readonly. One extra writable mapping must be created after the last instruction payload -area to be the CPI scratch pad, i.e. at address `0x10900000000` plus -`0x100000000` times the number of instructions in the transaction. Its purpose +area to be the CPI scratch pad, i.e. at address `0x10800000000` plus +`0x100000000` times the number of instructions in the transaction. Its purpose is for programs to write CPI instruction data directly to it and avoid copies. +### Instruction accounts area + +For each instruction, the runtime must map an array of `InstructionAccount` +(as previously defined) at address `0x14800000000` plus `0x100000000` times +the index of the instruction in the transaction. This mapped are is readonly. + +Each of these memory regions contain the following for each instruction: + +- For each account in instruction: + - `InstructionAccount`, consisting of: + - Index to transaction account: `u16` + - Signer flag: `u8` (1 for signer, 0 for non-singer) + - Writable flag: `u8` (1 for writable, 0 for readonly) + ### VM initialization During the initilization of the virtual machine, the runtime must load the From c022960d5fffb8e0e5a932cb03f1b400930e00aa Mon Sep 17 00:00:00 2001 From: Lucas Date: Tue, 6 Jan 2026 18:11:23 -0300 Subject: [PATCH 15/31] Nagisa's suggestions --- proposals/0177-program-runtime-abiv2.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/proposals/0177-program-runtime-abiv2.md b/proposals/0177-program-runtime-abiv2.md index 9a2f32219..8cbd0afb7 100644 --- a/proposals/0177-program-runtime-abiv2.md +++ b/proposals/0177-program-runtime-abiv2.md @@ -185,12 +185,13 @@ introduced in this SIMD (the return-data scratchpad and the CPI scratchpad) must be communicated via a new sycall `set_buffer_length`, with the following parameters: -- Address of region to be resized: `u64` +- Base address of region to be resized: `u64` - New length of region: `u64` -The syscall must check if the address belongs to either a writable account -payload or one of the scratchpads and return and error otherwise. Constrains -for the maximum resizable limits must also be verified (10 kb). +The syscall must check if the address matches the base address of either a +writable account payload mapping or one of the scratchpad mappings and return +an error otherwise. Constrains for the maximum resizable limits must also be +verified (10 kb). The verifier must reject SBPFv4 programs containing the `sol_invoke_signed_c` and `sol_invoke_signed_rust`, since they are not compatible with ABIv2. A new @@ -211,7 +212,7 @@ syscall `sol_invoke_signed_v2` must replace them. The parameters for Programs using `sol_get_return_data` and `sol_set_return_data` must be rejected by the verfier if ABI v2 is in use. -### Scratchpads managemnts +### Scratchpad management This SIMD introduces two scratch pad regions: the return-data scratchpad and the CPI scratchpad. At the beginning of every instruction, these scratchpads From c3303dbcd8080f690c0497edab043030bd5997e9 Mon Sep 17 00:00:00 2001 From: Lucas Date: Wed, 7 Jan 2026 16:04:41 -0300 Subject: [PATCH 16/31] Update instruction area with u16s --- proposals/0177-program-runtime-abiv2.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/proposals/0177-program-runtime-abiv2.md b/proposals/0177-program-runtime-abiv2.md index 8cbd0afb7..00377bb5a 100644 --- a/proposals/0177-program-runtime-abiv2.md +++ b/proposals/0177-program-runtime-abiv2.md @@ -102,9 +102,10 @@ The first one is a readonly region starting at `0x600000000`. It must be updated at each CPI call edge. The contents of this region are the following: - For each instruction in transaction: - - Index in transaction of program account to be executed: `u64` - - CPI nesting level: `u32` - - Index of parent instruction (`u32::MAX` for top-level instructions): `u32` + - Reserved filed for alignment and potential future usage: `u16` + - Index in transaction of program account to be executed: `u16` + - CPI nesting level: `u16` + - Index of parent instruction (`u32::MAX` for top-level instructions): `u16` - Reference to a slice of instruction accounts `&[InstructionAccount]`, consisting of: - Pointer to slice: `u64` From a94b38ce74f3038d8364aeaff238159f8c91c1ee Mon Sep 17 00:00:00 2001 From: Lucas Ste <38472950+LucasSte@users.noreply.github.com> Date: Mon, 2 Feb 2026 16:50:49 -0300 Subject: [PATCH 17/31] Update proposals/0177-program-runtime-abiv2.md Co-authored-by: Alex Kahn <43892045+alnoki@users.noreply.github.com> --- proposals/0177-program-runtime-abiv2.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/0177-program-runtime-abiv2.md b/proposals/0177-program-runtime-abiv2.md index 00377bb5a..64dbf4bb2 100644 --- a/proposals/0177-program-runtime-abiv2.md +++ b/proposals/0177-program-runtime-abiv2.md @@ -97,7 +97,7 @@ The contents for this region are as follow: #### Instruction area -For each transction, the program runtime must also preapre two memory regions. +For each transaction, the program runtime must also prepare two memory regions. The first one is a readonly region starting at `0x600000000`. It must be updated at each CPI call edge. The contents of this region are the following: From 2b641358ee80b884d5a81758c06de83b803a193e Mon Sep 17 00:00:00 2001 From: Lucas Date: Thu, 26 Feb 2026 16:59:45 -0300 Subject: [PATCH 18/31] Update variable names and their purpose --- proposals/0177-program-runtime-abiv2.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/proposals/0177-program-runtime-abiv2.md b/proposals/0177-program-runtime-abiv2.md index 64dbf4bb2..d5837c7eb 100644 --- a/proposals/0177-program-runtime-abiv2.md +++ b/proposals/0177-program-runtime-abiv2.md @@ -74,8 +74,9 @@ return data. The contents of this memory region are the following: - Pointer to CPI scratchpad: `u64` - Length of CPI scratchpad: `u64` - Index of current executing instruction: `u32` -- Number of instructions in transaction (including CPIs): `u32` -- Number of executed CPIs: `u32` +- Total number of instructions in transaction (including CPIs and top level + instructions): `u32` +- Number of CPIs in trace (under execution and finished): `u32` - The number of transaction accounts: `u32` From ccec371e3dc23252025a144ef50adec75b84a1b3 Mon Sep 17 00:00:00 2001 From: Lucas Ste <38472950+LucasSte@users.noreply.github.com> Date: Tue, 3 Mar 2026 11:58:14 -0300 Subject: [PATCH 19/31] Update proposals/0177-program-runtime-abiv2.md Co-authored-by: Alex Kahn <43892045+alnoki@users.noreply.github.com> --- proposals/0177-program-runtime-abiv2.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/0177-program-runtime-abiv2.md b/proposals/0177-program-runtime-abiv2.md index d5837c7eb..04e771d2e 100644 --- a/proposals/0177-program-runtime-abiv2.md +++ b/proposals/0177-program-runtime-abiv2.md @@ -143,7 +143,7 @@ must NOT be mapped. For each instruction, the runtime must map its payload at address `0x10800000000` plus `0x100000000` times the index of the instruction in the -trasaction. All instruction payload mappings are readonly. +transaction. All instruction payload mappings are readonly. One extra writable mapping must be created after the last instruction payload area to be the CPI scratch pad, i.e. at address `0x10800000000` plus From 47cb8f369692174b3c16f2feafe248a02ebd6a51 Mon Sep 17 00:00:00 2001 From: Lucas Ste <38472950+LucasSte@users.noreply.github.com> Date: Fri, 27 Mar 2026 16:16:46 -0300 Subject: [PATCH 20/31] Update proposals/0177-program-runtime-abiv2.md Co-authored-by: Alex Kahn <43892045+alnoki@users.noreply.github.com> --- proposals/0177-program-runtime-abiv2.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/0177-program-runtime-abiv2.md b/proposals/0177-program-runtime-abiv2.md index 04e771d2e..85e471ce3 100644 --- a/proposals/0177-program-runtime-abiv2.md +++ b/proposals/0177-program-runtime-abiv2.md @@ -115,7 +115,7 @@ updated at each CPI call edge. The contents of this region are the following: - Pointer to data: `u64` - Length of data: `u64` -Let `InstrucionAccount` contain the following fields: +Let `InstructionAccount` contain the following fields: - Index to transaction account: `u16` - Signer flag: `u8` (1 for signer, 0 for non-singer) From ebe6e360d511cc2bca2020011d66afb36a26d08b Mon Sep 17 00:00:00 2001 From: Lucas Date: Fri, 27 Mar 2026 16:56:08 -0300 Subject: [PATCH 21/31] Include recent suggestions --- proposals/0177-program-runtime-abiv2.md | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-) diff --git a/proposals/0177-program-runtime-abiv2.md b/proposals/0177-program-runtime-abiv2.md index 85e471ce3..f82e60a44 100644 --- a/proposals/0177-program-runtime-abiv2.md +++ b/proposals/0177-program-runtime-abiv2.md @@ -167,10 +167,16 @@ Each of these memory regions contain the following for each instruction: ### VM initialization During the initilization of the virtual machine, the runtime must load the -value `0x400000000` in register `R1`, value `0x500000000` in register `R2`, -and value `0x600000000` in register `R3`. These values represent addresses for -programs to easily find the areas to read information about the transaction -and the instructions. +following values to registers: + +1. Register R1: A pointer to the metadata of the instruction under execution. + (see section [Instruction area](#instruction-area)). +2. Register R2: A pointer to the instruction accounts slice for the + instruction under execution (see section + [Instruction accounts area](#instruction-accounts-area)). +3. Register R3: The number of instruction accounts for the instruction under + execution. + ### Changes to syscalls @@ -193,7 +199,10 @@ parameters: The syscall must check if the address matches the base address of either a writable account payload mapping or one of the scratchpad mappings and return an error otherwise. Constrains for the maximum resizable limits must also be -verified (10 kb). +verified for each region separetely. + +The `set_buffer_length` must charge a base cost (to be determined) plus the +same CU per byte ratio as the `memset` syscall. The verifier must reject SBPFv4 programs containing the `sol_invoke_signed_c` and `sol_invoke_signed_rust`, since they are not compatible with ABIv2. A new From 47f143516589fbfdf08acaf4c692ac95f49ec3c5 Mon Sep 17 00:00:00 2001 From: Lucas Date: Fri, 27 Mar 2026 17:05:30 -0300 Subject: [PATCH 22/31] Add sysvar accounts area --- proposals/0177-program-runtime-abiv2.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/proposals/0177-program-runtime-abiv2.md b/proposals/0177-program-runtime-abiv2.md index f82e60a44..d94c841ea 100644 --- a/proposals/0177-program-runtime-abiv2.md +++ b/proposals/0177-program-runtime-abiv2.md @@ -164,6 +164,20 @@ Each of these memory regions contain the following for each instruction: - Signer flag: `u8` (1 for signer, 0 for non-singer) - Writable flag: `u8` (1 for writable, 0 for readonly) +### Sysvar accounts area + +For each existing (non deprecated) sysvar account, the runtime must map its +payload at address `0x18800000000` plus `0x100000000` times the index of the +sysvar in the following order: + +0. Clock +1. Epoch rewards +2. Epoch Schdule +3. Last restart slot +4. Rent +5. Slot hashes +6. Stake history + ### VM initialization During the initilization of the virtual machine, the runtime must load the From da90cd3ce776331227673dfe4f1b14443f36faaa Mon Sep 17 00:00:00 2001 From: Lucas Ste <38472950+LucasSte@users.noreply.github.com> Date: Wed, 22 Apr 2026 12:13:34 -0300 Subject: [PATCH 23/31] Update proposals/0177-program-runtime-abiv2.md Co-authored-by: febo --- proposals/0177-program-runtime-abiv2.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/0177-program-runtime-abiv2.md b/proposals/0177-program-runtime-abiv2.md index d94c841ea..261d459d9 100644 --- a/proposals/0177-program-runtime-abiv2.md +++ b/proposals/0177-program-runtime-abiv2.md @@ -106,7 +106,7 @@ updated at each CPI call edge. The contents of this region are the following: - Reserved filed for alignment and potential future usage: `u16` - Index in transaction of program account to be executed: `u16` - CPI nesting level: `u16` - - Index of parent instruction (`u32::MAX` for top-level instructions): `u16` + - Index of parent instruction (`u16::MAX` for top-level instructions): `u16` - Reference to a slice of instruction accounts `&[InstructionAccount]`, consisting of: - Pointer to slice: `u64` From 53f2fd5f259d286b1484086655981c1e378c6d8f Mon Sep 17 00:00:00 2001 From: Lucas Date: Wed, 22 Apr 2026 18:10:21 -0300 Subject: [PATCH 24/31] @febo's suggestions --- proposals/0177-program-runtime-abiv2.md | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/proposals/0177-program-runtime-abiv2.md b/proposals/0177-program-runtime-abiv2.md index 261d459d9..56282c931 100644 --- a/proposals/0177-program-runtime-abiv2.md +++ b/proposals/0177-program-runtime-abiv2.md @@ -126,7 +126,7 @@ Let `InstructionAccount` contain the following fields: A writable memory region starting at `0x700000000` must be mapped in for the return-data scratchpad. -### Accounts area +#### Accounts area For each unique (meaning deduplicated) instruction account the payload must be mapped in at `0x800000000` plus `0x100000000` times the index of the @@ -139,7 +139,7 @@ The runtime must only map the payload for accounts that belong in the current executing instruction. The payload for accounts belonging to sibling instructions must NOT be mapped. -### Instruction payload area +#### Instruction payload area For each instruction, the runtime must map its payload at address `0x10800000000` plus `0x100000000` times the index of the instruction in the @@ -150,7 +150,7 @@ area to be the CPI scratch pad, i.e. at address `0x10800000000` plus `0x100000000` times the number of instructions in the transaction. Its purpose is for programs to write CPI instruction data directly to it and avoid copies. -### Instruction accounts area +#### Instruction accounts area For each instruction, the runtime must map an array of `InstructionAccount` (as previously defined) at address `0x14800000000` plus `0x100000000` times @@ -164,7 +164,7 @@ Each of these memory regions contain the following for each instruction: - Signer flag: `u8` (1 for signer, 0 for non-singer) - Writable flag: `u8` (1 for writable, 0 for readonly) -### Sysvar accounts area +#### Sysvar accounts area For each existing (non deprecated) sysvar account, the runtime must map its payload at address `0x18800000000` plus `0x100000000` times the index of the @@ -190,6 +190,9 @@ following values to registers: [Instruction accounts area](#instruction-accounts-area)). 3. Register R3: The number of instruction accounts for the instruction under execution. +4. Register R4: A pointer to the instruction payload of the instruction under + execution (see section [Instruction payload area](#instruction-payload-area)). +5. Register R5: The payload lenght for the instruction under execution. ### Changes to syscalls @@ -259,6 +262,8 @@ differently. At each CPI call, the runtime must perform the following actions: 1. Verify that all account indexes received in the `InstructionAccount` array belong in the current executing instruction. Likewise, the prgram ID index that should be called must also undergo the same verification. +2. Verify that accounts have the correct signer and writer flags set, avoiding + privelege promotion. 2. Append the slice `&[InstructionAccount]` passed as a parameter to the array kept at address `0x700000000`. 3. Append a new instruction at the end of the serialization array kept at From e5d8bf666ec002b5849cb856fcaf8fb7bcc1895c Mon Sep 17 00:00:00 2001 From: Lucas Date: Wed, 22 Apr 2026 18:27:46 -0300 Subject: [PATCH 25/31] Update CPI design --- proposals/0177-program-runtime-abiv2.md | 45 ++++++++++++++++++------- 1 file changed, 33 insertions(+), 12 deletions(-) diff --git a/proposals/0177-program-runtime-abiv2.md b/proposals/0177-program-runtime-abiv2.md index 56282c931..e2b6ac39d 100644 --- a/proposals/0177-program-runtime-abiv2.md +++ b/proposals/0177-program-runtime-abiv2.md @@ -73,6 +73,9 @@ return data. The contents of this memory region are the following: - The CPI scratchpad: `&[u8]`, which consists of: - Pointer to CPI scratchpad: `u64` - Length of CPI scratchpad: `u64` +- The CPI accounts scratchpad: `&[InstructionAccount]`, which consists of: + - Pointer to slice: `u64` + - Number of elements in slice: `u64` - Index of current executing instruction: `u32` - Total number of instructions in transaction (including CPIs and top level instructions): `u32` @@ -164,6 +167,11 @@ Each of these memory regions contain the following for each instruction: - Signer flag: `u8` (1 for signer, 0 for non-singer) - Writable flag: `u8` (1 for writable, 0 for readonly) +One extra writable mapping must be created after the last instruction accounts +area to be the CPI scratch pad, i.e. at address `0x14800000000` plus +`0x100000000` times the number of instructions in the transaction. Its purpose +is for programs to write CPI accounts directly to it and avoid copies. + #### Sysvar accounts area For each existing (non deprecated) sysvar account, the runtime must map its @@ -227,13 +235,6 @@ syscall `sol_invoke_signed_v2` must replace them. The parameters for `sol_invoke_signed_v2` are the following: - Index in transaction of program ID to be called: `u64`. -- A pointer to a slice `&[InstructionAccount]`, with each element - `InstructionAccount` - containing, as previously mentioned: - - Index to transaction account: `u16` - - Signer flag: `u8` (1 for signer, 0 for non-singer) - - Writable flag: `u8` (1 for writable, 0 for readonly) -- The length of the `&[InstructionAccount]` slice. - A pointer to the singer seeds of type `&[&[&[u8]]]`. - The length of the outer signer seeds slice in `&[&[&[u8]]]`. @@ -256,16 +257,37 @@ data it holds. ### CPIs -With ABIv2 and the new `sol_invoke_signed_v2` syscall, CPIs must be managed +#### Program side + +The workflow for cross program invokation on the program side will change. +Instead of programs themselves allocating memory on the heap or the stack for +CPI instruction data and CPI accounts, the program runtime must already +provide the pointers for programs to write to. + +The CPI data scratchpad is a region in the +[Instruction payload area](#instruction-payload-area), right after the space +reserved for the last intruction in the instruction trace. +In other words, `0x10800000000` plus `0x100000000` times the number of +instructions in the transaction. + +Likewise, the CPI accounts must be written to a runtime provided region +residing in the [Instruction accounts area](#instruction-accounts-area) +at `0x14800000000` plus `0x100000000` times the number of instructions in the +transaction. + +Both the aforementioned regions must begin with size zero, and programs must +resize them before writing to them using the `set_buffer_length` syscall. + +#### Runtime side + +With the new `sol_invoke_signed_v2` syscall, CPIs must be managed differently. At each CPI call, the runtime must perform the following actions: -1. Verify that all account indexes received in the `InstructionAccount` array +1. Verify that all account indexes received in the `InstructionAccount` area belong in the current executing instruction. Likewise, the prgram ID index that should be called must also undergo the same verification. 2. Verify that accounts have the correct signer and writer flags set, avoiding privelege promotion. -2. Append the slice `&[InstructionAccount]` passed as a parameter to the - array kept at address `0x700000000`. 3. Append a new instruction at the end of the serialization array kept at `0x600000000`. 4. Transform the caller CPI scratchpad into a readonly instruction payload @@ -284,7 +306,6 @@ When the CPI returns, the runtime must do the following: 2. Change the read and write permission for the account payload regions, according to potential changes in account ownership. 3. Update the index of current executing instruction. -4. No changes must be done in addresses `0x600000000` and `0x700000000`. ### Changes to CU metering From d5d1f7e563351d71895e4b40d3213f2b6afcf4fb Mon Sep 17 00:00:00 2001 From: Lucas Date: Wed, 22 Apr 2026 18:47:31 -0300 Subject: [PATCH 26/31] Use stable layout for CPI types --- proposals/0177-program-runtime-abiv2.md | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/proposals/0177-program-runtime-abiv2.md b/proposals/0177-program-runtime-abiv2.md index e2b6ac39d..6f40e124b 100644 --- a/proposals/0177-program-runtime-abiv2.md +++ b/proposals/0177-program-runtime-abiv2.md @@ -235,8 +235,15 @@ syscall `sol_invoke_signed_v2` must replace them. The parameters for `sol_invoke_signed_v2` are the following: - Index in transaction of program ID to be called: `u64`. -- A pointer to the singer seeds of type `&[&[&[u8]]]`. -- The length of the outer signer seeds slice in `&[&[&[u8]]]`. +- A pointer to the singer seeds of type `VmSlice>>`. +- The length of the outer signer seeds slice in + `VmSlice>>`. + +`VmSlice` is a stable layout type defined to share slices between the guest +and the host. It consists of: + +- `u64`: Pointer to the data. +- `u64`: Length of data (number of elements `T`) Programs using `sol_get_return_data` and `sol_set_return_data` must be rejected by the verfier if ABI v2 is in use. From f73bab9f4f91d61250f958b1f303217299150ebb Mon Sep 17 00:00:00 2001 From: Lucas Ste <38472950+LucasSte@users.noreply.github.com> Date: Fri, 1 May 2026 11:34:11 -0700 Subject: [PATCH 27/31] Update proposals/0177-program-runtime-abiv2.md Co-authored-by: Simonas Kazlauskas --- proposals/0177-program-runtime-abiv2.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/0177-program-runtime-abiv2.md b/proposals/0177-program-runtime-abiv2.md index 6f40e124b..c0ba8cb17 100644 --- a/proposals/0177-program-runtime-abiv2.md +++ b/proposals/0177-program-runtime-abiv2.md @@ -121,7 +121,7 @@ updated at each CPI call edge. The contents of this region are the following: Let `InstructionAccount` contain the following fields: - Index to transaction account: `u16` - - Signer flag: `u8` (1 for signer, 0 for non-singer) + - Signer flag: `u8` (1 for signer, 0 for non-signer) - Writable flag: `u8` (1 for writable, 0 for readonly) #### Return data scratchpad From 4a84d695724fede772220aa5299575bb5371dce3 Mon Sep 17 00:00:00 2001 From: Lucas Date: Fri, 8 May 2026 17:21:17 -0300 Subject: [PATCH 28/31] Add information about payer --- proposals/0177-program-runtime-abiv2.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/proposals/0177-program-runtime-abiv2.md b/proposals/0177-program-runtime-abiv2.md index c0ba8cb17..a80cddc4c 100644 --- a/proposals/0177-program-runtime-abiv2.md +++ b/proposals/0177-program-runtime-abiv2.md @@ -82,6 +82,11 @@ return data. The contents of this memory region are the following: - Number of CPIs in trace (under execution and finished): `u32` - The number of transaction accounts: `u32` +##### Payer information + +As accounts in this area sorted by their index in transaction, the payer +account must always be the account at index zero, as we surface runtime's +internal account ordering for programs. #### Account metadata area From 91ab453406f44127bc78631b0295771c636f3e7c Mon Sep 17 00:00:00 2001 From: Lucas Date: Fri, 8 May 2026 17:31:50 -0300 Subject: [PATCH 29/31] Update InstructionAccount and CPI --- proposals/0177-program-runtime-abiv2.md | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-) diff --git a/proposals/0177-program-runtime-abiv2.md b/proposals/0177-program-runtime-abiv2.md index a80cddc4c..11fccd55e 100644 --- a/proposals/0177-program-runtime-abiv2.md +++ b/proposals/0177-program-runtime-abiv2.md @@ -128,6 +128,8 @@ Let `InstructionAccount` contain the following fields: - Index to transaction account: `u16` - Signer flag: `u8` (1 for signer, 0 for non-signer) - Writable flag: `u8` (1 for writable, 0 for readonly) + - Pointer to the account metadata: `u64` (see [Account metadata area] + (#account-metadata-area)). #### Return data scratchpad @@ -169,8 +171,9 @@ Each of these memory regions contain the following for each instruction: - For each account in instruction: - `InstructionAccount`, consisting of: - Index to transaction account: `u16` - - Signer flag: `u8` (1 for signer, 0 for non-singer) - - Writable flag: `u8` (1 for writable, 0 for readonly) + - Signer flag: `u8` (1 for signer, 0 for non-singer) + - Writable flag: `u8` (1 for writable, 0 for readonly) + - Pointer to the account metadata: `u64` (see [Account metadata area](#account-metadata-area)). One extra writable mapping must be created after the last instruction accounts area to be the CPI scratch pad, i.e. at address `0x14800000000` plus @@ -297,7 +300,9 @@ differently. At each CPI call, the runtime must perform the following actions: 1. Verify that all account indexes received in the `InstructionAccount` area belong in the current executing instruction. Likewise, the prgram ID index - that should be called must also undergo the same verification. + that should be called must also undergo the same verification. The pointer + field in `InstructionAccount` must be filled by runtime, so programs may + leave it blank. 2. Verify that accounts have the correct signer and writer flags set, avoiding privelege promotion. 3. Append a new instruction at the end of the serialization array kept at @@ -319,6 +324,11 @@ When the CPI returns, the runtime must do the following: according to potential changes in account ownership. 3. Update the index of current executing instruction. +CPIs between ABIv0/v1 and ABIv2 program must be allowed, but costs will difer. +A CPI from an ABIv2 to another ABIv2 program must cost less than a CPI from an +ABIv2 to an ABIv0/v1 program, due to the decreased work overhead from program +runtime. + ### Changes to CU metering CPI will no longer charge CUs for the length of account payloads. Instead TBD From a510cdd7f6e12ded44fcb5aa0264b95c948123b3 Mon Sep 17 00:00:00 2001 From: Lucas Date: Tue, 19 May 2026 17:24:42 -0300 Subject: [PATCH 30/31] Clarify syscall parameters --- proposals/0177-program-runtime-abiv2.md | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/proposals/0177-program-runtime-abiv2.md b/proposals/0177-program-runtime-abiv2.md index 11fccd55e..d6600d70f 100644 --- a/proposals/0177-program-runtime-abiv2.md +++ b/proposals/0177-program-runtime-abiv2.md @@ -216,18 +216,21 @@ following values to registers: Changes to the account metadata must now be communicated with specific syscalls, as detailed below: -- `sol_assign_owner`: Dst account, new owner as `&[u8; 32]` -- `sol_transfer_lamports`: Dst account, src account, amount as `u64` - -The account parameters are the index of the account in the transaction. +- `sol_assign_owner(u64, *const [u8; 32])`. + - `u64`: Index in transaction of the account whose owner is changing, + - `*const [u8; 32]`: Pointer to the public key of the new owner. +- `sol_transfer_lamports(u64, u64, u64)`: + - `u64`: Index in transaction of the destination account. + - `u64`: Index in transaction of the source account. + - `u64`: Lamports amount. Changes to the account payload length and all the scratchpads sections introduced in this SIMD (the return-data scratchpad and the CPI scratchpad) -must be communicated via a new sycall `set_buffer_length`, with the following -parameters: +must be communicated via a new sycall `set_buffer_length(u64, u64)`, with the +following parameters: -- Base address of region to be resized: `u64` -- New length of region: `u64` +- `u64`: Base address of region to be resized. +- `u64`: New length of region. The syscall must check if the address matches the base address of either a writable account payload mapping or one of the scratchpad mappings and return From 050f8e735bcbb5e3e3bb9129c0f9060e3de37eba Mon Sep 17 00:00:00 2001 From: Lucas Date: Fri, 22 May 2026 11:48:57 -0300 Subject: [PATCH 31/31] Remove pointer field --- proposals/0177-program-runtime-abiv2.md | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/proposals/0177-program-runtime-abiv2.md b/proposals/0177-program-runtime-abiv2.md index d6600d70f..8d89886c0 100644 --- a/proposals/0177-program-runtime-abiv2.md +++ b/proposals/0177-program-runtime-abiv2.md @@ -128,8 +128,6 @@ Let `InstructionAccount` contain the following fields: - Index to transaction account: `u16` - Signer flag: `u8` (1 for signer, 0 for non-signer) - Writable flag: `u8` (1 for writable, 0 for readonly) - - Pointer to the account metadata: `u64` (see [Account metadata area] - (#account-metadata-area)). #### Return data scratchpad @@ -173,7 +171,6 @@ Each of these memory regions contain the following for each instruction: - Index to transaction account: `u16` - Signer flag: `u8` (1 for signer, 0 for non-singer) - Writable flag: `u8` (1 for writable, 0 for readonly) - - Pointer to the account metadata: `u64` (see [Account metadata area](#account-metadata-area)). One extra writable mapping must be created after the last instruction accounts area to be the CPI scratch pad, i.e. at address `0x14800000000` plus @@ -303,9 +300,7 @@ differently. At each CPI call, the runtime must perform the following actions: 1. Verify that all account indexes received in the `InstructionAccount` area belong in the current executing instruction. Likewise, the prgram ID index - that should be called must also undergo the same verification. The pointer - field in `InstructionAccount` must be filled by runtime, so programs may - leave it blank. + that should be called must also undergo the same verification. 2. Verify that accounts have the correct signer and writer flags set, avoiding privelege promotion. 3. Append a new instruction at the end of the serialization array kept at