Skip to content

flamenco, runtime: implement replace_spl_token_with_p_token#8921

Merged
topointon-jump merged 1 commit into
mainfrom
firedancer-replace-spl-token-with-p-token
Mar 20, 2026
Merged

flamenco, runtime: implement replace_spl_token_with_p_token#8921
topointon-jump merged 1 commit into
mainfrom
firedancer-replace-spl-token-with-p-token

Conversation

@topointon-jump
Copy link
Copy Markdown
Contributor

Copilot AI review requested due to automatic review settings March 16, 2026 19:49
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Implements the replace_spl_token_with_p_token feature transition in the Flamenco runtime by upgrading the SPL Token (loader v2) program to loader v3 using a new Core BPF migration helper, and wiring in the required buffer address + feature ID plumbing.

Changes:

  • Adds fd_upgrade_loader_v2_program_with_loader_v3_program() and supporting target checks/ELF validation in Core BPF migration code.
  • Introduces the ptoken program buffer address constant and ID bytes in system IDs.
  • Registers the replace_spl_token_with_p_token feature and triggers the migration at feature activation.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
src/flamenco/runtime/program/fd_builtin_programs.h Declares the new loader v2 → loader v3 upgrade helper.
src/flamenco/runtime/fd_system_ids_pp.h Adds PTOKEN_PROG_BUFFER_ID macro bytes.
src/flamenco/runtime/fd_system_ids.h Exposes fd_solana_ptoken_program_buffer_address.
src/flamenco/runtime/fd_system_ids.c Defines fd_solana_ptoken_program_buffer_address.
src/flamenco/runtime/fd_runtime.c Triggers the migration when replace_spl_token_with_p_token activates.
src/flamenco/runtime/fd_core_bpf_migration.c Implements the migration logic and ELF validation.
src/flamenco/features/feature_map.json Adds the new feature name/pubkey mapping.
src/flamenco/features/fd_features_generated.h Updates generated feature count/set ID and adds the feature field.
src/flamenco/features/fd_features_generated.c Adds the feature ID entry and perfect-hash mapping entries.

You can also share your feedback on Copilot code review. Take the survey.

Comment thread src/flamenco/runtime/fd_core_bpf_migration.c
Comment thread src/flamenco/runtime/fd_system_ids_pp.h Outdated
@topointon-jump topointon-jump force-pushed the firedancer-replace-spl-token-with-p-token branch from 2bf49a9 to 8d381b3 Compare March 16, 2026 19:56
Copilot AI review requested due to automatic review settings March 16, 2026 19:58
@topointon-jump topointon-jump force-pushed the firedancer-replace-spl-token-with-p-token branch from 8d381b3 to e11aa03 Compare March 16, 2026 19:58
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Implements the replace_spl_token_with_p_token feature activation path in the Flamenco runtime by adding a Loader-v2 → Loader-v3 program upgrade routine and wiring it into epoch-boundary feature activation, along with the required new IDs/constants.

Changes:

  • Add fd_upgrade_loader_v2_program_with_loader_v3_program() and supporting target checks in core BPF migration code.
  • Introduce the ptoken program buffer system ID and expose it via fd_system_ids.{h,c} for use at runtime.
  • Add the replace_spl_token_with_p_token feature to the feature map and generated feature tables, and invoke the upgrade when the feature activates.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
src/flamenco/runtime/program/fd_builtin_programs.h Exposes new upgrade helper prototype to runtime.
src/flamenco/runtime/fd_system_ids_pp.h Adds PTOKEN program buffer ID bytes.
src/flamenco/runtime/fd_system_ids.h Declares new ptoken program buffer address constant.
src/flamenco/runtime/fd_system_ids.c Defines new ptoken program buffer address constant.
src/flamenco/runtime/fd_runtime.c Triggers Loader v2→v3 upgrade on replace_spl_token_with_p_token activation.
src/flamenco/runtime/fd_core_bpf_migration.c Implements target checks and upgrade flow; adds ELF validation step.
src/flamenco/features/feature_map.json Adds new feature pubkey mapping.
src/flamenco/features/fd_features_generated.h Updates feature count/set ID and adds the new feature bitfield entry.
src/flamenco/features/fd_features_generated.c Adds new feature ID entry and perfect-hash lookup/static asserts.

You can also share your feedback on Copilot code review. Take the survey.

Comment thread src/flamenco/runtime/fd_core_bpf_migration.c Outdated
Comment thread src/flamenco/runtime/fd_core_bpf_migration.c Outdated
Comment thread src/flamenco/runtime/fd_system_ids_pp.h Outdated
@github-actions
Copy link
Copy Markdown

Performance Measurements ⏳

Suite Baseline New Change
backtest mainnet-368528500-perf per slot 0.060467 s 0.060563 s 0.159%
backtest mainnet-368528500-perf snapshot load 3.225 s 2.784 s -13.674%
backtest mainnet-368528500-perf total elapsed 60.467239 s 60.562513 s 0.158%
firedancer mem usage with mainnet.toml 985.41 GiB 985.41 GiB 0.000%

@topointon-jump topointon-jump force-pushed the firedancer-replace-spl-token-with-p-token branch from e11aa03 to 8f5477f Compare March 16, 2026 21:48
@github-actions
Copy link
Copy Markdown

Performance Measurements ⏳

Suite Baseline New Change
backtest mainnet-368528500-perf per slot 0.083109 s 0.082831 s -0.335%
backtest mainnet-368528500-perf snapshot load 3.867 s 3.416 s -11.663%
backtest mainnet-368528500-perf total elapsed 83.109279 s 82.831428 s -0.334%
firedancer mem usage with mainnet.toml 985.41 GiB 985.41 GiB 0.000%

Comment thread src/flamenco/runtime/fd_core_bpf_migration.c
Comment thread src/flamenco/runtime/fd_runtime_stack.h Outdated
Comment thread src/flamenco/runtime/fd_runtime_stack.h Outdated
Comment thread src/flamenco/runtime/fd_core_bpf_migration.c Outdated
Comment thread src/flamenco/runtime/fd_core_bpf_migration.c Outdated
Comment thread src/flamenco/runtime/fd_core_bpf_migration.c Outdated
Comment thread src/flamenco/runtime/fd_core_bpf_migration.c Outdated
Comment thread src/flamenco/runtime/fd_core_bpf_migration.c Outdated
Copilot AI review requested due to automatic review settings March 18, 2026 17:58
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Implements the replace_spl_token_with_p_token feature activation path in Flamenco by performing an epoch-boundary migration that upgrades the SPL Token program from a Loader v2 deployment to a Loader v3 deployment, using a ptoken source buffer.

Changes:

  • Add the replace_spl_token_with_p_token feature ID to the feature map and generated feature tables.
  • Introduce a ptoken program buffer system ID and invoke a new migration helper when the feature activates.
  • Implement fd_upgrade_loader_v2_program_with_loader_v3_program() and associated ELF validation checks in core BPF migration code, and add the new ledger backtest entry.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
src/flamenco/runtime/tests/run_backtest_all.sh Adds the new ledger backtest invocation for the feature scenario.
src/flamenco/runtime/program/fd_builtin_programs.h Declares the new loader-v2→loader-v3 migration helper.
src/flamenco/runtime/fd_system_ids_pp.h Adds PTOKEN_PROG_BUFFER_ID byte definition.
src/flamenco/runtime/fd_system_ids.h Exposes fd_solana_ptoken_program_buffer_address.
src/flamenco/runtime/fd_system_ids.c Defines the ptoken buffer pubkey constant.
src/flamenco/runtime/fd_runtime_stack.h Adds staging buffers for ELF validation during migrations.
src/flamenco/runtime/fd_runtime.c Triggers the migration at feature activation.
src/flamenco/runtime/fd_core_bpf_migration.c Implements the migration + deploy-equivalent validation checks.
src/flamenco/features/feature_map.json Adds the new feature name/pubkey entry.
src/flamenco/features/fd_features_generated.h Updates feature count/set ID and adds the new feature field.
src/flamenco/features/fd_features_generated.c Adds the new feature to the generated ID tables and layout asserts.

You can also share your feedback on Copilot code review. Take the survey.

Comment thread src/flamenco/runtime/fd_runtime_stack.h
Comment thread src/flamenco/runtime/fd_core_bpf_migration.c Outdated
Comment thread src/flamenco/runtime/fd_core_bpf_migration.c
Comment thread src/flamenco/runtime/fd_core_bpf_migration.c
Comment thread src/flamenco/runtime/fd_core_bpf_migration.c Outdated
Comment thread src/flamenco/runtime/fd_core_bpf_migration.c Outdated
Copilot AI review requested due to automatic review settings March 20, 2026 16:25
@topointon-jump topointon-jump force-pushed the firedancer-replace-spl-token-with-p-token branch from 511e395 to ba9d87c Compare March 20, 2026 16:32
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Implements the Solana feature transition for replace_spl_token_with_p_token by adding the new feature ID, wiring an epoch-boundary migration path that upgrades a Loader-v2 program to Loader-v3 using a staged buffer, and ensuring it runs in the backtest suite.

Changes:

  • Add replace_spl_token_with_p_token to the feature map and generated feature definitions.
  • Introduce fd_upgrade_loader_v2_program_with_loader_v3_program() and hook it into epoch feature activation, including ELF deploy-style validation.
  • Add the ptoken source buffer system ID and include the new scenario in the backtest-all script.

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
src/flamenco/runtime/tests/run_backtest_all.sh Adds the new ledger backtest scenario to the full backtest run list.
src/flamenco/runtime/program/fd_builtin_programs.h Exposes the new migration/upgrade API for Loader v2 → Loader v3 program replacement.
src/flamenco/runtime/fd_system_ids_pp.h Defines the ptoken program buffer ID bytes.
src/flamenco/runtime/fd_system_ids.h Declares the new fd_solana_ptoken_program_buffer_address.
src/flamenco/runtime/fd_system_ids.c Defines the new fd_solana_ptoken_program_buffer_address.
src/flamenco/runtime/fd_runtime_stack.h Adds staging memory used for ELF validation during epoch-boundary migrations.
src/flamenco/runtime/fd_runtime.c Triggers the new upgrade when the feature activates.
src/flamenco/runtime/fd_core_bpf_migration.c Implements Loader v2 → Loader v3 upgrade flow and adds deploy-style ELF validation checks.
src/flamenco/features/feature_map.json Adds the new feature pubkey entry.
src/flamenco/features/fd_features_generated.h Updates feature count/set ID and adds the new feature field.
src/flamenco/features/fd_features_generated.c Adds the new feature ID entry and updates perfect-hash/static asserts.
src/ballet/sbpf/fd_sbpf_loader.h Updates FD_SBPF_PROGRAM_FOOTPRINT sizing for stack/arena-backed program objects.

Comment thread src/flamenco/runtime/fd_core_bpf_migration.c Outdated
Comment thread src/flamenco/runtime/fd_core_bpf_migration.c Outdated
Comment thread src/flamenco/runtime/fd_core_bpf_migration.c Outdated
Comment thread src/flamenco/runtime/fd_core_bpf_migration.c Outdated
Comment thread src/flamenco/runtime/fd_core_bpf_migration.c
Comment thread src/ballet/sbpf/fd_sbpf_loader.h
@topointon-jump topointon-jump force-pushed the firedancer-replace-spl-token-with-p-token branch from ba9d87c to 3a9a446 Compare March 20, 2026 16:36
Copilot AI review requested due to automatic review settings March 20, 2026 16:38
@topointon-jump topointon-jump force-pushed the firedancer-replace-spl-token-with-p-token branch from 3a9a446 to ba8a578 Compare March 20, 2026 16:38
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated 3 comments.

Comment thread src/flamenco/runtime/fd_runtime_stack.h
Comment thread src/flamenco/runtime/fd_core_bpf_migration.c
Comment thread src/flamenco/runtime/fd_core_bpf_migration.c
@github-actions
Copy link
Copy Markdown

Performance Measurements ⏳

Suite Baseline New Change
backtest mainnet-406545575-perf per slot 0.135113 s 0.136771 s 1.227%
backtest mainnet-406545575-perf snapshot load 5.177 s 3.917 s -24.338%
backtest mainnet-406545575-perf total elapsed 135.112838 s 136.770875 s 1.227%
firedancer mem usage with mainnet.toml 1105.43 GiB 1105.43 GiB 0.000%

@github-actions
Copy link
Copy Markdown

Performance Measurements ⏳

Suite Baseline New Change
backtest mainnet-406545575-perf per slot 0.112514 s 0.112713 s 0.177%
backtest mainnet-406545575-perf snapshot load 4.075 s 2.488 s -38.945%
backtest mainnet-406545575-perf total elapsed 112.513657 s 112.7131 s 0.177%
firedancer mem usage with mainnet.toml 1105.43 GiB 1105.43 GiB 0.000%

Copilot AI review requested due to automatic review settings March 20, 2026 18:50
@topointon-jump topointon-jump force-pushed the firedancer-replace-spl-token-with-p-token branch from 491b1d2 to 42d1576 Compare March 20, 2026 18:50
@topointon-jump topointon-jump force-pushed the firedancer-replace-spl-token-with-p-token branch from 42d1576 to ba41dea Compare March 20, 2026 18:52
@topointon-jump topointon-jump force-pushed the firedancer-replace-spl-token-with-p-token branch from ba41dea to 1bd5cbb Compare March 20, 2026 18:53
@topointon-jump topointon-jump changed the title [WIP] flamenco, runtime: implement replace_spl_token_with_p_token flamenco, runtime: implement replace_spl_token_with_p_token Mar 20, 2026
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated 2 comments.

Comment thread src/flamenco/runtime/fd_core_bpf_migration.c
Comment thread src/flamenco/runtime/fd_core_bpf_migration.c
@github-actions
Copy link
Copy Markdown

Performance Measurements ⏳

Suite Baseline New Change
backtest mainnet-406545575-perf per slot 0.142875 s 0.142545 s -0.231%
backtest mainnet-406545575-perf snapshot load 7.206 s 4.209 s -41.590%
backtest mainnet-406545575-perf total elapsed 142.874987 s 142.545373 s -0.231%
firedancer mem usage with mainnet.toml 1105.43 GiB 1105.43 GiB 0.000%

#define FD_SBPF_TEXT_CNT_MAX (FD_RUNTIME_ACC_SZ_MAX / 8UL)
#define FD_SBPF_CALLDESTS_PRIVATE_WORD_CNT ( (FD_SBPF_TEXT_CNT_MAX +63UL)>>6 )
#define FD_SBPF_PROGRAM_FOOTPRINT (sizeof(fd_sbpf_calldests_private_t)-sizeof(ulong) + sizeof(ulong)*FD_SBPF_CALLDESTS_PRIVATE_WORD_CNT )
#define FD_SBPF_PROGRAM_FOOTPRINT (sizeof(fd_sbpf_program_t) + sizeof(fd_sbpf_calldests_private_t)-sizeof(ulong) + sizeof(ulong)*FD_SBPF_CALLDESTS_PRIVATE_WORD_CNT )
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

src/flamenco/runtime/tests/run_ledger_backtest.sh -l create_account_allow_prefund -y 1 -m 1000 -e 520
src/flamenco/runtime/tests/run_ledger_backtest.sh -l relax_programdata_account_check_migration -y 1 -m 1000 -e 260
src/flamenco/runtime/tests/run_ledger_backtest.sh -l mainnet-391824000-boundary -y 2 -m 2000000 -e 391824016
src/flamenco/runtime/tests/run_ledger_backtest.sh -l replace_spl_token_with_p_token -y 1 -m 1000 -e 720
Copy link
Copy Markdown
Contributor Author

@topointon-jump topointon-jump Mar 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This ledger:

  • Sets up the ptok6rngomXrDbWf5v5Mkmu5CEbB51hzSCPDoj9DrvF buffer account with the new token program executable, with the correct hash 435df1e70f1ca6258eb01a4507701688c7f1a22656e3e4ef361db473bf478075
  • Makes TokenkegQfeZyiNwAJbNbGKPFXCWuBvf9Ss623VQ5DA token transfers successfully pre-migration
  • Goes through the epoch boundary, exercising the feature and upgrading the program
  • Continues making token transfers
  • These fail for 1 slot due to the DelayVisibility checks
  • Then these succeed as normal

@topointon-jump topointon-jump merged commit 624f939 into main Mar 20, 2026
17 checks passed
@topointon-jump topointon-jump deleted the firedancer-replace-spl-token-with-p-token branch March 20, 2026 19:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants