Skip to content

chore(conformance): agave 4.0 upgrade#1314

Merged
dnut merged 35 commits intomainfrom
dnut/conformance/agave-4.0
Mar 30, 2026
Merged

chore(conformance): agave 4.0 upgrade#1314
dnut merged 35 commits intomainfrom
dnut/conformance/agave-4.0

Conversation

@dnut
Copy link
Copy Markdown
Collaborator

@dnut dnut commented Mar 13, 2026

This upgrades Sig's conformance harnesses, CI pipeline, and feature set to be compatible with Agave 4.0.

The diff is large, but the changes to existing sig code are small.

  • ~500 lines are the list of fixtures in failing.txt
  • ~3000 lines are a python package called parseout that I added to parse test outputs. See the readme for more information.

Current Test Results

All of the remaining failures are related to crypto programs. #1296 fixes the 80 zksdk instruction harness tests. The 102 txn harness failures are all related to the zk el gamal proof and secp256r1 programs, and we do not have fixes for those yet.

elf_loader/fixtures                               │ Pass  953 │ Fail    0 │ Skip    0
instr/fixtures/bpf-address-lookup-table           │ Pass    2 │ Fail    0 │ Skip    0
instr/fixtures/bpf-config                         │ Pass    1 │ Fail    0 │ Skip    0
instr/fixtures/bpf-loader                         │ Pass   10 │ Fail    0 │ Skip    0
instr/fixtures/bpf-loader-upgradeable-v1-programs │ Pass  371 │ Fail    0 │ Skip    0
instr/fixtures/bpf-loader-v1-programs             │ Pass    7 │ Fail    0 │ Skip    0
instr/fixtures/bpf-loader-v2                      │ Pass   18 │ Fail    0 │ Skip    0
instr/fixtures/bpf-loader-v2-programs             │ Pass    7 │ Fail    0 │ Skip    0
instr/fixtures/bpf-loader-v3                      │ Pass  349 │ Fail    0 │ Skip    0
instr/fixtures/bpf-loader-v3-programs             │ Pass  510 │ Fail    0 │ Skip    0
instr/fixtures/bpf-loader-v4                      │ Pass    7 │ Fail    0 │ Skip    0
instr/fixtures/compute-budget                     │ Pass  127 │ Fail    0 │ Skip    0
instr/fixtures/stake                              │ Pass 6933 │ Fail    0 │ Skip    0
instr/fixtures/system                             │ Pass 5324 │ Fail    0 │ Skip    0
instr/fixtures/unknown                            │ Pass   28 │ Fail    0 │ Skip    0
instr/fixtures/vote                               │ Pass 5987 │ Fail    0 │ Skip    0
instr/fixtures/zk_sdk                             │ Pass  168 │ Fail   80 │ Skip    0
syscall/fixtures/abort                            │ Pass    3 │ Fail    0 │ Skip    0
syscall/fixtures/alt_bn128                        │ Pass 3228 │ Fail    0 │ Skip    0
syscall/fixtures/blake3                           │ Pass   16 │ Fail    0 │ Skip    0
syscall/fixtures/cpi                              │ Pass  780 │ Fail    0 │ Skip    0
syscall/fixtures/create_program_address           │ Pass    6 │ Fail    0 │ Skip    0
syscall/fixtures/curve25519                       │ Pass  106 │ Fail    0 │ Skip    0
syscall/fixtures/get_epoch_schedule               │ Pass    2 │ Fail    0 │ Skip    0
syscall/fixtures/get_return_data                  │ Pass    2 │ Fail    0 │ Skip    0
syscall/fixtures/keccak256                        │ Pass   16 │ Fail    0 │ Skip    0
syscall/fixtures/log                              │ Pass   31 │ Fail    0 │ Skip    0
syscall/fixtures/log_data                         │ Pass   13 │ Fail    0 │ Skip    0
syscall/fixtures/memcmp                           │ Pass    3 │ Fail    0 │ Skip    0
syscall/fixtures/memcpy                           │ Pass   74 │ Fail    0 │ Skip    0
syscall/fixtures/memmove                          │ Pass   54 │ Fail    0 │ Skip    0
syscall/fixtures/memset                           │ Pass   88 │ Fail    0 │ Skip    0
syscall/fixtures/panic                            │ Pass   11 │ Fail    0 │ Skip    0
syscall/fixtures/poseidon                         │ Pass   42 │ Fail    0 │ Skip    0
syscall/fixtures/secp256k1                        │ Pass   33 │ Fail    0 │ Skip    0
syscall/fixtures/sha256                           │ Pass   16 │ Fail    0 │ Skip    0
syscall/fixtures/sol_get_sysvar                   │ Pass  236 │ Fail    0 │ Skip    0
syscall/fixtures/stack_height                     │ Pass    1 │ Fail    0 │ Skip    0
syscall/fixtures/try_find_program_address         │ Pass   15 │ Fail    0 │ Skip    0
syscall/fixtures/vm                               │ Pass    3 │ Fail    0 │ Skip    0
txn/fixtures                                      │ Pass 5142 │ Fail  102 │ Skip    0

Summary:
        Passed:  30723
        Failed:  182
        Skipped: 0

Overview

  1. Upgrade conformance dependencies: Latest versions are now specified in commits.env used for Agave 4.0 conformance.
  2. Wire format updates: Regenerated protobuf bindings (v1.pb.zig) and hand-wrote FlatBuffer serialization (elf_types.zig) to match Agave 4.0's protosol definitions.
  3. Harness adaptation: Updated txn_execute.zig, elf_loader.zig, utils.zig, and vm_interp.zig to consume/produce the new formats.
  4. Use text-vectors directly, since it actually contains the necessary expectations from agave, instead of downloading generated fixtures from our repository.
  5. Exclude failing tests from failures.txt and stop getting a list of fixtures from fixtures.txt because ci-run.sh can find all the fixtures in test-vectors.
  6. Feature set: Added new features to the feature set, but didn't implement the features yet. This fixed many tests in the instruction harness that panicked due to unknown features.

Caveats

  • Slot is hardcoded (10 for txn, 0 for instr/VM) rather than read from the fixture. This was previously the fallback behavior only when the slot is unspecified. Now the slot is never specified, so it's the only behavior.
  • The ELF loader types are now hand-maintained FlatBuffers -- upstream format changes require manual updates rather than protobuf regeneration.
  • The 2,303 entries in failing.txt represent existing conformance gaps that predate this PR. Fixing these is not in scope.

@github-project-automation github-project-automation Bot moved this to 🏗 In progress in Sig Mar 13, 2026
@codecov
Copy link
Copy Markdown

codecov Bot commented Mar 13, 2026

Codecov Report

❌ Patch coverage is 64.28571% with 5 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
src/replay/resolve_lookup.zig 33.33% 2 Missing ⚠️
src/runtime/program/address_lookup_table/state.zig 0.00% 1 Missing ⚠️
src/runtime/program/vote/execute.zig 50.00% 1 Missing ⚠️
src/runtime/transaction_execution.zig 80.00% 1 Missing ⚠️

❌ Your patch status has failed because the patch coverage (64.28%) is below the target coverage (85.00%). You can increase the patch coverage or adjust the target coverage.

Files with missing lines Coverage Δ
src/runtime/executor.zig 98.93% <100.00%> (+<0.01%) ⬆️
src/runtime/program/address_lookup_table/state.zig 93.33% <0.00%> (ø)
src/runtime/program/vote/execute.zig 95.83% <50.00%> (-0.07%) ⬇️
src/runtime/transaction_execution.zig 96.66% <80.00%> (ø)
src/replay/resolve_lookup.zig 85.90% <33.33%> (+0.37%) ⬆️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@dnut dnut marked this pull request as ready for review March 16, 2026 16:26
@dnut dnut requested review from ultd and yewman as code owners March 16, 2026 16:26
@dnut dnut force-pushed the dnut/conformance/agave-4.0 branch from b8471ed to b5b4692 Compare March 17, 2026 13:29
@dnut dnut requested review from InKryption March 17, 2026 13:43
@InKryption InKryption force-pushed the dnut/conformance/agave-4.0 branch from aba66b7 to b1883d8 Compare March 19, 2026 02:42
InKryption
InKryption previously approved these changes Mar 19, 2026
dnut and others added 16 commits March 20, 2026 16:31
3 rekeyed features:
- account_data_direct_mapping -> CR3dVN2Yoo95Y96kLSTaziWDAQT2MNEpiWh5cqVq2pNE
- enable_sbpf_v3_deployment_and_execution -> 5cC3foj77CWun58pC51ebHFUWavHWKarWyR5UUik7dnC
- reenable_zk_elgamal_proof_program -> zkexuyPRdyTVbZqEAREueqL2xvvoBhRgth9xGSc1tMN

19 new features:
- syscall_parameter_address_restrictions
- virtual_address_space_adjustments
- delay_commission_updates
- create_account_allow_prefund
- alt_bn128_little_endian
- bls_pubkey_management_in_vote_account
- enable_alt_bn128_g2_syscalls
- commission_rate_in_basis_points
- custom_commission_collector
- set_lamports_per_byte_to_6333
- set_lamports_per_byte_to_5080
- set_lamports_per_byte_to_2575
- set_lamports_per_byte_to_1322
- set_lamports_per_byte_to_696
- limit_instruction_accounts
- block_revenue_sharing
- vote_account_initialize_v2
- validate_chained_block_id
- validator_admission_ticket
- alpenglow
previously there were a bunch of scripts to manage these dependencies
and it was getting difficult to maintain them and make them generally
usable across different systems. nix flakes are intended for exactly
this use case
with previous versions of solana conformance you couldn't use later than python 3.11 without getting errors. now we only see errors on 3.14. 3.13 is working
the test will fail if the outputs are incorrect. if we don't know the feature but our outputs are correct, then it doesn't matter for that test that we didn't know the feature. our code still passed the validations of that test, so it should pass the test.

i converted the panics into prints so at least we can see when there are unknown features
fixes missing feature id 0xfc12b1cef363afa7 in conformance
yewman
yewman previously approved these changes Mar 23, 2026
Comment thread conformance/src/elf_loader.zig Outdated
Comment thread conformance/src/elf_loader.zig Outdated
Comment thread conformance/src/elf_loader.zig Outdated
Comment thread conformance/src/elf_loader.zig Outdated
Comment thread conformance/src/elf_types.zig
yewman
yewman previously approved these changes Mar 25, 2026
Fixes 894 transaction fixtures, breaks 10 others.
Comment thread conformance/src/txn_execute.zig
InKryption and others added 10 commits March 26, 2026 23:27
There was a work-around for this already, but apparently it was
insufficient.
Also moves the assertion below the index into the `dedupe_map`,
so that if this was to go OOB again somehow, the panic includes
the actual index and the length it went over.
Previous fix did fix behaviour in the tested build modes, but could
still potentially result in a similar or the same kind of bug given
different optimization heuristics.
This solves the problem definitively by avoiding potential invalidation
of the `prev` pointer entirely.
yewman
yewman previously approved these changes Mar 30, 2026
@github-project-automation github-project-automation Bot moved this from 🏗 In progress to 👀 In review in Sig Mar 30, 2026
@dnut dnut added this pull request to the merge queue Mar 30, 2026
Merged via the queue into main with commit 2121306 Mar 30, 2026
12 of 13 checks passed
@dnut dnut deleted the dnut/conformance/agave-4.0 branch March 30, 2026 17:21
@github-project-automation github-project-automation Bot moved this from 👀 In review to ✅ Done in Sig Mar 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

3 participants