Skip to content

Conversation

@jsdw
Copy link
Collaborator

@jsdw jsdw commented Nov 19, 2025

There are two main ways to decode some data that exist in Rust-land right now:

  1. via traits like scale_decode::DecodeAsType and scale_decode::DecodeAsFields
  2. viascale_decode::visitor::decode_with_visitor(..) and scale_decode::visitor::Visitor.

The first is the "higher level" approach, where in most APIs you simple give some type you want the bytes to be decoded into and get back that type if decoding works.

The second is "lower level" and is what the first is built upon. This isn't typically exposed, but in some cases can be very useful. One such case is addressing things like https://github.com/paritytech/polkadot-rest-api/pull/36#discussion_r2538499050, where we want to know the names of types for instance.

This PR adds a visit method to extrinsic fields in subxt_historic which allows them to be visitored with said Visitor type, and adds a usage of this in the extrinsics example so that you can see roughly how it's used.

This is somewhat of a workaround to the fact that we have to handle two different type ID / type resolver types in this version of subxt_historic. In future versions, either:

  1. the visit() method will be much simpler to implement internally, or
  2. it will not exist, but you'll be able to easily fetch everything you need to call scale_decode::visitor::decode_with_visitor(..) yourself anyway, which is all that is done in the visit method anyway.
  3. for cases where you just want type information and not to actually decode the type, you'll just be able to fetc hthe u32 type ID, and look up the type in the scale_info::PortableRegistry to find the information you need, without any visitor related stuff.

See examples/extrinsics.rs for an example of what we can do with a visitor.

Note: I also updated the subxt codegen CLI command to accept legacy metadatas; this is helpful to test what sort of types are generated (aside from possibly being useful in the future)

\cc @TarikGul

@jsdw jsdw requested a review from a team as a code owner November 19, 2025 16:54
@jsdw jsdw changed the title Allow visiting extrinsic fields [v0.50.0] Allow visiting extrinsic fields in subxt_historic Nov 19, 2025
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I updated the codegen to work with historic metadatas so that it's possible to see what sorts of types we get back for things in actual generated code.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a couple of tweaks to support the CLI command update; not really relevant for the historic example!

@TarikGul
Copy link
Member

@jsdw This is a huge win, thank you. So from what I understand this adds the visit() method from scale_decode onto the extrinsic fields.

So I think for us it would look something like:

  fn convert_extrinsic_fields_to_ss58(
      extrinsic: &Extrinsic,
      ss58_prefix: u16
  ) -> serde_json::Value {
      let mut converted_args = Map::new();

      for field in extrinsic.call().fields().iter() {
          let type_name = field
              .visit(GetTypeName::new())
              .ok()
              .flatten();

          let value = field.decode_as::<scale_value::Value>()?;

          let converted_value = if type_name == Some("AccountId32") {
              convert_account_id_to_ss58(&value, ss58_prefix)
          } else {
              // Recursively handle nested types (Is this correct?)
              convert_bytes_to_hex(value, ss58_prefix, Some(type_name))
          };

          converted_args.insert(field.name().to_string(), converted_value);
      }

      Value::Object(converted_args)
  }

@TarikGul
Copy link
Member

@jsdw Do you know if it will be possible for us to extend this to events as well?

impl Clone for AnyTypeId {
fn clone(&self) -> Self {
match self {
Self::A(arg0) => Self::A(*arg0),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At first I was confused by this by I guess the idea is u32 is a Copy, and LookupName requires the clone()?

Copy link
Collaborator Author

@jsdw jsdw Nov 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exactly! LookupName tries to avoid allocating in "normal" cases but alas requires cloning still since it can't guarantee to avoid them.

Copy link
Member

@TarikGul TarikGul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Super solid! 🚀

@jsdw
Copy link
Collaborator Author

jsdw commented Nov 20, 2025

I've added a visit method onto storage values, too, which I hope will be what you need w.r.t events :) The only thing it's not on now is storage keys, which I can also add it to if need be so just lemme know!

@jsdw jsdw merged commit 3bbba1b into v0.50.x Nov 21, 2025
23 of 27 checks passed
@jsdw jsdw deleted the jsdw-v0.50-visit-extrinsic-fields branch November 21, 2025 15:32
jsdw added a commit that referenced this pull request Nov 22, 2025
* v0.50.0: Integrate frame-decode, redo storage APIs and break up Error. (#2100)

* WIP integrating new frame-decode and working out new storage APIS

* WIP: first pass adding new storage things to subxt-core

* Second pass over Address type and start impl in Subxt

* WIP new storage APIs

* WIP New storage APIs roughly completed, lots of errors still

* Remove PlainorMap enum; plain and map values now use same struct to simplify usage

* Begin 'fixing' errors

* WIP splitting errors and tidying payload/address traits

* Get subxt-core compiling

* Small fixes in subxt-core and remove metadata mod

* subxt-core: cargo check --all-targets passes

* Fix test

* WIP starting to update subxt from subxt-core changes

* WIP splitting up subxt errors into smaller variants

* WIP errors: add DispatchError errors

* Port new Storage APIs to subxt-core

* cargo check -p subxt passes

* Quick-fix errors in subxt-cli (explore subcommand)

* fmt

* Finish fixing codegen up and start fixing examples

* get Subxt examples compiling and bytes_at for constants

* Add some arcs to limit lifetimes in subxt/subxt-core storage APIs

* A little Arcing to allow more method chaining in Storage APIs, aligning with Subxt

* Update codegen test

* cargo check --all-targets passing

* cargo check --features 'unstable-light-client' passing

* clippy

* Remove unused dep in subxt

* use published frame-decode

* fix wasm-example

* Add new tx extension to fix daily tests

* Remove unused subxt_core::dynamic::DecodedValue type

* Update book to match changes

* Update docs to fix more broken bits

* Add missing docs

* fmt

* allow larger result errs for now

* Add missing alloc imports in subxt-core

* Fix doc tests and fix bug getting constant info

* Fix V14 -> Metadata transform for storage & constants

* Fix parachain example

* Fix FFI example

* BlockLength decodes t ostruct, not u128

* use fetch/iter shorthands rather than entry in most storage tests

* Fix some integration tests

* Fix Runtime codegen tests

* Expose the dynamic custom_value selecter and use in a UI test

* Update codegen metadata

* Tidy CLI storage query and support (str,str) as a storage address

* Add (str,str) as valid constant address too

* Show string tuple in constants example

* Via the magic of traits, avoid needing any clones of queries/addresses and accept references to them

* clippy

* [v0.50] update scale-info-legacy and frame-decode to latest (#2119)

* bump scale-info-legacy and frame-decode to latest

* Remove something we don't need in this PR

* Fully remove unused for now dep

* [v0.50] Convert historic metadata to subxt::Metadata (#2120)

* First pass converting historic metadatas to our subxt::Metadata type

* use published frame-decode

* fmt and rename legacy metadata macro

* Enable legacy feature where needed in subxt_metadata so it compiles on its own

* Use cargo hack more in CI and fix subxt-metadata features

* Add tests for metadata conversion (need to optimise; some too expensive right now

* Address performance and equality issues in metadata conversion testing

* fmt

* fmt all

* clippy

* Fix a doc link

* Test codegen and fixes to make it work

* Remove local frame-decode patch

* bump frame-decode to latest

* [v0.50.0] Allow visiting extrinsic fields in subxt_historic (#2124)

* Allow visiting extrinsic fields

* fmt

* Don't use local scale-decode dep

* Clippy and tidy

* Extend 'subxt codegen' CLI to work with legacy metadatas

* Simplify historic extrinsics example now that AccountId32s have paths/names

* clippy

* clippy

* clippy..

* Allow visiting storage values, too, and clean up extrinsic visiting a little by narrowing lifetime

* Try to fix flaky test

* Add custom value decode to extrinsics example

* Remove useless else branch ra thought I needed

* Simplify examples

* Prep to release v0.0.5 (#2126)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants