Adopt [email protected]

[Keith-CY]

The method to parse the miner message changes since [email protected], the first 4 bytes will be the signal of softfork activation state.

            if let Ok(reader) = CellbaseWitnessReader::from_slice(&witness.raw_data()) {
                    let message = reader.message().to_entity();
                    if message.len() >= 4 {
                        if let Ok(raw) = message.raw_data()[..4].try_into() {
                            let version = u32::from_le_bytes(raw);
                            return ((version & VERSIONBITS_TOP_MASK) == VERSIONBITS_TOP_BITS)
                                && (version & self.mask()) != 0;
                        }
                    }
                }

Or we can fetch softfork state by PRC get_deployments_info which returns the version by a bit field. If bit is 1, the first 4 bytes of miner message should be treated as a signal.

Ref:
Code of versionbits: https://github.com/nervosnetwork/ckb/blob/de391ff6849f12bb01e05af9e634f4a8a6295bac/spec/src/versionbits/mod.rs#L310-L319
PR of versionbits: nervosnetwork/ckb#3515

[ShiningRay]

Oh, this is really tricky

[iamyates]

in ruby code, the core function which parse the witness shown as below: ( https://github.com/nervosnetwork/ckb-explorer/blob/3fed0a69d22ab74bb22b23b71458f21d8bd0c480/app/utils/ckb_utils.rb#L38-L66 )

  def self.parse_cellbase_witness(cellbase)
    cellbase_witness = cellbase.witnesses.first.delete_prefix("0x")
    cellbase_witness_serialization = [cellbase_witness].pack("H*")
    // this line indicates that the first 4 bits are not parsed
    script_offset = [cellbase_witness_serialization[4..7].unpack1("H*")].pack("H*").unpack1("V")
    message_offset = [cellbase_witness_serialization[8..11].unpack1("H*")].pack("H*").unpack1("V")
    script_serialization = cellbase_witness_serialization[script_offset...message_offset]
    code_hash_offset = [script_serialization[4..7].unpack1("H*")].pack("H*").unpack1("V")
    hash_type_offset = [script_serialization[8..11].unpack1("H*")].pack("H*").unpack1("V")
    args_offset = [script_serialization[12..15].unpack1("H*")].pack("H*").unpack1("V")
    message_bytes = cellbase_witness_serialization[message_offset..-1]
    message_serialization = message_bytes[4..-1]
    if message_serialization.size > 28
      message = "#{message_serialization[0..28]}#{message_serialization[29..-1]&.unpack1("H*")}".to_json.unpack1("H*")
    else
      message = message_serialization.unpack1("H*")
    end
    code_hash_serialization = script_serialization[code_hash_offset...hash_type_offset]
    hash_type_serialization = script_serialization[hash_type_offset...args_offset]
    args_serialization = script_serialization[hash_type_offset + 1..-1]
    args_serialization = args_serialization[4..-1]

    code_hash = "0x#{code_hash_serialization.unpack1('H*')}"
    hash_type_hex = "0x#{hash_type_serialization.unpack1('H*')}"
    args = "0x#{args_serialization.unpack1('H*')}"

    hash_type = hash_type_hex == "0x00" ? "data" : "type"
    lock = CKB::Types::Script.new(code_hash: code_hash, args: args, hash_type: hash_type)
    OpenStruct.new(lock: lock, message: "0x#{message}")
  end

the line script_offset = [cellbase_witness_serialization[4..7].unpack1("H*")].pack("H*").unpack1("V") seems indicates that the first 4 bits are not parsed.

However this function works under ckb node v0.104, but not work under v0.105, ( it can not parse the correct miner message) .

so I want to know where can I find the document showing which position of byte/bits belongs to which. e.g. code hash, lock, message. such as: https://github.com/nervosnetwork/rfcs/blob/master/rfcs/0017-tx-valid-since/0017-tx-valid-since.md

[Keith-CY]

@shaojunda who wrote this logic says there's no such a document of how to parse the miner message from the witness, he interpreted the code from ckb source code.

But the code snippet posted above recalls me that the data should be serialized by https://github.com/nervosnetwork/molecule

@quake could you do me a favor to add a link to the schema?

[Keith-CY]

@shaojunda who wrote this logic says there's no such a document of how to parse the miner message from the witness, he interpreted the code from ckb source code.

But the code snippet posted above recalls me that the data should be serialized by nervosnetwork/molecule

@quake could you do me a favor to add a link to the schema?

Got the code here https://github.com/nervosnetwork/ckb/blob/fafa37d979bd83f867358efb7aa176a0ec295bbd/util/types/src/generated/blockchain.rs#L8514-L8548 @iamyates

[Keith-CY]

I've found something new about the miner message and will update in this issue. The abnormal decoding could be a problem of frontend and testnet miner

[Keith-CY]

block number	raw message	displayed utf 8	expected utf 8	note
[7277508](https://pudge.explorer.nervos.org/block/7277508)	`0x0200000020302e3130352e312028202920deadbeef`	Fail to decode in front-end	`\x02\x00\x00\x00 0.105.1 ( ) ޭ��`	[hexToUtf8](https://github.com/nervosnetwork/ckb-explorer-frontend/blob/develop/src/utils/string.ts#L37-L45) method in front-end is incorrect so it fails to decode the message
[7000000](https://pudge.explorer.nervos.org/block/0xf9e1c6398f524c10b358dca7e000f59992004fda68c801453ed4da06bc3c6ecc)	`0x302e3130342e312028202920deadbeef`	Fail to decode in front-end	`0.104.1 ( ) ޭ��`	[hexToUtf8](https://github.com/nervosnetwork/ckb-explorer-frontend/blob/develop/src/utils/string.ts#L37-L45) method in front-end is incorrect so it fails to decode the message
[5277405](https://pudge.explorer.nervos.org/block/0xaed337b8313a1ba87cdb1dccb7347cd2036a0112121ab722fcca3a86ef705ab2)	`0x22302e3130322e302d70726520286632333539386620323032322d30342d3330333732393230646561646265656622`	`"0.102.0-pre (f23598f 2022-04-30372920deadbeef"`	`"0.102.0-pre (f23598f 2022-04-30372920deadbeef"`

The message is failed to be decoded for 2 reasons

1. deadbeef, which is a word in utf8 encoding, was decoded from the miner message before, but now it's appended in the raw message
2. hexToUtf8 method in the front-end is incorrect and an exception is thrown when deadbeef is in the string

So we should fix the decoding of message by 2 updates

1. fix the miner message encoding of testnet miner
2. fix the hex to utf8 method in frontend

[Keith-CY]

block number raw message displayed utf 8 expected utf 8 note
7277508 0x0200000020302e3130352e312028202920deadbeef Fail to decode in front-end \x02\x00\x00\x00 0.105.1 ( ) ޭ�� hexToUtf8 method in front-end is incorrect so it fails to decode the message
7000000 0x302e3130342e312028202920deadbeef Fail to decode in front-end 0.104.1 ( ) ޭ�� hexToUtf8 method in front-end is incorrect so it fails to decode the message
5277405 0x22302e3130322e302d70726520286632333539386620323032322d30342d3330333732393230646561646265656622 "0.102.0-pre (f23598f 2022-04-30372920deadbeef" "0.102.0-pre (f23598f 2022-04-30372920deadbeef"
The message is failed to be decoded for 2 reasons

1. deadbeef, which is a word in utf8 encoding, was decoded from the miner message before, but now it's appended in the raw message
2. hexToUtf8 method in the front-end is incorrect and an exception is thrown when deadbeef is in the string

So we should fix the decoding of message by 2 updates

1. fix the miner message encoding of testnet miner
2. fix the hex to utf8 method in frontend

Fixde by PR nervosnetwork/ckb-explorer-frontend#1105 which fixes the hexToUtf8 method.

Now 0x0200000020302e3130352e312028202920deadbeef will be decoded into \x02\x00\x00\x00 0.105.1 ( ) ޭ�� and the first 4 bytes will be rendered as invisible characters on the page.

I've confirmed the first 4 bytes are meaningless and can be omitted so rendering them as invisible characters is fine.