Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle bitcoin core 220 rpc output change #57

Merged
merged 2 commits into from
Dec 16, 2021

Conversation

veronicavero
Copy link
Contributor

it works for bitcoin core 22.0. It does not support backward compatibility for bitcoin core prior to core 22.0.
We can either use bitcoin version to support backward compatibility.

read "address" instead of "addresses" as core 22.0 changes the rpc output data structure.
keep 'multisig' type and make the address as nonstandard address
@veronicavero
Copy link
Contributor Author

@medvedev1088 can you help review? We verified the code change on bitcoin core 22.0. It works as expected.

@medvedev1088
Copy link
Member

Thank you,

It would be ideal if this PR included unit tests.

It seems like it should be possible to make it backward compatible by checking the addresses field first, and if it's empty, checking the address field. Is my understanding correct?

@veronicavero
Copy link
Contributor Author

@medvedev1088 I will take a look on the backward compatibility and update the PR.
For the change in transaction_output_mapper.py, we can check the addresses field.
I need to double confirm the change in btc_service.py, ideally we do not need the change in this file if addresses is present. I will take a further look.

@shivansh1905
Copy link

@veronicavero will this merge help to resolve the 'nonstandard' address type issue?

Let me know if we (team of crypto analysts and developers) can assist in any way to escalate this merge.

@prassee
Copy link

prassee commented Dec 16, 2021

We tested with aforementioned changes and its working. Below attached the JSON response for a enrich_transactions RPC call before and after the change.
Feel free to point out any further observations.
@aknirmal90 @saurabhdaga-merkle
input txn hash : 46cf0c70ad79af2c2b74baa83ea1cce67a6136e658fcd6f58579455b5f9496b1

Before
observe empty arrays values for inputs.addresses

{
    "transaction_id": "46cf0c70ad79af2c2b74baa83ea1cce67a6136e658fcd6f58579455b5f9496b1",
    "hash": "af0da28fdca03afa4d7792b2c473ce00a4e114e2eb785d1787551f75ead8c2ae",
    "block_number": 714384,
    "block_hash": "0000000000000000000a696a7011962f8a79c32026df0ef499fc6981fb77717c",
    "block_timestamp": 1639651902,
    "is_coinbase": false,
    "lock_time": 0,
    "size": 901,
    "virtual_size": 497,
    "weight": 1987,
    "version": 1,
    "index": 28,
    "input_count": 5,
    "output_count": 1,
    "input_value": 0,
    "output_value": 12309829,
    "inputs": [
        {
            "index": 0,
            "create_transaction_id": "1c051fc7f7f7303da003e8da1775676f76d840807e506a2e67661f831351947b",
            "spending_transaction_id": "46cf0c70ad79af2c2b74baa83ea1cce67a6136e658fcd6f58579455b5f9496b1",
            "create_output_index": 25,
            "sequence": 4294967295,
            "script_asm": "",
            "script_hex": "",
            "required_signatures": null,
            "addresses": [],
            "value": null,
            "type": null
        },
        {
            "index": 1,
            "create_transaction_id": "97a61972f058498396c715051ff8d01f3cef3ecd81b3ca0fce970e8f28a4799a",
            "spending_transaction_id": "46cf0c70ad79af2c2b74baa83ea1cce67a6136e658fcd6f58579455b5f9496b1",
            "create_output_index": 6,
            "sequence": 4294967295,
            "script_asm": "",
            "script_hex": "",
            "required_signatures": null,
            "addresses": [],
            "value": null,
            "type": null
        },
        {
            "index": 2,
            "create_transaction_id": "bfd20cc8be8f586149db3cb7ee4ac1702a44c4224905a6540c747341fa0771f0",
            "spending_transaction_id": "46cf0c70ad79af2c2b74baa83ea1cce67a6136e658fcd6f58579455b5f9496b1",
            "create_output_index": 51,
            "sequence": 4294967295,
            "script_asm": "",
            "script_hex": "",
            "required_signatures": null,
            "addresses": [],
            "value": null,
            "type": null
        },
        {
            "index": 3,
            "create_transaction_id": "d7e8916e6d73f65bef1937c3dfda2f47022db487c239e0cab9b6354c820886ed",
            "spending_transaction_id": "46cf0c70ad79af2c2b74baa83ea1cce67a6136e658fcd6f58579455b5f9496b1",
            "create_output_index": 213,
            "sequence": 4294967295,
            "script_asm": "",
            "script_hex": "",
            "required_signatures": null,
            "addresses": [],
            "value": null,
            "type": null
        },
        {
            "index": 4,
            "create_transaction_id": "fcb4591669cc7842c266b58371672d663fb2a9e598916b7c665af887f35a5c71",
            "spending_transaction_id": "46cf0c70ad79af2c2b74baa83ea1cce67a6136e658fcd6f58579455b5f9496b1",
            "create_output_index": 1,
            "sequence": 4294967295,
            "script_asm": "",
            "script_hex": "",
            "required_signatures": null,
            "addresses": [],
            "value": null,
            "type": null
        }
    ],
  ...
}

After

{
    "transaction_id": "46cf0c70ad79af2c2b74baa83ea1cce67a6136e658fcd6f58579455b5f9496b1",
    "hash": "af0da28fdca03afa4d7792b2c473ce00a4e114e2eb785d1787551f75ead8c2ae",
    "block_number": 714384,
    "block_hash": "0000000000000000000a696a7011962f8a79c32026df0ef499fc6981fb77717c",
    "block_timestamp": 1639651902,
    "is_coinbase": false,
    "lock_time": 0,
    "size": 901,
    "virtual_size": 497,
    "weight": 1987,
    "version": 1,
    "index": 28,
    "input_count": 5,
    "output_count": 1,
    "input_value": 12410429,
    "output_value": 12309829,
    "inputs": [
        {
            "index": 0,
            "create_transaction_id": "1c051fc7f7f7303da003e8da1775676f76d840807e506a2e67661f831351947b",
            "spending_transaction_id": "46cf0c70ad79af2c2b74baa83ea1cce67a6136e658fcd6f58579455b5f9496b1",
            "create_output_index": 25,
            "sequence": 4294967295,
            "script_asm": "",
            "script_hex": "",
            "required_signatures": null,
            "addresses": [
                "333ixMBzfwcqFpBGdpcpYShwMzUuwd4Wm5"
            ],
            "value": 4237500,
            "type": "scripthash"
        },
        {
            "index": 1,
            "create_transaction_id": "97a61972f058498396c715051ff8d01f3cef3ecd81b3ca0fce970e8f28a4799a",
            "spending_transaction_id": "46cf0c70ad79af2c2b74baa83ea1cce67a6136e658fcd6f58579455b5f9496b1",
            "create_output_index": 6,
            "sequence": 4294967295,
            "script_asm": "",
            "script_hex": "",
            "required_signatures": null,
            "addresses": [
                "3M5AcTQUEsEsYNGi1UHpP4MobM5m419krZ"
            ],
            "value": 4536429,
            "type": "scripthash"
        },
        {
            "index": 2,
            "create_transaction_id": "bfd20cc8be8f586149db3cb7ee4ac1702a44c4224905a6540c747341fa0771f0",
            "spending_transaction_id": "46cf0c70ad79af2c2b74baa83ea1cce67a6136e658fcd6f58579455b5f9496b1",
            "create_output_index": 51,
            "sequence": 4294967295,
            "script_asm": "",
            "script_hex": "",
            "required_signatures": null,
            "addresses": [
                "3MwJVgPcydvQrSrjeht9rxjZnb4Cxxvfdu"
            ],
            "value": 1875000,
            "type": "scripthash"
        },
        {
            "index": 3,
            "create_transaction_id": "d7e8916e6d73f65bef1937c3dfda2f47022db487c239e0cab9b6354c820886ed",
            "spending_transaction_id": "46cf0c70ad79af2c2b74baa83ea1cce67a6136e658fcd6f58579455b5f9496b1",
            "create_output_index": 213,
            "sequence": 4294967295,
            "script_asm": "",
            "script_hex": "",
            "required_signatures": null,
            "addresses": [
                "32Hyd8hNLZ91EMmkU6kbfbiNA6H5s7qvpB"
            ],
            "value": 1099000,
            "type": "scripthash"
        },
        {
            "index": 4,
            "create_transaction_id": "fcb4591669cc7842c266b58371672d663fb2a9e598916b7c665af887f35a5c71",
            "spending_transaction_id": "46cf0c70ad79af2c2b74baa83ea1cce67a6136e658fcd6f58579455b5f9496b1",
            "create_output_index": 1,
            "sequence": 4294967295,
            "script_asm": "",
            "script_hex": "",
            "required_signatures": null,
            "addresses": [
                "3CpaedyT46nugyMww3wubNs1eU1hTB8CGC"
            ],
            "value": 662500,
            "type": "scripthash"
        }
    ],

@medvedev1088 medvedev1088 merged commit 40c614d into blockchain-etl:master Dec 16, 2021
prassee added a commit to merklescience/bitcoin-etl that referenced this pull request Dec 16, 2021
re-appended more fixes for taproot from upstream (rather merged) PR
blockchain-etl#57
@medvedev1088
Copy link
Member

This PR along with #59 has been deployed to bigquery-public-data.crypto_bitcoin dataset. Let me know if you find any issues.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants