Skip to content

feat: automatically detect if genesis CAR is compressed#12885

Merged
masih merged 1 commit intomasterfrom
masih/gen-auto-zstd
Feb 11, 2025
Merged

feat: automatically detect if genesis CAR is compressed#12885
masih merged 1 commit intomasterfrom
masih/gen-auto-zstd

Conversation

@masih
Copy link
Copy Markdown
Member

@masih masih commented Feb 10, 2025

Related Issues

Proposed Changes

When --genesis path is set, automatically detect if the genesis file is ZSTD compressed and decompress it.

Additional Info

Checklist

Before you mark the PR ready for review, please make sure that:

Copy link
Copy Markdown

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@masih masih force-pushed the masih/gen-auto-zstd branch from 45e8853 to d1d301d Compare February 10, 2025 11:33
@masih masih changed the title Automatically detect if genesis CAR is compressed feat: automatically detect if genesis CAR is compressed Feb 10, 2025
@github-actions github-actions Bot dismissed their stale review February 10, 2025 11:33

PR title now matches the required format.

@masih masih force-pushed the masih/gen-auto-zstd branch 3 times, most recently from b9e5130 to 46b1c8e Compare February 10, 2025 11:40
@masih masih requested a review from rvagg February 10, 2025 11:41
@rjan90
Copy link
Copy Markdown
Contributor

rjan90 commented Feb 10, 2025

Did a test with this branch, and can confirm it works:

Before:

lotus daemon --genesis=/home/boost/lotus/build/genesis/mainnet.car.zst
2025-02-10T12:43:16.762+0100	INFO	main	lotus/daemon.go:230	lotus repo: /mnt/lotuschain/.lotus
2025-02-10T12:43:16.763+0100	INFO	paramfetch	go-paramfetch@v0.0.4/paramfetch.go:209	Parameter file /var/tmp/filecoin-proof-parameters/v28-empty-sector-update-merkletree-poseidon_hasher-8-0-0-92180959e1918d26350b8e6cfe217bbdd0a2d8de51ebec269078b364b715ad63.vk is ok
2025-02-10T12:43:16.763+0100	INFO	paramfetch	go-paramfetch@v0.0.4/paramfetch.go:209	Parameter file /var/tmp/filecoin-proof-parameters/v28-empty-sector-update-merkletree-poseidon_hasher-8-8-0-3b7f44a9362e3985369454947bc94022e118211e49fd672d52bec1cbfd599d18.vk is ok
--------
ERROR: initializing node: starting node: could not build arguments for function "github.com/filecoin-project/lotus/node".ConfigFullNode.StartListening.func10 (/home/boost/lotus/node/modules/lp2p/addrs.go:85): failed to build host.Host: could not build arguments for function "reflect".makeFuncStub (/usr/local/go/src/reflect/asm_amd64.s:28): failed to build lp2p.BaseIpfsRouting: could not build arguments for function "reflect".makeFuncStub (/usr/local/go/src/reflect/asm_amd64.s:28): failed to build dtypes.NetworkName: could not build arguments for function "reflect".makeFuncStub (/usr/local/go/src/reflect/asm_amd64.s:28): failed to build dtypes.AfterGenesisSet: received non-nil error from function "reflect".makeFuncStub (/usr/local/go/src/reflect/asm_amd64.s:28): getting expected genesis failed: loading genesis car file failed: invalid header: malformed stream: invalid appearance of int token; expected map key

After:

lotus daemon --genesis=/home/boost/lotus/build/genesis/mainnet.car.zst
2025-02-10T12:40:04.580+0100	INFO	lotus	lotus/daemon.go:230	lotus repo: /mnt/lotuschain/.lotus
2025-02-10T12:40:04.588+0100	INFO	paramfetch	go-paramfetch@v0.0.4/paramfetch.go:209	Parameter file /var/tmp/filecoin-proof-parameters/v28-proof-of-spacetime-fallback-merkletree-poseidon_hasher-8-0-0-0170db1f394b35d995252228ee359194b13199d259380541dc529fb0099096b0.vk is ok
2025-02-10T12:40:04.588+0100	INFO	paramfetch	go-paramfetch@v0.0.4/paramfetch.go:209	Parameter file /var/tmp/filecoin-proof-parameters/v28-proof-of-spacetime-fallback-merkletree-poseidon_hasher-8-0-0-0cfb4f178bbb71cf2ecfcd42accce558b27199ab4fb59cb78f2483fe21ef36d9.vk is ok
--------
25-02-10T12:40:18.013+0100	INFO	chain	chain/sync.go:646	block validation	{"took": 1.622022872, "height": "4696040", "age": 18.013856694}
2025-02-10T12:40:18.016+0100	INFO	chain	chain/sync.go:646	block validation	{"took": 1.623518005, "height": "4696040", "age": 18.016124076}
2025-02-10T12:40:18.016+0100	INFO	chain	chain/sync.go:646	block validation	{"took": 1.624685525, "height": "4696040", "age": 18.01613177}
2025-02-10T12:40:18.016+0100	INFO	chain	chain/sync.go:646	block validation	{"took": 1.624374797, "height": "4696040", "age": 18.01615072}
2025-02-10T12:40:18.018+0100	INFO	chain	chain/sync.go:646	block validation	{"took": 1.626626394, "height": "4696040", "age": 18.018491273}
2025-02-10T12:40:18.022+0100	INFO	chain	chain/sync.go:646	block validation	{"took": 1.631333797, "height": "4696040", "age": 18.022833674}

@masih masih force-pushed the masih/gen-auto-zstd branch 2 times, most recently from f126b2c to c77bc27 Compare February 10, 2025 11:59
Comment thread build/genesis.go Outdated
Comment thread build/genesis_test.go
Comment thread cli/lotus/daemon.go Outdated
Copy link
Copy Markdown
Member

@rvagg rvagg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👌 love it, we need more of this simple QoL sanity checking

When `--genesis` path is set, automatically detect if the genesis file
is ZSTD compressed and decompress it.
@masih masih force-pushed the masih/gen-auto-zstd branch from c77bc27 to ccc9751 Compare February 11, 2025 10:07
@masih masih merged commit 48d0ab2 into master Feb 11, 2025
@masih masih deleted the masih/gen-auto-zstd branch February 11, 2025 10:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: ☑️ Done (Archive)

Development

Successfully merging this pull request may close these issues.

3 participants