Skip to content

storing large file in bulletin#105

Merged
bkontur merged 22 commits into
paritytech:mainfrom
rosarp:big-data-example
Jan 15, 2026
Merged

storing large file in bulletin#105
bkontur merged 22 commits into
paritytech:mainfrom
rosarp:big-data-example

Conversation

@rosarp

@rosarp rosarp commented Nov 19, 2025

Copy link
Copy Markdown
Member

Closes: #77

Adds example according to #77

@rosarp rosarp requested a review from bkontur November 19, 2025 09:49
@rosarp

rosarp commented Nov 19, 2025

Copy link
Copy Markdown
Member Author

@bkontur the modified code is only common.js
The main function of store_chunked_data.js and store_big_data.js are same.
May be we should merge examples adding documentation instead.

@bkontur

bkontur commented Nov 20, 2025

Copy link
Copy Markdown
Collaborator

@bkontur the modified code is only common.js The main function of store_chunked_data.js and store_big_data.js are same. May be we should merge examples adding documentation instead.

Let's keep it as it is, but first, let's start to use PAPI here also, we merged PAPI example:

if (i > 0 && i % MAX_CHUNKS == 0) {
      await waitForNewBlock();
}

with something sophisticated, we will just push transactions/chunks in the loop until it returns some error "Block is full or similar error", then the loop will wait for another block and again continues to push chunks

Comment thread examples/common.js Outdated
Comment thread examples/common.js Outdated
Comment thread examples/api.js Outdated
Comment thread examples/common.js Outdated
Comment thread examples/common.js Outdated
@bkontur

bkontur commented Dec 17, 2025

Copy link
Copy Markdown
Collaborator

@rosarp please, could you refactor first, directly on the top of actual master as a separate PR:

  • move and reuse function fileToDisk to the examples/common.js
  • move and reuse class NonceManager { to the examples/common.js
  • create examples/cid_dag_metadata.js
  • move here:
    • function cidFromBytes
    • function convertCid
    • function buildUnixFSDagPB (the one from master)

then we will continue with this PR, please

@bkontur

bkontur commented Dec 17, 2025

Copy link
Copy Markdown
Collaborator

@rosarp please, please, rebase

@rosarp

rosarp commented Dec 18, 2025

Copy link
Copy Markdown
Member Author

@bkontur rebase completed

Comment thread .github/workflows/integration-test.yml Outdated
@bkontur bkontur enabled auto-merge (squash) January 14, 2026 23:51
@bkontur bkontur merged commit 00aa68c into paritytech:main Jan 15, 2026
8 checks passed
@rosarp rosarp deleted the big-data-example branch January 29, 2026 07:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Example] Store the “big data”

2 participants