Skip to content

network/kad: Increase memory store capacity for providers#9315

Merged
dmitry-markin merged 6 commits into
masterfrom
dm-kad-memstore-capacity
Jul 28, 2025
Merged

network/kad: Increase memory store capacity for providers#9315
dmitry-markin merged 6 commits into
masterfrom
dm-kad-memstore-capacity

Conversation

@dmitry-markin
Copy link
Copy Markdown
Contributor

Increase Kademlia memory store capacity for DHT content providers (used by parachain DHT-based bootnodes) and reduce provider republish interval & TTL. This is needed to support testnets with 1-minute fast runtime and up to 13 parachains.

Parameters set:

  • 10000 provider keys per node
  • 10h provider record TTL
  • 3.5h provider republish interval

Closes paritytech/litep2p#405.

@dmitry-markin dmitry-markin requested review from a team and lexnv July 24, 2025 12:30
@dmitry-markin dmitry-markin self-assigned this Jul 24, 2025
@dmitry-markin dmitry-markin added the T0-node This PR/Issue is related to the topic “node”. label Jul 24, 2025
@dmitry-markin
Copy link
Copy Markdown
Contributor Author

/cmd prdoc --audience node_dev --bump patch

Copy link
Copy Markdown
Contributor

@lexnv lexnv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice 🙏

Comment thread substrate/client/network/src/litep2p/discovery.rs Outdated
Copy link
Copy Markdown
Member

@bkchr bkchr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally I don't like that we add these things here as random constants. IMO it would be better if this could be communicated better to the discovery subsystem.

@dmitry-markin
Copy link
Copy Markdown
Contributor Author

dmitry-markin commented Jul 25, 2025

Generally I don't like that we add these things here as random constants. IMO it would be better if this could be communicated better to the discovery subsystem.

I am going to move the constants to network/src/config.rs. From there we can either import them or pass to discovery initialization as parameters. EDIT: done.

Or do you mean it should be communicated at runtime from the runtime? This would be an ideal solution, but neither litep2p nor libp2p doesn't support updating these parameters at runtime.

@paritytech-workflow-stopper
Copy link
Copy Markdown

All GitHub workflows were cancelled due to failure one of the required jobs.
Failed workflow url: https://github.com/paritytech/polkadot-sdk/actions/runs/16522931493
Failed job name: fmt

@dmitry-markin
Copy link
Copy Markdown
Contributor Author

/cmd fmt

@dmitry-markin dmitry-markin enabled auto-merge July 28, 2025 08:21
@bkchr
Copy link
Copy Markdown
Member

bkchr commented Jul 28, 2025

Or do you mean it should be communicated at runtime from the runtime? This would be an ideal solution, but neither litep2p nor libp2p doesn't support updating these parameters at runtime.

They could be exposed to the configuration to change them.

@dmitry-markin dmitry-markin added this pull request to the merge queue Jul 28, 2025
auto-merge was automatically disabled July 28, 2025 15:36

Pull Request is not mergeable

Merged via the queue into master with commit a15d066 Jul 28, 2025
236 of 238 checks passed
@dmitry-markin dmitry-markin deleted the dm-kad-memstore-capacity branch July 28, 2025 16:09
@github-project-automation github-project-automation Bot moved this to Blocked ⛔️ in Networking Jul 28, 2025
@dmitry-markin
Copy link
Copy Markdown
Contributor Author

Or do you mean it should be communicated at runtime from the runtime? This would be an ideal solution, but neither litep2p nor libp2p doesn't support updating these parameters at runtime.

They could be exposed to the configuration to change them.

Do you think we need to expose this as CLI options, or just for custom node impls to put their own values?

tmpolaczyk pushed a commit to moondance-labs/polkadot-sdk that referenced this pull request Oct 16, 2025
…#9315)

Increase Kademlia memory store capacity for DHT content providers (used
by parachain DHT-based bootnodes) and reduce provider republish interval
& TTL. This is needed to support testnets with 1-minute fast runtime and
up to 13 parachains.

Parameters set:
- 10000 provider keys per node
- 10h provider record TTL
- 3.5h provider republish interval

Closes paritytech/litep2p#405.

---------

Co-authored-by: cmd[bot] <41898282+github-actions[bot]@users.noreply.github.com>
alvicsam pushed a commit that referenced this pull request Oct 17, 2025
Increase Kademlia memory store capacity for DHT content providers (used
by parachain DHT-based bootnodes) and reduce provider republish interval
& TTL. This is needed to support testnets with 1-minute fast runtime and
up to 13 parachains.

Parameters set:
- 10000 provider keys per node
- 10h provider record TTL
- 3.5h provider republish interval

Closes paritytech/litep2p#405.

---------

Co-authored-by: cmd[bot] <41898282+github-actions[bot]@users.noreply.github.com>
tmpolaczyk pushed a commit to moondance-labs/polkadot-sdk that referenced this pull request Jan 15, 2026
…#9315)

Increase Kademlia memory store capacity for DHT content providers (used
by parachain DHT-based bootnodes) and reduce provider republish interval
& TTL. This is needed to support testnets with 1-minute fast runtime and
up to 13 parachains.

Parameters set:
- 10000 provider keys per node
- 10h provider record TTL
- 3.5h provider republish interval

Closes paritytech/litep2p#405.

---------

Co-authored-by: cmd[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

T0-node This PR/Issue is related to the topic “node”.

Projects

Status: Blocked ⛔️

Development

Successfully merging this pull request may close these issues.

kad/providers: High number of discarding record warnings

4 participants