ResourceManager: Adjust inbound connection limits depending on memory. #9593
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ajnavarro
force-pushed
the
fix/rm-use-memory-for-limits
branch
from
40f6876
to
75de158
Compare
Closed
|
Just checking in, any chance we're landing this today for the v0.18.1 release? |
|
@galargh I hope to land it today. |
BigLep
approved these changes
Jan 27, 2023
ajnavarro
force-pushed
the
fix/rm-use-memory-for-limits
branch
from
75de158
to
c98a9fc
Compare
lidel
reviewed
Jan 27, 2023
| @@ -1843,7 +1843,7 @@ This value is also used to scale the limit on various resources at various scope | |||
| when the default limits (discussed in [libp2p resource management](./libp2p-resource-management.md)) are used. | |||
| For example, increasing this value will increase the default limit for incoming connections. | |||
|
|
|||
| Default: `[TOTAL_SYSTEM_MEMORY]/4` | |||
| Default: `[TOTAL_SYSTEM_MEMORY]/2` | |||
There was a problem hiding this comment.
💭 (not familiar with prior discussions, someone else needs to review, but adding comment in favor of this change)
👍 I know @2color run into issues when he was running on a box that had 2 GB RAM on https://fly.io/ box, and his libp2p stack only got 1/4 (512MB).
This should improve default behavior in such setups (allocating 1GB).
BigLep
approved these changes
Jan 27, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is in response to libp2p/go-libp2p#2010 for simplification I think we want to be able to do this for Kubo.
I believe will allow the resource manager to get out of the way more but still protect against script-kiddies opening many connections/streams against a node.
In order for this work, established (non-transient) connections will need to register memory usage with the resource manager/accountant, ideally across all transports and muxers. Per libp2p/go-libp2p#2010 (comment), there are some cases where an established libp2p connection doesn't increase memory accounted for by the resource manager/accountant. In the meantime, we can set System.ConssInbound based on ResourceMgr.MaxMemory and then strip this out when libp2p/go-libp2p#2010 lands.
Change tested checking that we do not OOM after a stream flooding attack. Everything working as expected.
Updated defaults values to use totalMemory/2
It closes #9602