Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: option to not read size of blocks for want-have requests #672

Merged
merged 10 commits into from
Sep 27, 2024

Conversation

gammazero
Copy link
Contributor

@gammazero gammazero commented Sep 13, 2024

When the replace-have-with-want-blocks is disabled, the block sizes for blocks corresponding to have-want requests are not read and instead the blockstore is checked only to see if the block is present.

The replace-have-with-want-blocks feature can be disabled using the new WithReplaceHasWithBlockMaxSize(n) option which can be used with bitswap.New. It sets the maximum size of a block in bytes up to which we will replace a want-have with a want-block. Setting a size of 0 disables want-have replacement and means that block sizes are not read for want-have requests.

Aims to fix #657

Copy link

codecov bot commented Sep 13, 2024

Codecov Report

Attention: Patch coverage is 73.91304% with 24 lines in your changes missing coverage. Please review.

Project coverage is 60.33%. Comparing base (4af06fd) to head (d7f122c).
Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
...swap/server/internal/decision/blockstoremanager.go 64.51% 8 Missing and 3 partials ⚠️
bitswap/server/server.go 0.00% 7 Missing ⚠️
bitswap/server/internal/decision/engine.go 92.30% 3 Missing and 1 partial ⚠️
bitswap/options.go 0.00% 2 Missing ⚠️

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #672      +/-   ##
==========================================
+ Coverage   60.29%   60.33%   +0.03%     
==========================================
  Files         243      243              
  Lines       30889    30953      +64     
==========================================
+ Hits        18624    18674      +50     
- Misses      10603    10620      +17     
+ Partials     1662     1659       -3     
Files with missing lines Coverage Δ
bitswap/options.go 36.36% <0.00%> (-1.74%) ⬇️
bitswap/server/internal/decision/engine.go 91.44% <92.30%> (-0.35%) ⬇️
bitswap/server/server.go 63.35% <0.00%> (-1.29%) ⬇️
...swap/server/internal/decision/blockstoremanager.go 85.33% <64.51%> (-5.43%) ⬇️

... and 13 files with indirect coverage changes

@gammazero
Copy link
Contributor Author

See configuration in kubo: ipfs/kubo#10512

@gammazero gammazero changed the title feat: no not read size of blocks for want-have requests feat: do not read size of blocks for want-have requests Sep 18, 2024
Copy link
Member

@Wondertan Wondertan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Our integration tests passed with the option applied. I will test this E2E once the debug log is removed

bitswap/server/internal/decision/engine.go Outdated Show resolved Hide resolved
@Wondertan
Copy link
Member

I tested this E2E, and things have been going smoothly so far. Will see how it works over the weekend.

@Wondertan
Copy link
Member

Wondertan commented Sep 23, 2024

The cluster performs correctly, and the nodes are still in sync. We haven't tested for performance improvements or degradations and I think that's an approve from our side.

bare minimum to make it possible to discover current implicit default
by reading godocs
@lidel lidel changed the title feat: do not read size of blocks for want-have requests feat: option to not read size of blocks for want-have requests Sep 26, 2024
Copy link
Member

@lidel lidel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, pushed cosmetic docs changes (godoc explaining feature bit more + ability to read implicit default via godoc)

@gammazero thanks again, if text looks good to you, free to squash and merge 🚀

@gammazero gammazero merged commit 19a402b into main Sep 27, 2024
15 checks passed
gammazero added a commit to ipfs/kubo that referenced this pull request Sep 28, 2024
Allow configuration of the bitswap server's replace WantHave with WantBlock maximum block size using the Internal.Bitswap.WantHaveReplaceSize config item. This sets the maximum size of a block in bytes up to which we will replace a want-have with a want-block. Setting a size of 0 disables this replacement and means that block sizes are not read for WantHave requests.

See ipfs/boxo#672 for more details

Updated boxo to version with PR 672

---------

Co-authored-by: Marcin Rataj <[email protected]>
@gammazero gammazero deleted the feat/no-size-for-want-have branch September 28, 2024 10:16
wenyue pushed a commit to wenyue/boxo that referenced this pull request Oct 17, 2024
)

* Do not fetch size for WantHave blocks
* Option to set replaceHasWithBlockMaxSize
* Log if replace logic enabled/disabled
* docs: WithWantHaveReplaceSize

---------

Co-authored-by: Marcin Rataj <[email protected]>
@gammazero gammazero mentioned this pull request Oct 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

bitswap/server: WANT_HAVE requests should use Blockstore.Has
3 participants