-
Notifications
You must be signed in to change notification settings - Fork 71
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use ipnisync with libp2phttp #1638
Conversation
@gammazero A few questions about dependencies:
|
|
@gammazero Let me know once changes are done to downgrade Lotus dependency. After that, we can test in filcollins. |
@LexLuthr Is |
@gammazero Apologies! I completely missed that commit. I will start the deployment and testing on filcollins. |
I am merging this PR in a temporary branch. The reason for this approach:
|
* Use ipnisync with libp2phttp (#1638) * Use ipnisync with libp2phttp * Add config to fall back to legacy data-transfer sync * Update config doc * Need to sync entries chain * Fix expected error text in test case * Fix lint warning * Update comment * Fork mplex since libp2p no longer supports it * gostream relocated, deps updated * update circle CI config to use latest version of go * update libp2p * Do not import car/v2 from boxo * Update index-provider config * Update to latest index-provider * Update lotus, filecoin-ffi submodule, and lotus API parameter * doc gen * Update circle-ci image versions * Fix itest * cbor-gen * Update circieci lid-docker-compose image * Fix circieci image * update circleci docker version * update devnet golang docker image * Upgrade devnet docker to ubuntu 22.04 * Update go-car * Use lotus version [email protected] - Update ipni dependencies * rebase and mod tidy * add defaults, streamline go version * Update node/config/types.go Co-authored-by: dirkmc <[email protected]> * update UI * fix variable names * remove mplex lib and mux from bitswap * mod tidy --------- Co-authored-by: Andrew Gillis <[email protected]> Co-authored-by: dirkmc <[email protected]>
Overview
IPNI has a new capability to perform advertisement sync using HTTP over libp2p. This replaces the current data-transfer/graphsync based mechanism, and does not require any change in configuration.
For boost providers serving advertisements using plain HTTP, their current configurations will continue serving plain HTTP as well as HTTP over libp2p. Serving advertisements over libp2p can be disabled if needed.
Technical design
New capabilities within the ipni publisher, contained in the
go-libipni/dagsync
andindex-provider/engine
packages, serve advertisements using the boost libp2p stream host. A new protocol ID for HTTP is registered and libp2p stream host directs requests for this protocol to use the ipni publisher as a handler for incoming requests. The requests are handled as HTTP requests and the ipni publisher handles these requests the same way that it handles HTTP requests delivered by a plain HTTP server.This allows an existing boost provider to serve the new http-based ipni protocol over libp2p without making any changes to its existing configuration, which previously used data-transfer/graphsync to publish advertisements. Since communication is done using the existing libp2p host, there is no need to make any infrastructure changes such as exposing HTTP endpoints or configuring a proxy. The same addresses that were sent in announcement messages to announce new advertisements will continue to work with the new libp2phttp ipni protocol.
UI changes required
No UI changes are needed, unless necessary to give access to new configuration options. The new configuration options are documented in the documentation update.
Implementation
The only new implementation is a dependency update and a modification to fix an existing unit test.
Testing plan
The current tests are run using the default configuration, which serves advertisements using HTTP over libp2p.
Remaining Tasks