You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Using this as a quick way to add FAQ entries / notes while there is a lot of transition activity around this repo. In future, these will get moved to durable markdown files, but for now allowing edits here to reduce any friction with capturing this content.
What are the reasons/goals for doing repo consolidation?
We want to make it easier to contribute to PL's Go IPFS codebase. Why?
The IPFS Stewards, the maintainers of most IPFS org repos, are shifting focus to libraries.
Provide a first-class library experience, to enable users to build their own binaries/implementations instead of piling everything into Kubo
Existing libraries are hit-or-miss, with large amounts of tech debt (poor testing, unsafe APIs, multiple versions/forks, half-completed migrations, "experimental" features sitting unfinished, etc.). We want to refactor these but refactoring is very time consuming given the large number of repos and versions involved.
Library components should be tested together
We want to grow the Go IPFS community to include more than just PL engineers.
The large number of repos + versions is intimidating to newcomers
It is difficult to understand the impact of a change across all the repos+versions
There is no cross-repo testing feedback until you plumb everything up to Kubo
Why is the Boxo project consolidating repos and using example-driven development despite ecosystem friction?
We believe that the current go ipfs ecosystem is suboptimal due to issues like incompatible versions, complex APIs, and hard-to-navigate projects. We think a shared monorepo following an example-driven development will create a better
ecosystem. Although the transition causes friction, we developed a migration tool to help with migration. We believe the tradeoff is worthwhile for long-term ecosystem improvements, but the validation of this approach won't come until evaluating metrics and user anecdotes 3-6 months after.
Why do maintainers care about reducing the number of repos they maintain?
Repo maintenance costs include:
Keeping dependencies up-to-date
This is non-trivial as it often requires chasing down other dependencies in the dependency graph...mostly we don't do this until we have to
Releasing new versions as necessary
Making sure CI is still working
Migrating from Travis/CircleCI to Actions (still in progress)
Rolling out unified CI
Backporting changes across major versions as necessary
Manually testing impact of new code changes on downstream consumers
Monitoring issue trackers, PRs, etc.
Updating submodules
Commonly used for testing, example code, etc.
Often these contain circular module dependencies which complicate propagating breaking changes
What alternatives were considered and why were they ruled against?
Just focus on docs/examples in a repo with a known set of versions that play nice together
Given this isn't free for maintainers or those affected, what has been the cost/benefit analysis?
What are some concrete usecases where repo consolidation would have been beneficial?
2023-03-02 note: this will get extended in the next week. @BigLep only seeded with a few things.
Below are anecdotes from the last few months where EngRes IPFS Stewards wished for directly or saw need from others for consolidated repos:
Changing the fetcher API - @aschmahmann to fill in. @BigLep jotted this down during 2023-02-28 conversation.
Finding a bug in a sub-repo and not being able to handle
202302 security event handling where doing a fix spanned 3 repos instead of the ideal one. This mean 3 security advisories, releases, etc.
202302 Gala Games - After experiencing Kubo footguns that affected DHT performance, they raised their hands to be consumers of a project like boxo so they could assemble a IPFS implementation tailored to their needs.
In 2022 IPFS Stewards spent months plumbing contexts through 40+ repos, some having multiple incompatible versions. The complexity involved is quite high--you need to find an appropriate topological order of the dependency graph, make the changes & test & release repo versions incrementally, etc. And if you make a single mistake, you have to start over.
202303: blockservice & exchange & bitswap: add non variadic NotifyNewBlock #242 . This would have needed to be a complex DAG of 4 PR and go.mod bubbling for what is a simple API extension, this is now 1 PR. Those improvements would take an hour before, this was 5 minutes of a maintainer's time.
2023-03-31: Unsolicited feedback from engineers working on ipfs/gateway-conformance-testing: "❤️ the boxo consolidation. In conformance we went from importing ~8 packages to open a car file containing a unixfs node to 4 packages. The lib is easier to work with, it feels more consistent, and it’s less overwhelming to get started."
Copied or Migrated Repos FAQs
The FAQs below concern all the repos that were copied or migrated into boxo to bootstrap the library. Some of the repos are copies of repos that are maintained by others, but most are now not maintained.
What will happen if there is a security issue in one of the legacy ipfs/* repos that has been copied into boxo?
@ipfs/kubo-maintainers (which primarily maps to PL EngRes IPFS Stewards as of 202203) will certainly handle patching boxo. If there are maintainers for the original/source repos, they will need to handle patching/disclosing. @ipfs/kubo-maintainers will certainly coordinate and share work, but won't handle communication with or updating of the affected consumers of the original repos that boxo copied from. If there are no maintainers for the original/source repos, there will likely be internal private PL EngRes chats analyzing which projects are impacted, and it will then be up to those projects to determine whether they want to patch the existing repos or update to use boxo. (We understand and expect that the "update to boxo" option will often not be possible under tight timelines given boxo's plans to upgrade its dependencies frequently and refactor the existing code.)
I don't like the "deprecated" warnings in "not maintained" repos that have been moved to boxo. What can be done about this?
The intent of the "deprecated" warnings was to be clear to consumers about the status of the repo: that it isn't maintained that there is a repo where similar code is being maintained. If another maintainer comes along, then they can follow claim ownership and the state of the repo can be adjusted. This satisfies the requirement about being clear to users about who owns and its status.
How does one claim ownership of a "not maintained" repo?
A new maintainer can
create a PR that includes:
Revert of the deprecation types
Removal of the "not maintained" notice from the README
Assuming the new maintainer is known/trusted, @ipfs/kubo-maintainers will merge.
Will the "not maintained" ipfs/* repos be left around to rot?
PL EngRes @ipfs/kubo-maintainers don't want to leave a wasteland of unmaintained repos. We'll come back through and archive repos that haven't had a maintainer step up in 2023Q3. See #201
Logistics FAQs
What is the release process for boxo?
At least as of 2023-03-02 this is not well defined. This will be resolved in #170
What is the response time for issues or PRs filed?
What are some projects that depend on this project?
See #190 (comment) for latest
vvv Not reviewed or up-to-date vvv
Table of Contents
Purpose
Using this as a quick way to add FAQ entries / notes while there is a lot of transition activity around this repo. In future, these will get moved to durable markdown files, but for now allowing edits here to reduce any friction with capturing this content.
Rational FAQs
TODO: 2023-03-30 this needs to be expanded, including content from other issues and slack threads.
What are the reasons/goals for doing repo consolidation?
We want to make it easier to contribute to PL's Go IPFS codebase. Why?
Why is the Boxo project consolidating repos and using example-driven development despite ecosystem friction?
We believe that the current go ipfs ecosystem is suboptimal due to issues like incompatible versions, complex APIs, and hard-to-navigate projects. We think a shared monorepo following an example-driven development will create a better
ecosystem. Although the transition causes friction, we developed a migration tool to help with migration. We believe the tradeoff is worthwhile for long-term ecosystem improvements, but the validation of this approach won't come until evaluating metrics and user anecdotes 3-6 months after.
Why do maintainers care about reducing the number of repos they maintain?
Repo maintenance costs include:
What alternatives were considered and why were they ruled against?
Just focus on docs/examples in a repo with a known set of versions that play nice together
Given this isn't free for maintainers or those affected, what has been the cost/benefit analysis?
What are some concrete usecases where repo consolidation would have been beneficial?
2023-03-02 note: this will get extended in the next week. @BigLep only seeded with a few things.
Below are anecdotes from the last few months where EngRes IPFS Stewards wished for directly or saw need from others for consolidated repos:
https://filecoinproject.slack.com/archives/C02RLM3JHUY/p1677409745784779
Copied or Migrated Repos FAQs
The FAQs below concern all the repos that were copied or migrated into boxo to bootstrap the library. Some of the repos are copies of repos that are maintained by others, but most are now not maintained.
What repos have been copied into Boxo?
The authoritative list is in https://github.com/ipfs/boxo/blob/main/cmd/migrate/internal/config.go. Many of these were handled as part of #202.
What will happen if there is a security issue in one of the legacy ipfs/* repos that has been copied into boxo?
@ipfs/kubo-maintainers (which primarily maps to PL EngRes IPFS Stewards as of 202203) will certainly handle patching boxo. If there are maintainers for the original/source repos, they will need to handle patching/disclosing. @ipfs/kubo-maintainers will certainly coordinate and share work, but won't handle communication with or updating of the affected consumers of the original repos that boxo copied from. If there are no maintainers for the original/source repos, there will likely be internal private PL EngRes chats analyzing which projects are impacted, and it will then be up to those projects to determine whether they want to patch the existing repos or update to use boxo. (We understand and expect that the "update to boxo" option will often not be possible under tight timelines given boxo's plans to upgrade its dependencies frequently and refactor the existing code.)
I don't like the "deprecated" warnings in "not maintained" repos that have been moved to boxo. What can be done about this?
The intent of the "deprecated" warnings was to be clear to consumers about the status of the repo: that it isn't maintained that there is a repo where similar code is being maintained. If another maintainer comes along, then they can follow claim ownership and the state of the repo can be adjusted. This satisfies the requirement about being clear to users about who owns and its status.
How does one claim ownership of a "not maintained" repo?
A new maintainer can
Assuming the new maintainer is known/trusted, @ipfs/kubo-maintainers will merge.
Will the "not maintained" ipfs/* repos be left around to rot?
PL EngRes @ipfs/kubo-maintainers don't want to leave a wasteland of unmaintained repos. We'll come back through and archive repos that haven't had a maintainer step up in 2023Q3. See #201
Logistics FAQs
What is the release process for boxo?
At least as of 2023-03-02 this is not well defined. This will be resolved in #170
What is the response time for issues or PRs filed?
What are some projects that depend on this project?
The exhaustive list is https://github.com/ipfs/boxo/network/dependents. Some notable projects include:
Why is this named Boxo?
TODO: extract portions from #215
Good sources of information for more FAQ questions/answers that should get pulled in here
The text was updated successfully, but these errors were encountered: