Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handling "broken" pins. #3069

Closed
kevina opened this issue Aug 9, 2016 · 6 comments
Closed

Handling "broken" pins. #3069

kevina opened this issue Aug 9, 2016 · 6 comments
Labels
need/community-input Needs input from the wider community

Comments

@kevina
Copy link
Contributor

kevina commented Aug 9, 2016

In #2962 @whyrusleeping was uncomfortable with the idea of removing pinned blocks. I am guessing that is because removing indirectly pinned blocks can break the repo. This can be avoided in #2962 but the issue of broken pins will come up again in my filestore code ( #875 #2634) where parts of a recursive pin can just disappear if the backing file is changed or removed.

The question is what to do about it.

Assuming the contents of the pin are just not available (i.e., there are not available on another node),
the only thing that can be done is to either repair or remove the recursive pin. One way of repairing a recursive pin is by pinning the children that are still valid and optionally turning the original pin into a direct pin. This operation makes sense if one of the files of a pinned directory was removed but you want to keep the other files around. If the pinned block is part of a file than it may make more sense to just remove the recursive pin.

Thoughts?

@kevina
Copy link
Contributor Author

kevina commented Aug 9, 2016

Note: This is the result of an IRC converstation. See https://botbot.me/freenode/ipfs/msg/71044000/ for context.

@whyrusleeping
Copy link
Member

cc @jbenet @diasdavid @Kubuxu for input

@jakobvarmose
Copy link

jakobvarmose commented Aug 18, 2016

If the user has recursively pinned a block they clearly want to store this block and its children, so IPFS should try to keep what is still left, so deleting the pin seems like a bad idea. Adding new pins to compensate for the broken block may result in a lot of direct pins and would be very confusing for the user. So I guess the only option left is to simply keep the pin, but maybe mark it as broken.

@whyrusleeping whyrusleeping added the need/community-input Needs input from the wider community label Aug 18, 2016
@whyrusleeping whyrusleeping changed the title Handing "broken" pins. Handling "broken" pins. Aug 19, 2016
@kevina
Copy link
Contributor Author

kevina commented Aug 20, 2016

@jakobvarmose that would be similar to a best effort pin. There is some support for handling this in the garbage collector (see #2872), but currently there is no direct support for creating best effort pins.

@daviddias
Copy link
Member

Agreed, this is tricky. It seems that this belongs to mfs though, we should pin all of the mutations that happen on the virtual dir, but have a command to unpin all the 'debris' caused by mutating them (aka old nodes that we won't directly access anymore). This way, removing a file from a mfs dir that is pinned should work, the nodes will continue there and pinned, only with user request (i.e: ipfs mfs cleanup) will start unpinning all the ones we don't have a direct pointer in the virtual tree so that they can be cleaned by ipfs repo gc.

@kevina
Copy link
Contributor Author

kevina commented Aug 24, 2016

Since multiple people didn't like my idea of repairing pins. More direct support for best-effort pins seams to be the way to go. Closing in favor of #3069.

@kevina kevina closed this as completed Aug 24, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
need/community-input Needs input from the wider community
Projects
None yet
Development

No branches or pull requests

4 participants