Handling "broken" pins.

[kevina]

In #2962 @whyrusleeping was uncomfortable with the idea of removing pinned blocks. I am guessing that is because removing indirectly pinned blocks can break the repo. This can be avoided in #2962 but the issue of broken pins will come up again in my filestore code ( #875 #2634) where parts of a recursive pin can just disappear if the backing file is changed or removed.

The question is what to do about it.

Assuming the contents of the pin are just not available (i.e., there are not available on another node),
the only thing that can be done is to either repair or remove the recursive pin. One way of repairing a recursive pin is by pinning the children that are still valid and optionally turning the original pin into a direct pin. This operation makes sense if one of the files of a pinned directory was removed but you want to keep the other files around. If the pinned block is part of a file than it may make more sense to just remove the recursive pin.

Thoughts?

[kevina]

Note: This is the result of an IRC converstation. See https://botbot.me/freenode/ipfs/msg/71044000/ for context.

[whyrusleeping]

cc @jbenet @diasdavid @Kubuxu for input

[jakobvarmose]

If the user has recursively pinned a block they clearly want to store this block and its children, so IPFS should try to keep what is still left, so deleting the pin seems like a bad idea. Adding new pins to compensate for the broken block may result in a lot of direct pins and would be very confusing for the user. So I guess the only option left is to simply keep the pin, but maybe mark it as broken.

[kevina]

@jakobvarmose that would be similar to a best effort pin. There is some support for handling this in the garbage collector (see #2872), but currently there is no direct support for creating best effort pins.

[daviddias]

Agreed, this is tricky. It seems that this belongs to mfs though, we should pin all of the mutations that happen on the virtual dir, but have a command to unpin all the 'debris' caused by mutating them (aka old nodes that we won't directly access anymore). This way, removing a file from a mfs dir that is pinned should work, the nodes will continue there and pinned, only with user request (i.e: ipfs mfs cleanup) will start unpinning all the ones we don't have a direct pointer in the virtual tree so that they can be cleaned by ipfs repo gc.

[kevina]

Since multiple people didn't like my idea of repairing pins. More direct support for best-effort pins seams to be the way to go. Closing in favor of #3069.