-
Notifications
You must be signed in to change notification settings - Fork 26
Conversation
Unify the `string`/`ipld.Node` information a parent has about its children when updating a modified entry. This will be used to simplify the code around the `childCloser` interface and related logic.
Use newly introduced `child` structure whenever the logically tied `string`/`ipld.Node` pair was used to represent a child entry in a directory operation.
Rename the `childCloser` interface to `mutableParent` to better reflect the fact that the structures that implement it (`Directory` and `Root`) have the function of parents in the MFS hierarchy. Rename also the method of the interface from `closeChild` to `updateChildEntry` shifting the focus away from in which circumstance is the method being called (when closing a child) to actually what operation is being performed by the method (updating an entry in the parent to the new content).
Extract the `Republisher` structure and related logic from `system.go` (leaving that file mainly with the `Root` logic, and hence now renamed as `root.go`) to a new `repub.go` file. Even though the MFS root has support for republishing updates the logic behind those two structures is mostly decoupled, that should be reflected in the source code organization.
Remove the `Sync()` method from the `FileDescriptor` and also its implementation. Two different undocumented methods in `FileDescriptor` are offered to update the state of the MFS: `Flush()` and `Sync()`, both calling `updateChildEntry`. The only difference (the user is not aware of) is that one sets the `fullSync` argument and the other doesn't, that is, one does a full update of the filesystem and the other just updates the parent directory. This current situation is not clear to the consumer who may use one desiring the effect of the other. As a precautionary measure while redesigning the MFS API one method is removed to simplify its usage. The `Sync()` was chosen as the less "safe" alternative which didn't do a full update of the MFS and also doesn't seem to be used in the `go-ipfs` repository (the main consumer of the `mfs` package).
Unify the two caches in `Directory` that discriminated between files and directories (`files` and `childDirs`) into a single `entriesCache` since that distinction wasn't used anywhere and made the code unnecessarily more complex.
06cd484
to
9186d93
Compare
/cc @Stebalien |
@Stebalien ping :) |
dir.go
Outdated
@@ -405,7 +405,7 @@ func (d *Directory) AddChild(name string, nd ipld.Node) error { | |||
|
|||
// AddUnixFSChild adds a child to the inner UnixFS directory | |||
// and transitions to a HAMT implementation if needed. | |||
func (d *Directory) AddUnixFSChild(name string, node ipld.Node) error { | |||
func (d *Directory) AddUnixFSChild(c child) error { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Public functions shouldn't take private types.
system.go
Outdated
// TODO: What is `fullsync`? (unnamed `bool` argument) | ||
// TODO: There are two types of persistence/flush that need to be | ||
// distinguished here, one at the DAG level (when I store the modified | ||
// nodes in the DAG service) and one in the UnixFS/MFS level (when I modify | ||
// the entry/link of the directory that pointed to the modified node). | ||
type childCloser interface { | ||
closeChild(child, bool) error | ||
type mutableParent interface { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why not just parentHandle
, or even just parent
? I agree that childCloser
is not the right name but mutableParent
signals "this is a mutable parent, there are immutable versions".
dir.go
Outdated
// So, `fullSync` entails operations at two different layers: | ||
// 1. DAG: save the newly created directory node with the updated entry. | ||
// 2. MFS: propagate the update upwards repeating the whole process in the | ||
// parent. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's be concise and to the point.
dir.go
Outdated
@@ -237,12 +237,6 @@ func (d *Directory) Child(name string) (FSNode, error) { | |||
return d.childUnsync(name) | |||
} | |||
|
|||
func (d *Directory) Uncache(name string) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is important. We use it from the unixfs file adder in go-ipfs to avoid a massive memory leak.
This isn't ideal but we can't remove it without replacement. It was added to fix a critical bug.
Document the `Republisher` code in `repub.go` according to the explanation in ipfs/kubo#5092 (comment). Erring on the side of verbosity since this has been a part of the code with very little review up until now.
Unify all the `Republisher` receivers names to `rp` (avoiding `r` to distinguish it from the `Root` receivers).
1930199
to
cb36051
Compare
@Stebalien Addressed your comments in the added (last) 3 commits and dropped the one that removed the |
dir.go
Outdated
@@ -405,7 +403,7 @@ func (d *Directory) AddChild(name string, nd ipld.Node) error { | |||
|
|||
// AddUnixFSChild adds a child to the inner UnixFS directory | |||
// and transitions to a HAMT implementation if needed. | |||
func (d *Directory) AddUnixFSChild(name string, node ipld.Node) error { | |||
func (d *Directory) AddUnixFSChild(c Child) error { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As an internal helper type, child
is fine (although I still feel it's unecessary). However, I really don't want to see it leak into an external interface. All it does is change calls to:
d.AddUnixFSChild(name, node)
To:
d.AddUnxiFSChild(mfs.Child{Name: name, Node: node})
Which is strictly more annoying to use and provides no additional safety, documentation, etc.
(sorry, I should have commented on this the first round but I assumed you'd replace child
with the name/node in this case instead of exporting it)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for pointing me to it, actually AddUnixFSChild
shouldn't be exported in the first place, I'll also un-export child.
The name `mutableParent` signals "this is a mutable parent, there are immutable versions".
cb36051
to
6c4a00a
Compare
@Stebalien Fixed. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM.
@Stebalien Thanks for helping me get this PR in shape! |
Thanks for cleaning all this up! |
This PR contains a series of commits with diverse MFS code refactorings. The commits are loosely tied together here to make it easier to review and merge. This PR then should be reviewed on a commit-by-commit basis in chronological order.
This PR breaks the current API (which is something expected in the context of the milestone to review the MFS API) but only in evident ways, e.g., renaming/removing functions, it doesn't change the current behavior (those kind of changes will be discussed in separate issues/PRs).