Update help text to avoid confusion of where files can be added from #1826

daviddias · 2015-10-11T00:39:43Z

License: MIT
Signed-off-by: David Dias [email protected]

This used to lead to large refcount numbers, causing Flush to create a lot of IPFS objects, and merkledag to consume tens of gigabytes of RAM. License: MIT Signed-off-by: Jeromy <[email protected]>

OS X sed is documented as "-i SUFFIX", GNU sed as "-iSUFFIX". The one consistent case seems to be "-iSUFFIX", where suffix cannot empty (or OS X will parse the next argument as the suffix). This used to leave around files named `refsout=` on Linux, and was just confusing. License: MIT Signed-off-by: Jeromy <[email protected]>

License: MIT Signed-off-by: Jeromy <[email protected]>

These secondary copies were never actually queried, and didn't contain the indirect refcounts so they couldn't become the authoritative source anyway as is. New goal is to move pinning into IPFS objects. A migration will be needed to remove the old data from the datastore. This can happen at any time after this commit. License: MIT Signed-off-by: Jeromy <[email protected]>

License: MIT Signed-off-by: Jeromy <[email protected]>

Pinner had method GetManual that returned a ManualPinner, so every Pinner had to implement ManualPinner anyway. License: MIT Signed-off-by: Jeromy <[email protected]>

License: MIT Signed-off-by: Jeromy <[email protected]>

Platform-dependent behavior is not nice, and negative refcounts are not very useful. License: MIT Signed-off-by: Jeromy <[email protected]>

License: MIT Signed-off-by: Jeromy <[email protected]>

License: MIT Signed-off-by: Jeromy <[email protected]> sharness: Don't assume we know all things that can create garbage License: MIT Signed-off-by: Jeromy <[email protected]>

WARNING: No migration performed! That needs to come in a separate commit, perhaps amended into this one. This is the minimal rewrite, only changing the storage from JSON(+extra keys) in Datastore to IPFS objects. All of the pinning state is still loaded in memory, and written from scratch on Flush. To do more would require API changes, e.g. adding error returns. Set/Multiset is not cleanly separated into a library, yet, as it's API is expected to change radically. License: MIT Signed-off-by: Jeromy <[email protected]>

License: MIT Signed-off-by: Jeromy <[email protected]>

…lure License: MIT Signed-off-by: Jeromy <[email protected]>

There was doublewrapping with an unneeded msgio. given that we use a stream muxer now, msgio is only needed by secureConn -- to signal the boundaries of an encrypted / mac-ed ciphertext. Side note: i think including the varint length in the clear is actually a bad idea that can be exploited by an attacker. it should be encrypted, too. (TODO) License: MIT Signed-off-by: Jeromy <[email protected]>

License: MIT Signed-off-by: Jeromy <[email protected]>

* ID service stream * make the relay service use msmux * fix nc tests Note from jbenet: Maybe we should remove the old protocol/muxer and see what breaks. It shouldn't be used by anything now. License: MIT Signed-off-by: Jeromy <[email protected]> Signed-off-by: Juan Batiz-Benet <[email protected]>

The addition of a locking interface to the blockstore allows us to perform atomic operations on the underlying datastore without having to worry about different operations happening in the background, such as garbage collection. License: MIT Signed-off-by: Jeromy <[email protected]>

This commit improves (fixes) the FetchGraph call for recursively fetching every descendant node of a given merkledag node. This operation should be the simplest way of ensuring that you have replicated a dag locally. This commit also implements a method in the merkledag package called EnumerateChildren, this method is used to get a set of the keys of every descendant node of the given node. All keys found are noted in the passed in KeySet, which may in the future be implemented on disk to avoid excessive memory consumption. License: MIT Signed-off-by: Jeromy <[email protected]>

License: MIT Signed-off-by: Jeromy <[email protected]>

License: MIT Signed-off-by: Juan Batiz-Benet <[email protected]>

License: MIT Signed-off-by: Jeromy <[email protected]> dont GC blocks used by pinner License: MIT Signed-off-by: Jeromy <[email protected]> comment GC algo License: MIT Signed-off-by: Jeromy <[email protected]> add lock to blockstore to prevent GC from eating wanted blocks License: MIT Signed-off-by: Jeromy <[email protected]> improve FetchGraph License: MIT Signed-off-by: Jeromy <[email protected]> separate interfaces for blockstore and GCBlockstore License: MIT Signed-off-by: Jeromy <[email protected]> reintroduce indirect pinning, add enumerateChildren dag method License: MIT Signed-off-by: Jeromy <[email protected]>

License: MIT Signed-off-by: Jeromy <[email protected]>

whyrusleeping · 2015-10-11T00:59:13Z

core/commands/files/files.go

@@ -124,7 +124,7 @@ var FilesCpCmd = &cmds.Command{
 		Tagline: "copy files into mfs",
 	},
 	Arguments: []cmds.Argument{
-		cmds.StringArg("source", true, false, "source object to copy"),
+		cmds.StringArg("source", true, false, "source object to copy (e.g. '/ipfs/Qm...'"),


it can be either /ipfs/QmAbc.. or an absolute path within mfs itself.

GitCop · 2015-10-11T04:47:53Z

There were the following issues with your Pull Request

Commit: 46b34e2
- Invalid signoff. Commit message must end with
  License: MIT
  Signed-off-by: .* <.*>
Commit: a2ade40
- Invalid signoff. Commit message must end with
  License: MIT
  Signed-off-by: .* <.*>

Guidelines and a script are available to help. Your feedback on GitCop is welcome on this issue.

This message was auto-generated by https://gitcop.com

jbenet · 2015-10-11T10:14:37Z

@diasdavid run https://github.com/ipfs/community/blob/master/dev/tools/hooks/setup_commit_msg_hook.sh

GitCop · 2015-10-11T15:40:05Z

There were the following issues with your Pull Request

Commit: 46b34e2
- Invalid signoff. Commit message must end with
  License: MIT
  Signed-off-by: .* <.*>
Commit: a2ade40
- Invalid signoff. Commit message must end with
  License: MIT
  Signed-off-by: .* <.*>

Guidelines and a script are available to help. Your feedback on GitCop is welcome on this issue.

This message was auto-generated by https://gitcop.com

License: MIT Signed-off-by: David Dias <[email protected]>

daviddias · 2015-10-11T16:26:37Z

should be all good now (squashed with the sig on daviddias@a42daf2)

whyrusleeping · 2015-10-12T23:56:28Z

core/commands/files/files.go

@@ -24,7 +24,7 @@ var log = u.Logger("cmds/files")

 var FilesCmd = &cmds.Command{
 	Helptext: cmds.HelpText{
-		Tagline: "Manipulate unixfs files",
+		Tagline: "Manipulate mfs files",


I dont know that we actually want to call them 'mfs' files.. @jbenet?

ipfs unix or ipfs files

whyrusleeping · 2015-10-20T04:49:49Z

@diasdavid ping!

daviddias · 2015-10-20T05:03:42Z

@whyrusleeping pong! not sure what I am missing. I squashed all of the tiny commits into one and signed it daviddias@a42daf2 , isn't that good?

whyrusleeping · 2015-10-20T05:28:17Z

@diasdavid theres a bit of feedback on the phrasing of the helptext. we dont think it should be referred to as mfs

jbenet · 2015-10-20T19:45:15Z

it should be ipfs unix or ipfs files

License: MIT Signed-off-by: David Dias <[email protected]>

rht · 2016-01-23T04:29:28Z

This should be rewritten against master; can be closed.

Based on ipfs#1826 License: MIT Signed-off-by: rht <[email protected]>

tv42 and others added 30 commits September 11, 2015 10:09

pin: Guard against callers causing refcount underflow

80bcb16

This used to lead to large refcount numbers, causing Flush to create a lot of IPFS objects, and merkledag to consume tens of gigabytes of RAM. License: MIT Signed-off-by: Jeromy <[email protected]>

sharness: fusermount -u is the documented way to unmount FUSE on Linux

99ea9d4

License: MIT Signed-off-by: Jeromy <[email protected]>

pin: unexport NewIndirectPin, it's not useful and not used elsewhere

2a9cf2f

License: MIT Signed-off-by: Jeromy <[email protected]>

blocks/set: Remove now-unused NewDBWrapperSet

763553d

License: MIT Signed-off-by: Jeromy <[email protected]>

Simplify Pinner interface by folding ManualPinner into Pinner

c9872d0

Pinner had method GetManual that returned a ManualPinner, so every Pinner had to implement ManualPinner anyway. License: MIT Signed-off-by: Jeromy <[email protected]>

pin: Remove dead code

da6e8a6

License: MIT Signed-off-by: Jeromy <[email protected]>

pin: Remove double bookkeeping of refcount keys

6fc2fb9

License: MIT Signed-off-by: Jeromy <[email protected]>

Use uint64 for indirect pin refcounts

ad4e305

Platform-dependent behavior is not nice, and negative refcounts are not very useful. License: MIT Signed-off-by: Jeromy <[email protected]>

Typo

778ecf8

License: MIT Signed-off-by: Jeromy <[email protected]>

sharness: Don't assume we know all things that can create garbage

7eaebfa

License: MIT Signed-off-by: Jeromy <[email protected]> sharness: Don't assume we know all things that can create garbage License: MIT Signed-off-by: Jeromy <[email protected]>

pin: Future-proof against refcount marshaled size changes

bbacfe1

License: MIT Signed-off-by: Jeromy <[email protected]>

bump fsrepo version to 3

2884244

License: MIT Signed-off-by: Jeromy <[email protected]>

pin: Do not accidentally delete indirect pins on Flush

c3732b9

License: MIT Signed-off-by: Jeromy <[email protected]>

dagmodifier: Don't lose pin if old and new key happen to be equal

16bf686

License: MIT Signed-off-by: Jeromy <[email protected]>

dagmodifier test: Add TODO note about how bad luck can cause test fai…

c47432f

…lure License: MIT Signed-off-by: Jeromy <[email protected]>

buffer msgio

2f3f582

License: MIT Signed-off-by: Jeromy <[email protected]>

address concerns from PR

f415763

License: MIT Signed-off-by: Jeromy <[email protected]>

move locking out of GC branch

cae166e

License: MIT Signed-off-by: Jeromy <[email protected]>

lock blockstore for pin add

55b5ab8

License: MIT Signed-off-by: Jeromy <[email protected]>

renamed {R,}Lock -> {Pin,GC}Lock

a85f58f

License: MIT Signed-off-by: Juan Batiz-Benet <[email protected]>

break up GC logic

7d8f458

License: MIT Signed-off-by: Jeromy <[email protected]>

addressing comments from CR

b3deca0

License: MIT Signed-off-by: Jeromy <[email protected]>

whyrusleeping reviewed Oct 11, 2015
View reviewed changes

update text on help commands

a42daf2

License: MIT Signed-off-by: David Dias <[email protected]>

daviddias force-pushed the patch-2 branch from c1df9df to a42daf2 Compare October 11, 2015 16:23

whyrusleeping reviewed Oct 12, 2015
View reviewed changes

whyrusleeping added status/in-progress In progress and removed backlog labels Oct 20, 2015

change all mfs references to unixfs

65013ab

License: MIT Signed-off-by: David Dias <[email protected]>

whyrusleeping force-pushed the dev0.4.0 branch 3 times, most recently from 764bef9 to 1bbc472 Compare November 11, 2015 18:04

ghost added topic/docs-ipfs Topic docs-ipfs topic/files Topic files labels Dec 22, 2015

whyrusleeping force-pushed the dev0.4.0 branch 2 times, most recently from 68b9745 to b0a8591 Compare December 28, 2015 14:36

whyrusleeping force-pushed the dev0.4.0 branch from cf514d8 to 3224ae0 Compare January 12, 2016 16:23

rht closed this Jan 23, 2016

rht added a commit to rht/go-ipfs that referenced this pull request Jan 23, 2016

Change all mfs references to unixfs

e4522a1

Based on ipfs#1826 License: MIT Signed-off-by: rht <[email protected]>

rht mentioned this pull request Jan 23, 2016

Change all mfs references to unixfs #2233

Closed

rht added a commit to rht/go-ipfs that referenced this pull request Jan 23, 2016

Change all mfs references to unixfs

4ccb094

Based on ipfs#1826 License: MIT Signed-off-by: rht <[email protected]>

jbenet mentioned this pull request Jul 29, 2016

Unite the Files API 🗂 ipfs/specs#98

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update help text to avoid confusion of where files can be added from #1826

Update help text to avoid confusion of where files can be added from #1826

daviddias commented Oct 11, 2015

whyrusleeping Oct 11, 2015

GitCop commented Oct 11, 2015

jbenet commented Oct 11, 2015

GitCop commented Oct 11, 2015

daviddias commented Oct 11, 2015

whyrusleeping Oct 12, 2015

jbenet Oct 13, 2015

whyrusleeping commented Oct 20, 2015

daviddias commented Oct 20, 2015

whyrusleeping commented Oct 20, 2015

jbenet commented Oct 20, 2015

rht commented Jan 23, 2016

Update help text to avoid confusion of where files can be added from #1826

Update help text to avoid confusion of where files can be added from #1826

Conversation

daviddias commented Oct 11, 2015

whyrusleeping Oct 11, 2015

Choose a reason for hiding this comment

GitCop commented Oct 11, 2015

jbenet commented Oct 11, 2015

GitCop commented Oct 11, 2015

daviddias commented Oct 11, 2015

whyrusleeping Oct 12, 2015

Choose a reason for hiding this comment

jbenet Oct 13, 2015

Choose a reason for hiding this comment

whyrusleeping commented Oct 20, 2015

daviddias commented Oct 20, 2015

whyrusleeping commented Oct 20, 2015

jbenet commented Oct 20, 2015

rht commented Jan 23, 2016