Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[WIP] Filestore Implementation #2634

Closed
wants to merge 209 commits into from
Closed

[WIP] Filestore Implementation #2634

wants to merge 209 commits into from

Commits on Jun 2, 2016

  1. In the measure package don't return ErrInvalidType in batch Put. 

    None of the other methods in the measure package return this error,
instead they only call RecordValue() when the value is []byte.  This
change makes batch Put consistent with the other methods and allows
non []byte data to be passed though the measure datastore.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    929b49f View commit details
    Browse the repository at this point in the history

  2. New "multi" datastore. 

    The datastore has an optional "advanced" datastore that handles Put
requests for non []byte values, a "normal" datastore that handles all
other put requests, and then any number of other datastore, some of
them that can be designated read-only.  Delete requests are passed on
to all datastore not designed read-only.

For now, querying will only work on a "normal" datastore.

Note: Only tested in the case of just a "normal" datastore and the
case of an "advanced" and "normal" datastore.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    d8c1ccf View commit details
    Browse the repository at this point in the history

  3. Create a new AdvReader interface. 

    Create a new AdvReader interface that returns an ExtraInfo object that
contains extra information about the the filehandle.  This includes
the current offset and the absolute path of a file.

Also modify chunk.Splitter to return a Bytes struct in the NextBytes()
method that in addition to the raw bytes returns the ExtraInfo object
from the AdvReader that contains the current offset in the file
(amoung other information).

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    a4ffdaf View commit details
    Browse the repository at this point in the history

  4. Basic implementation of "add --no-copy". 

    This involved:

1) Maintaing a DataPtr in merkledag.Node and blocks.Block.  The
DataPtr points to the location of the data within a file on the file
system.  It the node is a leaf it also contains an alternative
serialization of the Node or Block that does not contain the data.

3) A new datastore "filestore" that stores just the information in
DataPtr.  When retrieving blocks the Merkle-DAG node is reconstructed
from combining AltData with the data from the file in the file system.
Because the datastore needs to reconstruct the node it needs access to
the Protocol Buffers for "merkledag" and "unixfs" and thus, for now,
lives in go-ipfs instead of go-datastore.

The filestore uses another datastore to store the protocol buffer
encoded DataPtr.  By default this is the leveldb datastore, as the size
for the encoded DataPtr is small.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    f4ad3ba View commit details
    Browse the repository at this point in the history

  5. Add basic tests for "add --no-copy". 

    Note that as per issue ipfs#2259 content is not verified for local
retrieval so changing the file content will not always be detected.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    28d18ce View commit details
    Browse the repository at this point in the history

  6. When reconstructing block always verify the result. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    0f3f2e9 View commit details
    Browse the repository at this point in the history

  7. Add Basic Query and "Direct" commands to filestore. Needs Testing. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    7f1bf41 View commit details
    Browse the repository at this point in the history

  8. Save ref to Filestore in repo for future direct access. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    cc30fa6 View commit details
    Browse the repository at this point in the history

  9. Add Self() method to be able to get to FSRepo. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    be4649b View commit details
    Browse the repository at this point in the history

  10. Add "filestore" commands to list contents and verify filestore. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    7853241 View commit details
    Browse the repository at this point in the history

  11. Disable failing test. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    4fc46db View commit details
    Browse the repository at this point in the history

  12. Refactor. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    725fb99 View commit details
    Browse the repository at this point in the history

  13. Add temp. utility command to find dangling pins. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    5e1897f View commit details
    Browse the repository at this point in the history

  14. Add "filestore rm" command. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    cc65b61 View commit details
    Browse the repository at this point in the history

  15. "filestore ls": add "--quiet" option 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    4a2807f View commit details
    Browse the repository at this point in the history

  16. "filestore verify": change "invalid" status to "changed". 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    a7a2e55 View commit details
    Browse the repository at this point in the history

  17. Check if the WholeFile flag should be set for leaf nodes in filestore… 

    ….Put.

If the dataObj.Offset is zero and the WholeFile flag is not set, check
if it should be.  That is check if the file size is the same as
dataObj.Size, and if it is, set dataObj.WholeFile.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    889f470 View commit details
    Browse the repository at this point in the history

  18. Add "filestore rm-invalid" command. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    d028645 View commit details
    Browse the repository at this point in the history

  19. "filestore ls": add help text and rework output 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    958e344 View commit details
    Browse the repository at this point in the history

  20. "filestore verify": only verify leaf nodes, add help text 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    07a292c View commit details
    Browse the repository at this point in the history

  21. Add sharness tests for new "filestore" commands. 

    Add tests for:
  filestore ls
  filestore verify
  filestore rm-invalid
  filestore rm

Also rename t0046-add-no-copy.sh to t0260-filestore.sh.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    b0a6468 View commit details
    Browse the repository at this point in the history

  22. Report an error when trying to use "--no-copy" when daemon is online. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    a2610b2 View commit details
    Browse the repository at this point in the history

  23. Fix bug in LevelDB datastore's Delete() method. 

    LevelDB Delete() method will not return ErrNotFound so check that the key
exists first by calling Has() and return ds.ErrNotFound if Has() returns
false.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    1203021 View commit details
    Browse the repository at this point in the history

  24. In the filestore, reconstruct the block directly when retrieving. 

    Use the new function reconstructDirect that will reconstruct the block
directly without any intermediate data structures and without
performing any unnecessary copies of the block data from the file.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    b5a89f1 View commit details
    Browse the repository at this point in the history

  25. Refactor filestore utility commands. 

    Move most of the code from core/command/filestore.go into
filestore/util.go.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    caf89fb View commit details
    Browse the repository at this point in the history

  26. Enhance "filestore rm" to give better output. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    5e20d47 View commit details
    Browse the repository at this point in the history

  27. Filestore: Change DataObj struct to use bit flags. 

    This will simplify adding new flags.

Also modify the related Protocol Buffer.  The old format will still be
recognized for now.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    92170c9 View commit details
    Browse the repository at this point in the history

  28. Filestore: Only verify contents of block if the file's mod-time has c… 

    …hanged.

In DataObj add two fields: the file's modification time, and a flag to
mark the content as invalid.  If the modification time of the file is
the same as the one in the DataObj, skip validation and return the
cached value of the invalid flag.  If the modification time differs
than recheck the hash of the block and update the modification time
and invalid flag accordingly.

With this change retrieval of blocks from the filestore is as fast as
retrieving a block from the normal datastore.

Note that when a block is first added the modification time is unset
so the first retrial will be slower since the contents need to be
verified.  After the first retrieval no additional verification needs
to be done until the file changed.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    8f136fd View commit details
    Browse the repository at this point in the history

  29. Filestore: Get the modification time from the file before it is first… 

    … read.

When adding a file to the filestore get the modification time of a
file before we read any data from it and use that time to set the
initial timestamp in the DataObj.  This avoids the need for verifying a
file when it is retrieved for the first time.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    c7f9757 View commit details
    Browse the repository at this point in the history

  30. Move filestore utility commands into their own package. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    e649788 View commit details
    Browse the repository at this point in the history

  31. Redo "filestore verify" to check more than if blocks can be read. 

    In addition to checking if blocks can be read also check for other
problems.  (1) For each block representing a file root, check that all
the blocks in the merkledag tree can be read and the contents of the
file can be reconstructed.  (2) Check for orphan nodes, those are
nodes that are a child of another node that was not found in the
filestore. (3) Check if a file was appended too as the block is still
valid but no longer represents the contents of the file.

The "missing" status, which was used indicate that the file is not
found, is now "no-file" and "missing" is now used to indicate when a
block could not be found in the datastore.

The original verify behavior is still available by using the "--basic"
option.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    85cae36 View commit details
    Browse the repository at this point in the history

  32. New command "filestore clean", replaces obsolete "... rm-invalid" com… 

    …mand.

The new "filestore clean" uses the output of "filestore verify" to
remove invalid or useless blocks.  The new command can now also removes
incomplete or orphan nodes.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    ebca489 View commit details
    Browse the repository at this point in the history

  33. Add "filestore fix-pins", remove obsolete "filestore find-dangling-pi… 

    …ns".

The new command will repair broken pins by either removing or
reconstructing the pin so that invalid blocks are not pinned.  If the
pin points to a block that no longer exists it is simply removed.  If
the pin is a recursive pin that points to a valid block but some of
the nodes of the merkle tree are invalid the recursive pin is
converted to a direct pin and new recursive pins are created for the
nodes that are still valid.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    d9f3844 View commit details
    Browse the repository at this point in the history

  34. New command: "filestore repin". 

    The new command pins any un-pined nodes in a filestore that represent
whole files.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    38eb47c View commit details
    Browse the repository at this point in the history

  35. Tweak help text for filestore commands. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    43bb5f4 View commit details
    Browse the repository at this point in the history

  36. Filestore: Refactor 

    Refactor some of the functionality of Repin into a new walkPins function.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    1f2fda7 View commit details
    Browse the repository at this point in the history

  37. Rewrite "filestore rm" for greater functionality and to correctly han… 

    …dle pines.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    9022879 View commit details
    Browse the repository at this point in the history

  38. Change "repin" to "unpinned" and just list the unpinned objects. 

    Just listing the unpinned objects is more flexible.  For the old
functionally just use something like:
  ipfs filestore unpinned | xargs ipfs pin add

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    f08ea6b View commit details
    Browse the repository at this point in the history

  39. "filestore rm": Avoid checking for indirect pins when "--force" is used. 

    If "--force" is used than indirect pins are ignored so save time and
don't check for them in the first place.  This will also produce
cleaner output when using "filestore clean" as it basically calls
"filestore rm --force --direct".

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    6f97efd View commit details
    Browse the repository at this point in the history

  40. New command "filestore rm-dups" 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    7db64c4 View commit details
    Browse the repository at this point in the history

  41. Filestore: Todo list. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    ee9d941 View commit details
    Browse the repository at this point in the history

  42. Enhance filestore testcases. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    70f4453 View commit details
    Browse the repository at this point in the history

  43. "filestore repin": Add --skip-root option. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    53f499f View commit details
    Browse the repository at this point in the history

  44. Filestore: Add README. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    df10f8e View commit details
    Browse the repository at this point in the history

  45. Fix a compile error in a test case. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    906320f View commit details
    Browse the repository at this point in the history

  46. Do not use separate absolute Path, just make the FullPath absolute. 

    In the various components of the "files" package do not store a
separate absolute path, just make the FullPath absolute when possible.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    f9896fa View commit details
    Browse the repository at this point in the history

  47. Add way to add files on the API server's filesystem without sending t… 

    …he data.

If the option API.ServerSideAdds is enabled than the new command
"add-ss" will add files to ipfs by sending the filename, rather than
the contents of the file, to the API server.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    c2df48e View commit details
    Browse the repository at this point in the history

  48. Filestore: Refactor test cases. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    f37c326 View commit details
    Browse the repository at this point in the history

  49. Filestore: Update README and add testcase using new add-ss functionally 

    With the new "add-ss" command it is now possible to add files to the
filestore with the daemon online.  Update README to reflect this
fact and create some new test cases to test the new functionally.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    7991036 View commit details
    Browse the repository at this point in the history

  50. Filestore: Refactor fsutil.List 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    4313c4f View commit details
    Browse the repository at this point in the history

  51. "filestore rm": Do not delete blocks that are shared with another pin… 

    …ned node.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    2b09cb4 View commit details
    Browse the repository at this point in the history

  52. New command "filestore ls-files" 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    d97b42e View commit details
    Browse the repository at this point in the history

  53. Add Filestore.Verify config option to control when block are verified. 

    The default value of "ifchanged" and blocks are only verified when
the backing file has changed.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    1ded141 View commit details
    Browse the repository at this point in the history

  54. Filestore: Remove todo list. 

    Issues now on https://github.com/ipfs-filestore/go-ipfs/issues

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    9f689fa View commit details
    Browse the repository at this point in the history

  55. Gofmt 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    f40117f View commit details
    Browse the repository at this point in the history

  56. Filestore: Expand tests. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    ff4ee54 View commit details
    Browse the repository at this point in the history

  57. Create an alternative blockstore for use with the filestore. 

    Restore blockstore, write_cache to original unmodified versions.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    af94888 View commit details
    Browse the repository at this point in the history

  58. Factor out filestore specific code from DagServices. 

    This also eliminate the need for AddOpts and ExtraInfo interface.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    3d3b544 View commit details
    Browse the repository at this point in the history

  59. Keep track of offsets in the DAG builder and not the splitter. 

    This completely elements the need for AdvReader and cleans up a lot of
other code.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    6153f43 View commit details
    Browse the repository at this point in the history

  60. Filestore: Change FileRoot constant to Internal 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    c0fe98f View commit details
    Browse the repository at this point in the history

  61. Simplify "multi" datastore. Move logic into filestore's blockstore. 

    The datastore now only sends Put and Delete requests to the first
datastore.  Adding to other datastores is done directly.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    72fb3c6 View commit details
    Browse the repository at this point in the history

  62. In the fsrepo, provide ways to get sub-datastore by name. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    976635a View commit details
    Browse the repository at this point in the history

  63. Gofmt and clean up diff noise. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    47962cc View commit details
    Browse the repository at this point in the history

  64. New command "filestore upgrade" 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    c82a9cf View commit details
    Browse the repository at this point in the history

  65. New command: "filestore mv" 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    951df45 View commit details
    Browse the repository at this point in the history

  66. Filestore: Minor Refactor. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    490026c View commit details
    Browse the repository at this point in the history

  67. "filestore ls": enhance 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    d168f04 View commit details
    Browse the repository at this point in the history

  68. Filestore: res.Request() => req 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    d44c892 View commit details
    Browse the repository at this point in the history

  69. "filestore verify": add ability to specify hashes to verify 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 2, 2016
    Configuration menu
    Copy the full SHA
    2fe38b3 View commit details
    Browse the repository at this point in the history

Commits on Jun 3, 2016

  1. Filestore: Prevent server crash when DataPtr.PosInfo is not set. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 3, 2016
    Configuration menu
    Copy the full SHA
    fd077d3 View commit details
    Browse the repository at this point in the history

Commits on Jun 6, 2016

  1. Fix multi-file add so it works as expected. 

    If neither "-w" or "-r" is specified don't use an mfs-root.  Add and
pin each file separately.

Towards ipfs#2811

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 6, 2016
    Configuration menu
    Copy the full SHA
    293e848 View commit details
    Browse the repository at this point in the history

  2. Filestore: Cleanup. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 6, 2016
    Configuration menu
    Copy the full SHA
    aafc61d View commit details
    Browse the repository at this point in the history

  3. Filestore: change "add --no-copy" to "filestore add" 

    Also change "add-ss --no-copy" to "filestore add-ss"  and remove "add-ss".

The API.ServerSideAdds option is now Filestore.APIServerSidePaths.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 6, 2016
    Configuration menu
    Copy the full SHA
    b643748 View commit details
    Browse the repository at this point in the history

  4. Rework "filestore add" command. Absolute paths now always required. 

    Rework "filestore add" command to now always accept string arguments for
better control on how filenames are handled and eliminate the need
for a separate "add-ss" command.

For now absolute paths are always required.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 6, 2016
    Configuration menu
    Copy the full SHA
    0cdf270 View commit details
    Browse the repository at this point in the history

  5. Handle errors returned by the splitter in the DAG builder. 

    Implements: "// TODO: handle err (which wasn't handled either when
the splitter was channeled".

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 6, 2016
    Configuration menu
    Copy the full SHA
    30a11ce View commit details
    Browse the repository at this point in the history

  6. Filestore: Add support for "porcelain" output. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 6, 2016
    Configuration menu
    Copy the full SHA
    d4f1638 View commit details
    Browse the repository at this point in the history

  7. Filestore: Add another '-v' level to "filestore verify". 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 6, 2016
    Configuration menu
    Copy the full SHA
    07376d8 View commit details
    Browse the repository at this point in the history

  8. Filestore: Allow adds when server is online. 

    Directory adds not supported yet.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 6, 2016
    Configuration menu
    Copy the full SHA
    7ddf150 View commit details
    Browse the repository at this point in the history

  9. Filestore: Don't Store Empty Files in Filestore 

    Empty files don't have a DataPtr for some reason and thus will cause an
error when added to the filestore.  So, for now just don't add them to
the filestore.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 6, 2016
    Configuration menu
    Copy the full SHA
    3676047 View commit details
    Browse the repository at this point in the history

  10. Filestore: new func CleanPath like filepath.Clean but leaves '..' alone 

    Use the new function to clean the path before adding it to the filestore.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 6, 2016
    Configuration menu
    Copy the full SHA
    4827fa7 View commit details
    Browse the repository at this point in the history

  11. Filestore: Readd support for realtive directories with '-P' and '-l' … 

    …options.

Add the '-P' (--physical) and '-l' (--logical) options to filestore
add to create absolute directories from relative ones.  The '-P'
options uses the physical (no symbolic links) location of the current
directory and the '-l' option uses the PWD env. variable if it is
available and if it names the current directory.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 6, 2016
    Configuration menu
    Copy the full SHA
    c449829 View commit details
    Browse the repository at this point in the history

  12. Filestore: Update README. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 6, 2016
    Configuration menu
    Copy the full SHA
    e69a33a View commit details
    Browse the repository at this point in the history

  13. Filestore: Add example script. Update README. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 6, 2016
    Configuration menu
    Copy the full SHA
    6eb4f0d View commit details
    Browse the repository at this point in the history

Commits on Jun 9, 2016

  1. Filestore: Use filepath.IsAbs() not path.IsAbs(). 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 9, 2016
    Configuration menu
    Copy the full SHA
    53cb291 View commit details
    Browse the repository at this point in the history

  2. Filestore: Fix -P to work as expected on Windows. 

    Requires some windows specific stubs.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 9, 2016
    Configuration menu
    Copy the full SHA
    ea3716f View commit details
    Browse the repository at this point in the history

  3. Filestore: Fix add-dir.sh script 

    (Accidental deleted some lines before commit.)

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 9, 2016
    Configuration menu
    Copy the full SHA
    26416a4 View commit details
    Browse the repository at this point in the history

  4. Filestore: Update README 

    Update README to reflect the fact that the filestore has now been tested
on Windows.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 9, 2016
    Configuration menu
    Copy the full SHA
    2bf9260 View commit details
    Browse the repository at this point in the history

Commits on Jun 12, 2016

  1. Filestore: Make sure to explicitly close files to avoid a file handle… 

    … leak.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 12, 2016
    Configuration menu
    Copy the full SHA
    0106446 View commit details
    Browse the repository at this point in the history

Commits on Jun 13, 2016

  1. Move multi-datastore out of go-datastore. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 13, 2016
    Configuration menu
    Copy the full SHA
    629803e View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'upstream/master' into kevina/filestore 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 13, 2016
    Configuration menu
    Copy the full SHA
    d3bf502 View commit details
    Browse the repository at this point in the history

  3. Filestore: Eliminate need for DataPtr 

    Eliminate the need for DataPtr by reconstruct the the Node before
sending it to the Filestore.

This eliminates the need for some bookkeeping at the cost of doing
some unnecessary work further down.

Informal test show the performance is about the same.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 13, 2016
    Configuration menu
    Copy the full SHA
    d5ef180 View commit details
    Browse the repository at this point in the history

  4. Filestore: Generalize Reconstruct() function 

    Generalize Reconstruct() function so that either the reference or the
optimized version of reconstruct can be used interchangeably.  For now
the reference implementation is used as the optimized version does not
work when blockDataSize == 0.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 13, 2016
    Configuration menu
    Copy the full SHA
    b454ba0 View commit details
    Browse the repository at this point in the history

  5. Filestore: Rewrite Optimized version of Reconstruct(). 

    The old version could only handle adding the data component to a
protocol buffer encoded node.  The new version adds the ability to
also remove the data component.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 13, 2016
    Configuration menu
    Copy the full SHA
    7a64523 View commit details
    Browse the repository at this point in the history

  6. Store PosInfo in BasicBlock, eliminate NodeToBlock hook in DAGService. 

    This also eliminates the need for FilestoreBlock

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 13, 2016
    Configuration menu
    Copy the full SHA
    1135469 View commit details
    Browse the repository at this point in the history

  7. Diff noise cleanup. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 13, 2016
    Configuration menu
    Copy the full SHA
    e976430 View commit details
    Browse the repository at this point in the history

Commits on Jun 14, 2016

  1. Filestore: Cleanup. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 14, 2016
    Configuration menu
    Copy the full SHA
    6ff54cf View commit details
    Browse the repository at this point in the history

  2. Filestore: Cleanup 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 14, 2016
    Configuration menu
    Copy the full SHA
    b5c3561 View commit details
    Browse the repository at this point in the history

  3. Filestore: Simplify the specialized Blockstore 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 14, 2016
    Configuration menu
    Copy the full SHA
    382e1a6 View commit details
    Browse the repository at this point in the history

Commits on Jun 17, 2016

  1. Filestore: Clean up Errors and Command Line help text. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 17, 2016
    Configuration menu
    Copy the full SHA
    02a6f2a View commit details
    Browse the repository at this point in the history

Commits on Jun 18, 2016

  1. Filestore: More Error message cleanup. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 18, 2016
    Configuration menu
    Copy the full SHA
    cdb8ea5 View commit details
    Browse the repository at this point in the history

  2. Filestore: Clean up Errors and documentation. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 18, 2016
    Configuration menu
    Copy the full SHA
    a9cc988 View commit details
    Browse the repository at this point in the history

  3. Filestore: Remove spurious ".EnableStdin()." 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 18, 2016
    Configuration menu
    Copy the full SHA
    36a9948 View commit details
    Browse the repository at this point in the history

Commits on Jun 19, 2016

  1. Filestore: Use LevelDB directly. 

    Rather than working on top of another generic Datastore, use LevelDB
directly.  This increase the speed of "ipfs filestore ls" by an order
of magnitude.

The datastore Query() method adds a lot unnecessary overhead, by
allowing filestore_util.List direct access to the database its speed
is increased by an order of magnitude when used with a filter function
to only list file roots.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Jun 19, 2016
    Configuration menu
    Copy the full SHA
    09e3da1 View commit details
    Browse the repository at this point in the history

Commits on Jul 6, 2016

  1. Filestore: Documentation: Change README 

    License: MIT
Signed-off-by: Dominic Della Valle <[email protected]>
    @djdv @kevina
    djdv authored and kevina committed Jul 6, 2016
    Configuration menu
    Copy the full SHA
    a861ad7 View commit details
    Browse the repository at this point in the history

Commits on Aug 4, 2016

  1. Merge remote-tracking branch 'upstream/master' into kevina/filestore 

    Conflicts:
	commands/cli/parse.go
	core/coreunix/add.go
	importer/helpers/helpers.go
	repo/fsrepo/defaultds.go

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Aug 4, 2016
    Configuration menu
    Copy the full SHA
    f82b39e View commit details
    Browse the repository at this point in the history

Commits on Aug 5, 2016

  1. Filestore: Enhance "filestore upgrade" to convert keys to base32. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Aug 5, 2016
    Configuration menu
    Copy the full SHA
    3c3f02c View commit details
    Browse the repository at this point in the history

Commits on Aug 19, 2016

  1. Merge remote-tracking branch 'upstream/master' into kevina/filestore 

    Conflicts:
	blocks/blocks.go

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Aug 19, 2016
    Configuration menu
    Copy the full SHA
    3ffc5f1 View commit details
    Browse the repository at this point in the history

Commits on Aug 20, 2016

  1. Replace the multi-datastore with a multi-blockstore. 

    Instead of using a multi-datastore, mount each datastore under a
different mount point and use a multi-blockstore to check each mount
point for the block.

The first mount checked of the multi-blockstore is considered the
"cache", all others are considered read-only.  This implies that the
garbage collector only removes block from the first mount.

This change also factors out the pinlock from the blockstore into its
own structure.  Only the multi-datastore now implements the
GCBlockstore interface.  In the future this could be separated out
from the blockstore completely.

For now caching is only done on the first mount, in the future this
could be reworked.  The bloom filter is the most problematic as the
read-only mounts are not necessary immutable and can be changed by
methods outside of the blockstore.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Aug 20, 2016
    Configuration menu
    Copy the full SHA
    270e402 View commit details
    Browse the repository at this point in the history

  2. Implement multiblockstore.AllKeysChan(), GC now uses the first mount. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Aug 20, 2016
    Configuration menu
    Copy the full SHA
    a482ee1 View commit details
    Browse the repository at this point in the history

  3. Add DAGService.GetLinks() method and use it in the GC and elsewhere. 

    This method will use the (also new) LinkService if it is available to
retrieving just the links for a MerkleDAG without necessary having to
retrieve the underlying block.

For now the main benefit is that the pinner will not break when a block
becomes invalid due to a change in the backing file.  This is possible
because the metadata for a block (that includes the Links) is stored
separately and thus always available even if the backing file changes.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Aug 20, 2016
    Configuration menu
    Copy the full SHA
    5a73226 View commit details
    Browse the repository at this point in the history

  4. Filestore: Don't pin by default when adding files. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Aug 20, 2016
    Configuration menu
    Copy the full SHA
    7697929 View commit details
    Browse the repository at this point in the history

Commits on Aug 21, 2016

  1. Add "block locate" command. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Aug 21, 2016
    Configuration menu
    Copy the full SHA
    a8f0c9f View commit details
    Browse the repository at this point in the history

  2. Pinner: Providing Pinned.String() method and use it in "block rm" 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Aug 21, 2016
    Configuration menu
    Copy the full SHA
    8bcc429 View commit details
    Browse the repository at this point in the history

  3. Filestore: Simplify "filestore rm" removing pinning commands. 

    Simplify "filestore rm" to always check pins (to be consistent with
"block rm") and to not attempt to remove pins (as it should no longer
be necessary as filestore objects are now pinned by default).

Also remove the filestore utility commands as they are no longer
necessary.  "filestore unpinned" is no longer needed as filestore
objects will not get garbage collected.  "filestore fix-pins" is no
longer needed as the pin check is required in "filestore rm", and
pinned invalid blocks will no longer break pinning related operations.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Aug 21, 2016
    Configuration menu
    Copy the full SHA
    c4fefd5 View commit details
    Browse the repository at this point in the history

  4. Filestore: Update README to reflect reality. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Aug 21, 2016
    Configuration menu
    Copy the full SHA
    75baf15 View commit details
    Browse the repository at this point in the history

Commits on Aug 22, 2016

  1. "blockstore rm": move core functionally into blockstore_util package 

    Also enhance remove command to remove from optionally sub-blockstore.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Aug 22, 2016
    Configuration menu
    Copy the full SHA
    33099b2 View commit details
    Browse the repository at this point in the history

  2. Filestore: Use "block rm" to remove blocks. 

    That is replace the specialized "filestore rm" with "block rm" that removes
blocks from the from the filestore mount in the blockstore.

"filestore clean" also uses the generic "block rm" thereby eliminating the
need for the special case removal code in filestore_util.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Aug 22, 2016
    Configuration menu
    Copy the full SHA
    f9acb13 View commit details
    Browse the repository at this point in the history

  3. "block/filestore rm": Allow removal of pinned but duplicate blocks. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Aug 22, 2016
    Configuration menu
    Copy the full SHA
    f68bba7 View commit details
    Browse the repository at this point in the history

  4. "add": add "--allow-dup" option. 

    This option adds a files the the primary blockstore even if the block
is in another blockstore such as the filestore.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Aug 22, 2016
    Configuration menu
    Copy the full SHA
    e9ba1fb View commit details
    Browse the repository at this point in the history

  5. Filestore: kill "rm-dups" replace it by a more flexable "dups". 

    The output of "filestore dups" can be passed into "block rm" to remove
duplicates.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Aug 22, 2016
    Configuration menu
    Copy the full SHA
    68f7240 View commit details
    Browse the repository at this point in the history

  6. Filestore: Update README. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Aug 22, 2016
    Configuration menu
    Copy the full SHA
    9542730 View commit details
    Browse the repository at this point in the history

Commits on Aug 27, 2016

  1. Filestore: Fix typos in README.md 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Aug 27, 2016
    Configuration menu
    Copy the full SHA
    5917f56 View commit details
    Browse the repository at this point in the history

  2. Filestore: Remove note about managled hashed from README. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Aug 27, 2016
    Configuration menu
    Copy the full SHA
    68ece81 View commit details
    Browse the repository at this point in the history

  3. Filestore: Enhance "filestore ls-files" command. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Aug 27, 2016
    Configuration menu
    Copy the full SHA
    ec3ac46 View commit details
    Browse the repository at this point in the history

  4. Filestore: Add (failing) tests that paths are not modified. 

    (Note failed test are marked as such)

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Aug 27, 2016
    Configuration menu
    Copy the full SHA
    0b6dc58 View commit details
    Browse the repository at this point in the history

  5. Revert "Resolve symlink if it is directly referenced in cli (ipfs#2897)" 

    This reverts commit fe7b01f.

Conflicts:
	commands/cli/parse.go

Revert "Merge pull request ipfs#3023 from ipfs/feature/eval-symlink-windows"

This reverts commit 16c5a89, reversing
changes made to 8c77ff8.

Conflicts:
	commands/cli/parse.go

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Aug 27, 2016
    Configuration menu
    Copy the full SHA
    9090771 View commit details
    Browse the repository at this point in the history

  6. Filestore: Path tests now work. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Aug 27, 2016
    Configuration menu
    Copy the full SHA
    6c99d37 View commit details
    Browse the repository at this point in the history

Commits on Aug 28, 2016

  1. Filestore: Output full paths when added. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Aug 28, 2016
    Configuration menu
    Copy the full SHA
    0c3c9b5 View commit details
    Browse the repository at this point in the history

  2. Bug fix for "Add DAGService.GetLinks() method..." 

    Bug fix for 5a73226:
  Kevin Atkinson <[email protected]>  2016-08-19 15:52:27
  Add DAGService.GetLinks() method and use it in the GC and elsewhere.

Eventually Squash me.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Aug 28, 2016
    Configuration menu
    Copy the full SHA
    8b396fd View commit details
    Browse the repository at this point in the history

  3. Filestore: Add better example script. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Aug 28, 2016
    Configuration menu
    Copy the full SHA
    d8a17f0 View commit details
    Browse the repository at this point in the history

Commits on Sep 1, 2016

  1. Filestore: Add new verification level, Refactor. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Sep 1, 2016
    Configuration menu
    Copy the full SHA
    df3f170 View commit details
    Browse the repository at this point in the history

  2. "filestore verify": adjust porcelain output. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Sep 1, 2016
    Configuration menu
    Copy the full SHA
    75e61f3 View commit details
    Browse the repository at this point in the history

  3. Filestore: Fix new example script so it is more correct. 

    That is use the new level of "ipfs verify" to find any broken objects
as a change in one backing file can break any number of other objects.

Also use access(path,R_OK) on each file before attempting to add it to
minimize the possibility of a file not being able to be added.  This
extra step can likely be removed if "ipfs filestore add" can simply
skip files it can't read rather than abort.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Sep 1, 2016
    Configuration menu
    Copy the full SHA
    12631ee View commit details
    Browse the repository at this point in the history

  4. Filestore: Minor Refactor 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Sep 1, 2016
    Configuration menu
    Copy the full SHA
    f4d6a9f View commit details
    Browse the repository at this point in the history

Commits on Sep 2, 2016

  1. Distinguish between Offline and Local Mode. 

    This fixes filestore operations when the daemon is started in offline
mode.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Sep 2, 2016
    Configuration menu
    Copy the full SHA
    b34fd9c View commit details
    Browse the repository at this point in the history

Commits on Sep 3, 2016

  1. Filestore: Store empty files in the filestore itself. 

    Requires some special casing as the actual path to the file in not
available.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Sep 3, 2016
    Configuration menu
    Copy the full SHA
    5fd8d6a View commit details
    Browse the repository at this point in the history

  2. Filestore: Minor enhancement to example script. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Sep 3, 2016
    Configuration menu
    Copy the full SHA
    cdb4951 View commit details
    Browse the repository at this point in the history

Commits on Sep 4, 2016

  1. Require filestore to be enabled with "filestore enable". 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Sep 4, 2016
    Configuration menu
    Copy the full SHA
    83f9ec8 View commit details
    Browse the repository at this point in the history

Commits on Sep 7, 2016

  1. Blockstore: Minor refactor of RmBlocks. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Sep 7, 2016
    Configuration menu
    Copy the full SHA
    5e2cac2 View commit details
    Browse the repository at this point in the history

Commits on Sep 10, 2016

  1. Adder: Don't hold the PinLock when not pinning. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Sep 10, 2016
    Configuration menu
    Copy the full SHA
    ebf86f2 View commit details
    Browse the repository at this point in the history

  2. Filestore: Bug fix when adding files in "online" mode. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Sep 10, 2016
    Configuration menu
    Copy the full SHA
    ba8060d View commit details
    Browse the repository at this point in the history

Commits on Sep 15, 2016

  1. Filestore: Fix a race condition when updating DataObj 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Sep 15, 2016
    Configuration menu
    Copy the full SHA
    8f342a7 View commit details
    Browse the repository at this point in the history

  2. Filestore: Better logic in Update() method 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Sep 15, 2016
    Configuration menu
    Copy the full SHA
    84e73c3 View commit details
    Browse the repository at this point in the history

  3. Filestore: Fix maintenance commands so they are safe to run Online 

    And do so while avoiding the need for holding onto locks.

Maintenance commands are run on a snapshot of the database that is
guaranteed to be in a consistent state.  Basically a snapshot is taken
before a file is added.

To avoid a race condition when deleting a block that has since become
valid, the value of the snapshot is compared with the current value in
the database and if they don't match then the block is not deleted.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Sep 15, 2016
    Configuration menu
    Copy the full SHA
    5cd6a93 View commit details
    Browse the repository at this point in the history

  4. Filestore: Remove OrigData from ListRes and use iterators instead 

    The recent commit "Filestore: Fix a race condition when updating
DataObj" introduced a rather serious bug as the bytes returned through
a leveldb iterator Value() method could change on the next call to
Next() and I was using that result (without making a copy) in a
buffered channel.

Fix that bug by using Iterators instead when that field is required.
Iterators are more efficient than channels anyway.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Sep 15, 2016
    Configuration menu
    Copy the full SHA
    38ad631 View commit details
    Browse the repository at this point in the history

  5. Filestore: Use iterator in VerifyFull. Other refactoring. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Sep 15, 2016
    Configuration menu
    Copy the full SHA
    e318684 View commit details
    Browse the repository at this point in the history

  6. Filestore: Rework list filtering so it also works on verify command. 

    Refactor filename filtering used by the "ls" command so that filename
filtering can also be used by the "verify" command.

Other related changes to the filter functions in filestore/util.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Sep 15, 2016
    Configuration menu
    Copy the full SHA
    602abc0 View commit details
    Browse the repository at this point in the history

  7. Filestore: Enhance add-dir.py script to work without cache file. 

    Old shell should should likely be retired.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Sep 15, 2016
    Configuration menu
    Copy the full SHA
    c75bca3 View commit details
    Browse the repository at this point in the history

  8. Filestore: Clarify verify levels, minor refactor 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Sep 15, 2016
    Configuration menu
    Copy the full SHA
    3ac9b1a View commit details
    Browse the repository at this point in the history

  9. Filestore: Remove unused function. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Sep 15, 2016
    Configuration menu
    Copy the full SHA
    5238994 View commit details
    Browse the repository at this point in the history

  10. Filestore: Redo "clean" command so it works again and related changes. 

    The orignal "clean" command worked by doing up to three separate
passes and removing problem blocks after each pass.  The three-pass
approach depended on the blocks being deleted from the previous pass.
The change introduced in "Filestore: Fix maintenance commands so they
are safe to run Online" broke this by using a single snapshot for all
passes.  This commit reworks the "verify" command so that the
three-pass approach is no longer required.

Blocks are now normally only reported "incomplete" when one the child
nodes is missing, if a child node is just unreadable the status is now
reported as "problem".  The flag "incomplete-when" can change this
behavior.  This change eliminates the need for the second (incomplete)
pass.

To eliminate the need for the thired (orphan) pass in clean, a new
"verify-post-orphan" command is created.  This command works like
"verify" but in addition checks for orphans that would be created if
"incomplete" blocks are removed.

Also add more comprehensives tests of the "clean" command to the
sharness tests to make sure the "clean" does not break again.

Other various changes to the "verify" command including adding of a
"--no-obj-info" flag that only reports the status and hash.

Also fix special "append" status so that it is reported correctly.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Sep 15, 2016
    Configuration menu
    Copy the full SHA
    4c8950e View commit details
    Browse the repository at this point in the history

  11. "filestore clean": Avoid blocking while holidng the GCLock 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Sep 15, 2016
    Configuration menu
    Copy the full SHA
    c515ca0 View commit details
    Browse the repository at this point in the history

  12. Filestore: Bug fix to example script. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Sep 15, 2016
    Configuration menu
    Copy the full SHA
    c27121b View commit details
    Browse the repository at this point in the history

  13. "block rm": Just return "error" in ProcRmOutput. 

    The RmError structure is no longer needed.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Sep 15, 2016
    Configuration menu
    Copy the full SHA
    4f6a43a View commit details
    Browse the repository at this point in the history

  14. "block rm": Document RemovedBlock, rename CheckPins to FilterPinned. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Sep 15, 2016
    Configuration menu
    Copy the full SHA
    5ab877b View commit details
    Browse the repository at this point in the history

Commits on Sep 16, 2016

  1. Filestore: Rename example script. Update README. Document Filestore.V… 

    …erify.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Sep 16, 2016
    Configuration menu
    Copy the full SHA
    9871144 View commit details
    Browse the repository at this point in the history

  2. Adder: Make sure errors from addAllAndPin make it to the client 

    Before the error would be printed by the daemon but not make it to the
client.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Sep 16, 2016
    Configuration menu
    Copy the full SHA
    15e6ec7 View commit details
    Browse the repository at this point in the history

  3. Filestore: Fail cleanly when adding symlinks with daemon running. 

    Before adding a symlink could crash the daemon.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Sep 16, 2016
    Configuration menu
    Copy the full SHA
    6857aec View commit details
    Browse the repository at this point in the history

Commits on Sep 22, 2016

  1. Merge commit '19779b3707827355f3b9d6c5fbd17e924ea42e91' into kevina/f… 

    …ilestore

Conflicts:
        core/builder.go

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Sep 22, 2016
    Configuration menu
    Copy the full SHA
    4cf2e8c View commit details
    Browse the repository at this point in the history

  2. Merge commit '8ffab98d55628cf590f363ab203da2d4ba160ba2' into kevina/f… 

    …ilestore

Conflicts:
        core/commands/add.go

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Sep 22, 2016
    Configuration menu
    Copy the full SHA
    43bdd07 View commit details
    Browse the repository at this point in the history

  3. Merge commit '8830aae9bcef257b11401b7c9e834a3aa107386a' into kevina/f… 

    …ilestore

Conflicts:
	core/builder.go

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Sep 22, 2016
    Configuration menu
    Copy the full SHA
    580fb9f View commit details
    Browse the repository at this point in the history

  4. Merge commit '531b89abc980d2a0080baa0baa0b41638e559b3a' into kevina/f… 

    …ilestore

Conflicts:
	core/commands/block.go
	repo/fsrepo/defaultds.go

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Sep 22, 2016
    Configuration menu
    Copy the full SHA
    157055b View commit details
    Browse the repository at this point in the history

  5. Redo 582e5de 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Sep 22, 2016
    Configuration menu
    Copy the full SHA
    7728d47 View commit details
    Browse the repository at this point in the history

  6. Merge branch 'blockservice-fix' into kevina/filestore 

    Conflicts:
	blocks/blockstore/blockstore.go

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Sep 22, 2016
    Configuration menu
    Copy the full SHA
    97f1ca2 View commit details
    Browse the repository at this point in the history

Commits on Sep 23, 2016

  1. Merge commit '87ecb92fceb58497268f7460ac1c877a97b056df' into blockser… 

    …vice-fix

Conflicts:
	blocks/blockstore/blockstore.go
	blockservice/blockservice.go

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Sep 23, 2016
    Configuration menu
    Copy the full SHA
    43d7d8e View commit details
    Browse the repository at this point in the history

  2. Merge branch 'blockservice-fix' into kevina/filestore 

    Conflicts:
	blocks/blocks.go
	blocks/blockstore/blockstore.go
	blocks/blockstore/util/remove.go
	core/builder.go
	core/commands/block.go
	core/commands/pin.go
	core/coreunix/add.go
	core/coreunix/add_test.go
	merkledag/merkledag.go
	merkledag/merkledag_test.go
	merkledag/node.go
	pin/gc/gc.go
	pin/pin.go
	repo/fsrepo/defaultds.go
	unixfs/mod/dagmodifier_test.go

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Sep 23, 2016
    Configuration menu
    Copy the full SHA
    9709cb5 View commit details
    Browse the repository at this point in the history

  3. Merge remote-tracking branch 'upstream/master' into kevina/filestore 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Sep 23, 2016
    Configuration menu
    Copy the full SHA
    3f6214b View commit details
    Browse the repository at this point in the history

Commits on Sep 25, 2016

  1. Run gofmt on some files, other diff noise cleanup. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Sep 25, 2016
    Configuration menu
    Copy the full SHA
    f478548 View commit details
    Browse the repository at this point in the history

Commits on Oct 14, 2016

  1. Don't use a separate LinkService for DAGService.GetLinks() 

    Instead make LinkService a part of DAGService.  The LinkService is now
simply an interface that DAGService implements.  Also provide a
GetOfflineLinkService() method that the GC uses to get an offline
instance.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Oct 14, 2016
    Configuration menu
    Copy the full SHA
    73130fe View commit details
    Browse the repository at this point in the history

  2. Fix EnumerateChildren & hasChild to take a *cid.Cid instead of []*mda… 

    …g.Link

Author: Kevin Atkinson <[email protected]>

Fix EnumerateChildren & hasChild to take a *cid.Cid instead of []*mdag.Link

Author: Jeromy Johnson <[email protected]>

make FetchGraph use a cid

pin: fix TestPinRecursiveFail

License: MIT
Signed-off-by: Jeromy <[email protected]>

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Oct 14, 2016
    Configuration menu
    Copy the full SHA
    3af102a View commit details
    Browse the repository at this point in the history

  3. Revert "Adder: Make sure errors from addAllAndPin make it to the client" 

    This reverts commit 15e6ec7.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Oct 14, 2016
    Configuration menu
    Copy the full SHA
    739e27c View commit details
    Browse the repository at this point in the history

Commits on Oct 15, 2016

  1. Revert "Redo 582e5de" 

    This reverts commit 7728d47

Undoes: bitswap: redo: don't re-provide blocks we've provided very recently

Breaks lots of filestore tests, need fixing.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Oct 15, 2016
    Configuration menu
    Copy the full SHA
    24cff3a View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'upstream/master' into kevina/filestore 

    This also involved fixing the filestore code to use Cids.

Conflicts:
	blocks/blocks.go
	blocks/blockstore/blockstore.go
	blocks/blockstore/blockstore_test.go
	core/builder.go
	core/commands/block.go
	core/commands/dht.go
	core/core.go
	importer/helpers/helpers.go
	merkledag/merkledag.go
	merkledag/node.go
	pin/gc/gc.go
	repo/fsrepo/fsrepo.go

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Oct 15, 2016
    Configuration menu
    Copy the full SHA
    a855ab3 View commit details
    Browse the repository at this point in the history

  3. Filestore: Use the new write-though block service. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Oct 15, 2016
    Configuration menu
    Copy the full SHA
    e74f619 View commit details
    Browse the repository at this point in the history

  4. Filestore: Disable config.Filestore.APIServerSidePaths for now. 

    Disable config.Filestore.APIServerSidePaths for now due to security concerns.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Oct 15, 2016
    Configuration menu
    Copy the full SHA
    241d10d View commit details
    Browse the repository at this point in the history

Commits on Oct 17, 2016

  1. Clean up diff noise. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Oct 17, 2016
    Configuration menu
    Copy the full SHA
    51f71e1 View commit details
    Browse the repository at this point in the history

  2. Clarify reason for Has() in Put() in MultiBlockstore. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Oct 17, 2016
    Configuration menu
    Copy the full SHA
    0e045c6 View commit details
    Browse the repository at this point in the history

  3. In NewBlockstoreWPrefix don't assume "" is the default value. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Oct 17, 2016
    Configuration menu
    Copy the full SHA
    5c3b48b View commit details
    Browse the repository at this point in the history

  4. Add test that PosInfo() is getting set. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Oct 17, 2016
    Configuration menu
    Copy the full SHA
    3d0f2e0 View commit details
    Browse the repository at this point in the history

  5. blockstore: use helper func to convert from Cid to DsKey and the reverse 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Oct 17, 2016
    Configuration menu
    Copy the full SHA
    ba807ea View commit details
    Browse the repository at this point in the history

  6. Adder: Disallow adding multiple directories unless "-w" is used. 

    Adding multiple directories without "-w" will not produce the expected
result, so for now it is best to disallow it.

This also added a NumFiles() method to SliceFile.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Oct 17, 2016
    Configuration menu
    Copy the full SHA
    d3f0a59 View commit details
    Browse the repository at this point in the history

  7. Adder: fixups 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Oct 17, 2016
    Configuration menu
    Copy the full SHA
    85908c2 View commit details
    Browse the repository at this point in the history

  8. Filestore: Accommodate systems without sub-second mod-times in tests 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Oct 17, 2016
    Configuration menu
    Copy the full SHA
    15fa70a View commit details
    Browse the repository at this point in the history

  9. Filestore: bug fix, go 'case' does not fall-through 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Oct 17, 2016
    Configuration menu
    Copy the full SHA
    a32f091 View commit details
    Browse the repository at this point in the history

  10. Filestore: Change the default value of Filestore.Verify to 'Always' 

    The previous default of 'IfChanged' can be unreliable on MacOS as it
does not have sub-second mod-times.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Oct 17, 2016
    Configuration menu
    Copy the full SHA
    b9ad427 View commit details
    Browse the repository at this point in the history

  11. Filestore: Verify level 6 behavior now depends on Filestore.Verify 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Oct 17, 2016
    Configuration menu
    Copy the full SHA
    db6c9e8 View commit details
    Browse the repository at this point in the history

  12. Filestore: Fix sharness test. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Oct 17, 2016
    Configuration menu
    Copy the full SHA
    97933ac View commit details
    Browse the repository at this point in the history

  13. Filestore: Fix comment. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Oct 17, 2016
    Configuration menu
    Copy the full SHA
    2c58788 View commit details
    Browse the repository at this point in the history

Commits on Nov 4, 2016

  1. Merge remote-tracking branch 'upstream/kevina/posinfo-2' into kevina/… 

    …filestore

Conflicts:
	blocks/blockstore/blockstore.go
	blocks/blockstore/blockstore_test.go
	commands/files/file.go
	core/commands/add.go
	core/coreunix/add.go
	core/coreunix/add_test.go
	importer/balanced/builder.go
	importer/helpers/dagbuilder.go
	importer/helpers/helpers.go
	merkledag/node.go

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Nov 4, 2016
    Configuration menu
    Copy the full SHA
    d283e7e View commit details
    Browse the repository at this point in the history

  2. Gofmt and other diff noise cleanup. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Nov 4, 2016
    Configuration menu
    Copy the full SHA
    85f1089 View commit details
    Browse the repository at this point in the history

  3. Filestore: Support Raw Blocks. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Nov 4, 2016
    Configuration menu
    Copy the full SHA
    9e6636d View commit details
    Browse the repository at this point in the history

  4. Merge remote-tracking branch 'upstream/master' into kevina/filestore 

    Conflicts:
	blocks/blockstore/blockstore.go
	core/builder.go
	core/core.go

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Nov 4, 2016
    Configuration menu
    Copy the full SHA
    f080bdb View commit details
    Browse the repository at this point in the history

  5. Filestore: Enable default LevelDB Compression By Default 

    It can be disabled with the Filestore.NoDDBCompression option.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Nov 4, 2016
    Configuration menu
    Copy the full SHA
    0313956 View commit details
    Browse the repository at this point in the history

  6. Filestore: Move custom version of filepath.Clean() into its own package. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Nov 4, 2016
    Configuration menu
    Copy the full SHA
    77bf0c3 View commit details
    Browse the repository at this point in the history

  7. Filestore: Use the CidToDsKey form ds-help package. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Nov 4, 2016
    Configuration menu
    Copy the full SHA
    f3d5399 View commit details
    Browse the repository at this point in the history

Commits on Nov 7, 2016

  1. Filestore: Implement Key struct for future multi DataObj per hash sup… 

    …port.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Nov 7, 2016
    Configuration menu
    Copy the full SHA
    3a2b2a0 View commit details
    Browse the repository at this point in the history

  2. Filestore: Move snapshot related code into its own file. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Nov 7, 2016
    Configuration menu
    Copy the full SHA
    b028088 View commit details
    Browse the repository at this point in the history

  3. Filestore: Remove support for old format and Upgrade command. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Nov 7, 2016
    Configuration menu
    Copy the full SHA
    c69cd67 View commit details
    Browse the repository at this point in the history

  4. Filestore: Prep work for multi DataObj per hash support. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Nov 7, 2016
    Configuration menu
    Copy the full SHA
    cdd0b69 View commit details
    Browse the repository at this point in the history

  5. Filestore: Enhance Key structure. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Nov 7, 2016
    Configuration menu
    Copy the full SHA
    b0e2d0d View commit details
    Browse the repository at this point in the history

  6. Filestore: Basic support for multiple DataObjs per hash 

    Some Filestore clean operations are broken.  Needs more tests.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Nov 7, 2016
    Configuration menu
    Copy the full SHA
    4bd48f8 View commit details
    Browse the repository at this point in the history

  7. Filestore: Fix maintenance commands. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Nov 7, 2016
    Configuration menu
    Copy the full SHA
    4da34e4 View commit details
    Browse the repository at this point in the history

  8. Filestore: Remove seperate veirfy-post-orphan command. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Nov 7, 2016
    Configuration menu
    Copy the full SHA
    6d292b8 View commit details
    Browse the repository at this point in the history

  9. Filestore: enhance "ls" and "verify" command 

    Add --full-key option to "filestore ls" command.

Add "ls" format option to "filestore verify".

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Nov 7, 2016
    Configuration menu
    Copy the full SHA
    dfc5342 View commit details
    Browse the repository at this point in the history

  10. Filestore: Enhance "rm" command. 

    Also fix bug discovered in "verify".

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Nov 7, 2016
    Configuration menu
    Copy the full SHA
    4bd3d40 View commit details
    Browse the repository at this point in the history

  11. Filestore: Documentation updates and improvements. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Nov 7, 2016
    Configuration menu
    Copy the full SHA
    12d62f4 View commit details
    Browse the repository at this point in the history

  12. Filestore "ls": Enhance "w/type" format. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Nov 7, 2016
    Configuration menu
    Copy the full SHA
    c64c0e6 View commit details
    Browse the repository at this point in the history

  13. Filestore: Improve shareness test descriptions. 

    License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Nov 7, 2016
    Configuration menu
    Copy the full SHA
    ae0f4a9 View commit details
    Browse the repository at this point in the history

  14. Revert "Filestore: Disable config.Filestore.APIServerSidePaths for now." 

    This reverts commit 241d10d.

License: MIT
Signed-off-by: Kevin Atkinson <[email protected]>
    @kevina
    kevina committed Nov 7, 2016
    Configuration menu
    Copy the full SHA
    9215a1e View commit details
    Browse the repository at this point in the history