Skip to content
This repository has been archived by the owner on Feb 12, 2024. It is now read-only.

track: Files API + MFS #60

Closed
6 of 14 tasks
daviddias opened this issue Jan 26, 2016 · 11 comments
Closed
6 of 14 tasks

track: Files API + MFS #60

daviddias opened this issue Jan 26, 2016 · 11 comments
Labels
exp/expert Having worked on the specific codebase is important help wanted Seeking public contribution on this issue status/deferred Conscious decision to pause or backlog

Comments

@daviddias
Copy link
Member

daviddias commented Jan 26, 2016

The files API offers an interface to work with unixfs, a Merklelized Data Structure that represents a typical FileSystem used in a Unix OS. with files, dirs and symlinks. This API is what enables IPFS to add and retrieve files without having to ask the users to convert their data into a graph data structure.

To learn more about the Files API, you can

Implementation roadmap

  • Files API spec Files API spec specs#88
  • Unixfs Spec
  • jsipfs files add (current ipfs add)
  • jsipfs files cat (current ipfs cat)
    • cat subfiles
  • jsipfs files get (current ipfs get)
  • jsipfs files mv - Move files.
  • jsipfs files cp - Copy files into mfs.
  • jsipfs files ls - List directories.
  • jsipfs files mkdir - Make directories.
  • jsipfs files stat - Display file status.
  • jsipfs files rm ... - Remove a file.
  • jsipfs files read - Read a file in a given mfs.
  • jsipfs files write - Write to a mutable file in a given filesystem.
@daviddias daviddias self-assigned this Jan 26, 2016
@daviddias daviddias added exp/expert Having worked on the specific codebase is important feature labels Feb 23, 2016
@daviddias daviddias changed the title feature: files files Feb 23, 2016
@daviddias
Copy link
Member Author

Updated this issue

@nginnever
Copy link
Member

jsipfs files get

Taking a note from @noffle and a stab at jsipfs files get

Relevant go-ipfs links:
https://github.com/ipfs/go-ipfs/blob/master/core/commands/get.go

ARGUMENTS:

<ipfs-path> - The path to the IPFS object(s) to be outputted

OPTIONS:

-o, --output            string - The path where output should be stored
-a, --archive           bool   - Output a TAR archive
-C, --compress          bool   - Compress the output with GZIP compression
-l, --compression-level int    - The level of compression (1-9)

STEPS

1. Get the gzip compression options, bail if err (TODO: Look into js gzip modules) - return NoCompression if compress bool is false - return DefaultCompression if compress bool is true but compress level is undefined - return NoCompression and error if bool is true and compress level is not between 1-9 - return compress level if none of the above conditions are met 2. Get the node from the command request context. 3. Get the path from the argument . - TODO: List what happens here 4. Resolve the path, bail if err 5. Set archive to the option --archive provided. 6. DagArchive (this is equivalent to `ipfs getdag $hash | maybe_tar | maybe_gzip`) - TODO: list what happens here 7. create a readable stream of the dagnode 8. Check the stream bail if error 9. Check the path option. If it does not exist, defaut path to the multihash 10. Get the compression level from options 11. Get the archive boolean from options 12. Pass the readable stream and path to a write() function - If archive or compression options != gzip.NoCompress then pass to writeArchive() function - else pass the stream and path to writeExtracted() function - **If writeArchive()** - check archive is set to true - if it is, check if the ending of the path is `path + '.tar'` or `path + '.tar.gz'` - if it does not then add .tar to path name - adjust path name to .gz if compression level != gzip.NoCompress - create the file on disk - print to console `Saving archive to $path` - show a progress bar - **if writeExtracted()** - print to console `Saving file(s) to $path` - show progress bar - TODO: figure out what extractor is

@nginnever
Copy link
Member

jsipfs files cat

A look at jsipfs files cat

Relevant go-ipfs links:
https://github.com/ipfs/go-ipfs/blob/master/core/commands/cat.go
https://github.com/ipfs/go-ipfs/blob/master/core/coreunix/cat.go

ARGUMENTS:

<ipfs-path>... - The path to the IPFS object(s) to be outputted

DESCRIPTION:

Retrieves the object named by <ipfs-or-ipns-path> and outputs the data
it contains.

STEPS

1. Get the node from the command request context. 2. If node.onlineMode = false, setupOfflineRouting() - TODO: List setupOfflineRouting steps 3. call cat() function passing in the context, the node, and the path(s) from the arguments - create read streams array for each path provided - for each path, call coreunix.Cat(ctx, node, fpath) - resolve the dagnode - return a readable stream - append the stream from resolve to the read streams array - collect the total length by adding each buffer length returned from resolve - return the readers array, length, and null or error 4. Do a conditional GC 5. set the length in the response 6. combine the readers streams into one read stream 7. check if the length < the smallest progress bar size 8. create the progress bar if not 9. set the res to the read stream.

@daviddias daviddias changed the title files files / add Mar 21, 2016
@daviddias
Copy link
Member Author

merged #67

@daviddias daviddias removed their assignment Mar 21, 2016
@daviddias daviddias changed the title files / add files API Mar 30, 2016
@daviddias daviddias modified the milestone: milestone 1 - js-ipfs on the browser Apr 4, 2016
@daviddias daviddias added the help wanted Seeking public contribution on this issue label Apr 4, 2016
@daviddias
Copy link
Member Author

@nginnever I know you are working on this. Can you make a PR tagged with WIP from a branch under this project?

@daviddias
Copy link
Member Author

-> #156

@daviddias daviddias removed this from the milestone 1 - js-ipfs on the browser milestone Jul 27, 2016
@daviddias daviddias changed the title files API track: jsipfs files -The files + mfs API Aug 15, 2016
@daviddias daviddias changed the title track: jsipfs files -The files + mfs API track: jsipfs files - The files + mfs API Aug 15, 2016
@daviddias
Copy link
Member Author

The remaining items (specific to mfs) go into the Milestone 6 - Feature parity

@daviddias
Copy link
Member Author

//cc @victorbjelkholm

@daviddias daviddias mentioned this issue Dec 5, 2016
@daviddias daviddias added the status/ready Ready to be worked label Dec 5, 2016
@victorb victorb mentioned this issue Dec 28, 2016
@daviddias daviddias changed the title track: jsipfs files - The files + mfs API track: Files API + MFS Jan 9, 2017
@daviddias
Copy link
Member Author

Now Files Add work as expected with the option to do trickle-dag too! #707

@daviddias
Copy link
Member Author

cat/get subtrees is now there too :)

@daviddias
Copy link
Member Author

Now tracked here https://github.com/ipfs/ipfs/blob/master/IMPLEMENTATION_STATUS.md

tl;dr; Files is done, MFS isn't

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
exp/expert Having worked on the specific codebase is important help wanted Seeking public contribution on this issue status/deferred Conscious decision to pause or backlog
Projects
None yet
Development

No branches or pull requests

3 participants