Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Support UnixFS mode and modification times in ipld dag and mfs #658

Merged
merged 5 commits into from
Aug 13, 2024

Conversation

gammazero
Copy link
Contributor

@gammazero gammazero commented Aug 13, 2024

Adds support for storing and retrieving file mode and last modification time.

Support added for:

  • Files
  • LinkFiles
  • Webfiles
  • Directories

Tar archives are supported by the parent branch.

Replaced PRs:

Adds support for storing and retrieving file mode and last modification time.

Support added to:

Support added to:
- Files
- LinkFiles
- Webfiles
- Directories

Tar archives are supported by the parent branch.
@gammazero gammazero requested a review from a team as a code owner August 13, 2024 02:59
@gammazero gammazero requested a review from lidel August 13, 2024 03:00
Copy link

codecov bot commented Aug 13, 2024

Codecov Report

Attention: Patch coverage is 60.00000% with 146 lines in your changes missing coverage. Please review.

Project coverage is 60.10%. Comparing base (0338750) to head (1b5bb03).

Files Patch % Lines
mfs/dir.go 5.55% 51 Missing ⚠️
ipld/unixfs/pb/unixfs.pb.go 27.77% 26 Missing ⚠️
mfs/file.go 64.61% 16 Missing and 7 partials ⚠️
mfs/ops.go 7.69% 12 Missing ⚠️
ipld/unixfs/importer/helpers/dagbuilder.go 69.69% 8 Missing and 2 partials ⚠️
ipld/unixfs/unixfs.go 91.66% 5 Missing and 3 partials ⚠️
ipld/unixfs/importer/balanced/builder.go 68.42% 4 Missing and 2 partials ⚠️
ipld/unixfs/file/unixfile.go 73.33% 3 Missing and 1 partial ⚠️
ipld/unixfs/importer/trickle/trickledag.go 72.72% 2 Missing and 1 partial ⚠️
ipld/unixfs/mod/dagmodifier.go 75.00% 2 Missing and 1 partial ⚠️

Impacted file tree graph

@@                Coverage Diff                @@
##           feat/unixfs15     #658      +/-   ##
=================================================
+ Coverage          60.03%   60.10%   +0.07%     
=================================================
  Files                240      240              
  Lines              30265    30583     +318     
=================================================
+ Hits               18170    18383     +213     
- Misses             10460    10556      +96     
- Partials            1635     1644       +9     
Files Coverage Δ
ipld/unixfs/io/dagreader.go 81.22% <100.00%> (+0.77%) ⬆️
mfs/root.go 40.50% <ø> (ø)
ipld/unixfs/importer/trickle/trickledag.go 53.97% <72.72%> (+0.38%) ⬆️
ipld/unixfs/mod/dagmodifier.go 58.94% <75.00%> (+1.00%) ⬆️
ipld/unixfs/file/unixfile.go 72.03% <73.33%> (-1.20%) ⬇️
ipld/unixfs/importer/balanced/builder.go 63.85% <68.42%> (+5.39%) ⬆️
ipld/unixfs/unixfs.go 77.99% <91.66%> (+6.16%) ⬆️
ipld/unixfs/importer/helpers/dagbuilder.go 59.90% <69.69%> (-0.45%) ⬇️
mfs/ops.go 39.39% <7.69%> (-3.09%) ⬇️
mfs/file.go 62.19% <64.61%> (+1.58%) ⬆️
... and 2 more

... and 15 files with indirect coverage changes

gammazero added a commit to ipfs/kubo that referenced this pull request Aug 13, 2024
Replaces #7754 written by @kstuart

- ipfs/boxo#653
- ipfs/boxo#658

- [X] Can `ipfs add` with preserved mode and/or last modification time
  - [X] on files
  - [X] on directories
- [X] Can `ipfs add` with custom mode and/or last modification time
  - [X] on files
  - [X] on directories
- [X] Can `ipfs get` restoring mode and/or last modification time
  - [X] on files
  - [X] on directories
  - [X] in archives
- [X] Can `ipfs files chmod` to change mode
  - [X] on files
  - [X] on directories
- [X] Can `ipfs files touch` to change last modification time
  - [X] on files
  - [X] on directories
- [X] Automatically update the last modification time when file data is changed or truncated (e.g. `ipfs files write`)
- [X] Can add files and directories with mode and/or modification time using multipart-form data
- [X] `ipfs files stat` reports mode and last modification time

**Note:**
- [X] Adds support to `kubo/core/rpc` (may require additional tests).

- ~ipfs/interface-go-ipfs-core/pull/66~ replace by this PR
- ~ipfs/go-unixfs/pull/85~ replaced by: ipfs/boxo#658
- ~ipfs/go-mfs/pull/93~ replaced by: ipfs/boxo#658
- ~ipfs/go-ipfs-files/pull/31~ replaced by: ipfs/boxo#653
- ~ipfs/tar-utils/pull/11~ replaced by: ipfs/boxo#653

- When adding files and directories without opting to store a mode or modification time the same CIDs are generated that would have been created before this feature was implemented (opt-in).
- The Go runtime currently has no native support for restoring file mode and modification time on symbolic-links, support for restoring the last modification time has been added for Linux distributions and the following BSDs: freebsd, netbsd, openbsd, dragonflybsd.
- Automatically updating a modification time will only occur if a modification time was previously stored.
- When creating an archive, for compatibility, time resolution is to the second; Nanoseconds are not supported.

The `ipfs add` options `--preserve-mode` and `--preserve-mtime` are used to store the original mode and last modified time of the file being added, the options `--mode`, `--mtime` and `--mtime-nsecs` are used to store custom values, a custom value of 0 is a no-op as is providing `--mtime-nsecs` without `--mtime`.

The preserve flags and custom options are mutually exclusive, if both are provided the custom options take precedence.

---

Closes #6920
@gammazero gammazero merged commit 1062062 into feat/unixfs15 Aug 13, 2024
15 checks passed
@gammazero gammazero deleted the feat/unixfs-mode-mtime-in-dag-and-files branch August 14, 2024 00:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant