ARROW-6622: [R] Normalize paths for filesystem API on Windows

[nealrichardson]

I fixed this in R, though I wonder if that's not enough. At a minimum, the C++ docs should note this requirement (@pitrou maybe you know the best place in the docs to add this?), and I still think it would be nice if all of this path normalization were handled in C++ (cf. https://issues.apache.org/jira/browse/ARROW-6324).

[wesm]

I think I'm with @pitrou here that introducing automatic path normalization into the C++ filesystem API might be a bit too much scope creep -- we could provide a built-in path normalization function for consumers that don't have it (e.g. Python has pathlib, R has its own function here)

[nealrichardson]

I can accept that, though we should document the C++ API's requirements accordingly. I created https://issues.apache.org/jira/browse/ARROW-6629 because I didn't see any filesystem docs at all, so nowhere to just add a note in this patch.

[nealrichardson]

This is failing, though differently than before: https://ci.appveyor.com/project/nealrichardson/arrow/builds/27536178/job/gu10af0vck7gum81#L4342

The path normalization is making absolute paths with respect to the current working directory in the R process, and that can clash with whatever the FileSystem object has as its path. I'll need to rethink the approach.

[codecov-io]

Codecov Report

❗ No coverage uploaded for pull request base (master@2c7fb24). Click here to learn what that means.
The diff coverage is 80%.

@@            Coverage Diff            @@
##             master    #5445   +/-   ##
=========================================
  Coverage          ?   77.18%           
=========================================
  Files             ?       58           
  Lines             ?     4011           
  Branches          ?        0           
=========================================
  Hits              ?     3096           
  Misses            ?      915           
  Partials          ?        0

Impacted Files	Coverage Δ
r/R/io.R	77.14% <50%> (ø)
r/R/filesystem.R	70.45% <82.6%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2c7fb24...515d710. Read the comment docs.

[nealrichardson]

This is passing and ready for review @romainfrancois

[romainfrancois]

Should we use fs:: here ?

[romainfrancois]

This looks like things potentially better fixed with fs:: cc @jimhester. I know fs has been downgraded to Suggests here, but I'm just worried we'd be missing cases with the implementation below.

[nealrichardson]

I didn't see the fs function that does the winslash normalization we need here, but more than happy to use it if it exists.

[jimhester]

You should probably use .Platform$file.sep

[jimhester]

Also not sure about the name of this function, it is not really doing anything to compute a relative path, you could probably just use normalizePath(mustWork = FALSE), assuming path doesn't actually have to exist on the filesystem.

[nealrichardson]

Re: .Platform$file.sep, I intended to but then on my Windows VM, .Platform$file.sep reported "/" while tempfile() still created paths with "\" 🤷‍♂

I originally did normalizePath but unfortunately for this use case, normalizePath does path expansion, even with mustWork=FALSE. We need to keep the path relative (to whatever the FileSystem object holds as its cwd) and just need Windows \ to be converted to /. Regular expressions were my last resort and I'm happy to use something better or more robust if it exists.

[pitrou]

It looks a bit weird to do a no-op replacement on Unix, though I suppose it doesn't work.

[pitrou]

This doesn't look correct to me. The simple solution should be to start with /-separated slashes and only use that.

[pitrou]

Are you sure? Will it work correctly if base_dir is e.g. a path in an S3 filesystem, or will it produce bogus results?

[pitrou]

Same question for all such cases below.

[pitrou]

So you should simply /-normalize td above. Should be easy :-)

[nealrichardson]

Should be!

[pitrou]

At worse you simply string-replace \ with / (but only when on Windows :-)).

[nealrichardson]

That's where I ended up: https://github.com/apache/arrow/pull/5445/files#diff-3bd936ee39b8abe149e5399681610c75R255

[pitrou]

Ah, then you shouldn't call it fs_relative_path...

[pitrou]

Also note that SubTreeFileSystem is not really needed by end-users. It's mostly useful for internal C++ testing, because there is a sizable generic filesystem test suite that expects to write at the root of an empty filesystem.

[nealrichardson]

I renamed the functions to clean_path_abs() and clean_path_rel(). Hopefully that makes their intent and output more clear.

I wouldn't be surprised if we had to revisit this in the future, but for now this seems like a step forward, so I'll merge unless there are objections.