Make sure path is in native encoding

[jennybc]

Fixes #476 CP1251 char set in file name

Fixes #370 read_excel on Windows fails to open a file from a UTF-8 encoded path containing special characters

This is a new problem seen with R 3.5, we believe as a result of measures taken in response to Bug 17120 :

In a non-UTF-8 locale (basically Windows?), path.expand() returns a UTF-8 encoded path, even for non UTF-8 input. readxl pre-processes xls paths via normalizePath(), which calls path.expand(). And then libxls uses fopen(), which expects the path in the native encoding.

We have two choices:

1. Stop using normalizePath() on xls files. It's not clear to me why we do this. This code goes back to Day 3 of readxl development, in 2015 fc076c4. The history doesn't indicate if there is a reason to use normalizePath(). I explored "stop calling normalizePath()" in 945178d, which passes tests. If we choose this, I will obviously remove more even code vs. comment it out.
2. Keep using normalizePath() on xls files, but process immediately with enc2native(). Explored in 8a8c061, which also passes tests.

Either way, I think we should consider incorporating the new test. However I decided skip_on_cran() might be wise.

[yutannihilation]

FYI: using Rf_translateChar() may be another choice. (e.g. https://github.com/tidyverse/readr/pull/838/files)

IMHO, converting Rcpp::String directly to std::string might not do the right thing, as it doesn't care about the encoding, whereas base R provides useful functions that considers the encoding (Rf_translateChar(), Rf_translateChar0() and Rf_translateCharUTF8()). But, I don't know how to do this with Rcpp properly... (I've asked on RStudio Community, but no one has answered so far)

[jimhester]

There is no Rcpp::String created in this case, the results from the R function calls are just SEXP objects and coercing to std::string should just do the equivalent of std::string(CHAR()).

[yutannihilation]

There is no Rcpp::String created in this case

Thanks for pointing out, I was wrong at this point. I meant, it might be also good if we can use the encoding attribute in C++ codes. But, as it seems difficult, just ignore me. Sorry for the noise...

[jennybc]

@hadley Do you have any memory of why xls paths are sent through normalizePath()? But xlsx are not.

@jimhester Which approach do you think I should take?

I now introduce a 3rd option:

We should normalize all paths and do enc2native() on the R side, for xls and xlsx.

Why?
I now see there was a pre-existing issue about xlsx and encoding of the path: #370
Also a new xls one that's probably this: #478

[jimhester]

I think doing enc2native(normalizePath()) on the R side for both xls and xlsx seems the best option.

[hadley]

I have no recollection of why there is a difference.

[jennybc]

I went with a different option:

• Add a test with a filename containing non-ASCII characters. Apply it to both xls and xlsx.
• Stop normalizing xls paths.
• Apply enc2native() to both xls and xlsx paths on the R side.