pandoc: openFile: does not exist (No such file or directory)

[harryprince]

I am trying to server Rmd using shiny server. However, it seems rmarkdown or pandoc is buggy.
It works well on Rstudio but fails deploy on rstudio connect too.

yaml config:

output:
  flexdashboard::flex_dashboard:
    social: menu
    source: embed
    storyboard: yes

Rmarkdown version:

 packageVersion("rmarkdown")
[1] ‘1.8’

shiny-server --version
Shiny Server v1.5.3.838
Node.js v6.10.0

pandoc version

/opt/shiny-server/ext/pandoc/pandoc --version
pandoc 1.12.3
Compiled with texmath 0.6.6, highlighting-kate 0.5.6.

sessionInfo():

R version 3.3.2 (2016-10-31)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 14.04.5 LTS

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C
 [9] LC_ADDRESS=C               LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base

here is the shiny server log:

processing file: index.Rmd
output file: /tmp/rmarkdown/e01d3e5bbe48ef364e37eb0580046563/index.knit.md

pandoc: /tmp/RtmpWH829n/rmarkdown-str67a94810a85d.html: openFile: does not exist (No such file or directory)
Warning: Error in : pandoc document conversion failed with error 1
Stack trace (innermost first):
    115: pandoc_convert
    114: convert
    113: <Anonymous>
    112: do.call
    111: contextFunc
    110: .getReactiveEnvironment()$runWith
    109: shiny::maskReactiveContext
    108: <reactive>
     95: doc
     94: shiny::renderUI
     93: func
     92: force
     91: withVisible
     90: withCallingHandlers
     89: globals$domain$wrapSync
     88: promises::with_promise_domain
     87: captureStackTraces
     83: tryCatch
     82: do
     81: hybrid_chain
     80: origRenderFunc
     79: output$__reactivedoc__
      3: <Anonymous>
      2: do.call
      1: rmarkdown::run

[ADraginda]

Just FYI I'm also seeing this error (trying to use rmarkdown with flexdashboard). Using the dev version unblocked things but I'm still seeing the error.

[yihui]

@ADraginda Are you also on Shiny Server? Is it reproducible locally?

[NilsOle]

I have the same issue, but only when I run rmarkdown::render(...) in parallel on multiple cores. When I do
registerDoParallel(cores=1) it works without error. When running in parallel, out of many calls of the function, only some return errors as described here, and also not consistently (appears random at first sight).
Is that information helpful?

To reproduce, install the following R package and

1. run reproducemarkdownerror:hello(3) for 3 cores respectively. This should give you error messages.
2. reproducemarkdownerror:hello(1) should not give you any errors.

reproducermarkdownerror_0.1.0.tar.gz

Output (operation produces only 92 files although 100 expected):

> reproducermarkdownerror::hello(3)
Loading required package: foreach
Loading required package: iterators
Loading required package: parallel
pandoc: /tmp/RtmpmiRJfV/rmarkdown-str124b12f90895.html: openFile: does not exist (No such file or directory)
pandoc: /tmp/RtmpmiRJfV/rmarkdown-str124d5a2948a2.html: openFile: does not exist (No such file or directory)
pandoc: /tmp/RtmpmiRJfV/rmarkdown-str124b1d015295.html: openFile: does not exist (No such file or directory)
pandoc: /tmp/RtmpmiRJfV/rmarkdown-str124c3a9fd602.html: openFile: does not exist (No such file or directory)
pandoc: /tmp/RtmpmiRJfV/rmarkdown-str124b191fe7b1.html: openFile: does not exist (No such file or directory)
pandoc: /tmp/RtmpmiRJfV/rmarkdown-str124d70f14424.html: openFile: does not exist (No such file or directory)
pandoc: /tmp/RtmpmiRJfV/rmarkdown-str124b586db239.html: openFile: does not exist (No such file or directory)
pandoc: /tmp/RtmpmiRJfV/rmarkdown-str124b142b5ab8.html: openFile: does not exist (No such file or directory)

 Error in do_rpt(r) : 
  task 7 failed - "pandoc document conversion failed with error 1"
 

Script taken from here:
https://gist.github.com/hrbrmstr/17bc21af55392f23f012f57bb2fda51c

Version information:

> devtools::session_info("rmarkdown")
 setting  value                       
 version  R version 3.4.4 (2018-03-15)
 system   x86_64, linux-gnu           
 ui       RStudio (1.1.383)           
 language (EN)                        
 collate  en_US.UTF-8                 
 tz       Europe/Berlin               
 date     2018-06-04                  

 package   * version date       source        
 backports   1.1.2   2017-12-13 CRAN (R 3.4.3)
 base64enc   0.1-3   2015-07-28 CRAN (R 3.4.3)
 digest    * 0.6.15  2018-01-28 CRAN (R 3.4.3)
 evaluate    0.10.1  2017-06-24 CRAN (R 3.4.3)
 glue        1.2.0   2017-10-29 CRAN (R 3.4.3)
 graphics  * 3.4.4   2018-03-16 local         
 grDevices * 3.4.4   2018-03-16 local         
 highr       0.6     2016-05-09 CRAN (R 3.4.3)
 htmltools   0.3.6   2017-04-28 CRAN (R 3.4.3)
 jsonlite    1.5     2017-06-01 CRAN (R 3.4.3)
 knitr       1.20    2018-02-20 CRAN (R 3.4.3)
 magrittr    1.5     2014-11-22 CRAN (R 3.4.3)
 markdown    0.8     2017-04-20 CRAN (R 3.4.3)
 methods   * 3.4.4   2018-03-16 local         
 mime        0.5     2016-07-07 CRAN (R 3.4.3)
 Rcpp        0.12.17 2018-05-18 CRAN (R 3.4.4)
 rmarkdown   1.9     2018-03-01 CRAN (R 3.4.3)
 rprojroot   1.3-2   2018-01-03 CRAN (R 3.4.4)
 stats     * 3.4.4   2018-03-16 local         
 stringi     1.2.2   2018-05-02 CRAN (R 3.4.4)
 stringr     1.3.1   2018-05-10 CRAN (R 3.4.4)
 tools       3.4.4   2018-03-16 local         
 utils     * 3.4.4   2018-03-16 local         
 yaml        2.1.19  2018-05-01 CRAN (R 3.4.4)

[goldingn]

I can reproduce this error locally. Using rmarkdown::pandoc_convert() to convert a file in the same directory works, but trying to convert it one directory down fails.

Create directory structure and file

dir <- tempdir()
sub_dir <- tempfile(tmpdir = dir)
f <- tempfile(tmpdir = sub_dir)
dir.create(sub_dir)
writeLines("This is a test", f)
cwd <- getwd()
library(rmarkdown)

This works

setwd(sub_dir)
fname <- basename(f)
readLines(fname)
#> [1] "This is a test"
pandoc_convert(fname, to = "html")

This doesn't:

setwd(dir)
fname <- file.path(basename(sub_dir), basename(f))
readLines(fname)
#> [1] "This is a test"
pandoc_convert(fname, to = "html")
#> Error: pandoc document conversion failed with error 1

setwd(cwd)
sessionInfo()
#> R version 3.5.1 (2018-07-02)
#> Platform: x86_64-apple-darwin15.6.0 (64-bit)
#> Running under: macOS Sierra 10.12.6
#> 
#> Matrix products: default
#> BLAS: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRblas.0.dylib
#> LAPACK: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRlapack.dylib
#> 
#> locale:
#> [1] en_GB.UTF-8/en_GB.UTF-8/en_GB.UTF-8/C/en_GB.UTF-8/en_GB.UTF-8
#> 
#> attached base packages:
#> [1] stats     graphics  grDevices utils     datasets  methods   base     
#> 
#> other attached packages:
#> [1] rmarkdown_1.10.12
#> 
#> loaded via a namespace (and not attached):
#>  [1] compiler_3.5.1  magrittr_1.5    htmltools_0.3.6 tools_3.5.1    
#>  [5] yaml_2.2.0      Rcpp_0.12.18    stringi_1.2.4   knitr_1.20     
#>  [9] htmldeps_0.1.1  digest_0.6.16   stringr_1.3.1   evaluate_0.11

[AndrewManHayChiu]

I had a similar problem, where I could knit in some directories, but not in others (on a mapped network drive).

I removed pandoc 2.4, to direct RStudio to use the in-built pandoc as per @yihui 's comment at #1184.

[vthivierge]

Following @pauvasquezh, @NilsOle or @goldingn were you able to solve this?

I solved it by changing my RStudio global options and setting the R version to be the local C:\ drive and not R on the network drive.

[crossxwill]

If the Rmd file is stored on a network drive, the knitting fails. However, if the Rmd file is stored on the local C:\ drive, the knitting works fine.

YAML

---
title: "Untitled"
author: "William Chiu"
date: "7/22/2019"
output: powerpoint_presentation
---

Network drive

output file: ppt_test.knit.md

"C:/Users/wchiu/AppData/Local/RStudio/bin/pandoc/pandoc" +RTS -K512m -RTS ppt_test.utf8.md --to pptx --from markdown+autolink_bare_uris+ascii_identifiers+tex_math_single_backslash+smart --output ppt_test.pptx 
pandoc.exe: ppt_test.utf8.md: openBinaryFile: does not exist (No such file or directory)
Error: pandoc document conversion failed with error 1
Execution halted

Local C:\ drive

"C:/Users/wchiu/AppData/Local/RStudio/bin/pandoc/pandoc" +RTS -K512m -RTS ppt_test.utf8.md --to pptx --from markdown+autolink_bare_uris+ascii_identifiers+tex_math_single_backslash+smart --output ppt_test.pptx 
output file: ppt_test.knit.md


Output created: ppt_test.pptx

Session Info

> sessionInfo()
R version 3.6.1 (2019-07-05)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 7 x64 (build 7601) Service Pack 1

Matrix products: default

locale:
[1] LC_COLLATE=English_United States.1252  LC_CTYPE=English_United States.1252   
[3] LC_MONETARY=English_United States.1252 LC_NUMERIC=C                          
[5] LC_TIME=English_United States.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
 [1] compiler_3.6.1  htmltools_0.3.6 tools_3.6.1     yaml_2.2.0      Rcpp_1.0.1      rmarkdown_1.14 
 [7] knitr_1.23      xfun_0.8        digest_0.6.20   evaluate_0.14  

RStudio Version

$citation

To cite RStudio in publications use:

  RStudio Team (2018). RStudio: Integrated Development for R. RStudio, Inc., Boston, MA URL
  http://www.rstudio.com/.

A BibTeX entry for LaTeX users is

  @Manual{,
    title = {RStudio: Integrated Development Environment for R},
    author = {{RStudio Team}},
    organization = {RStudio, Inc.},
    address = {Boston, MA},
    year = {2018},
    url = {http://www.rstudio.com/},
  }


$mode
[1] "desktop"

$version
[1] ‘1.2.1335’

[lamontbeau]

Just save the Rmd file to the C:\ drive rather than desktop (Network Drive). This solved my problem while I have struggled a month...
Very simple solution. Thank you guys.

[crossxwill]

@yihui - although there is a very simple work around (store Rmd file on C:\ rather than network drive), some of us are not allowed to save files on our C:\ (per company policy and/or using RStudio Server Pro with mounted network folders). Is there a plan to fix rmarkdown or knitr such that Rmd files that are stored on a network drive can still knit?

[cderv]

I think this is the same issue as in #701 where there is already a long discussion.

An issue was even opened to pandoc (jgm/pandoc#1326) as it is in fact a more general issue with pandoc than just Rmarkdown. it was conclude that it could come from the drive directly, and that it is not specific to R or Rmarkdown tools (#701 (comment))

Currently solution is to copy locally somewhere (not necessary in C:) before doing the conversion. see for example #701 (comment) or other comments.

To share experience, on a RStudio Server Pro (linux) with NFS mounted drive, we have no issue with that.
Before I had issues with older system, and specifically with self_contained=TRUE document.
Also, there was recently a specific fix for WINDOWS environment in jgm/pandoc#5127 that is included in Pandoc 2.7.3. I don't know if it could help.

At the time, I manage to change some directory argument from Rmarkdown (#701 (comment)) to get the intermediaries files on local drive. (my home directory was on local drive).

although there is a very simple work around (store Rmd file on C:\ rather than network drive), some of us are not allowed to save files on our C:\ (per company policy and/or using RStudio Server Pro with mounted network folders)

I think any local folder where you have access right will work. Not just C:/ . You could have a home directory or even a temp directory to move your file before knitting. (R has one in tempdir())
Moving files there before rendering could solve the issue with pandoc. Updating pandoc could help too maybe (rmarkdown::pandoc_version() to see what is shipped with your RStudio)

Hope it helps.

[vnijs]

FYI Seeing the same issue using Rstudio in a windows VM. Setting the default working directory in Rstudio to a drive letter (e.g., Z:) in "Tools > Global options > General" rather than the default "~" resolved the issue.

[arielfuentes]

I'm trying to do multiple reports (https://www.reed.edu/data-at-reed/software/R/markdown_multiple_reports.html) and with mtcars works fine, but if I do the same with my own dataframe loaded from a *.csv doesn't work. I don't know how to fix it nor I understand why and how is happening, I'm on a local computer.

sessionInfo()
R version 3.5.0 (2018-04-23)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 7 x64 (build 7601) Service Pack 1

Matrix products: default

locale:
[1] LC_COLLATE=Spanish_Spain.1252 LC_CTYPE=Spanish_Spain.1252 LC_MONETARY=Spanish_Spain.1252
[4] LC_NUMERIC=C LC_TIME=Spanish_Spain.1252

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] readr_1.1.1 stringr_1.3.1 leaflet_2.0.2 dplyr_0.8.3 ggplot2_3.1.1 rmarkdown_1.10 markdown_0.8
[8] knitr_1.20

loaded via a namespace (and not attached):
[1] Rcpp_1.0.1 pillar_1.3.1 compiler_3.5.0 later_0.7.5 plyr_1.8.4 tools_3.5.0
[7] digest_0.6.18 evaluate_0.10.1 tibble_2.1.3 gtable_0.2.0 pkgconfig_2.0.2 rlang_0.4.0
[13] cli_1.0.1 shiny_1.1.0 rstudioapi_0.7 crosstalk_1.0.0 yaml_2.1.19 withr_2.1.2
[19] hms_0.4.2 htmlwidgets_1.3 rprojroot_1.3-2 grid_3.5.0 tidyselect_0.2.5 glue_1.3.1
[25] R6_2.2.2 fansi_0.4.0 purrr_0.2.5 magrittr_1.5 promises_1.0.1 scales_1.0.0
[31] backports_1.1.2 htmltools_0.3.6 rsconnect_0.8.15 assertthat_0.2.0 xtable_1.8-3 mime_0.5
[37] colorspace_1.3-2 httpuv_1.4.5 utf8_1.1.4 stringi_1.2.2 lazyeval_0.2.1 munsell_0.5.0
[43] crayon_1.3.4

[Joe-Wasserman]

I encountered the same error knitting to HTML from an .Rmd on a network drive. My working directory was already local. However, clearing the knitr cache resolved the issue.

[jacobgellman]

The solution by @vthivierge worked for me.

[knjpollard]

100% fix

when you open a project on a mapped drive do not open it through the shortcut to the mapped drive

#this works and happens automatically when you open the Rstudio project through that path
rmarkdown::render('H:/Mediasonic/John/tester.Rmd', encoding = 'UTF-8');

this is what happens when you open it wrong the knit command cannot operate on this path
rmarkdown::render('//offfice//H//Mediasonic RAID1//John/tester.Rmd', encoding = 'UTF-8');

[knjpollard]

Watch out used open file in recents and in "Quick Access" in windows 10 when opening Rstudio projects always brows to the actual mapped drive letter or set up a short cut that explicitly does that.

[vliublinska]

Watch out used open file in recents and in "Quick Access" in windows 10 when opening Rstudio projects always brows to the actual mapped drive letter or set up a short cut that explicitly does that.

Thank you!! This solved it for me.

[vickibendus]

I'm having the same issue on a Mac. Would anyone be able to assist on how to map the files properly to overcome this glitch since there is no C:/ drive? It seems like the solution is possible but I can't get it to work on my system! Thank you!

[Joe-Wasserman]

You can use "~/" as a shortcut for home directory, or you can refer to the absolute path, which will be something like /Users/Vicki , /Users/[your username on the machine] Joe Wasserman [email protected] https://www.joewasserman.com

…

On Thu, Jan 2, 2020 at 3:56 PM vickibendus ***@***.***> wrote: I'm having the same issue on a Mac. Would anyone be able to assist on how to map the files properly to overcome this glitch since there is no C:/ drive? It seems like the solution is possible but I can't get it to work on my system! Thank you! — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1268?email_source=notifications&email_token=AH4TPUVHQKESIT2NTR3MX43Q3ZIHBA5CNFSM4ESHCZKKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEH7MM3I#issuecomment-570345069>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AH4TPURW7GAVR35JJ7ZDU53Q3ZIHBANCNFSM4ESHCZKA> .

[vickibendus]

Shoot that doesn't seem to be a solution for me then. My current working directory is "/Users/vickibendus/Desktop/R Materials/Soccer Reports", which appears to be the correct type of path. I'll have to keep searching for a way around this problem.

[cderv]

@MTF-UMass thanks a lot ! That is really good to know - that means unfortunately Pandoc still have the issue.
Can you paste the error you get with Pandoc 2.11 to see if anything has changed ?

[MTF-UMass]

@cderv absolutely, sorry about that

"C:/Program Files/RStudio/bin/pandoc/pandoc" +RTS -K512m -RTS 03-params.utf8.md --to html4 --from markdown+autolink_bare_uris+tex_math_single_backslash --output 03-params.html --lua-filter "\\umaaffp\users\mtaftferguso\R\win-library\4.0\rmarkdown\rmarkdown\lua\pagebreak.lua" --lua-filter "\\umaaffp\users\mtaftferguso\R\win-library\4.0\rmarkdown\rmarkdown\lua\latex-div.lua" --email-obfuscation none --self-contained --standalone --section-divs --template "\\umaaffp\users\mtaftferguso\R\win-library\4.0\rmarkdown\rmd\h\default.html" --no-highlight --variable highlightjs=1 --variable "theme:bootstrap" --include-in-header "C:\Users\MTAFTF~1\AppData\Local\Temp\RtmpKomXKh\rmarkdown-str42503354e80.html" --mathjax --variable "mathjax-url:https://mathjax.rstudio.com/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML" 
pandoc.exe: \\: openBinaryFile: does not exist (No such file or directory)
Error: pandoc document conversion failed with error 1
Execution halted

[cderv]

Thanks a lot for your help !

[jmarshallnz]

Just chiming in with my experiences with RStudio 1.4+ and pandoc 2.14.1. 2.14.1 has fixed the ".Rmd file located in the user roaming profile (i.e. UNC path) issue", so if RStudio updates to this version of pandoc then that is solved as far as I can tell.

However, there is a related problem: If the .libPaths() is on a roaming profile (UNC) location, and rmarkdown is installed into that location then you get basically the same error, presumably due to pandoc not loading the template files etc correctly. This is supposed to be fixed in pandoc 2.14.1:

jgm/pandoc@26ed7fb

But it seems that it isn't completely fixed. Am not sure exactly which bit is failing, output is like this:

"C:/Program Files/RStudio/bin/pandoc/pandoc" +RTS -K512m -RTS blah.knit.md --to html4 --from markdown+autolink_bare_uris+tex_math_single_backslash --output blah.html --lua-filter "\\tur-homes2.massey.ac.nz\jcmarsha\My Documents\R\win-library\4.1\rmarkdown\rmarkdown\lua\pagebreak.lua" --lua-filter "\\tur-homes2.massey.ac.nz\jcmarsha\My Documents\R\win-library\4.1\rmarkdown\rmarkdown\lua\latex-div.lua" --self-contained --variable bs3=TRUE --standalone --section-divs --template "\\tur-homes2.massey.ac.nz\jcmarsha\My Documents\R\win-library\4.1\rmarkdown\rmd\h\default.html" --no-highlight --variable highlightjs=1 --variable theme=bootstrap --include-in-header "C:\Users\jcmarsha\AppData\Local\Temp\3\Rtmpya8M4d\rmarkdown-strd784b525dcd.html" --mathjax --variable "mathjax-url:https://mathjax.rstudio.com/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML" 

In addition, two warnings are generated, suggesting it might be fixable within rmarkdown maybe?

Warning messages:
1: In normalizePath(path.expand(path), winslash, mustWork) :
  path[1]="\\tur-homes2.massey.ac.nz": The specified path is invalid
2: In normalizePath(path.expand(path), winslash, mustWork) :
  path[1]="\\tur-homes2.massey.ac.nz": The specified path is invalid

Everything works fine, and the warnings are not generated if the UNC path is mapped to a drive and then R_LIBS_USER is set accordingly.

[kiwifb]

It seems like windows people have some kind of work around. But non-windows using drive over cifs are not so lucky.

I have seen a similar issue on jupyter because by default they tried to enforce atomic write which didn't work so well on remote file system. It kind of works the first time but after that you have some kind of ghost files preventing things from working. I got the following message from the r markdown window

Error in abs_path(input) : 
  The file 'simulating-the-Roy-Rubin-Model-final.Rmd' does not exist.
Calls: <Anonymous> -> setwd -> dirname -> abs_path
In addition: Warning messages:
1: In file.create(to[okay]) :
  cannot create file 'simulating-the-Roy-Rubin-Model-final.Rmd', reason 'No such file or directory'
2: In normalizePath(path, winslash = winslash, mustWork = mustWork) :
  path[1]="simulating-the-Roy-Rubin-Model-final.Rmd": No such file or directory
Execution halted

when I look at my file system I have

-rwx--x--x 1 frb15 domain users 817988 Jul 28 23:42  simulating-the-Roy-Rubin-Model-final.html*
-????????? ? ?     ?                 ?            ?  simulating-the-Roy-Rubin-Model-final.knit.md
-rwx--x--x 1 frb15 domain users  17041 Jul 28 23:41 'simulating-the-Roy-Rubin-Model final.Rmd'*
-????????? ? ?     ?                 ?            ?  simulating-the-Roy-Rubin-Model-final.Rmd

But I feel one the reason things don't work, is that temporary files are created in place instead of a proper temporary location, /tmp on unix and whatever it would be in windows.

[cderv]

@jmarshallnz @kiwifb Thanks a lot that is very helpful ! With your comment, I believe we have something to reproduce. It require to setup an environment with UNC path, so it won't be straightforward for me.

Also, we still don't know for sure if this is Pandoc or rmarkdown. It is probably both.

@jabranham the commit you linked to (jgm/pandoc@26ed7fb) is in a release version since Pandoc 2.11. When you say

2.14.1 has fixed the ".Rmd file located in the user roaming profile (i.e. UNC path) issue"

Do you have another fix in Pandoc in mind ? FWIW, next RStudio IDE version should be shipped with 2.14.1

@kiwifb The error you get is clearly not from Pandoc and it different. If you can reproduce, can you share also the traceback() after the error ? What abs_path() does is calling file.exists(). With rmarkdown, everything happens by default relatively to the input path, included intermediates. I believe mainly because those intermediates needs to be found by Pandoc and we would need to deal with absolute path everywhere including in the generated md file. Note that some resources are created in a tmp folder like special HTML headers to include.

We'll try to make quicker progress with your information, thanks for sharing ! We definitely need to reproduce this in order to better dig into the issue with file managements and computation with network drive.

Thanks again

[jmarshallnz]

@cderv Yes, you're right about that commit being in 2.11. Strange. There definitely is a difference in behaviour between pandoc 2.11 and 2.14.1 with RStudio 1.4. With v2.11 I can reproduce (everytime) the fault where the .Rmd is stored in a UNC path it fails to build. With 2.14.1 this is gone, except for the case where rmarkdown is also installed on a UNC path.

I can't really see anything obvious in pandoc other than that commit, so not sure what fixed (part of) the problem.

[cderv]

Then it is possible that something in RStudio IDE helped fix this issue. There was also non-pandoc issue with RStudio IDE and UNC path I believe.

With what you say, it could mean that next version of RStudio IDE with Pandoc 2.14.1 should be better.

We would need to look then more at the issue with rmarkdown package on network drive. For this, I am thinking we may need to copy the resources included into the package locally so that Pandoc does not need to access them through network drive (if that is the issue). Currently, some resource are passed to pandoc using absolute path to rmarkdown installation folder.

[kiwifb]

My setup is a rstudio server running on ubuntu 20.04 with some remote folders mounted via cifs. The previous error I think had some extra layers of indirections because the filename had space in it, which lead to extra normalization.
The first time it knits OK

output file: simulating-the-Roy-Rubin-Modelfinal.knit.md

/usr/lib/rstudio-server/bin/pandoc/pandoc +RTS -K512m -RTS simulating-the-Roy-Rubin-Modelfinal.knit.md --to html4 --from markdown+autolink_bare_uris+tex_math_single_backslash --output simulating-the-Roy-Rubin-Modelfinal.html --lua-filter /usr/lib/R/library/rmarkdown/rmarkdown/lua/pagebreak.lua --lua-filter /usr/lib/R/library/rmarkdown/rmarkdown/lua/latex-div.lua --self-contained --variable bs3=TRUE --standalone --section-divs --template /usr/lib/R/library/rmarkdown/rmd/h/default.html --highlight-style tango --variable theme=united --include-in-header /tmp/RtmpaGRfIM/rmarkdown-str14a4f44e57f07c.html --mathjax --variable 'mathjax-url:https://mathjax.rstudio.com/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML' 

Output created: simulating-the-Roy-Rubin-Modelfinal.html

But the second time (or sometimes more time, it is a bit finicky) I now get

label: unnamed-chunk-28 (with options) 
List of 1
 $ echo: logi TRUE

  |......................................................................| 100%
  ordinary text without R code


Error in file(con, "w") : cannot open the connection
Calls: <Anonymous> -> <Anonymous> -> write_utf8 -> writeLines -> file
In addition: Warning message:
In file(con, "w") :
  cannot open file 'simulating-the-Roy-Rubin-Modelfinal.knit.md': No such file or directory
Execution halted

I am doing everything from the rstudio interface at the moment, how do I run things to get to the backtrace you want. I can run R directly from the command line if needed.

[kiwifb]

Figured it out by running rmarkdown::render from rstudio's console.

5: file(con, "w") 
4: writeLines(enc2utf8(text), con, ..., useBytes = TRUE)
3: write_utf8(res, output)
2: knitr::knit(knit_input, knit_output, envir = envir, quiet = quiet) 
1: rmarkdown::render("/home/frb15/Pdrive/simulating-the-Roy-Rubin-Modelfinal.Rmd",         "html_document")

[cderv]

That is a bit disturbing to me that this is sometimes working, and sometimes not.

Thanks for sharing the setup. I'll see if I can get my hand of such configuration.

[kiwifb]

I think there is a lot to be done around some remote file system. What personally gets my attention as a programer in other language and context is that the intermediary file (in .knit.md) shouldn't be created there. It is transient and disappear once you are done. On most OS there are special place for those in /tmp (or windows equivalent) or ~/.cache or other special folders if you are more sophisticated. There is no absolute guarantee of locality but it is way cleaner than leaving your stuff next to your input and output - unless you are debugging.

[cderv]

I understand you point, and I think what you describe applies to more standard temporary content. However, for the specific usage here, we need to consider the way Pandoc is working. For example, it is highly possible that .knit.md will contain references to resource relatively to your Rmd document. If the resulting md file is elsewhere, those resources linked in the md doc won't be located relatively to the source file anymore and Pandoc won't find them during the conversion to HTML. Some processing would need to happen to handle absolute vs relative path correctly. That is just an example. Also, rmarkdown does write to tempdir() already some files that does not have this type of concerns.

We are a bit off topic here. You can open a new issue if you want to discuss some ways to improve that (Knitr + Pandoc + LaTeX interaction needs to be considered here). Thanks !

[kiwifb]

Ha, it just proves that I am completely ignorant of the context and that I don't know what I am talking about. I have been spending a few hours looking at the code earlier today and I couldn't really untangle the various abstractions. R is not the language I am the most familiar with.

I have come to that conversation because I am supporting a lecturer for whom I have organised the particular setup. The cifs mount is to access our university shared file system for convenience. As it turns out it is not currently convenient to run a number of things including knitting directly on it :)

[cderv]

Thanks for chiming in then !

We really appreciate having also external contributors with different background to share views and thoughts on the ecosystem because it brings new perspective. If you still want to look deeper with your setup (and I think that would us), you could try without R and rmarkdown, using directly Pandoc to convert a document with different scenarios. For example :

• Using Pandoc on your cifs mount to convert a doc
• Using Pandoc to convert from local storage but using a resource from external storage

If this does not work well at this level, then it is why we have a hard time looking at it from our level 🤔

[cacsfre]

Just in case: I was getting this error with a learnr tutorial (shiny_prerendered). It was working locally but it kept throwing this error when running through docker. I kept trying to figure what was wrong with the Dockerfile but later realized the problem was due to the index.html file being copied into the Docker version of the tutorial. The index.html file referenced a third file (footer.html) using the wrong location, which was being copied from the locally run tutorial.

Solution: don't track/copy index.html but let rmarkdown::run or rmarkdown::render generate it each time.

[EPriske]

Following @pauvasquezh, @NilsOle or @goldingn were you able to solve this?

I solved it by changing my RStudio global options and setting the R version to be the local C:\ drive and not R on the network drive.

This solved my problem with R Markdown. Thank you so much