renv::use() requires R restart after updating loaded packages

[kobusbosman]

Dear reader,

I have an issue with (advanced) use of renv in our organization. We develop a package in a project with renv enabled, which works great for the package developers. The users will use the built package, so they do not open the package project itself like the developers do. For reproducibility of their results, over time as well as between users, we seek to extend renv to the context of the users as well.

We therefore equip the package with a script or Markdown template that includes an renv::use() call, which was created using renv::embed(). The renv::use() call sets up a temporary library path in which it installs the packages listed in renv::use(), for example renv::use("[email protected]").

This approach works well in principle, but leads to confusing behavior when the packages listed in the renv::use() call are of a different version and attached when renv::use() is called. R will then display a message in the console that says: "The following loaded package(s) have been updated: ... Restart your R session to use the new versions." Of course this is standard behavior so that R can reload the namespaces of the updated packages. However in the context of renv::use(), an R restart leads to the temporary library path to be removed and renv to exit the temporary environment and revert back to the environment prior to the renv::use() call.

I have tried to wrap the renv::use() call with a function that calls unloadNamespace but there are a few intricacies that make it feel like a bit of a hack. It works for the most part but still asks the user to restart R to update the namespace of renv itself. That in turn can be prevented by not including the renv package in the renv::use() call, but it makes me wonder why this is not included in the logic of renv::use().

My questions are:

• Is there a reason why renv::use() does not unload the loaded namespaces automatically?
• How can I make sure that the loaded namespaces are updated as part of the renv::use() call so that the message "Restart your R session to use the new versions" is not displayed to the user?

How to reproduce:

1 - install.packages("renv")
2 - library(renv)
3 - packageVersion("renv") # version 1.0.7 at time of writing
4 - install.packages("dplyr")
5 - library(dplyr)
6 - packageVersion("dplyr") # version 1.1.4 at time of writing
7 - renv::use("[email protected]")
8 - packageVersion("dplyr")

Observed behavior: at step 7, the console prints a message asking to restart the R session and, at step 8, dplyr is still at the newest version and not 1.0.6.
Expected behavior: at step 7, the console does not ask the user to restart the R session and, at step 8, dplyr is at version 1.0.6.

Thank you very much for your time!

Session Info:
R version 4.1.1 (2021-08-10)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Red Hat Enterprise Linux 9.4 (Plow)
Matrix products: default
BLAS/LAPACK: /usr/lib64/libopenblasp-r0.3.21.so
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C LC_TIME=en_US.UTF-8
[4] LC_COLLATE=en_US.UTF-8 LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=en_US.UTF-8 LC_NAME=C LC_ADDRESS=C
[10] LC_TELEPHONE=C LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets methods base
loaded via a namespace (and not attached):
[1] compiler_4.1.1 tools_4.1.1

[kevinushey]

Thanks for the bug report! I think this is ultimately a limitation of R -- while in some narrow cases it's possible to unload a package and load a new, separate version of that package, this unfortunately does not work in general. We could try to make this work with renv::use(), but I worry that this is not something we could rely on. For reference, ?detach has this documentation (which is also referenced from ?unloadNamespace):

If a package has a namespace, detaching it does not by default unload the namespace (and may not even with unload = TRUE), and detaching will not in general unload any dynamically loaded compiled code (DLLs); see getLoadedDLLs and library.dynam.unload. Further, registered S3 methods from the namespace will not be removed, and because S3 methods are not tagged to their source on registration, it is in general not possible to safely un-register the methods associated with a given package. If you use library on a package whose namespace is loaded, it attaches the exports of the already loaded namespace. So detaching and re-attaching a package may not refresh some or all components of the package, and is inadvisable. The most reliable way to completely detach a package is to restart R.

And also, as an example of potentially negative effects:

Unloading some namespaces has undesirable side effects: e.g. unloading grid closes all graphics devices, and on some systems tcltk cannot be reloaded once it has been unloaded and may crash R if this is attempted.

This typically implies that, if you want to use renv::use(), it should be run as early as possible in a fresh R session. Otherwise, you could run into exactly the issues you describe here.

Perhaps renv::use() should be stricter here, and print a more informative warning (or error) if one attempts to call renv::use() with a different version of a package that has already been loaded?

[kobusbosman]

Thank you for your elaborate reply, I appreciate it. It would work for us to tell our users to always run the template that contains the renv::use() call in a fresh R session. Indeed it would be more clear for a future release of renv to not print the prompt to restart the session, as it would confuse the user.