You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is it currently possible for users to set up a callr command such as callr::r_bg to spawn background tasks across different nodes of a compute cluster?
If not, could the existing workings behind callr::r_bg be by any chance easily applied across multiple nodes?
I've been trying to figure out how to make this potentially work on a large HPC cluster which runs on openmpi as the interface for inter-node communication. From the documentation, it seems like callr::r_bg is a wrapper for processx::process, which is itself based around calling system so it theoretically it should work right ...? however, I am not familiar enough with what callr::bg, for example, adds on top of the existing processx::process function that may make this harder or easier.
I would be interested to know any thoughts about this / suggested ways we can currently achieve the goal
cheers!
The text was updated successfully, but these errors were encountered:
processx::process does not use system. More importantly the IPC in processx is a Unix socket or (a Windows named pipe), so it is not going to work across machines.
Good day! 😊
Is it currently possible for users to set up a
callr
command such ascallr::r_bg
to spawn background tasks across different nodes of a compute cluster?If not, could the existing workings behind
callr::r_bg
be by any chance easily applied across multiple nodes?I've been trying to figure out how to make this potentially work on a large HPC cluster which runs on
openmpi
as the interface for inter-node communication. From the documentation, it seems likecallr::r_bg
is a wrapper forprocessx::process
, which is itself based around callingsystem
so it theoretically it should work right ...? however, I am not familiar enough with whatcallr::bg
, for example, adds on top of the existingprocessx::process
function that may make this harder or easier.I would be interested to know any thoughts about this / suggested ways we can currently achieve the goal
cheers!
The text was updated successfully, but these errors were encountered: