You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I was thinking about the extension of the file caching in Sisyphus. I would like to make an interface available so that the user can read and write using local file caching.
The idea would be to extend Sisyphus to support three user-defined functions. One is already available via file_caching. The other two would support copying between remote and local paths.
The Sisyphus AbstractPath Object would need additional functions to then facilitate remote/local copy operations.
This would leave the actual usage of file caching in the hands of the user because this would need to be implemented inside the jobs. This gives the user more direct control over the file caching.
@curufinwe@critias Do you think this is a good approach? Or do you think it would be better to abstract everything away? So hide inside Sisyphus? I have extended the API to support the above idea but wanted to discuss the general approach before making a pull request.
The text was updated successfully, but these errors were encountered:
I'm not quite sure what you mean. Are you talking about something like this: some_path.copy_from_server() do_stuff(some_path) some_path.copy_back_to_server()
Overall I'm not really happy with the current state of file_caching, so I'm very open for suggestions how to improve it.
I was thinking about the extension of the file caching in Sisyphus. I would like to make an interface available so that the user can read and write using local file caching.
The idea would be to extend Sisyphus to support three user-defined functions. One is already available via
file_caching
. The other two would support copying between remote and local paths.The Sisyphus
AbstractPath
Object would need additional functions to then facilitate remote/local copy operations.This would leave the actual usage of file caching in the hands of the user because this would need to be implemented inside the jobs. This gives the user more direct control over the file caching.
@curufinwe @critias Do you think this is a good approach? Or do you think it would be better to abstract everything away? So hide inside Sisyphus? I have extended the API to support the above idea but wanted to discuss the general approach before making a pull request.
The text was updated successfully, but these errors were encountered: