Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

extend file caching #79

Open
christophmluscher opened this issue Apr 13, 2022 · 1 comment
Open

extend file caching #79

christophmluscher opened this issue Apr 13, 2022 · 1 comment
Assignees
Labels
enhancement New feature or request

Comments

@christophmluscher
Copy link
Contributor

I was thinking about the extension of the file caching in Sisyphus. I would like to make an interface available so that the user can read and write using local file caching.

The idea would be to extend Sisyphus to support three user-defined functions. One is already available via file_caching. The other two would support copying between remote and local paths.

The Sisyphus AbstractPath Object would need additional functions to then facilitate remote/local copy operations.

This would leave the actual usage of file caching in the hands of the user because this would need to be implemented inside the jobs. This gives the user more direct control over the file caching.

@curufinwe @critias Do you think this is a good approach? Or do you think it would be better to abstract everything away? So hide inside Sisyphus? I have extended the API to support the above idea but wanted to discuss the general approach before making a pull request.

@christophmluscher christophmluscher added the enhancement New feature or request label Apr 13, 2022
@christophmluscher christophmluscher self-assigned this Apr 13, 2022
@critias
Copy link
Contributor

critias commented Apr 14, 2022

I'm not quite sure what you mean. Are you talking about something like this:
some_path.copy_from_server()
do_stuff(some_path)
some_path.copy_back_to_server()

Overall I'm not really happy with the current state of file_caching, so I'm very open for suggestions how to improve it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants