Use sync instead of copy for blob storage #6298
Conversation
| dest_folders, dest_files = self.listdir(self._dirpath(destination)) | ||
| for folder in dest_folders: | ||
| if folder not in copied_dirs: | ||
| self.delete_directory(self.join(destination, folder)) |
There was a problem hiding this comment.
I changed this, because set1 - set2 creates another set. not in is O(1).
|
|
||
| tree = [ | ||
| # Cloud storage generally doesn't consider empty directories to exist | ||
| ('api', []), |
There was a problem hiding this comment.
I was about to fix this to delete all dirs on FileSystemStorage, but looks like we expect that and probably isn't a big deal.
readthedocs.org/readthedocs/rtd_tests/tests/test_build_storage.py
Lines 39 to 41 in 43463b4
- Linting fix - Check against suspicious operations - Additional logging
There was a problem hiding this comment.
I fixed the linting error and added a few other cleanup parts. This is good to go and should fix the lingering storage issues. Thanks for doing this @stsewd!
No completely, we still need to remove files when a version is deactivated, but I'm making another PR about that. |
We use
rsyncfor our normal storage, but for blob storage we are just doing a copy.Full explanation at #6186 (comment)
This is heavily borrowed from a snippet from @davidfischer :)
Fixes #6069
Closes #6186