fix: synced cache file by adding support for filelock #359
Conversation
|
I’m trying to use filelock>=3.12.2 due to compatibility issues with the Python version. |
| def port_target_cache(cache_dir) -> t.Dict[str, str]: | ||
| """Session scoped port-target cache, for esp only""" | ||
| _cache_file_path = os.path.join(cache_dir, 'port_target_cache') | ||
| lock = filelock.FileLock(f'{_cache_file_path}.lock') |
There was a problem hiding this comment.
One question, should we check the lock file at the beginning of the whole process, and remove it if it exists? (fixture session)
Will the lock file be left over if the last pytest process got terminated for whatever reason?
There was a problem hiding this comment.
As you mentioned, the main problem with pytest-xdist is that it runs independent processes. I guess if we try to remove files manually, there might be a race condition. If the processes are killed, the system automatically releases the file lock, so I think it's okay. This library uses system calls to achieve synchronization.
|
LGTM. I like the idea with the file lock which could work on all OSes. |
Description
We had an issue with a cache file race condition while running pytest-xdist. One possible solution is to add filelock, which ensures synchronized access.
Related
Fixed #354
Checklist
Before submitting a Pull Request, please ensure the following: