Skip to content

Commit

Permalink
Do not re-populate directory structure from disk on pending entry change
Browse files Browse the repository at this point in the history
When updating a dentry (e.g., adding a new file), the parent
directory will be re-read from disk on lazy-loading. This
causes the pending change to be reverted, leading to missing
newly added files.

To fix this, the directory entries __dirs list will only
be populated once.
  • Loading branch information
nathanhi committed Oct 16, 2023
1 parent 4e56534 commit 6eb9026
Show file tree
Hide file tree
Showing 3 changed files with 22 additions and 2 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.rst
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ Changed
- Introduce ``lazy_load`` parameter to allow restoring previous behavior
- `PR #32 <https://github.com/nathanhi/pyfatfs/pull/32>`_: Fix tree iteration on non-lazy load by `@zurcher <https://github.com/zurcher>`_ / `@Microsoft <https://github.com/Microsoft>`_
- `PR #33 <https://github.com/nathanhi/pyfatfs/pull/33>`_: Fix missing parent directory entry link on lazy-load by `@zurcher <https://github.com/zurcher>`_ / `@Microsoft <https://github.com/Microsoft>`_
- `PR #33 <https://github.com/nathanhi/pyfatfs/pull/33>`_: Do not re-populate directory structure from disk on pending entry change by `@zurcher <https://github.com/zurcher>`_ / `@Microsoft <https://github.com/Microsoft>`_

Removed
~~~~~~~
Expand Down
3 changes: 1 addition & 2 deletions pyfatfs/FATDirectoryEntry.py
Original file line number Diff line number Diff line change
Expand Up @@ -378,6 +378,7 @@ def __populate_dirs(self):
self.__dirs = self.__fs.parse_dir_entries_in_cluster_chain(clus)
for dir_entry in self.__dirs:
dir_entry._add_parent(self)
self.__lazy_load = False

def _get_entries_raw(self):
"""Get a full list of entries in current directory."""
Expand Down Expand Up @@ -497,8 +498,6 @@ def remove_dir_entry(self, name):
**NOTE:** This will also remove special entries such
as ».«, »..« and volume labels!
"""
self._verify_is_directory()
self.__populate_dirs()
# Iterate all entries
for dir_entry in self._get_entries_raw():
sn = dir_entry.get_short_name()
Expand Down
20 changes: 20 additions & 0 deletions tests/test_PyFatFS.py
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,26 @@ def test_lazy_load_dentry_parent_update(self):
assert foo_dentry.get_full_path() == "foo"
assert foobar_dentry.get_full_path() == "foo/bar"

def test_update_dentry_no_repopulate(self):
"""#33: Verify that update_dentry doesn't re-read entries from disk.
This is only problematic in case of lazy-loading, where
directory entries can be dynamically loaded, even when there
is a pending directory entry change; ultimately overwriting
the pending change.
"""
fs, in_memory_fs = _make_fs(self.FAT_TYPE, lazy_load=True)
fs.makedirs("/foo")
fs.touch("/foo/bar")
assert fs.listdir("/foo") == ['bar']

in_memory_fs.seek(0)
fs = PyFatBytesIOFS(BytesIO(in_memory_fs.read()),
encoding='UTF-8', lazy_load=True)
fs.touch("/foo/baz")
fs.remove("/foo/bar")
assert fs.listdir("/foo") == ['baz']

def test_lazy_vs_nonlazy_tree(self):
"""Compare directory tree between lazy and non-lazy loading."""
fs1, in_memory_fs = _make_fs(self.FAT_TYPE, lazy_load=False)
Expand Down

0 comments on commit 6eb9026

Please sign in to comment.