[Feature Request] fsspec/universal_pathlib integration

[Mithrandir2k18]

As far as I understand it, this package cannot be used with universal pathlib right away, but all the building block are already there. All that's missing is registering the AbstractFileSystem subclasses with fsspec and creating fsspec/universal_pathlib subclasses and registering them with their registry.

This seems simple enough and seems to work in my testing:

from fsspec import AbstractFileSystem
from upath import UPath
from upath.registry import register_implementation

from morefs.memory import MemFS

# local instance for inspectability during testing
mem_fs = MemFS()


class MorefsMemPath(UPath):
    @classmethod
    def _fs_factory(
        cls, urlpath: str, protocol: str, storage_options: dict
    ) -> AbstractFileSystem:
        return mem_fs  # this should be MemFS(...)


fsname = "customfs"
register_implementation(fsname, MorefsMemPath)


def test_upath():
    testtext = "Hello from MemFS!"
    testfile = "/example.txt"
    p = UPath(f"{fsname}:///{testfile}")
    p.write_text(testtext)
    assert testtext == p.read_text()
    assert mem_fs.ls("/") == [testfile]

[skshetry]

I think universal-pathlib is supposed to work right out of the box without requiring explicit registration (which I am not against, PR welcome).

So, UPath("memfs://") should just work. Although, compared to fsspec's MemoryFileSystem, morefs's memfs and dictfs are not global. So I see that every child paths get created with different filesystem instances.

[skshetry]

Also see: https://github.com/fsspec/universal_pathlib/blob/31573984674f6188c430f552276228ec7b922ef7/upath/_flavour_sources.py#L684C7-L684C19.

[Mithrandir2k18]

Thanks for the speedy reply. Yeah I am new to fsspec and totally missed that there was a mechanism for this upstream already.
For my usecase, memfs filesystems not being global works out well anyway.

I wouldn't mind creating a PR, but since it's already implemented as a flavour upstream, what benefit would explicitly registering bring? And registering would require adding a (optional) dependency to the universal_pathlib repo, for as I see it, hardly any benefit/noticeable change? Maybe it could introduce a global variant?