[FR] --filter-file via a python script

[mo-han]

I'm trying to achieve some job via complex and long filters, thus wondering if it's a good idea to use an external python script a filter?

At first it's a very simple idea, just like --filter "str(id) not in open('/path/to/list.txt').read()". But then I think, "why not push it to the edge?"

[mikf]

You can import functions, values, etc via the globals option
and then use those in --filter / image-filter.

[mo-han]

So if globals is configured, the GLOBALS in util.py is ignored. Is that correct?

It reminds me of something else about how the downloader checks existing file, which I always want to customize. Is there any option like globals that can help?

Thx.

[mikf]

So if globals is configured, the GLOBALS in util.py is ignored. Is that correct?

No, the symbols from globals get added to util.py GLOBALS (and overwrite them if they have the same name).

It reminds me of something else about how the downloader checks existing file, which I always want to customize. Is there any option like globals that can help?

Not yet. I'll eventually add a way to customize these checks, but for now you'll have to add this functionality yourself by modifying a bit of code.

gallery-dl/gallery_dl/job.py

Line 315 in 0bbdeaa

if pathfmt.exists():

gallery-dl/gallery_dl/path.py

Line 160 in 0bbdeaa

def exists(self):

[mo-han]

gallery-dl/gallery_dl/path.py

Line 162 in 8ea7520

if self.extension and os.path.exists(self.realpath):

self.realpath is the final and full abs path, is that correct?

[mikf]

It is the path which gallery-dl uses to access and open files.

It is always absolute when path-extended is enabled on Windows.
Otherwise it could be relative depending on base-directory.

[mo-han]

i cooked something like this and it fills stomach :D

    def exists(self):
        """Return True if the file exists on disk"""
        if self.extension and os.path.exists(self.realpath):
            return self.check_file()
        if 'sankaku' in self.realpath:
            import sys
            lib_dir = os.path.expanduser('~/gallery-dl')
            sys.path.append(lib_dir)
            # while lib_dir in sys.path:
            #     sys.path.remove(lib_dir)
            import modkit

            return modkit.check_exist_and_update(self.realpath)
        return False