[Feature Request] Optionally never write personally identifiable information to metadata files

[Scripter17]

Sometimes when sharing archives of accounts, the metadata can snitch to people what account you're using or if you're following the person and whatnot

Since this would affect so many extractors it should probably be added as a postprocessor

A basic implementation of the idea:

sanitizeRules=[
    {"key":"is_blocked"   , "value":False, "category":"deviantart", "subcategory":""},
    {"key":"is_favourited", "value":False, "category":"deviantart", "subcategory":""},
    {"key":"is_watching"  , "value":False, "category":"deviantart", "subcategory":""},
]
def sanitizeMetadata(metadata):
    for rule in sanitizeRules:
        catMatch   =not rule["category"   ] or rule["category"   ]==metadata["category"   ]
        subCatMatch=not rule["subcategory"] or rule["subcategory"]==metadata["subcategory"]
        if catMatch and subCatMatch:
            if rule["key"] in metadata:
                metadata[rule["key"]]=rule["value"]
    return metadata

Of course the hard part is getting a list of all metadata keys that can be used to identify you

[mikf]

I've already had the idea of adding options to add, modify/change, or remove metadata fields to the existing metadata post processor, so this would pretty much fit right in, wouldn't it?

Something like this:

{
    "name": "metadata",

    "delete": ["thumbnails", "preview"],
    "set"   : {
        "is_blocked"   : false,
        "is_favourited": false,
        "is_watching"  : false
    },
}

[Scripter17]

It's possible there's an edge case where, for example, one site has user be your username and another has user be the person being downloaded

But until someone runs into that this should work perfectly!

[Scripter17]

Just realized you said this was an idea and not already added

Gonna go get some tea to wake myself up

[nisehime]

Couldn't you just remove metadata fields with custom content format?

[mikf]

This has been implemented quite some time ago, but I never got back to this issue and I'm also not sure if this implementation is any good. It's needlessly verbose but rather flexible because of that.

f3de6b7 and 0c73914 add two additional modees for the metadata post processor that allow to delete or modify/add fields from the current metadata dict.

"deviantart": {
    "postprocessors": [
        {
            "name": "metadata",
            "mode": "delete",
            "fields": [
                "is_blocked",
                "is_favourited"
            ]
        },
        {
            "name": "metadata",
            "mode": "modify",
            "fields": {
                "is_watching": "\fE False",
                "custom_field": "{index!s}-{extension}"
            }
        },
        {
            "name": "metadata",
            "#": "write metadata"
        }
    ]
}