Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cached thumb penalty with S3 backend #662

Closed
lorinkoz opened this issue Dec 30, 2020 · 3 comments
Closed

Cached thumb penalty with S3 backend #662

lorinkoz opened this issue Dec 30, 2020 · 3 comments

Comments

@lorinkoz
Copy link

lorinkoz commented Dec 30, 2020

👋🏼 Good work with this package!


Because of this code:

class LazyStorage(LazyObject):
def _setup(self):
self._wrapped = get_module_class(data['storage'])()
image_file = ImageFile(data['name'], LazyStorage())

If multiple thumbs with S3 backend are fetched from cache, the fact that they will all use a different storage will incur in a time penalty when generating presigned URLs for each one of those, as the object backend.connection will not be initialized every time.

@dirigeant
Copy link

same issue with #301 and #508

krukas added a commit to krukas/sorl-thumbnail that referenced this issue Jan 24, 2023
krukas added a commit to krukas/sorl-thumbnail that referenced this issue Jan 24, 2023
@krukas
Copy link
Contributor

krukas commented Jan 24, 2023

I created a PR to fix this #719

In the mean time a fix for others with this problem. I have used the Redis KVStore, this should work for all stores.

from sorl.thumbnail.kvstores.redis_kvstore import KVStore
from sorl.thumbnail.kvstores.base import add_prefix
from sorl.thumbnail.helpers import get_module_class, deserialize
from sorl.thumbnail.images import ImageFile

STORAGES_CACHE = {}


def get_or_create_storage(storage):
    if storage not in STORAGES_CACHE:
        STORAGES_CACHE[storage] = get_module_class(storage)()
    return STORAGES_CACHE[storage]


class SorlThumbnailRedisKVSTore(KVStore):
    def _get(self, key, identity='image'):
        """
        Deserializing, prefix wrapper for _get_raw
        """
        value = self._get_raw(add_prefix(key, identity))

        if not value:
            return None

        if identity == 'image':
            return self._deserialize_image_file(value)

        return deserialize(value)

    def _deserialize_image_file(self, s):
        data = deserialize(s)

        image_file = ImageFile(data['name'], get_or_create_storage(data['storage']))
        image_file.set_size(data['size'])
        return image_file

settings

THUMBNAIL_KVSTORE = "<your module path>.SorlThumbnailRedisKVSTore"

krukas added a commit to krukas/sorl-thumbnail that referenced this issue Aug 14, 2023
@claudep
Copy link
Contributor

claudep commented Aug 15, 2023

Duplicate of #301

@claudep claudep marked this as a duplicate of #301 Aug 15, 2023
@claudep claudep closed this as completed Aug 15, 2023
claudep pushed a commit that referenced this issue Aug 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants