[REQUEST] Limiting cache

[Toricane]

What is the feature you're proposing?
As of right now, caching has no limit.
I feel like there should be a way to set a limit for each type of cache, preferably in the Client instance.

Something like:

bot = interactions.Client(
    ...,
    cache_limits=interactions.CacheLimits(
        messages=100,
        guilds=50,
        ...
    )
)

This is the best thing that I can think of off the top of my head.
Another way is to do what dis-snek did do something with kwargs and have something like this:

bot = interactions.Client(
    ...,
    message_cache=100,
    guild_cache=50,
    ...
)

Additional Information

• My feature request is related to an existing Issue.
  • Issue (if referenceable):

[i0bs]

Garbage collection is a pretty bashed topic in terms of data in a library being handled. For the most part, the cache is only on-demand so it has to be done through a listened event, or by creating something through the Web API. I'm not sure if we particularly need limits because while yes, they do help with memory size, it's also hard locking how big a cache can be.

I think TTLing unused data in the cache would be a better approach, not limiting how big the cache can be. However, TTLing information would need to have some work done through how the data is accessed. If we want to reliably check this, we'd have to either force users checking the cache to use the current view() method, or using the attrs module which I have been looking into.

[Toricane]

Ok I see
What I was thinking was to remove the first item(s) in the OrderedDict when the cache hit the limit.
I don't know what TTLing is, but I like the current method of using get(), but view() could work I guess.

[tagptroll1]

Ok I see What I was thinking was to remove the first item(s) in the OrderedDict when the cache hit the limit. I don't know what TTLing is, but I like the current method of using get(), but view() could work I guess.

TTL (Time to live), something like store a timestamp when it was stored / accessed and either have a job/task that checks these dates in intervals to invalidate / remove from cache. Or check each entry everytime the cache is accessed. Hope that helps :)

[Toricane]

Ohh I see, thanks!

[i0bs]

Proposal may be amended to #909 , please make a comment referencing there.

[AstreaTSS]

I'd say while #909 mostly fixes it, it may still be useful having something like this for messages, as they can grow more or less unbounded in most servers since many messages are never deleted.

[holdur-ground]

Ideally for me, you would be able to configure the cache size (with a reasonable default) and use the last accessed time as above or a combination of number of times accessed and last accessed time to determine what to purge when the cache limit is hit.

So when adding an item to the cache,

1. If cache is full, then find least used item and remove
2. Add new item to cache

This way you don't need to have another task responsible for pruning the cache and you don't unnecessarily remove items from the cache if there is no reason to.

Thanks