Feature request: non-blocking invalidate() in AsyncLoadingCache

[adamw]

Right now the only way to invalidate a key in AsyncLoadingCache is through the synchronous interface, which provides a blocking method. It would be great if there was an asynchronous version, which would return a CompletableFuture<Void>, which would complete when the key is invalidated.

[ben-manes]

It is blocking only to perform the hash table operation. The value isn't returned, so it won't block on the future result. I don't think that deserves an asynchronous wrapping, do you?

[adamw]

According to my experiments it blocks if a concurrent load is happening for a given key, see my test: https://github.com/softwaremill/cache-get-or-create/blob/master/src/main/java/com/softwaremill/TestCaffeine.java (the loader artificially sleeps for one second)

[ben-manes]

Yes, but an async cache performs a load that returns a future which is very cheap. Your test is for a sync cache where the load can be slow due to performing the user operation.

[adamw]

Ah! True :) And even though invalidate returns immediately, the value is invalidated after the cache completes (test here: https://github.com/softwaremill/cache-get-or-create/blob/master/src/main/java/com/softwaremill/TestCaffeineAsync.java) - which was the original problem that lead me to Caffeine.

I'll put a blog on that up tomorrow probably :) - thanks for the project

[ben-manes]

Yep, it's removed and the removal notification is chained on the future. The asMap().remove(k) does block on the future since it returns the result, so invalidating demonstrates an optimization by the more opinionated API.

[adamw]

Managed to finish it today, here's the blog I mentioned: https://softwaremill.com/race-condition-cache-guava-caffeine/

[ben-manes]

Cool, thanks!

You might want to use getIfPresent in your code examples, as get() was confusing if familiar with the interfaces.

There is a Guava bug on the invalidation problem. Since the entry isn't loaded Guava let's removal skip, which simplifies the locking. Caffeine has the same problem on a clear since ConcurrentHashMap suppresses the keys being loaded when we iterate.

[adamw]

Thanks for the pointer, adding a note to the blog.

I think get is the correct one, as getIfPresent would suggest to skip the loading if the value isn't already in the cache?

[ben-manes]

In your first example get doesn't have a loader to demonstrate the race. But some libraries, like JSR107, only have get and behave differently if a loader is attached. I find that more confusing so the explicit getIfPresent and get (load) are more obvious to me, at least.

[adamw]

Ah there, correct, thanks, I was thinking of the later examples :)

[ben-manes]

You can use invalidate(key) from the synchronous view.

This is equivalent to the async asMap() remove(key) except doesn’t return the prior future mapping. The synchronous asMap().remove(key) does block on the future as that view has to return the value and wait until it’s resolved.

[Inego]

Thanks, Ben!
My bad, I deleted my question because I had found an answer to it in a different issue here, but you still managed to answer it 😅