[API Proposal]: Soft References #63113
Description
Background and motivation
MemoryCache is the "standard" way to cache things in .NET, but its behavior is unintuitive and it does not guarantee that it will evict cache entries quickly enough to prevent OutOfMemoryExceptions, among other issues such as badly bloated cache entries. Plus, it apparently relies on some kind of black magic (the documentation for which I have not been able to locate) to detect the size in bytes of objects in the cache, so that using it correctly is difficult: even if I am using it correctly, it's difficult to be confident of that! I would like to be able to put objects in a cache that contain two kinds of references: (1) references to "owned" subobjects that should be counted as part of the parent object, and (2) references to (large) shared objects that can never be evicted. I can't imagine how anything except the garbage collector would be able to detect that (1) is only reachable via the cache and so should be counted for "eviction" purposes, while (2) cannot be GC'd.
Finally, if the goal is to prevent memory exhaustion, MemoryCache is problematic because multiple cache instances can exist that do not coordinate with one another.
Weak references tend to be collected far too quickly to be used in caches. Soft references would solve this problem. Soft references are like weak references, but garbage-collected much less aggressively.
API Proposal
An obvious interface would be to replicate WeakReference<T>:
namespace System;
public sealed class SoftReference<T> : IWeakReference<T>
{
public SoftReference(T target);
~SoftReference();
public void SetTarget(T target);
public bool TryGetTarget([MaybeNullWhen(false)][NotNullWhen(true)] out T target);
}
// An interface implemented by WeakReference<T> and SoftReference<T>
public interface IWeakReference<T>
{
bool TryGetTarget([MaybeNullWhen(false)][NotNullWhen(true)] out T target);
void SetTarget(T target);
}API Usage
SoftReference<MyEntity>> _entity;
void Cache(MyEntity entity) => _entity = new SoftReference<MyEntity>(entity);
// Later on...
if (_entity != null && _entity.TryGetTarget(out MyEntity e))
Console.WriteLine("We've still got it!");
else
Console.WriteLine("Ain't got it!");Alternative Designs
Another obvious design is to define SoftReference<T> as a derived class of WeakReference<T>. A third possibility is to add "softness" as a feature of the existing WeakReference class.
WeakReferences have a "track resurrection" feature. It's not clear to me that this would add value to a soft reference, but IMO this feature should be supported if it does not add significant complexity to the GC.
It should be kept in mind that after this feature is introduced, many applications could be dominated by soft references (i.e. at any given time, most objects are reachable only through soft references). Therefore, perhaps there should be a property of GC to control the preferred total memory usage of the process, which would affect the aggressiveness of soft-reference collection.
// Sets a limit for memory usage that the GC should attempt to enforce
// by collecting more aggressively near the limit. This particularly affects
// the degree to which objects referenced via soft references are collected.
public long SoftMemoryLimit { get; set; }(I would very much like a hard memory limit too, but that's another story.)
Ideally, the GC would not collect all soft-referenced object when memory pressure is encountered, but instead prioritize which objects to collect first according to some kind of "priority". I believe the most commonly-desired way to prioritize would be by recency: to first get rid of objects that have not been used recently. To that end there could be a LastUsed property:
// Controls garbage collection priority; unreachable objects with
// lower values for LastUsed tend to be collected first.
// - To artificially delay GC for a soft reference, increase it (eg add 24 hours)
// - To artificially encourage GC for a soft reference, decrease it (eg subtract
// 24 hours; or use DateTime.MinValue to treat soft ref like WeakReference)
// - Setter could convert all dates to UTC so that the GC can directly compare
// LastUsed.Ticks of different soft references.
public DateTime LastUsed { get; set; }
// Variant of TryGetTarget that sets LastUsed = DateTime.UtcNow if target is alive
public bool TryGetTargetAndSetLastUsed([MaybeNullWhen(false)][NotNullWhen(true)] out T target) {
if (TryGetTarget(out target)) {
LastUsed = DateTime.UtcNow;
return true;
}
return false;
}Rather than adding bool updateLastUsed as a parameter on TryGetTarget, it could be a separate boolean property so that it is possible to configure IWeakReference.TryGetTarget() to update LastUsed.
It is possible that there are multiple soft references to the same object. It is tempting to put the last-used timestamp on the object itself so that there can only be a single timestamp:
// Any class could implement this interface in order to control GC priority
public interface IGCSoftReferencePriority
{
DateTime LastUsed { get; set; }
}However this approach would have major disadvantages:
- Users may certainly wish to hold soft references to objects that don't implement
IGCSoftReferencePriority - The very act of checking whether
IGCSoftReferencePriorityis implemented might be too expensive inside the GC - I expect that user-defined code cannot be called during GC's stop-the-world phase, and the property could do strange things like allocate memory, loop indefinitely, return a different value each time it is called, etc.
Risks
I have no idea how difficult it would be to implement this in the GC.