ObjectPool<T>

[benaadams]

Background and Motivation

Follow up to #23700 (comment)

Expose the bounded ConcurrentQueueSegment<T> as an ObjectPool<T> type as using ConcurrentQueue<T> as a bounded object pool is too hard as the counting mechanism becomes separate from the queue so its at best eventually consistent; either adding too many^[1] (which is then removed), or pooling too few^[2] due to the race between counting and enqueuing/dequeuing; depending if the decrement is before¹ .TryDequeue (when it has to be a Interlocked.CompareExchange spin) or after² (which is a less expensive Interlocked.Decrement)

Additionally ConcurrentQueue<T> has already paid the cost for strict bounding; which then a second phase of "atomic"ish bounding has to be shimmed onto; so there is additional cost in memory and indirections from the extra in ConcurrentQueue<T> plus the inexactness of the counting; which is unsatisfactory, since all the work has already been done for the scenario its just inaccessable.

Proposed API

namespace System.Collections.Concurrent
{
    /// <summary>
    /// Provides a multi-producer, multi-consumer thread-safe bounded pool.  When the pool is full,
    /// enqueues fail and return false.  When the pool is empty, dequeues fail and return null.
    /// </summary>
    [DebuggerDisplay("Capacity = {Capacity}")]
    public sealed class ObjectPool<T>
    {
        /// <summary>Creates the pool.</summary>
        /// <param name="boundedLength">
        /// The maximum number of elements the pool can contain.  Must be a power of 2.
        /// </param>
        public ObjectPool(int boundedLength);

        /// <summary>Gets the number of elements this pool can store.</summary>
        public int Capacity;

        /// <summary>
        /// Tries to dequeue an element from the pool. If successful, the item will set
        /// from the pool and true will be returned; otherwise, the item will be null, and false
        /// will be returned.
        /// </summary>
        public bool TryRent([MaybeNullWhen(false)] out T item);

        /// <summary>
        /// Attempts to enqueue the item.  If successful, the item will be stored
        /// in the pool and true will be returned; otherwise, the item won't be stored, and false
        /// will be returned.
        /// </summary>
        public bool TryReturn(T item);
    }
}

Usage Examples

static readonly ObjectPool<MyObject> _pool = new (32);

public MyObject Rent() => _pool.TryRent(out var value) ? value : new ();

public void Return(MyObject value) 
{
    value.Reset();
    _pool.TryRetrun(value);
}

Alternative Designs

Always enqueue, but overwrite first (oldest) in queue; however that introduces extra contention...

Risks

People might want collection interfaces and enumeration; but they are fungible objects, so why?

[ghost]

Tagging subscribers to this area: @eiriktsarpalis
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Background and Motivation

Follow up to #23700 (comment)

Expose the bounded ConcurrentQueueSegment<T> as an ObjectPool<T> type as using ConcurrentQueue<T> as a bounded object pool is too hard as the counting mechanism become separate from the queue so its at best eventually consistent; either adding too many (which is then removed), or pooling too few due to the race between counting and enqueuing/dequeuing.

Additionally ConcurrentQueue<T> has already paid the cost for strict bounding; which then a second phase of atomic bounding has to be shimmed onto.

Proposed API

namespace System.Collections.Concurrent
{
    /// <summary>
    /// Provides a multi-producer, multi-consumer thread-safe bounded pool.  When the pool is full,
    /// enqueues fail and return false.  When the pool is empty, dequeues fail and return null.
    /// </summary>
    [DebuggerDisplay("Capacity = {Capacity}")]
    internal sealed class ObjectPool<T>
    {
        /// <summary>Creates the pool.</summary>
        /// <param name="boundedLength">
        /// The maximum number of elements the pool can contain.  Must be a power of 2.
        /// </param>
        public ObjectPool(int boundedLength);

        /// <summary>Gets the number of elements this pool can store.</summary>
        public int Capacity;

        /// <summary>Tries to dequeue an element from the pool.</summary>
        public bool TryDequeue([MaybeNullWhen(false)] out T item);

        /// <summary>
        /// Attempts to enqueue the item.  If successful, the item will be stored
        /// in the pool and true will be returned; otherwise, the item won't be stored, and false
        /// will be returned.
        /// </summary>
        public bool TryEnqueue(T item);
    }
}

Usage Examples

static readonly ObjectPool<MyObject> _pool = new (32);

public MyObject Rent() => _pool.TryDequeue(out var value) ? value : new ();

public void Return(MyObject value) => _pool.TryEnqueue(value);

Alternative Designs

None

Risks

People might want collection interfaces and enumeration.

Author:	benaadams
Assignees:	-
Labels:	api-suggestion, area-System.Collections, untriaged
Milestone:	-

[omariom]

Will it be fully thread safe or single-producer-single-consumer only?

update: I confused ConcurrentQueueSegment with another type.

[eiriktsarpalis]

When I was writing classes like this I would usually have the pool constructor accept a Func<T> parameter. That would eliminate the need for Try* methods and would have semantics akin to ArrayPool<T>. Is that something we could consider for this design?

[FiniteReality]

Is there any reason why the "return" method is named TryEnqueue? To me, TryReturn seems like a better name.

[davidfowl]

I think we should draw inspiration from https://github.com/dotnet/aspnetcore/tree/main/src/ObjectPool/src.

• The ObjectPool is an abstraction. It dictates the pooling strategy (bounded, dropping elements, max size, LIFO, FIFO etc)
• There's a policy abstraction that delegates creation and returning to the pool. Here's an example of a policy
• There's a simple default implementation of the pool in the box.

Splitting the responsibility between policy and pool allows consumers to change if an object is pool without changing how its stored.