Weak reference concurrent pool #10787

gregw · 2023-10-25T23:16:34Z

The Pool is now a list of Holder instances, each with a WeakReference to an Entry and an optional hard reference.

gregw · 2023-10-25T23:19:19Z

@sbordet @lorban What is wrong with this approach?

Once an entry is acquired, the pool only holds a weak reference to it. Once it is released it creates a hard reference to keep it in GC.

I do not think there is a race, as before... what am I missing?

The Pool is now a list of Holder instances, each with a WeakReference to an Entry and an optional hard reference. back link may be hard

The Pool is now a list of Holder instances, each with a WeakReference to an Entry and an optional hard reference. free on acquire

The Pool is now a list of Holder instances, each with a WeakReference to an Entry and an optional hard reference. free on acquire removed ThreadLocal cache

The Pool is now a list of Holder instances, each with a WeakReference to an Entry and an optional hard reference. free on acquire added Leak test

The Pool is now a list of Holder instances, each with a WeakReference to an Entry and an optional hard reference. free on acquire added warnings

…tty-12/WeakReference-ConcurrentPool

The Pool is now a list of Holder instances, each with a WeakReference to an Entry and an optional hard reference. free on acquire added warnings

gregw · 2023-10-26T05:40:12Z

@sbordet @lorban OK it was not as simple as it looked at first.... but I think I found the race (could be acquired and freed before it was held after release). I think I have fixed it with a spin test (scary). It now works for multiplexed pools as well. I removed the cache because I think it is an unnecessary complexity.

…g configurable Signed-off-by: Ludovic Orban <[email protected]>

The Pool is now a list of Holder instances, each with a WeakReference to an Entry and an optional hard reference. free on acquire added warnings

gregw · 2023-10-26T23:03:16Z

@lorban I reverted to use a weak pool by default (with system property to change default). Whilst we are an experimental branch, we want to test it as much as possible. Perhaps once we merge, we switch the default (and have a better way of setting it).
I'm also using concrete Holder classes rather than if(weak) tests.

jetty-core/jetty-util/src/test/java/org/eclipse/jetty/util/ConcurrentPoolTest.java

lorban · 2023-10-27T08:57:16Z

@gregw that's cool, and I currently have no strong opinion about whether we should set the pool as weak by default or not.

The reason why I made weak ref configurable is to measure the perf impact of this PR, so I want to compare in order:

mainline vs this PR with hard refs only
this PR with hard refs only vs this PR with weak refs
mainline vs this PR with weak refs
to make sure this doesn't regress perf in any meaningful way, as the concurrent pool really is perf critical.

I don't mind your modifications; as I can configure an ArrayByteBufferPool with weak or strong references at will the default doesn't impact my perf testing.

lorban · 2023-10-27T09:07:15Z

Perf report (so far):

This change has no measurable impact, no matter if weak referencing is enabled or not. The JIT inlines the entire buffer acquisition code in a single blob and most of the time is spent, in order: iterating the list, CASing the BiInteger in tryAcquire, updating the ref counter of the RetainableByteBuffer.

Basically, the cost stayed where it was expected to be, so we can concentrate on making sure the feature works. 👍

The Pool is now a list of Holder instances, each with a WeakReference to an Entry and an optional hard reference. free on acquire added warnings no race in tests

gregw · 2023-10-28T02:59:23Z

@lorban so I'm doing a risk analysis for this, and I think the fundamental approach is a lot safer than before, as we rely on less moving parts.
I don't think there is any risk if we keep a strong link accidentally, as that is just case we have today and we expect the application to release.
Ditto if we don't keep a strong link when we should, then we might cleanup an entry we did not need to... but only if the application doesn't have a strong link. So it would just be an idle entry shrink... not harm.

The only problem I can see is the spin wait to see a string link before it is cleaned. If we get that wrong, then we spin forever! My concern is what if between one spin and the next, the entry is acquired, released, acquired and released? Do we need a similar spin on the hold method, so that it waits to see the previous free? That will prevent multiple acquire/release cycles from happening during a spin.... actually I think I will implement that now!

The Pool is now a list of Holder instances, each with a WeakReference to an Entry and an optional hard reference. free on acquire added warnings no race in tests

lorban

I think this logic is correct, and unless I missed something, the race condition that you fix with a spin loop in hoder can only happen under one cirmunstance: when a release makes the multiplex counter go to 0 at the same time an acquire moves it to 1.

In that case, hold() must execute first so the concurrent free() call stays in the spin loop to wait for that moment before doing its job.

But I see no reason that would make hold() spin as I can't see a race condition that would trigger this need.

Wether I'm right or wrong, these new states and their transitions should be very carefully documented.

lorban · 2023-10-30T10:22:38Z

jetty-core/jetty-util/src/main/java/org/eclipse/jetty/util/ConcurrentPool.java

+    {
+        Pool.Entry<P> getEntry();
+
+        void hold();


How about renaming this method harden()?

I prefer release() as it matches where it is called from.

@sbordet but it is not a release. A release only reduces the multiplexed reference count and only if that goes to zero do we create a non-weak reference. So it is called from release, but is not the same.
Maybe lastRelease or fullyReleased ? I don't mind harden

lorban · 2023-10-30T10:22:49Z

jetty-core/jetty-util/src/main/java/org/eclipse/jetty/util/ConcurrentPool.java

+
+        void hold();
+
+        void free();


How about renaming this method weaken()?

I prefer acquire() as it matches where it is called from.

@sbordet again it is not an acquire. perhaps firstAcquire? I don't mind weaken

jetty-core/jetty-util/src/main/java/org/eclipse/jetty/util/ConcurrentPool.java

sbordet · 2023-10-30T18:03:25Z

jetty-core/jetty-util/src/main/java/org/eclipse/jetty/util/ConcurrentPool.java

+    {
+        Pool.Entry<P> getEntry();
+
+        void hold();


I prefer release() as it matches where it is called from.

sbordet · 2023-10-30T18:03:40Z

jetty-core/jetty-util/src/main/java/org/eclipse/jetty/util/ConcurrentPool.java

+
+        void hold();
+
+        void free();


I prefer acquire() as it matches where it is called from.

sbordet · 2023-10-30T18:05:15Z

jetty-core/jetty-util/src/main/java/org/eclipse/jetty/util/ConcurrentPool.java

+            holder = pool.weak ? new WeakHolder<>(this) : new StrongHolder<>(this);
+        }
+
+        private Holder<E> getHolder()


I'd just reference the final field and remove the method.

jetty-core/jetty-util/src/main/java/org/eclipse/jetty/util/ConcurrentPool.java

sbordet · 2023-10-30T18:20:13Z

jetty-core/jetty-io/src/main/java/org/eclipse/jetty/io/ArrayByteBufferPool.java

+     * @param maxDirectMemory the max direct memory in bytes, -1 for unlimited memory or 0 to use default heuristic
+     * @param weakPool true if the pooled buffers should be weakly referenced upon acquisition, false otherwise
+     */
+    public ArrayByteBufferPool(int minCapacity, int factor, int maxCapacity, int maxBucketSize, long maxHeapMemory, long maxDirectMemory, boolean weakPool)


Don't really like the telescopic additional "weakPool" parameter -- seems an implementation detail that leaks many layers up, and if we change our mind it'll be part of the public APIs for a long time.

@sbordet Isn't this at odds with your previous comment about preferring "explicit configuration". How would you configure this if not by constructor parameter? Should we just have a different classes for Weak pools?

@gregw the comments above are now out of date, and the weakPool parameter is never used, so please remove it.

The Pool is now a list of Holder instances, each with a WeakReference to an Entry and an optional hard reference. free on acquire added warnings

lorban · 2023-10-31T08:48:50Z

@gregw @sbordet I think we now reached the point where we all agree that this idea is technically sound, will work, and is worth implementing.

Besides naming (I'll skip that subject), there are still two points of disagreement or uncertainties:

What should be done when a leak is detected to help report the problem?
How this new feature should be made configurable?

Here are my current positions:

When a leak is detected, we shouldn't do more than incrementing a counter exposed via JMX. The kind of information we can get from the ConcurrentPool is IMHO insufficient to track the source of a leak. At least, in the context of the retainable buffers pool where acquire(), retain() and release() calls all need to be tracked to understand where the cause of the leak might be.
I'm tempted by the idea that weak referencing shouldn't be made configurable at all, and ConcurrentPool should have this feature always on. My top reason for this preference is that ConcurrentPool already is a complex piece of lock-free code that we should keep as simple as possible.

Thoughts?

gregw · 2023-10-31T20:50:35Z

@gregw @sbordet I think we now reached the point where we all agree that this idea is technically sound, will work, and is worth implementing.

+1

Besides naming (I'll skip that subject), there are still two points of disagreement or uncertainties:

On naming I can live with any of hold/free, strengthen/weaken, lastRelease/firstAcquire but not release/acquire

What should be done when a leak is detected to help report the problem?

How this new feature should be made configurable?

Here are my current positions:

When a leak is detected, we shouldn't do more than incrementing a counter exposed via JMX. The kind of information we can get from the ConcurrentPool is IMHO insufficient to track the source of a leak. At least, in the context of the retainable buffers pool where acquire(), retain() and release() calls all need to be tracked to understand where the cause of the leak might be.

I think that is a reasonable default mechanism.

However, I do think we should at least include something like my DebugWeakHolder, so that extra info can be obtained without the need to change code. However, I'd probably change it to make it a static determination of debug, to give the JIT every opportunity to optimize away virtual calls (does that work?). But for now, I think we should make this debug mechanism private and not extensible, as once protected it is part of the API.

I'm tempted by the idea that weak referencing shouldn't be made configurable at all, and ConcurrentPool should have this feature always on. My top reason for this preference is that ConcurrentPool already is a complex piece of lock-free code that we should keep as simple as possible.

+1

So long as we provide alternate classes that do simple or null pooling, then it is best to make our main ConcurrentPool implement the best algorithm we know and to not try to be too configurable unless there is good reason.

…ncurrentPool

The Pool is now a list of Holder instances, each with a WeakReference to an Entry and an optional hard reference. free on acquire added warnings

…ncurrentPool

lorban · 2023-11-01T21:53:14Z

@gregw I don't think DebugWeakHolder adds much value, if any, so I don't think it's worth the extra complexity in ConcurrentPool. Better leave leak tracking to the layer above, like ArrayByteBufferPool.Tracking IMHO.

…ncurrentPool

The Pool is now a list of Holder instances, each with a WeakReference to an Entry and an optional hard reference. free on acquire added warnings

* Removed unused boolean parameter "weakPool" in ArrayByteBufferPool and updated javadocs. * Improved implementation of ConcurrentPool.Entry.tryEnable(). * Added javadocs for ConcurrentPool.Holder.free() and hold(). Signed-off-by: Simone Bordet <[email protected]>

sbordet · 2023-11-09T07:59:22Z

jetty-core/jetty-io/src/main/java/org/eclipse/jetty/io/ArrayByteBufferPool.java

+     * @param maxDirectMemory the max direct memory in bytes, -1 for unlimited memory or 0 to use default heuristic
+     * @param weakPool true if the pooled buffers should be weakly referenced upon acquisition, false otherwise
+     */
+    public ArrayByteBufferPool(int minCapacity, int factor, int maxCapacity, int maxBucketSize, long maxHeapMemory, long maxDirectMemory, boolean weakPool)


@gregw the comments above are now out of date, and the weakPool parameter is never used, so please remove it.

jetty-core/jetty-io/src/main/java/org/eclipse/jetty/io/ArrayByteBufferPool.java

sbordet · 2023-11-09T08:57:16Z

jetty-core/jetty-util/src/main/java/org/eclipse/jetty/util/ConcurrentPool.java

+            if (!acquire)
+                getHolder().hold();


In tryAcquire(), the call to free() is done after the CAS. I think we should build the Holder with a strong reference by default, and only when we successfully CAS to 1 we call free().
In this way the two behaviors (here and in tryAcquire()) are normalized.

@sbordet Your change was a little different to this comment, but I think it is good.

jetty-core/jetty-util/src/main/java/org/eclipse/jetty/util/ConcurrentPool.java

…ConcurrentPool'. Signed-off-by: Simone Bordet <[email protected]>

Experiment with a weakly referenced ConcurrentPool

acbca5a

The Pool is now a list of Holder instances, each with a WeakReference to an Entry and an optional hard reference.

gregw requested review from sbordet and lorban October 25, 2023 23:16

gregw added 8 commits October 26, 2023 10:20

Experiment with a weakly referenced ConcurrentPool

9eebabb

The Pool is now a list of Holder instances, each with a WeakReference to an Entry and an optional hard reference. back link may be hard

Experiment with a weakly referenced ConcurrentPool

4fbcb0a

The Pool is now a list of Holder instances, each with a WeakReference to an Entry and an optional hard reference. free on acquire

Experiment with a weakly referenced ConcurrentPool

86ecaa2

The Pool is now a list of Holder instances, each with a WeakReference to an Entry and an optional hard reference. free on acquire removed ThreadLocal cache

Experiment with a weakly referenced ConcurrentPool

9e3d183

The Pool is now a list of Holder instances, each with a WeakReference to an Entry and an optional hard reference. free on acquire added Leak test

Experiment with a weakly referenced ConcurrentPool

9218c7d

The Pool is now a list of Holder instances, each with a WeakReference to an Entry and an optional hard reference. free on acquire added warnings

Experiment with a weakly referenced ConcurrentPool

9014d68

The Pool is now a list of Holder instances, each with a WeakReference to an Entry and an optional hard reference. free on acquire added warnings

Merge remote-tracking branch 'origin/jetty-12.0.x' into experiment/je…

db414fd

…tty-12/WeakReference-ConcurrentPool

Experiment with a weakly referenced ConcurrentPool

1f91daf

The Pool is now a list of Holder instances, each with a WeakReference to an Entry and an optional hard reference. free on acquire added warnings

gregw requested a review from lachlan-roberts October 26, 2023 05:40

lorban and others added 2 commits October 26, 2023 11:04

Modify ArrayByteBufferPool and ConcurrentPool to make weak-referencin…

ae9b7e8

…g configurable Signed-off-by: Ludovic Orban <[email protected]>

Experiment with a weakly referenced ConcurrentPool

c0eaa4a

The Pool is now a list of Holder instances, each with a WeakReference to an Entry and an optional hard reference. free on acquire added warnings

lorban reviewed Oct 27, 2023

View reviewed changes

jetty-core/jetty-util/src/test/java/org/eclipse/jetty/util/ConcurrentPoolTest.java Show resolved Hide resolved

Experiment with a weakly referenced ConcurrentPool

8ce0260

The Pool is now a list of Holder instances, each with a WeakReference to an Entry and an optional hard reference. free on acquire added warnings no race in tests

gregw marked this pull request as ready for review October 29, 2023 21:02

Experiment with a weakly referenced ConcurrentPool

11a3dff

The Pool is now a list of Holder instances, each with a WeakReference to an Entry and an optional hard reference. free on acquire added warnings no race in tests

lorban reviewed Oct 30, 2023

View reviewed changes

sbordet requested changes Oct 30, 2023

View reviewed changes

Experiment with a weakly referenced ConcurrentPool

0d54890

The Pool is now a list of Holder instances, each with a WeakReference to an Entry and an optional hard reference. free on acquire added warnings

gregw requested review from sbordet and lorban October 31, 2023 02:48

gregw added 3 commits November 1, 2023 07:55

Merge branch 'jetty-12.0.x' into experiment/jetty-12/WeakReference-Co…

c58905a

…ncurrentPool

A weakly referenced ConcurrentPool

bbcc930

The Pool is now a list of Holder instances, each with a WeakReference to an Entry and an optional hard reference. free on acquire added warnings

Merge branch 'jetty-12.0.x' into experiment/jetty-12/WeakReference-Co…

2604eb6

…ncurrentPool

lorban added this to the 12.0.x milestone Nov 1, 2023

lorban linked an issue Nov 1, 2023 that may be closed by this pull request

Jetty 12: leak tracking of Pool entries #9148

Closed

lorban added the Enhancement label Nov 1, 2023

gregw and others added 3 commits November 2, 2023 09:39

Merge branch 'jetty-12.0.x' into experiment/jetty-12/WeakReference-Co…

c28979f

…ncurrentPool

A weakly referenced ConcurrentPool

dd53992

The Pool is now a list of Holder instances, each with a WeakReference to an Entry and an optional hard reference. free on acquire added warnings

sbordet approved these changes Nov 9, 2023

View reviewed changes

lorban approved these changes Nov 9, 2023

View reviewed changes

Merged branch 'jetty-12.0.x' into 'experiment/jetty-12/WeakReference-…

3d0fb16

…ConcurrentPool'. Signed-off-by: Simone Bordet <[email protected]>

sbordet merged commit e800631 into jetty-12.0.x Nov 9, 2023
4 checks passed

sbordet deleted the experiment/jetty-12/WeakReference-ConcurrentPool branch November 9, 2023 21:32

sbordet mentioned this pull request Nov 11, 2023

Jetty 12: leak tracking of Pool entries #9148

Closed

lorban mentioned this pull request Mar 13, 2024

ConcurrentPool should not use the thread-local cache when running on a virtual thread #10568

Closed

lorban mentioned this pull request Sep 23, 2024

Investigate leak-resistant ConcurrentPool #10569

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Weak reference concurrent pool #10787

Weak reference concurrent pool #10787

gregw commented Oct 25, 2023

gregw commented Oct 25, 2023

gregw commented Oct 26, 2023

gregw commented Oct 26, 2023

lorban commented Oct 27, 2023 •

edited

Loading

lorban commented Oct 27, 2023

gregw commented Oct 28, 2023

lorban left a comment

lorban Oct 30, 2023

sbordet Oct 30, 2023

gregw Oct 30, 2023 •

edited

Loading

lorban Oct 30, 2023

sbordet Oct 30, 2023

gregw Oct 30, 2023 •

edited

Loading

sbordet Oct 30, 2023

sbordet Oct 30, 2023

sbordet Oct 30, 2023

gregw Oct 31, 2023

sbordet Oct 30, 2023

gregw Oct 30, 2023 •

edited

Loading

sbordet Nov 9, 2023

lorban commented Oct 31, 2023

gregw commented Oct 31, 2023

lorban commented Nov 1, 2023

sbordet Nov 9, 2023

sbordet Nov 9, 2023

gregw Nov 9, 2023

Weak reference concurrent pool #10787

Weak reference concurrent pool #10787

Conversation

gregw commented Oct 25, 2023

gregw commented Oct 25, 2023

gregw commented Oct 26, 2023

gregw commented Oct 26, 2023

lorban commented Oct 27, 2023 • edited Loading

lorban commented Oct 27, 2023

gregw commented Oct 28, 2023

lorban left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gregw Oct 30, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gregw Oct 30, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gregw Oct 30, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lorban commented Oct 31, 2023

gregw commented Oct 31, 2023

lorban commented Nov 1, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lorban commented Oct 27, 2023 •

edited

Loading

gregw Oct 30, 2023 •

edited

Loading

gregw Oct 30, 2023 •

edited

Loading

gregw Oct 30, 2023 •

edited

Loading