[🐛 Bug]: Selector manager threads are leaking when a proxy is passed and remoteWebDriver instantiation fails

[Camill06]

What happened?

Since I'm using the JDK http client, i'm facing an issue with selectorManager threads that are hanging forever and do not finish. After several calls, I can notice that the number of hanging threads are increasing and the Java garbage collector does not delete them.
This issue is only happening when those two conditions are reunited:

• A proxy is passed to the client config
• The remoteWebDriver instantiation fails. Whatever the reason of the failure : timeout, proxy not reachable, url not reachable, wrong driver version....

If the driver instantiation succeeds or if no proxy is passed, the selectorManager threads are not stuck and will be able to end.

How can we reproduce the issue?

// dummy proxy to have a failing instantiation      
Proxy myProxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress("localhost", 1234));

        ClientConfig config = null;
        URL myUrl = null;
        try {
            myUrl = new URL("http://myseleniumserver:1234");
        } catch (MalformedURLException e) {
            LOG.error("Remote URL is not correct", e);
        }
        config = ClientConfig.defaultConfig().baseUrl(myUrl).proxy(myProxy);

        HttpCommandExecutor myExecutor = null;
        Factory factory = Factory.createDefault();
        myExecutor = new HttpCommandExecutor(new HashMap<String, CommandInfo>(), config, factory);

        ChromeOptions copt = new ChromeOptions();
        try {
            WebDriver driver = new RemoteWebDriver(myExecutor, copt);
        } catch (WebDriverException e) {
            LOG.error("Driver could not be created", e);
        }

Relevant log output

NA

Operating System

macOS Sonoma 14.3.1

Selenium version

Java 4.18.1

What are the browser(s) and version(s) where you see this issue?

Chrome 122

What are the browser driver(s) and version(s) where you see this issue?

Chromedriver 122

Are you using Selenium Grid?

No response

[github-actions]

@Camill06, thank you for creating this issue. We will troubleshoot it as soon as we can.

---

Info for maintainers

Triage this issue by using labels.

If information is missing, add a helpful comment and then I-issue-template label.

If the issue is a question, add the I-question label.

If the issue is valid but there is no time to troubleshoot it, consider adding the help wanted label.

If the issue requires changes or fixes from an external project (e.g., ChromeDriver, GeckoDriver, MSEdgeDriver, W3C), add the applicable G-* label, and it will provide the correct link and auto-close the issue.

After troubleshooting the issue, please add the R-awaiting answer label.

Thank you!

[diemol]

@joerg1985 do you know if this was fixed with your recent commits?

[joerg1985]

@diemol No this has not been fixed, the httpclient inside the commandexecutor is not shutdown in the code above.
But the threads should terminate as soon as the httpclient is collected by the GC and the cleaners are running. This might take a while when alot of memory is available.

@Camill06 what JDK version are you using? There have been alot changes in the recent JDK versions releasing the threads.

[Camill06]

Hello @joerg1985, I''m using JDK version 17.0.8.1
In the case that I explained above, httpclient is never collected by the GC, even if there is no more memory available.

[joerg1985]

@Camill06 I can reproduce this and confirm the JdkHttpClient instances are never GC'ed and this is the cause for the selector threads not to stop.

From looking at a memory dump, i could not find the reason for the instances are never GC'ed.
When compiling selenium to a JDK 21+ jar the issue is gone, so i would guess this might be a JDK bug.
I think there might be a easy workaround, but i have to check this later in detail.

[joerg1985]

@Camill06 With the latest nightly build the issue is fixed for me, could you please retest it with the latest nightly build?

[Camill06]

Hi @joerg1985, thanks for your quick fix !
I tested and indeed It solves the issue with selector manager threads that were never closed before. Thanks !
I noticed that a new thread “Jdk-http-client-**” is now opened for each driver instantiation and is not closed right away, even with a GC. It was not the case in the previous version if I am not mistaken.
But, it seems that the number of “Jdk-http-client” threads simultaneously open does not exceed 30. So this should not create a memory issue.

[joerg1985]

These jdk-http-client threads are the renamed pool-X-thread-X threads created in the past. So i will close this as completed.

[github-actions]

This issue has been automatically locked since there has not been any recent activity since it was closed. Please open a new issue for related bugs.