Debugger connection appears to time out

[scottkurz]

Reports from internal user of the debugger connection timing out when using the Liberty Tools Eclipse "Debug" operation.

This is running w/ Semeru Java 17 on Win 11,
Eclipse OpenJ9 VM 17.0.7.0 (build openj9-0.38.0, JRE 17 Windows 11 amd64-64-Bit ...

Interestingly enough, this particular user was able to successfully establish a Debug connection with the same app on the same system, using the WDT/LDT feature in another Eclipse install.

I wonder if there is anything to consider here?:
https://github.com/OpenLiberty/open-liberty-tools/blob/integration/dev/com.ibm.ws.st.core/src/com/ibm/ws/st/core/internal/launch/BaseLibertyLaunchConfiguration.java

Note: The solution here could possibly overlap #400 as this issue suggests the need for a new "Liberty Tools" Run/Debug config type of our own.

[scottkurz]

Similar error running Windows 11 plus a local build of PR #437

!ENTRY org.eclipse.jdt.debug 4 125 2023-10-16 18:05:29.230
!MESSAGE Internal error logged from JDI Debug: 
!STACK 0
org.eclipse.jdi.TimeoutException: Timeout occurred while waiting for packet 21412.
	at org.eclipse.jdi.internal.connect.PacketReceiveManager.getReply(PacketReceiveManager.java:196)
	at org.eclipse.jdi.internal.connect.PacketReceiveManager.getReply(PacketReceiveManager.java:207)
	at org.eclipse.jdi.internal.MirrorImpl.requestVM(MirrorImpl.java:208)
	at org.eclipse.jdi.internal.MirrorImpl.requestVM(MirrorImpl.java:243)
	at org.eclipse.jdi.internal.VirtualMachineImpl.classesBySignature(VirtualMachineImpl.java:658)
	at org.eclipse.jdi.internal.VirtualMachineImpl.classesByName(VirtualMachineImpl.java:688)
	at org.eclipse.jdt.internal.debug.core.model.JDIDebugTarget.jdiClassesByName(JDIDebugTarget.java:1956)
	at org.eclipse.jdt.internal.debug.core.model.JDIDebugTarget.filterNotLoadedTypes(JDIDebugTarget.java:3318)
	at org.eclipse.jdt.internal.debug.core.hcr.JavaHotCodeReplaceManager.doHotCodeReplace(JavaHotCodeReplaceManager.java:443)
	at org.eclipse.jdt.internal.debug.core.hcr.JavaHotCodeReplaceManager.lambda$0(JavaHotCodeReplaceManager.java:317)
	at org.eclipse.debug.core.DebugPlugin$AsynchRunner.run(DebugPlugin.java:1186)
	at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:45)
	at org.eclipse.debug.core.DebugPlugin$AsynchRunner.async(DebugPlugin.java:1173)
	at org.eclipse.debug.core.DebugPlugin$EventDispatchJob.run(DebugPlugin.java:489)
	at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)

WDT debug works fine with the same app and JVM

[scottkurz]

User recreated with this in in .log:

eclipse.buildId=4.29.0.20230907-1200

java.fullversion=17.0.8.1+1
JRE 17 Windows 11 amd64-64-Bit Compressed References 20230824_473 (JIT enabled, AOT enabled)
OpenJ9 - d12d10c9e
OMR - e80bff83b
JCL - 87c2977201e based on jdk-17.0.8.1+1

[scottkurz]

To recap (the info received over DM plus earlier comments in this issue):

• The problem persists after switching from Remote Java App type of Debug config to our custom Liberty Tools debug config added in 23.0.12
• The problem does NOT seem to be relieved simply by increasing the debugger timeout values from preferences (as in the screenshot):
  
• It is unexplained how the problem doesn't happen for the same JRE, the same machine, and the same app, but using WDT instead of LDT. This could be interesting to look at. In moving from the Remote Java App config to Liberty Tools config we made the paths a bit more common, arguably, but there is still some difference.
• The problem seems HCR-specific .. it seems to happen specifically when an HCR update is happening, as opposed to other operations, stepping into, past breakpoints, etc.

org.eclipse.jdi.TimeoutException: Timeout occurred while waiting for packet 12766.
at org.eclipse.jdi.internal.connect.PacketReceiveManager.getReply(PacketReceiveManager.java:196)
at org.eclipse.jdi.internal.connect.PacketReceiveManager.getReply(PacketReceiveManager.java:207)
at org.eclipse.jdi.internal.MirrorImpl.requestVM(MirrorImpl.java:208)
at org.eclipse.jdi.internal.MirrorImpl.requestVM(MirrorImpl.java:243)
at org.eclipse.jdi.internal.VirtualMachineImpl.classesBySignature(VirtualMachineImpl.java:658)
at org.eclipse.jdi.internal.VirtualMachineImpl.classesByName(VirtualMachineImpl.java:688)
at org.eclipse.jdt.internal.debug.core.model.JDIDebugTarget.jdiClassesByName(JDIDebugTarget.java:1956)
at org.eclipse.jdt.internal.debug.core.model.JDIDebugTarget.filterNotLoadedTypes(JDIDebugTarget.java:3318)
at org.eclipse.jdt.internal.debug.core.hcr.JavaHotCodeReplaceManager.doHotCodeReplace(JavaHotCodeReplaceManager.java:443)
at org.eclipse.jdt.internal.debug.core.hcr.JavaHotCodeReplaceManager.lambda$0(JavaHotCodeReplaceManager.java:317)
at org.eclipse.debug.core.DebugPlugin$AsynchRunner.run(DebugPlugin.java:1187)
at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:45)
at org.eclipse.debug.core.DebugPlugin$AsynchRunner.async(DebugPlugin.java:1174)
at org.eclipse.debug.core.DebugPlugin$EventDispatchJob.run(DebugPlugin.java:490)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)

[scottkurz]

Moving out to the next milestone.

Some thoughts:

Is it conceivable that the concurrent compiles, from both the IDE and from dev mode, could be triggering this problem? Could that be a reason this is never seen in LDT/WDT for the same app?

Possible things to look at:

• see if a similar problem can be recreated running liberty:run (not using an LT Run/Debug config obviously) with a Remote Java App Debug connection active, and the IDE doing an incremental compilation. If so.. it's not dev mode to blame.
• If dev mode does still seem like a candidate for the cause, consider, as we investigate Dev mode slowdown with high CPU usage on Windows & Intel Mac with project with large number of files ci.maven#1709 (disabling dev mode compile within "IDE or lite dev mode", trying to make a run with the app provoking this problem against our prototype
• If we remain stuck try an issue https://github.com/eclipse-jdt/eclipse.jdt.debug

Note there is a separate more straightforward angle to pursue wrt jdt.debug... there is a timeout, why isn't it taking effect?

It is useful to consider if the double-compile could be relevant, but we could consider the jdt.debug code and its timeout independently perhaps.