Add support for JMX connection retries [SPR-4910]

[spring-projects-issues]

Matt Wright opened SPR-4910 and commented

Add support in MBeanClientInterceptor to allow for the server connection to be reconnected after a connection failure. This would be similar to what has already been done for the JaxRpcPortClientInterceptor.

---

Affects: 2.5.5

[spring-projects-issues]

Juergen Hoeller commented

As far as I can tell, Sun's standard JMX connectors are actually able to recover from an MBeanServer restart on the fly... That's the main reason why we haven't built a recovery mechanism ourselves there. I could imagine that the RMI connector might fail to recover in some scenarios, though. It's generally the responsibility of the MBeanServerConnection to perform such recovery... we might possibly provide an optional decorator there - for MBeanServerConnections that cannot recover natively.

Where exactly did you experience a recovery problem?

Juergen

[spring-projects-issues]

Matt Wright commented

I have noticed that if the remote mbean is restarted that the cached connection is not restored. Any subsequent calls receive a ConnectException, Here is the stacktrace:

org.springframework.jmx.access.InvocationFailureException: I/O failure during JMX access; nested exception is java.rmi.ConnectException: Connection refused to host: xxx.xxx.xxx.xxx; nested exception is:
java.net.ConnectException: Connection refused: connect
org.springframework.jmx.access.MBeanClientInterceptor.invoke(MBeanClientInterceptor.java:374)
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)

This happens both when a mbeanserver connection is used as well as a serviceUrl

[spring-projects-issues]

Juergen Hoeller commented

Actually, Sun's RemoteMBeanServerConnection implementation (as used by the RMIConnector) contains full-fledged recovery code already, even using an async recovery thread... In case of any IOException raised by the underlying RMI connection, it automatically reinitializes the stub and retries the invocation! I've verified this in both the JDK 1.6 and 1.5 source code.

Which exact RMI stack are you using there? Try debugging a bit into the JMX RMI connector and see what happens there in your scenario...

Juergen

[spring-projects-issues]

Matt Wright commented

I have spent some time looking into your suggestions. The Sun RMIConnection does have an async thread that will tests the connection and reconnect if necessary; however, if it fails to recover the connection, the thread will stop and never start again. I am working with IBM WebSphere, but I am not sure of their implementation.

[spring-projects-issues]

Juergen Hoeller commented

I've added a "refreshOnConnectFailure" property to MBeanClientInterceptor, allowing for automatic reconnect in case of an IOException thrown from an MBeanServerConnection invocation. This will work on top of any recovery support that the underlying JMX connector may already provide, whether it is Sun's or IBM's.

Juergen

[spring-projects-issues]

Juergen Hoeller commented

This will be available in tonight's 2.5.6 snapshot (http://static.springframework.org/downloads/nightly/snapshot-download.php?project=SPR). Please give it a try and let me know whether it works for you...

Juergen