servlet: ignore IllegalStateException from AsyncContext.complete()

[panchenko]

There are errors logged in case of DEADLINE_EXCEEDED which I would like to avoid.

2025-01-13 11:46:01.942 ERROR [grpc-timer-0]                 SerializingExecutor           Exception while executing runnable io.grpc.servlet.ServletServerStream$Sink$$Lambda$2200/0x0000000801a1cab0@3580f36e
java.lang.IllegalStateException: Calling [asyncComplete()] is not valid for a request with Async state [COMPLETING]
	at org.apache.coyote.AsyncStateMachine.asyncComplete(AsyncStateMachine.java:344)
	at org.apache.coyote.AbstractProcessor.action(AbstractProcessor.java:507)
	at org.apache.coyote.Request.action(Request.java:505)
	at org.apache.catalina.core.AsyncContextImpl.complete(AsyncContextImpl.java:92)
	at io.grpc.servlet.ServletServerStream$Sink.lambda$cancel$2(ServletServerStream.java:303)
	at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133)
	at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:31)
	at io.grpc.internal.SerializingExecutor.schedule(SerializingExecutor.java:102)
	at io.grpc.internal.SerializingExecutor.execute(SerializingExecutor.java:95)
	at io.grpc.servlet.ServletServerStream$ServletTransportState.runOnTransportThread(ServletServerStream.java:146)
	at io.grpc.servlet.ServletServerStream$Sink.cancel(ServletServerStream.java:302)
	at io.grpc.internal.AbstractServerStream.cancel(AbstractServerStream.java:148)
	at io.grpc.internal.ServerImpl$ServerTransportListenerImpl$1HandleServerCall$1ServerStreamCancellationListener.cancelled(ServerImpl.java:632)
	at io.grpc.Context$ExecutableListener.run(Context.java:1065)
	at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:31)
	at io.grpc.Context$ExecutableListener.deliver(Context.java:1057)
	at io.grpc.Context$CancellableContext.notifyAndClearListeners(Context.java:860)
	at io.grpc.Context$CancellableContext.cancel(Context.java:833)
	at io.grpc.Context$CancellableContext$1CancelOnExpiration.run(Context.java:696)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
	at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.base/java.lang.Thread.run(Unknown Source)

[kannanjgithub]

I tried reproing by inducing a timeout and I see that the org.eclipse.jetty.server.HttpChannelState is getting set to EXPIRE

if (this._requestState == HttpChannelState.RequestState.ASYNC) {
          this._requestState = HttpChannelState.RequestState.EXPIRE;

timeout:615, HttpChannelState (org.eclipse.jetty.server)
run:148, AsyncContextEvent (org.eclipse.jetty.server)
call:539, Executors$RunnableAdapter (java.util.concurrent)
run:264, FutureTask (java.util.concurrent)
run:304, ScheduledThreadPoolExecutor$ScheduledFutureTask (java.util.concurrent)
runWorker:1136, ThreadPoolExecutor (java.util.concurrent)
run:635, ThreadPoolExecutor$Worker (java.util.concurrent)
run:833, Thread (java.lang)

and that the exception thrown from asyncContext.complete(); has the message

s=HANDLING rs=EXPIRE os=OPEN is=IDLE awp=false se=false i=false al=1

It did not have the state COMPLETING.

[panchenko]

• The message is different in different servers (Tomcat / Jetty), as state names is an implementation detail.
• It happens multiple times when running tests for this module, like connection closed by client after deadline exceeded, etc
• I have tried to add more checks before this call, some exceptions can be avoided with that, but then decided it does not worth it

[ejona86]

IllegalStateException means we've either called something incorrectly (so we should fix it) or there's a bug in the servlet container (and we should file a bug). I don't think we should have something like this without a tracking issue to fix whatever the actual bug is, as this should be removed medium-term.

Needs more investigation.

[panchenko]

Adding related checks in #12296