Jetty http client metrics not pegging for http2 (plain text) request/response.

[Niteshkumar152]

jetty Version=11.0.17
Java Version = 17

I am using the http2 Client of Jetty(plain text).
Following is the code snipped of initializing the HTTP client:

public void init(){

     SslContextFactory sslContextFactory = new SslContextFactory.Client(true);
     ClientConnector clientConnector= new ClientConnector() {
         protected void configure(SelectableChannel selectable) throws IOException {
            super.configure(selectable);
	    if (selectable instanceof NetworkChannel) {
			NetworkChannel channel = (NetworkChannel)selectable;
			channel.setOption(java.net.StandardSocketOptions.SO_KEEPALIVE,
					tcpConfigOptionProvider.getTcpKeepalive().getEnable());
                        // Set keepalive parameters only if it is enabled
					if(tcpConfigOptionProvider.getTcpKeepalive().getEnable()) {
						channel.setOption(jdk.net.ExtendedSocketOptions.TCP_KEEPIDLE,
								Integer.parseInt(StringUtils.chop(tcpConfigOptionProvider.getTcpKeepalive().getTime())));
						channel.setOption(jdk.net.ExtendedSocketOptions.TCP_KEEPINTERVAL,
								Integer.parseInt(StringUtils.chop(tcpConfigOptionProvider.getTcpKeepalive().getInterval())));
						channel.setOption(jdk.net.ExtendedSocketOptions.TCP_KEEPCOUNT,
								tcpConfigOptionProvider.getTcpKeepalive().getProbes());
					}
                                        tcpKeepaliveChannelCofigDetails(channel);

				}
			}
		};
		
		clientConnector.setSslContextFactory((SslContextFactory.Client) sslContextFactory);

		HTTP2Client http2Client = new HTTP2Client(clientConnector);
		http2Client.setMaxConcurrentPushedStreams(maxConcurrentPushedStreams);

		HttpClientTransportOverHTTP2 transport = new HttpClientTransportOverHTTP2(http2Client);
		transport.setUseALPN(false);
		this.httpClient = new HttpClient(transport);
		httpClient.addBean(new JettyConnectionMetrics(SoothsayerMetrics.getInstance().getRegistry()));
		this.httpClient.setIdleTimeout(idleTimeout);
		this.httpClient.setConnectTimeout(httpClientConnectionTimeout);

		this.httpClient.setMaxRequestsQueuedPerDestination(maxRequestsQueuedPerDestination);
		this.httpClient.setMaxConnectionsPerDestination(maxConnectionsPerDestination);
		this.httpClient.setFollowRedirects(false);
		this.httpClient.setRemoveIdleDestinations(true);
		//httpClient.getRequestListeners().add(new JettyClientRequestMetrics());
		httpClient.addBean(new ConnectionStatistics());
		httpClient.addBean(new JettyConnectionMetrics(SoothsayerMetrics.getInstance().getRegistry()));
		this.httpClient.setUserAgentField(null);

		logger.info("Jetty Max Requests: configured value: {}, set value: {}", maxRequestsQueuedPerDestination,
				this.httpClient.getMaxRequestsQueuedPerDestination());
		logger.info("Jetty Idle Timeout: configured value: {}, set value: {}", idleTimeout,
				this.httpClient.getIdleTimeout());
		logger.info("Jetty Max Connections Per Destination: configured value: {}, set value: {}",
				maxConnectionsPerDestination, this.httpClient.getMaxConnectionsPerDestination());
		logger.info("Jetty Http Client Connection Timeout: configured value: {}, set value: {}",
				httpClientConnectionTimeout, this.httpClient.getConnectTimeout());
		try {
			httpClient.start();
			httpClient.getContentDecoderFactories().clear();
		} catch (Exception e) {
			
		}

		ClientHttpConnector httpConnector = new JettyClientHttpConnector(httpClient);
		this.httpDestWebClient = WebClient.builder().codecs(configurer -> configurer.defaultCodecs()
						.maxInMemorySize(2048))
				.clientConnector(httpConnector).baseUrl(extUrl).build();
		
		}

I am trying to enable jetty client metrics for http2(plain text) for the following:
1.) http2 client connection open/close
2.) http2 request sent out and response received

	I did the following to enable jetty client metrics:

// Implemented listener interface
		public class JettyClientRequestMetrics implements Listener {
	
        private static Logger logger = LogManager.getLogger(JettyClientRequestMetrics.class);
       
           
        public JettyClientRequestMetrics() {
        }
       
        @Override
        public void onQueued(Request request)
        {
       	       logger.debug("JettyClientRequestMetrics::onQueued IN"); 
			   Metrics.counter("jetty.client.requests", "eventtype","onQueued").increment();
               logger.debug("JettyClientRequestMetrics::onQueued OUT");  
        }

...overriden all the required methods.
Added this class as a bean to http client as well as http2 client beans.

httpClient.getRequestListeners().add(new JettyClientRequestMetrics());

Looks like this is pegging metrics only for http1.1 request, not http2 request. Also tried by adding the following two beans:

httpClient.addBean(new ConnectionStatistics());
httpClient.addBean(new JettyConnectionMetrics(SoothsayerMetrics.getInstance().getRegistry()));

http2Client.addBean(new JettyConnectionMetrics(SoothsayerMetrics.getInstance().getRegistry()));
http2Client.addBean(new JettyClientRequestMetrics());
http2Client.addBean(new ConnectionStatistics());

With above changes, the http dump doesn't show correct data.
Pasted Snippet below:

+= JettyConnectionMetrics@254ab3d7{STARTED} - STARTED
+= ConnectionStatistics@33625338 - STARTED
|  +> Stats[total]
|  |  +> connections=CounterStatistic@80aafd{c=4,m=4,t=4}
|  |  +> durations=SampleStatistic@7b09ddd1{count=0,max=0,mean=0.000000,total=0,stddev=0.000000}
|  |  +> bytes in/out=0/0
|  |  +> messages in/out=0/0
|  +> Stats[org.eclipse.jetty.http2.client.HTTP2ClientConnectionFactory$HTTP2ClientConnection]
|     +> connections=CounterStatistic@bad1d6c{c=4,m=4,t=4}
|     +> durations=SampleStatistic@2af764b2{count=0,max=0,mean=0.000000,total=0,stddev=0.000000}
|     +> bytes in/out=0/0
|     +> messages in/out=0/0
+= JettyConnectionMetrics@2f25f453{STARTED} - STARTED

Could you please help in fixing this problem?

[Niteshkumar152]

Hi team,

This is a gentle reminder to please help me fix, the above-stated problem.

I appreciate any help you can provide.

[joakime]

Jetty 11 is now at End of Community Support.

• EoCS (End of Community Support) for Jetty 10 / Jetty 11 - January 2024 #10485
• EoCS (End of Community Support) for Jetty 9 - June 2022 #7958

You should be using a supported version of Jetty (Jetty 12.0.x) at this point in time.

[Niteshkumar152]

Ok, sure

Thanks @joakime !!