[APIM 4.2.0] - apim_metrics.log file is not updating

[dilshanfardil]

Description

Hi Team,

Enable the ELK with APIM 4.2.0. You can follow this documentation [1]. Then we are getting the error while invoking an API

[2023-12-28 14:42:00,439]  WARN - SourceHandler I/O error: Received fatal alert: certificate_unknown
[2023-12-28 14:42:00,536]  INFO - LogMediator {api:PizzaShackAPI:v1.0.0} To: /pizzashack/1.0.0/menu, MessageID: urn:uuid:7564f0b5-dce4-4806-b1fd-d7230ddd4770, correlation_id: 7564f0b5-dce4-4806-b1fd-d7230ddd4770, Direction: request, MESSAGE = MESSAGE, Envelope: <?xml version='1.0' encoding='utf-8'?><soapenv:Envelope xmlns:soapenv="http://www.w3.org/2003/05/soap-envelope"><soapenv:Body/></soapenv:Envelope>
[2023-12-28 14:42:00,554] ERROR - AnalyticsMetricsHandler Error Occurred when collecting data
java.lang.NullPointerException: null
	at org.wso2.carbon.apimgt.gateway.handlers.analytics.SynapseAnalyticsDataProvider.getBackendLatency_aroundBody54(SynapseAnalyticsDataProvider.java:394) ~[org.wso2.carbon.apimgt.gateway_9.28.116.106.jar:?]
	at org.wso2.carbon.apimgt.gateway.handlers.analytics.SynapseAnalyticsDataProvider.getBackendLatency(SynapseAnalyticsDataProvider.java:1) ~[org.wso2.carbon.apimgt.gateway_9.28.116.106.jar:?]
	at org.wso2.carbon.apimgt.gateway.handlers.analytics.SynapseAnalyticsDataProvider.getLatencies_aroundBody18(SynapseAnalyticsDataProvider.java:205) ~[org.wso2.carbon.apimgt.gateway_9.28.116.106.jar:?]
	at org.wso2.carbon.apimgt.gateway.handlers.analytics.SynapseAnalyticsDataProvider.getLatencies(SynapseAnalyticsDataProvider.java:1) ~[org.wso2.carbon.apimgt.gateway_9.28.116.106.jar:?]
	at org.wso2.carbon.apimgt.common.analytics.collectors.impl.SuccessRequestDataCollector.collectData(SuccessRequestDataCollector.java:71) ~[org.wso2.carbon.apimgt.common.analytics_9.28.116.jar:?]
	at org.wso2.carbon.apimgt.common.analytics.collectors.impl.GenericRequestDataCollector.collectData(GenericRequestDataCollector.java:46) ~[org.wso2.carbon.apimgt.common.analytics_9.28.116.jar:?]
	at org.wso2.carbon.apimgt.gateway.handlers.analytics.AnalyticsMetricsHandler.handleResponseOutFlow_aroundBody8(AnalyticsMetricsHandler.java:95) ~[org.wso2.carbon.apimgt.gateway_9.28.116.106.jar:?]
	at org.wso2.carbon.apimgt.gateway.handlers.analytics.AnalyticsMetricsHandler.handleResponseOutFlow(AnalyticsMetricsHandler.java:1) ~[org.wso2.carbon.apimgt.gateway_9.28.116.106.jar:?]
	at org.apache.synapse.core.axis2.Axis2Sender.sendBack(Axis2Sender.java:188) ~[synapse-core_4.0.0.wso2v20_25.jar:4.0.0-wso2v20.25]
	at org.apache.synapse.mediators.builtin.RespondMediator.mediate(RespondMediator.java:46) ~[synapse-core_4.0.0.wso2v20_25.jar:4.0.0-wso2v20.25]
	at org.apache.synapse.mediators.AbstractListMediator.mediate(AbstractListMediator.java:126) ~[synapse-core_4.0.0.wso2v20_25.jar:4.0.0-wso2v20.25]
	at org.apache.synapse.mediators.AbstractListMediator.mediate(AbstractListMediator.java:74) ~[synapse-core_4.0.0.wso2v20_25.jar:4.0.0-wso2v20.25]
	at org.apache.synapse.mediators.base.SequenceMediator.mediate(SequenceMediator.java:158) ~[synapse-core_4.0.0.wso2v20_25.jar:4.0.0-wso2v20.25]
	at org.wso2.carbon.apimgt.gateway.handlers.ext.APIManagerExtensionHandler.mediate_aroundBody0(APIManagerExtensionHandler.java:78) ~[org.wso2.carbon.apimgt.gateway_9.28.116.106.jar:?]
	at org.wso2.carbon.apimgt.gateway.handlers.ext.APIManagerExtensionHandler.mediate(APIManagerExtensionHandler.java:1) ~[org.wso2.carbon.apimgt.gateway_9.28.116.106.jar:?]
	at org.wso2.carbon.apimgt.gateway.handlers.ext.APIManagerExtensionHandler.handleRequest_aroundBody2(APIManagerExtensionHandler.java:112) ~[org.wso2.carbon.apimgt.gateway_9.28.116.106.jar:?]
	at org.wso2.carbon.apimgt.gateway.handlers.ext.APIManagerExtensionHandler.handleRequest(APIManagerExtensionHandler.java:1) ~[org.wso2.carbon.apimgt.gateway_9.28.116.106.jar:?]
	at org.apache.synapse.api.API.process(API.java:403) ~[synapse-core_4.0.0.wso2v20_25.jar:4.0.0-wso2v20.25]
	at org.apache.synapse.api.AbstractApiHandler.apiProcessNonDefaultStrategy(AbstractApiHandler.java:108) ~[synapse-core_4.0.0.wso2v20_25.jar:4.0.0-wso2v20.25]
	at org.apache.synapse.api.AbstractApiHandler.identifyAPI(AbstractApiHandler.java:128) ~[synapse-core_4.0.0.wso2v20_25.jar:4.0.0-wso2v20.25]
	at org.apache.synapse.api.AbstractApiHandler.dispatchToAPI(AbstractApiHandler.java:60) ~[synapse-core_4.0.0.wso2v20_25.jar:4.0.0-wso2v20.25]
	at org.apache.synapse.api.rest.RestRequestHandler.dispatchToAPI(RestRequestHandler.java:90) ~[synapse-core_4.0.0.wso2v20_25.jar:4.0.0-wso2v20.25]
	at org.apache.synapse.api.rest.RestRequestHandler.process(RestRequestHandler.java:76) ~[synapse-core_4.0.0.wso2v20_25.jar:4.0.0-wso2v20.25]
	at org.apache.synapse.rest.RESTRequestHandler.process(RESTRequestHandler.java:54) ~[synapse-core_4.0.0.wso2v20_25.jar:4.0.0-wso2v20.25]
	at org.apache.synapse.core.axis2.Axis2SynapseEnvironment.injectMessage(Axis2SynapseEnvironment.java:350) ~[synapse-core_4.0.0.wso2v20_25.jar:4.0.0-wso2v20.25]
	at org.apache.synapse.core.axis2.SynapseMessageReceiver.receive(SynapseMessageReceiver.java:101) ~[synapse-core_4.0.0.wso2v20_25.jar:4.0.0-wso2v20.25]
	at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180) ~[axis2_1.6.1.wso2v83_3.jar:?]
	at org.apache.synapse.transport.passthru.ServerWorker.processNonEntityEnclosingRESTHandler(ServerWorker.java:399) ~[synapse-nhttp-transport_4.0.0.wso2v20_20.jar:?]
	at org.apache.synapse.transport.passthru.ServerWorker.run(ServerWorker.java:213) ~[synapse-nhttp-transport_4.0.0.wso2v20_20.jar:?]
	at org.apache.axis2.transport.base.threads.NativeWorkerPool$1.run(NativeWorkerPool.java:172) ~[axis2_1.6.1.wso2v83_3.jar:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
	at java.lang.Thread.run(Thread.java:829) [?:?]
[2023-12-28 14:51:57,636]  WARN - SourceHandler I/O error: Received fatal alert: certificate_unknown
[2023-12-28 14:52:08,949]  INFO - LogMediator {api:PizzaShackAPI:v1.0.0} To: /pizzashack/1.0.0/menu, MessageID: urn:uuid:991a3a8b-49e6-4f8c-b4f3-7c6a0f225758, correlation_id: 991a3a8b-49e6-4f8c-b4f3-7c6a0f225758, Direction: request, MESSAGE = MESSAGE, Envelope: <?xml version='1.0' encoding='utf-8'?><soapenv:Envelope xmlns:soapenv="http://www.w3.org/2003/05/soap-envelope"><soapenv:Body/></soapenv:Envelope>

[1] https://apim.docs.wso2.com/en/latest/api-analytics/on-prem/elk-installation-guide/

Steps to Reproduce

You can follow this documentation [1].

1. Adding this to the deployment.toml.

[apim.analytics]
enable = true
type = "elk"

2. Adding this to the log4j2.properties

Add APIM_METRICS_APPENDER to the appenders list:

appenders = APIM_METRICS_APPENDER, .... (list of other available appenders)

Add the following configuration after the appenders:

appender.APIM_METRICS_APPENDER.type = RollingFile
appender.APIM_METRICS_APPENDER.name = APIM_METRICS_APPENDER
appender.APIM_METRICS_APPENDER.fileName = /repository/logs/apim_metrics.log
appender.APIM_METRICS_APPENDER.filePattern = /repository/logs/apim_metrics-%d{MM-dd-yyyy}-%i.log
appender.APIM_METRICS_APPENDER.layout.type = PatternLayout
appender.APIM_METRICS_APPENDER.layout.pattern = %d{HH:mm:ss,SSS} [%X{ip}-%X{host}] [%t] %5p %c{1} %m%n
appender.APIM_METRICS_APPENDER.policies.type = Policies
appender.APIM_METRICS_APPENDER.policies.time.type = TimeBasedTriggeringPolicy
appender.APIM_METRICS_APPENDER.policies.time.interval = 1
appender.APIM_METRICS_APPENDER.policies.time.modulate = true
appender.APIM_METRICS_APPENDER.policies.size.type = SizeBasedTriggeringPolicy
appender.APIM_METRICS_APPENDER.policies.size.size=1000MB
appender.APIM_METRICS_APPENDER.strategy.type = DefaultRolloverStrategy
appender.APIM_METRICS_APPENDER.strategy.max = 10

Add a reporter to the loggers list:

loggers = reporter, ...(list of other available loggers)

Add the following configurations after the loggers:

logger.reporter.name = org.wso2.am.analytics.publisher.reporter.elk
logger.reporter.level = INFO
logger.reporter.additivity = false
logger.reporter.appenderRef.APIM_METRICS_APPENDER.ref = APIM_METRICS_APPENDER

3. Add a mediation to the insequnce with the mediator in the mediation policy and invoke the API.

<?xml version="1.0" encoding="UTF-8"?>
<sequence xmlns="http://ws.apache.org/ns/synapse" name="PizzaShackAPI:v1.0.0--In">
   <property name="switchExpression"
             expression="fn:concat($ctx:REST_METHOD,'_',$ctx:API_ELECTED_RESOURCE)"/>
   <switch source="$ctx:switchExpression">
      <case regex="GET_\/menu">
         <log level="full">
            <property name="MESSAGE" value="MESSAGE"/>
         </log>
      </case>
   </switch>
   <respond/>
</sequence>

[1] https://apim.docs.wso2.com/en/latest/api-analytics/on-prem/elk-installation-guide/

Affected Component

APIM

Version

4.2.0

Environment Details (with versions)

No response

Relevant Log Output

No response

Related Issues

No response

Suggested Labels

No response