Duplicated log events in AppInsights dashboard

[Sherkdavid]

We tried using AppInsights agent 3.0.0-PREVIEW.7 to push log4j events for a JavaApplication server to see INFO level logs reported, the events are logged as expected but we are now seeing duplicated entries in the AppInsights traces.

Expected behavior

Server logs appear in the AppInsights dashboard at all levels

Actual behavior

Server logs appear but with a duplicate written for each

To Reproduce

1. Start the java application server using Log4j2 with jvm arg -javaagent applicationinsights-agent-3.0.0-PREVIEW.7.jar
2. Observe that events can be seen duplicated in ApplicationInsights traces

Log4j configuration

We thought this issue might have something to do with having multiple loggers receiving events with additivity="false" but it's been reproducible with a fully stripped back configuration comprising a single Logger and Appender.

<Configuration status="error" name="foo" monitorInterval="300">
  
  <Appenders>
    <RollingFile name="foo" fileName="${env:LOG_DIR:-${sys:app.home:-/tmp}/logs}/foo-server.log"
                 filePattern="$${env:LOG_DIR:-$${sys:app.home:-/tmp}/logs}/foo-server-%d{yyyy-MM-dd'T'HH:mm:ss}{UTC}.log">
      <PatternLayout pattern="%d{ISO8601}{UTC}|%-5p|%t|%c{1.}|$${ctx:CORRELATION_ID:-N/A}|%m%n" charset="UTF-8"/>
      <Policies>
        <OnStartupTriggeringPolicy minSize="1"/>
        <TimeBasedTriggeringPolicy interval="86400" modulate="true"/>
        <SizeBasedTriggeringPolicy size="200 MB"/>
      </Policies>
      <DefaultRolloverStrategy>
        <Delete basePath="${env:LOG_DIR:-${sys:app.home:-/tmp}/logs}">
          <IfFileName glob="foo-server-*.log"/>
          <IfAccumulatedFileCount exceeds="26"/>
        </Delete>
      </DefaultRolloverStrategy>
    </RollingFile>
  </Appenders>
  
  <Loggers>
    <Root level="${env:ROOT_LOG_LEVEL:-INFO}">
      <AppenderRef ref="foo"/>
    </Root>
  </Loggers>
  
</Configuration>

System information

Please provide the following information:

• SDK Version: 3.0.0-PREVIEW.7
• OS type and version: MacOS Mojave 10.14.6
• Application Server type and version (if applicable): Java (Jetty 9.4.19.v20190610)
• Using spring-boot? No
• Additional relevant libraries (with version, if applicable): Log4j 2.12.1

Screenshots

Unsure at what point it is events are being duplicated, ingest or egest. I'm monitoring this issue and can follow through on producing any useful debugging on the server side. We had been pushing our logging successfully using com.microsoft.azure:applicationinsights-logging-log4j2:2.6.1 prior.

[trask]

Hi @Sherkdavid! Can post a repro that we can use to troubleshoot this?

[Sherkdavid]

Hey @trask, yeah after trying to reproduce this with a minimal jar it look like duplicates are only produced when an application is run through an IDE (IntelliJ 2019-1-4 Community)..

Running with java -jar ... -javaagent:applicationinsights-agent-3.0.0-PREVIEW.7.jar seems to have the expected behavior

I've attached a zip of the gradle project if it's of any use still.
ai-dupe.zip