[CI] TransformUsageIT.testUsage fails with incorrect stats call count

[dakrone]

In https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+master+multijob+fast+part2/3953/consoleFull / https://gradle-enterprise.elastic.co/s/nobrqofhrecye/tests/vkcr26w2ldob6-j74o4svwlg7nm

The TransformUsageIT.testUsage test failed with:

java.lang.AssertionError: 
Incorrect stat trigger_count expected:<3.0> but was:<2.0>
at __randomizedtesting.SeedInfo.seed([5671C4D5CB7154:BCF1ED1069B7A9DD]:0)
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:834)
at org.junit.Assert.assertEquals(Assert.java:553)
at org.elasticsearch.xpack.transform.integration.TransformUsageIT.lambda$testUsage$2(TransformUsageIT.java:105)
at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:881)
...

I was unable to reproduce this on master with:

./gradlew ':x-pack:plugin:transform:qa:single-node-tests:integTestRunner' --tests "org.elasticsearch.xpack.transform.integration.TransformUsageIT.testUsage" -Dtests.seed=5671C4D5CB7154 -Dtests.security.manager=true -Dtests.locale=th-TH -Dtests.timezone=Africa/Bissau -Dcompiler.java=13

[elasticmachine]

Pinging @elastic/ml-core (:ml/Transform)

[hendrikmuhs]

Thanks for the report. I took a look and it seems to be a bad test. trigger_count is a counter for how many times the transform has been triggered, this is not fully under control by the test but runs in a separate thread (scheduler thread). The scheduler is configured to fire every second, which is probably what happened. I will take out the test for trigger_count or change it to test "greater than or equal".

[davidkyle]

Another failure, reopening this ticket

Incorrect stat trigger_count
Expected: a value equal to or greater than <3.0>
     but: <2.0> was less than <3.0>

at __randomizedtesting.SeedInfo.seed([78BA13B22EB61635:C41D8F6692CACEBC]:0)
at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:18)
at org.junit.Assert.assertThat(Assert.java:956)
at org.elasticsearch.xpack.transform.integration.TransformUsageIT.lambda$testUsage$2(TransformUsageIT.java:110)
at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:887)
at org.elasticsearch.xpack.transform.integration.TransformUsageIT.testUsage(TransformUsageIT.java:95)

https://gradle-enterprise.elastic.co/s/zxok2tigiaaks/tests/vkcr26w2ldob6-j74o4svwlg7nm
https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+master+multijob+fast+part2/4257/console

[mayya-sharipova]

Another failure on master intake: https://gradle-enterprise.elastic.co/s/zxok2tigiaaks

Doesn't reproduce for me:
./gradlew ':x-pack:plugin:transform:qa:single-node-tests:integTestRunner' --tests "org.elasticsearch.xpack.transform.integration.TransformUsageIT.testUsage" -Dtests.seed=78BA13B22EB61635 -Dtests.security.manager=true -Dtests.locale=es-CO -Dtests.timezone=Pacific/Tongatapu -Dcompiler.java=13

  | 2> java.lang.AssertionError: Incorrect stat trigger_count |  
  | Expected: a value equal to or greater than <3.0> |  
  | but: <2.0> was less than <3.0> |  
  | at __randomizedtesting.SeedInfo.seed([78BA13B22EB61635:C41D8F6692CACEBC]:0) |  
  | at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:18) |  
  | at org.junit.Assert.assertThat(Assert.java:956) |  
  | at org.elasticsearch.xpack.transform.integration.TransformUsageIT.lambda$testUsage$2(TransformUsageIT.java:110)