[CI] MachineLearningLicensingTests] [testAutoCloseJobWithDatafeed fails with NPE

[henningandersen]

MachineLearningLicensingTests.testAutoCloseJobWithDatafeed fails rarely with NPE. Can be reproduced by adding a sleep in the onResponse method here and using following repro line:

./gradlew ':x-pack:plugin:ml:test' --tests "org.elasticsearch.license.MachineLearningLicensingTests.testAutoCloseJobWithDatafeed" -Dtests.seed=8505B129BBE04142 -Dtests.security.manager=true -Dtests.locale=en-SG -Dtests.timezone=Asia/Hovd -Dcompiler.java=13

It failed on a couple of PR builds, for instance this one:

https://gradle-enterprise.elastic.co/s/doocabu3vhmdm/tests/7bifs6bt3pims-kf5sbxwq5fdpm

The assertion error in that build is new code, but caused by the NullPointerException:

Caused by: java.lang.NullPointerException
	at org.elasticsearch.xpack.ml.datafeed.DatafeedManager.getJobId(DatafeedManager.java:271)
	at org.elasticsearch.xpack.ml.datafeed.DatafeedManager.getJobState(DatafeedManager.java:275)
	at org.elasticsearch.xpack.ml.datafeed.DatafeedManager$TaskRunner.runWhenJobIsOpened(DatafeedManager.java:498)
	at org.elasticsearch.xpack.ml.datafeed.DatafeedManager$1.onResponse(DatafeedManager.java:94)
	at org.elasticsearch.xpack.ml.datafeed.DatafeedManager$1.onResponse(DatafeedManager.java:91)
	at org.elasticsearch.action.ActionListener$4.onResponse(ActionListener.java:162)

[elasticmachine]

Pinging @elastic/ml-core (:ml)

[droberts195]

@henningandersen how long did you sleep for to make it easily reproducible? I tried a variety of values from 20ms to 9 seconds and none of them reproduced the problem. (10 seconds or more causes an unrelated failure because an assertBusy wait would need extending with such a long sleep.)

Having said that I can see a theoretical flaw in the code, and a reason why adding a sleep would make it more likely to happen. So I am happy to fix that today.

[droberts195]

I think #51302 will fix this, but it's hard to be certain as I couldn't reproduce it locally.

[henningandersen]

Thanks @droberts195 , you are right, it is only surfaced due to the added assertion in my PR (sorry, should have double checked against master before reporting here). It looks like the resulting NPE is ignored in AllocatedPersistentTask.completeAndNotifyIfNeeded since the task is already completed.

I used a sleep of 1 second.

I checked after merging in your changes locally and it seems to work fine, thanks for fixing this.