[DataFrame] batch dataframe transform fails to start after it stopped at runtime #45339
Description
Repro:
- create a batch data frame transform with sufficient amount of data
- start it
- stop it before it can finish, progress should be between 0 and 1
- try to start it again
-> fails with NPE, see stack trace below
regression introduced in #44219
The issue also applies to continuous data frames, however only during the bootstrap phase until checkpoint 1 has been reached.
stacktrace from 8.0, also applies to 7.3
[2019-08-08T14:45:39,802][INFO ][o.e.x.d.t.DataFrameTransformTask] [xyz] Updating state for data frame transform [fq1405] to [{"task_state":"started","indexer_state":"stopped","position":{"indexer_position":{"@timestamp":1549675260000,"airline":"KLM"}},"checkpoint":0,"progress":{"total_docs":86274,"docs_remaining":50410,"percent_complete":41.56988200384821}}]
[2019-08-08T14:45:39,878][INFO ][o.e.x.d.t.DataFrameTransformPersistentTasksExecutor] [xyz] Successfully completed and scheduled task in node operation
[2019-08-08T14:45:39,887][WARN ][o.e.x.d.t.DataFrameTransformTask] [xyzl] Data frame transform [fq1405] encountered an exception:
java.lang.NullPointerException: null
at org.elasticsearch.xpack.dataframe.transforms.DataFrameIndexer.determineRunStateAtStart(DataFrameIndexer.java:488) ~[data-frame-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
at org.elasticsearch.xpack.dataframe.transforms.DataFrameIndexer.onStart(DataFrameIndexer.java:159) [data-frame-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
at org.elasticsearch.xpack.dataframe.transforms.DataFrameTransformTask$ClientDataFrameIndexer.lambda$onStart$3(DataFrameTransformTask.java:633) [data-frame-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:62) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
at org.elasticsearch.xpack.dataframe.transforms.DataFrameTransformTask$ClientDataFrameIndexer.onStart(DataFrameTransformTask.java:663) [data-frame-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]