[Bug]: compass did not parse to completion dolphinscheduler log

[woxiangwoshi123]

Contact Details

No response

Is there an existing issue for this?

• I have searched all the existing issues

Priority

high

Current Behavior

I used ds2hdfs.py provided by the author to upload logs.
It is currently found that compass is inaccurate when reading kafak data to parse the dolphinscheduler log uploaded to hdfs by flume. When the log generates multiple hdfs files, sometimes the last file is not read.
After investigation, it was found that the last tmp file was not generated when compass parsed the log. The delay in flume generating files caused the last file to be missing when parsing the log.

Expected Behavior

No response

Steps To Reproduce

No response

Compass Log

No response

Anything else? (Additional Context)

No response