TiDB unexpected exit with "MANIFEST set current failed: open /tmp/tidb/tmp_ddl-4000/xxx/duplicates/temporary.000001.dbtmp: no such file or directory"

[lance6716]

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

check out to #52467 and run TestFirstLitSlowStart

2. What did you expect to see? (Required)

not exit

3. What did you see instead (Required)

4. What is your TiDB version? (Required)

bug timeline:

1. ddl pick job 100
|\
| 2. job 100 start to create the lightning local backend. Now pebble DB passed the mkdir check
3. somehow ddl owner is resigned, the resign callback reset the runningJobs
4. ddl becomes owner again, and pick job 101, cleanup all duplicate directory which is less than 101
| 5. job 100 pebble DB writes the necessary files, found work direcctory is missing and call log.Fatal

[ywqzzy]

Could you show the timeline?

[lance6716]

Could you show the timeline?

updated