[SPARK-38005][core] Support cleaning up merged shuffle files and state from external shuffle service#38560

yabola wants to merge 3 commits intoapache:masterapache/spark:masterfrom

yabola:external_shuffle_cleanCopy head branch name to clipboard