[SPARK-35672][CORE][YARN] Pass user classpath entries to executors using config instead of command line.#32810

Closed

xkrogen wants to merge 6 commits intoapache:masterfrom

xkrogen:xkrogen-SPARK-35672-classpath-scalable

Commits on Jun 24, 2021

SPARK-35672 [YARN] Pass user classpath entries to executors using config instead of command line.
xkrogen
committed
Use the correct configuration to address test failures.
xkrogen
committed
Simplify passing the classpath: Allow the executor to leverage the existing SECONDARY_JARS and APP_JAR configs to construct the classpath using the same (now shared) logic as the ApplicationMaster …
xkrogen
committed
Change `getUserClasspathUrls` to properly resolve `local:`-style paths which leverage the gateway/replacment path functionality. Enhance test cases in `YarnClusterSuite` for this case, and add test…
xkrogen
committed
Expand test cases and extract some shared constants. Simplify logic in getUserClasspathUrls and make the assumptions more clear via an assert
xkrogen
committed
empty commit to trigger build
xkrogen
committed