[spark] Validation of the Ray-on-Spark-on-YARN mode to enable it to run.

[CaiZhanqi]

Description

Add spark master model validation to let Ray run on Spark-On-YARN mode.

Why need this?

If we directly run Ray on a YARN cluster, we need to do more tests and integration, and also need to setup related tools and environments. If support ray-on-spark-on-yarn and we already have Spark envs setup, we don't need to do other things, can use Spark and let the user run pyspark.

[gemini-code-assist]

Code Review

This pull request adds support for running Ray on Spark with a YARN master by updating the validation logic. The change is straightforward and correct. I've suggested a minor improvement to the error message to include the detected Spark master, which will enhance debuggability for users encountering configuration issues.

[gemini-code-assist]

To improve debuggability, consider including the detected spark_master value in the error message. This will help users quickly identify misconfigurations when they are using an unsupported Spark master.

Suggested change

	raise RuntimeError(
	"Ray on Spark only supports spark cluster in standalone mode, "
	"local-cluster mode or spark local mode."
	"local-cluster mode, spark on yarn mode or spark local mode."
	)
	raise RuntimeError(
	"Ray on Spark only supports spark cluster in standalone mode, "
	"local-cluster mode, spark on yarn mode or spark local mode. "
	f"Detected spark master is '{spark_master}'."
	)

[edoakes]

@CaiZhanqi have you already tested running Ray on Spark-on-YARN mode?

[CaiZhanqi]

@edoakes yes, have tested Spark 3.2 by removing Spark version check, and Apache Spark 3.5 as well.