[SPARK-32335][K8S][TESTS] Remove Python2 test from K8s IT

[dongjoon-hyun]

What changes were proposed in this pull request?

This PR aims to remove Python 2 test case from K8s IT.

Why are the changes needed?

Since Apache Spark 3.1.0 dropped Python 2.7, 3.4 and 3.5 support officially via SPARK-32138, K8s IT fails.

KubernetesSuite:
- Run SparkPi with no resources
- Run SparkPi with a very long application name.
- Use SparkLauncher.NO_RESOURCE
- Run SparkPi with a master URL without a scheme.
- Run SparkPi with an argument.
- Run SparkPi with custom labels, annotations, and environment variables.
- All pods have the same service account by default
- Run extraJVMOptions check on driver
- Run SparkRemoteFileTest using a remote data file
- Run SparkPi with env and mount secrets.
- Run PySpark on simple pi.py example
- Run PySpark with Python2 to test a pyfiles example *** FAILED ***
  The code passed to eventually never returned normally. Attempted 113 times over 2.0014854648999996 minutes. Last failure message: false was not true. (KubernetesSuite.scala:370)
- Run PySpark with Python3 to test a pyfiles example
- Run PySpark with memory customization
- Run in client mode.
- Start pod creation from template
- PVs with local storage
- Launcher client dependencies
- Test basic decommissioning
- Run SparkR on simple dataframe.R example
Run completed in 11 minutes, 15 seconds.
Total number of tests run: 20
Suites: completed 2, aborted 0
Tests: succeeded 19, failed 1, canceled 0, ignored 0, pending 0
*** 1 TEST FAILED ***

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Pass Jenkins K8s IT.

[dongjoon-hyun]

Could you review this when you have some time, @liyinan926 ?
cc @HyukjinKwon

[liyinan926]

LGTM. Thanks!

[dongjoon-hyun]

Wow. Thank you for quick review!

[dongjoon-hyun]

Merged to master.
(This is a removal of integration test case which is irrelevant to Jenkins and GitHub Action.)

[SparkQA]

Test build #125995 has finished for PR 29136 at commit 542cb2a.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/30609/
Test FAILed.

[HyukjinKwon]

Nice LGTM