Skip to content

Conversation

@dongjoon-hyun
Copy link
Member

@dongjoon-hyun dongjoon-hyun commented Sep 12, 2024

What changes were proposed in this pull request?

This PR aims to re-enable test_create_dataframe_from_pandas_with_day_time_interval in PyPy3.9.

Why are the changes needed?

This was disabled at PyPy3.8, but we dropped Python 3.8 support and the test passed with PyPy3.9.

BEFORE: Skipped with Fails in PyPy Python 3.8, should enable. message

$ python/run-tests.py --python-executables pypy3 --testnames pyspark.sql.tests.test_creation
Running PySpark tests. Output is in /Users/dongjoon/APACHE/spark-merge/python/unit-tests.log
Will test against the following Python executables: ['pypy3']
Will test the following Python tests: ['pyspark.sql.tests.test_creation']
pypy3 python_implementation is PyPy
pypy3 version is: Python 3.9.19 (a2113ea87262, Apr 21 2024, 05:41:07)
[PyPy 7.3.16 with GCC Apple LLVM 15.0.0 (clang-1500.1.0.2.5)]
Starting test(pypy3): pyspark.sql.tests.test_creation (temp output: /Users/dongjoon/APACHE/spark-merge/python/target/58e26724-5c3e-4451-80f8-cabdb36f0901/pypy3__pyspark.sql.tests.test_creation__n448ay57.log)
Finished test(pypy3): pyspark.sql.tests.test_creation (6s) ... 3 tests were skipped
Tests passed in 6 seconds

Skipped tests in pyspark.sql.tests.test_creation with pypy3:
    test_create_dataframe_from_pandas_with_day_time_interval (pyspark.sql.tests.test_creation.DataFrameCreationTests) ... skipped 'Fails in PyPy Python 3.8, should enable.'
    test_create_dataframe_required_pandas_not_found (pyspark.sql.tests.test_creation.DataFrameCreationTests) ... skipped 'Required Pandas was found.'
    test_schema_inference_from_pandas_with_dict (pyspark.sql.tests.test_creation.DataFrameCreationTests) ... skipped '[PACKAGE_NOT_INSTALLED] PyArrow >= 10.0.0 must be installed; however, it was not found.'

AFTER

$ python/run-tests.py --python-executables pypy3 --testnames pyspark.sql.tests.test_creation
Running PySpark tests. Output is in /Users/dongjoon/APACHE/spark-merge/python/unit-tests.log
Will test against the following Python executables: ['pypy3']
Will test the following Python tests: ['pyspark.sql.tests.test_creation']
pypy3 python_implementation is PyPy
pypy3 version is: Python 3.9.19 (a2113ea87262, Apr 21 2024, 05:41:07)
[PyPy 7.3.16 with GCC Apple LLVM 15.0.0 (clang-1500.1.0.2.5)]
Starting test(pypy3): pyspark.sql.tests.test_creation (temp output: /Users/dongjoon/APACHE/spark-merge/python/target/1f0db01f-0beb-4ee2-817f-363eb2f2804d/pypy3__pyspark.sql.tests.test_creation__2w4gy9u1.log)
Finished test(pypy3): pyspark.sql.tests.test_creation (13s) ... 2 tests were skipped
Tests passed in 13 seconds

Skipped tests in pyspark.sql.tests.test_creation with pypy3:
    test_create_dataframe_required_pandas_not_found (pyspark.sql.tests.test_creation.DataFrameCreationTests) ... skipped 'Required Pandas was found.'
    test_schema_inference_from_pandas_with_dict (pyspark.sql.tests.test_creation.DataFrameCreationTests) ... skipped '[PACKAGE_NOT_INSTALLED] PyArrow >= 10.0.0 must be installed; however, it was not found.'

Does this PR introduce any user-facing change?

No, this is a test only change.

How was this patch tested?

Manual tests with PyPy3.9.

Was this patch authored or co-authored using generative AI tooling?

No.

@xinrong-meng
Copy link
Member

LGTM, thank you!

@dongjoon-hyun
Copy link
Member Author

Thank you, @xinrong-meng .
Merged to master for Apache Spark 4.0.0-preview2.

@dongjoon-hyun dongjoon-hyun deleted the SPARK-43354 branch September 13, 2024 03:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants