Skip to content

Conversation

@zhengruifeng
Copy link
Contributor

What changes were proposed in this pull request?

Why are the changes needed?

Fix the doctest of pivot, to make sure the example works

Does this PR introduce any user-facing change?

doc-only change

How was this patch tested?

enabled doc-test

Was this patch authored or co-authored using generative AI tooling?

no

Copy link
Member

@dongjoon-hyun dongjoon-hyun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, could you check the pivot test failure?

File "/__w/spark/spark/python/pyspark/sql/connect/group.py", line 309, in pyspark.sql.connect.group.GroupedData.pivot
Failed example:
    df2.groupBy("sales.year").pivot(
        "sales.course"
    ).sum("sales.earnings").sort("year").show()
Exception raised:
    Traceback (most recent call last):
      File "/usr/lib/python3.11/doctest.py", line 1355, in __run
        exec(compile(example.source, filename, "single",
      File "<doctest pyspark.sql.connect.group.GroupedData.pivot[7]>", line 3, in <module>
        ).sum("sales.earnings").sort("year").show()
          ^^^^^^^^^^^^^^^^^^^^^
      File "/__w/spark/spark/python/pyspark/sql/connect/group.py", line 208, in sum
        return self._numeric_agg("sum", list(cols))
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/__w/spark/spark/python/pyspark/sql/connect/group.py", line 175, in _numeric_agg
        raise PySparkTypeError(
    pyspark.errors.exceptions.base.PySparkTypeError: [NOT_NUMERIC_COLUMNS] Numeric aggregation function can only be applied on numeric columns, got ['sales.earnings'].
**********************************************************************
   1 of   8 in pyspark.sql.connect.group.GroupedData.pivot
***Test Failed*** 1 failures.

Copy link
Member

@dongjoon-hyun dongjoon-hyun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, LGTM (Pending CIs).

dongjoon-hyun pushed a commit that referenced this pull request Nov 3, 2025
### What changes were proposed in this pull request?

### Why are the changes needed?
Fix the doctest of `pivot`, to make sure the example works

### Does this PR introduce _any_ user-facing change?
doc-only change

### How was this patch tested?
enabled doc-test

### Was this patch authored or co-authored using generative AI tooling?
no

Closes #52814 from zhengruifeng/py_test_pivot.

Authored-by: Ruifeng Zheng <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
(cherry picked from commit 2063c36)
Signed-off-by: Dongjoon Hyun <[email protected]>
@dongjoon-hyun
Copy link
Member

Merged to master. Thank you.

@zhengruifeng zhengruifeng deleted the py_test_pivot branch November 4, 2025 00:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants