fix(pyspark): fix a bug in spark's to_parquet_dir method

[chloeh13q]

Description of changes

Fix a bug in spark's to_parquet_dir method. Spark's write method only accepts strings as paths, and the current implementation will throw something like

AttributeError: 'PosixPath' object has no attribute '_get_object_id'

I was already doing this in streaming mode, just forgot to do this in batch mode.

xref: #9781

[cpcloud]

Do we have a test for this?

[chloeh13q]

@cpcloud @ncclementi is adding one in #9781, so I didn't add one.

[ncclementi]

@chloeh13q when I try to run the test with the change you suggest I get a different error now.

    def test_table_to_parquet_dir(tmp_path, backend, awards_players):
        outparquet_dir = tmp_path
>       awards_players.to_parquet_dir(outparquet_dir)

ibis/backends/tests/test_export.py:221: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
ibis/expr/types/core.py:636: in to_parquet_dir
    self._find_backend(use_default=True).to_parquet_dir(self, path, **kwargs)
ibis/backends/pyspark/__init__.py:1329: in to_parquet_dir
    return self._to_filesystem_output(expr, "parquet", path, params, limit, options)
ibis/backends/pyspark/__init__.py:1289: in _to_filesystem_output
    df.save(os.fspath(path)) #df.save(path)
/Users/naty/mambaforge/envs/ibis-dev/lib/python3.11/site-packages/pyspark/sql/readwriter.py:1463: in save
    self._jwrite.save(path)
/Users/naty/mambaforge/envs/ibis-dev/lib/python3.11/site-packages/py4j/java_gateway.py:1322: in __call__
    return_value = get_return_value(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

a = ('xro392', <py4j.clientserver.JavaClient object at 0x13015a9d0>, 'o391', 'save'), kw = {}, converted = AnalysisException()

    def deco(*a: Any, **kw: Any) -> Any:
        try:
            return f(*a, **kw)
        except Py4JJavaError as e:
            converted = convert_exception(e.java_exception)
            if not isinstance(converted, UnknownException):
                # Hide where the exception came from that shows a non-Pythonic
                # JVM exception message.
>               raise converted from None
E               pyspark.errors.exceptions.captured.AnalysisException: [PATH_ALREADY_EXISTS] Path file:/private/var/folders/ky/_zw0jkyd7hl3f9hfy6c994zw0000gn/T/pytest-of-naty/pytest-27/test_table_to_parquet_dir_pysp0 already exists. Set mode as "overwrite" to overwrite the existing path.

/Users/naty/mambaforge/envs/ibis-dev/lib/python3.11/site-packages/pyspark/errors/exceptions/captured.py:185: AnalysisException

I tried adding .mode("overwrite") here df = df.write.format(format) that bypasses the error, although I don't know if that's the desired thing. But the test fails anyways because of assertion errors. I'll push this to the original PR and we can discuss there since we will have the test failure to see the problems.

[cpcloud]

I think this was fixed in @ncclementi's PR, closing!