Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for List as a data type in the writer #451

Closed
mkleinbort-ic opened this issue Feb 20, 2024 · 1 comment · Fixed by #458
Closed

Add support for List as a data type in the writer #451

mkleinbort-ic opened this issue Feb 20, 2024 · 1 comment · Fixed by #458

Comments

@mkleinbort-ic
Copy link

Feature Request / Improvement

This code does not work:

import polars as pl
df = pl.DataFrame({'x':[[1,2,3]]}, schema={'x':pl.List(pl.Int8)}).to_arrow()

# pyarrow.Table
# x: large_list<item: int8>
#    child 0, item: int8
# ----
# x: [[[1,2,3]]]

table = catalog.create_table(
    "default.test_table_02",
    schema=df.schema,
)

With error

TypeError: Expected primitive type, got: <class 'pyarrow.lib.LargeListType'>

I'd be good to support List data types

@hussein-awala
Copy link
Member

The list data type is supported in PyIceberg, but there is a bug in the mapping method, I believe #458 will fix the issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants