feat: Added struct namespace with field method.

[osoucy]

What type of PR is this? (check all applicable)

• 💾 Refactor
• ✨ Feature
• 🐛 Bug Fix
• 🔧 Optimization
• 📝 Documentation
• ✅ Test
• 🐳 Other

Related issues

• Related issue #<issue number>
• Closes #<issue number>

Checklist

• Code follows style guide (ruff)
• Tests added
• Documented the changes

If you have comments or can explain your changes, please do so below

Introduction of the strut namespace for applicable expressions and series with an initial field function that returns a field of a struct.

[MarcoGorelli]

Instead of using apply can we use https://pandas.pydata.org/pandas-docs/dev/reference/api/pandas.Series.struct.field.html and only support this for pyarrow-backed dtypes?

[osoucy]

I feel the majority of Pandas users don't leverage pyarrow-backed dtypes. I added a switch to check and use the struct namespace if available and fallbacks on the apply function if it's not the case.

[EdAbati]

This looks great! Thank you for doing this 🙌🏼

I've added a couple of comments 'on the go' (I'll check the rest later)

[FBruzzesi]

Thanks @osoucy - it generally looks very promising. Left an additional comment to those of Edo and Marco

[FBruzzesi]

Output is missing and doctest check would fail without it

[osoucy]

The output has to be manually generated? There is no command that I can use to generate it?

[osoucy]

Are we good to merge? Anything else to add?

[MarcoGorelli]

thanks @osoucy !

i'll check the evaluate_output_names / alias_output_names more closely, I'm not 100% sure about those

[MarcoGorelli]

For all implementations, instead of overwriting evaluate output names, can we just use .alias?

A good test would be nw.col('a').struct.field('b').name.keep(). I think for polars the resulting column name would be 'a' we should check that we do the same

[osoucy]

It's odd. That's what I tried initially, but it caused some errors when evaluating (mismatching expected and actual something). I must have forgotten something. I removed the changes made to the _from_call and reuse_series_namespace_implementation functions.

[MarcoGorelli]

you can't mutate self._compliant_series, you'll need alias here too

[osoucy]

Done.

[dangotbanned]

For the Arrow* stuff I'd suggest using .compliant and .native

• refactor: adds _Stores(Native|Compliant) #2130

They weren't available when you started the PR @osoucy

[dangotbanned]

Some more context #2130 (comment)

[MarcoGorelli]

this is fine as a follow-up

[osoucy]

At least the current implementation

    def field(self: Self, name: str) -> ArrowSeries:
        return self._compliant_series._from_native_series(
            pc.struct_field(self._compliant_series.alias(name)._native_series, name),
        )

avoids mutating the compliant series. Maybe I can look into @dangotbanned in a future PR?

[dangotbanned]

All good @osoucy 😊

[osoucy]

So we are good to merge? :)

[MarcoGorelli]

there's some unnecessary alias calls - I tried making a commit and pushing to your branch, but was denied access

could you check that the "allow edits by maintainers" checkbox is checked? you should see something like this

[osoucy]

there's some unnecessary alias calls - I tried making a commit and pushing to your branch, but was denied access

could you check that the "allow edits by maintainers" checkbox is checked? you should see something like this

This option is not available because the fork is created inside of my organization instead of inside my personnal account. I added you as a contributor to the organization. You should be able to push to the branch now.

[MarcoGorelli]

typo

[MarcoGorelli]

outdated

[MarcoGorelli]

thanks @osoucy ! great feature 🙌

[osoucy]

Awesome! Thanks for the guidance and review!