feat: IbisLazyFrame support

[rwhitten577]

What type of PR is this? (check all applicable)

• 💾 Refactor
• ✨ Feature
• 🐛 Bug Fix
• 🔧 Optimization
• 📝 Documentation
• ✅ Test
• 🐳 Other

Related issues

• Closes Request for contributions: Ibis support #1886

Checklist

• Code follows style guide (ruff)
• Tests added
• Documented the changes

If you have comments or can explain your changes, please do so below

[NickCrews]

I want ibis support in narwhals and can help to get this landed. Ping me if you want a review or help!

Cc for visibility @cpcloud, the maintainer of ibis

[MarcoGorelli]

thanks @NickCrews for your help! sure - we're in the middle of some large refactors so it may be prudent to wait a bit to avoid too many merge conflicts, but we will get to this

[dangotbanned]

ibis.selectors

I'm interested in how we might be able to take advantage of ibis.selectors.
I noted in (#2064 (comment)) ibis is an outlier because of their native support

Initial thoughts are that our adaptation could work quite differently to the other backends - maybe even being more performant

Since (#2064) every backend besides polars is sharing a lot of code - which recently moved to nw._compliant.selectors.
Most of this is simply performing operations on nw.Schema

[MarcoGorelli]

Hey @rwhitten577 - we're done with the big refactors, so if it interests you to continue, we'd love to ship this 🚢 🚀

I'd suggest not worrying about doing anything different for selectors just yet (it looks to me like they call .schema anyway?), mirroring what we do for duckdb / sqlframe / pyspark should be fine 👍

[afrisgaard]

How would this land? (Just considering to be ready when ibis ships to Narwhals :) )

Something like this:
ibis_frame = ibis.use_backend("BigQuery"/"DuckDb"/etc)

NarwhalsFrame = from_native(ibis_frame)

.. do narwhals operations

[MarcoGorelli]

yup,that's right! we'd just translate to the ibis api, so whatever backend is set there would be used

[rwhitten577]

Hey @MarcoGorelli, I will try to pick this back up in the next few weeks. I haven't looked through all the refactor changes yet, but do you think I'd be better off rebasing this old branch or are the changes significant enough where I'd want to start over and use duckdb or spark as an example?

[MarcoGorelli]

awesome, thanks @rwhitten577 !

i think it might be salvageable to continue from here, but still using _spark_like / _duckdb as a reference. looks like most of the merge conflicts are in the tests anyway

[dangotbanned]

Thanks for all your work on this @rwhitten577!

With everything in (#2000 (review)) resolved - I'm approving ❤️

Definitely would need an approval from @MarcoGorelli or @FBruzzesi as well - but I'm happy with where this is at 🎉

[MarcoGorelli]

Amazing, thanks @rwhitten577 and @dangotbanned for reviewing!

Got some mostly minor comments. I've split out some follow-ups in #2525

The most important comments from my end are:

• not casting to string for numerical ops
• not repeatedly calling .drop but instead collecting the items to drop into a list first (in join). In theory I think Ibis should be able to optimise this away, but I don't know if they do, so it's probably safer to just call drop once with everything we need to drop

Once this is addressed, we can ship it

[MarcoGorelli]

can we just use col(name) here, like we do for spark-like / duckdb?

[rwhitten577]

We can only do getattr(col, name) where col == ibis._. This leads to test errors though where Ibis cannot determine the data type of a deferred value when done this way. Accessing from the native df is the only way I've found to reference the actual column with type info.

[MarcoGorelli]

cool, thanks for explaining

aiming for a full review today, hopefully we can ship it

[MarcoGorelli]

thanks @rwhitten577, and @dangotbanned for reviewing

happy with the code now, and we've opened issues with Ibis for missing features on their end

let's ship it! 2 approval gifs here as this was an extra-large one

[rwhitten577]

Awesome! Thanks @dangotbanned and @MarcoGorelli for reviewing!