enh: Start narwhals.stable.v2 and enforce deprecations

[MarcoGorelli]

From the downstream tests, the only breakage is in:

• hierarchical forecast is using native_namespace in from_dict

That's...all, from what I can tell.

I've made a PR to them to update this: Nixtla/hierarchicalforecast#381

I don't want to be disruptive towards them, so perhaps we could take a bit of a middle-ground approach:

• for deprecated functions/params which don't show up in any downstream test, just enforce the deprecation
• for native_namespace in from_dict, keep it around longer in the main Narwhals namespace at least until there's been a new hierarchicalforecast

todo:

• for from_native_test, separate out some tests to just v1, and see what we need to test for v2 as well

Aiming to merge this (and make a Narwhals 2.0 release) by August

[dangotbanned]

Hey @MarcoGorelli 👋

IIUC, would the motivation for removing deprecated members from Compliant* be that extenders of narwhals aren't required to implement them?

If so, this is a tricky problem to solve 🤔

Note

TL;DR: I'd recommend keeping deprecations in the protocols

Typing

The main downside for us is that the loss of typing information makes it more likely that we unknowingly regress on the v1 promise in the future.
Despite the deprecation, we can clearly see which things are deprecated atm, and have some IDE help with:

Finding all references

Finding all implementations

This one probably should've had a @deprecated 🤦‍♂️

New [attr-defined]s make me think that this link between layers is now (statically) murky?

If the goal were just to keep a type checker happy, then the two options are:

1. Add Compliant*V1 protocols, use them in *V1 narwhals-level and implement both in the backends that support V1
2. Keep deprecations in Compliant* protocols, punt the issue for now

Runtime

This is the trickier one for me ...

Would a new internal backend be expected to support v1?

For example, in the daft PR, we can see the typing distinction between things that aren't yet and won't ever be implemented.

https://github.com/MarcoGorelli/narwhals/blob/f918282f14feb43e0a9687d82e16eed79b3832f7/narwhals/_daft/dataframe.py#L333-L340

    gather_every = not_implemented.deprecated(
        "`LazyFrame.gather_every` is deprecated and will be removed in a future version."
    )
    join_asof = not_implemented()
    tail = not_implemented.deprecated(
        "`LazyFrame.tail` is deprecated and will be removed in a future version."
    )
    explode = not_implemented()

At runtime we give the same message to users (I don't know why I made that decision (#2232 (comment)) 😅)

A new backend won't get a type checker yelling to add a not_implemented, which at runtime will mean we just get an AttributeError 😢

[MarcoGorelli]

we've got a file which specifically tests all the functionality from v1 (tests/v1_test.py), if we're removing the functionality then I think it looks a bit odd to keep it but marked as deprecated

Would a new internal backend be expected to support v1?

no, we only need to care about preserving functionality for code that's already been written

[dangotbanned]

We can move level to v1-only can't we?

Or at least "interchange" I thought was on the chopping block?

[MarcoGorelli]

yes true, i just think it'd require a larger refactor

it's not user-facing though, so not necessarily pressing?

[dangotbanned]

Yeah that seems fair

How about in this PR, try removing v2.typing.DataFrameLike and adding some tests using a mock w/ __dataframe__?

I think making sure the typing on v2 rejects interchange only should be doable 🙂

We can punt the rest to another issue

[MarcoGorelli]

Just tried v2 out in Bokeh, and it all passes as-is

In Altair it requires some code changes because eager_or_interchange_only was used there, but with those in it seems fine