chore: Add EagerSeries*Namespace protocols

[dangotbanned]

Closes #2230

What type of PR is this? (check all applicable)

• 💾 Refactor
• ✨ Feature
• 🐛 Bug Fix
• 🔧 Optimization
• 📝 Documentation
• ✅ Test
• 🐳 Other

Related issues

• refactor: adds _Stores(Native|Compliant) #2130
• chore: Update ._native_series refs #2293
• Formalise Compliant* protocols #2230
• feat: Spec'd-out CompliantExpr #2119
• Tracking: Drop support for python 3.8 #2084

Checklist

• Code follows style guide (ruff)
• Tests added
• Documented the changes

If you have comments or can explain your changes, please do so below

• Figure out <3.11 compatible approach
  • Venture into this one at your own peril (chore: Add EagerSeries*Namespace protocols #2294 (comment))
• PandasLike*
  • CatNamespace
  • DateTimeNamespace
  • ListNamespace
  • StringNamespace
  • StructNamespace
• Un-break generate_backend_completeness
  • (chore: Add EagerSeries*Namespace protocols #2294 (comment))
  • (3ff77a0)

[dangotbanned]

Now that these are in the right place, I can collect my thoughts on the 3 solutions we can use for Protocol definintions with an __init__.

Problem

Defining a constructor in a Protocol can produce two different bugs.
I've documented that in (#2064 (comment)) - but the short of it is 3.8 and 3.(9|10) have issues.

Solutions

1. Protocol38

This was the first one I discovered (for #2064) and I really dislike it due to the complexity.

"Fixing" 3.8 requires tricking the type checker into thinking typing.Generic is typing.Protocol.

Imports

narwhals/narwhals/_compliant/expr.py

Lines 40 to 48 in 697bb2c

	if not TYPE_CHECKING: # pragma: no cover
	if sys.version_info >= (3, 9):
	from typing import Protocol as Protocol38
	else:
	from typing import Generic as Protocol38
	else: # pragma: no cover
	# TODO @dangotbanned: Remove after dropping `3.8` (#2084)
	# - https://github.com/narwhals-dev/narwhals/pull/2064#discussion_r1965921386
	from typing import Protocol as Protocol38

The side-effect is every sub-protocol is then required to use that Protocol38 definition or face runtime errors - that can no-longer be caught statically

E   TypeError: Protocols can only inherit from other protocols, got <class 'narwhals._compliant.series.EagerSeriesNamespace'>

"Fixing" 3.(9|10) requires redefining __init__ anyway in the impl.
Which mostly negates the benefit of defining in the Protocol 🤦‍♂️

Repeat __init__

narwhals/narwhals/_compliant/expr.py

Lines 332 to 345 in 697bb2c

	def __init__(
	self: Self,
	call: Callable[[EagerDataFrameT], Sequence[EagerSeriesT]],
	*,
	depth: int,
	function_name: str,
	evaluate_output_names: Callable[[EagerDataFrameT], Sequence[str]],
	alias_output_names: Callable[[Sequence[str]], Sequence[str]] | None,
	implementation: Implementation,
	backend_version: tuple[int, ...],
	version: Version,
	call_kwargs: dict[str, Any] | None = None,
	) -> None: ...

narwhals/narwhals/_arrow/expr.py

Lines 33 to 55 in 697bb2c

	def __init__(
	self: Self,
	call: Callable[[ArrowDataFrame], Sequence[ArrowSeries]],
	*,
	depth: int,
	function_name: str,
	evaluate_output_names: Callable[[ArrowDataFrame], Sequence[str]],
	alias_output_names: Callable[[Sequence[str]], Sequence[str]] | None,
	backend_version: tuple[int, ...],
	version: Version,
	call_kwargs: dict[str, Any] | None = None,
	implementation: Implementation | None = None,
	) -> None:
	self._call = call
	self._depth = depth
	self._function_name = function_name
	self._depth = depth
	self._evaluate_output_names = evaluate_output_names
	self._alias_output_names = alias_output_names
	self._backend_version = backend_version
	self._version = version
	self._call_kwargs = call_kwargs or {}
	self._metadata: ExprMetadata | None = None

narwhals/narwhals/_pandas_like/expr.py

Lines 69 to 91 in 697bb2c

	def __init__(
	self: Self,
	call: Callable[[PandasLikeDataFrame], Sequence[PandasLikeSeries]],
	*,
	depth: int,
	function_name: str,
	evaluate_output_names: Callable[[PandasLikeDataFrame], Sequence[str]],
	alias_output_names: Callable[[Sequence[str]], Sequence[str]] | None,
	implementation: Implementation,
	backend_version: tuple[int, ...],
	version: Version,
	call_kwargs: dict[str, Any] | None = None,
	) -> None:
	self._call = call
	self._depth = depth
	self._function_name = function_name
	self._evaluate_output_names = evaluate_output_names
	self._alias_output_names = alias_output_names
	self._implementation = implementation
	self._backend_version = backend_version
	self._version = version
	self._call_kwargs = call_kwargs or {}
	self._metadata: ExprMetadata | None = None

2. Define the constructor another way

(#2261) introduced a different solution - but still needed to use Protocol38 to avoid the metaclass issues of the first solution.

I like this a lot more as it feels more intuitive from the outside - given the clear chain of @classmethod and @property definitions.

when-then

narwhals/narwhals/_arrow/namespace.py

Lines 241 to 242 in 697bb2c

	def when(self: Self, predicate: ArrowExpr) -> ArrowWhen:
	return ArrowWhen.from_expr(predicate, context=self)

narwhals/narwhals/_arrow/namespace.py

Lines 282 to 285 in 697bb2c

	class ArrowWhen(EagerWhen[ArrowDataFrame, ArrowSeries, ArrowExpr, "ArrowChunkedArray"]):
	@property
	def _then(self) -> type[ArrowThen]:
	return ArrowThen

narwhals/narwhals/_compliant/when_then.py

Lines 72 to 82 in 697bb2c

	@classmethod
	def from_expr(cls, condition: ExprT, /, *, context: _FullContext) -> Self:
	obj = cls.__new__(cls)
	obj._condition = condition
	obj._then_value = None
	obj._otherwise_value = None
	obj._implementation = context._implementation
	obj._backend_version = context._backend_version
	obj._version = context._version
	return obj

narwhals/narwhals/_compliant/when_then.py

Lines 63 to 64 in 697bb2c

	@property
	def _then(self) -> type[CompliantThen[FrameT, SeriesT, ExprT]]: ...

narwhals/narwhals/_compliant/when_then.py

Lines 67 to 70 in 697bb2c

	def then(
	self, value: IntoExpr[SeriesT, ExprT], /
	) -> CompliantThen[FrameT, SeriesT, ExprT]:
	return self._then.from_when(self, value)

narwhals/narwhals/_compliant/when_then.py

Lines 94 to 115 in 697bb2c

	@classmethod
	def from_when(
	cls,
	when: CompliantWhen[FrameT, SeriesT, ExprT],
	then: IntoExpr[SeriesT, ExprT],
	/,
	) -> Self:
	when._then_value = then
	obj = cls.__new__(cls)
	obj._call = when
	obj._when_value = when
	obj._depth = 0
	obj._function_name = "whenthen"
	obj._evaluate_output_names = getattr(
	then, "_evaluate_output_names", lambda _df: ["literal"]
	)
	obj._alias_output_names = getattr(then, "_alias_output_names", None)
	obj._implementation = when._implementation
	obj._backend_version = when._backend_version
	obj._version = when._version
	obj._call_kwargs = {}
	return obj

The only downside I could see is __new__ being a less-understood method than __init__.
Personally it took me a while to understand when it makes sense to use.
Even now that I do - I didn't think of it as a solution until a month after (#2064) merged 😅

3. Avoid constructors entirely in Protocol(s)

That is the solution this comment is attached to.

It might be easy to miss, but _SeriesNamespace is a Protocol and EagerSeriesNamespace is a Generic class.

The interesting aspect of that is the Protocol(s) represent the interface - but the concrete impl is separate.
This approach makes sense when the interface and implementation differ greatly.
Here it is ArrowSeries and PandasLikeSeries - but we go fully structural for all of Polars

[dangotbanned]

https://github.com/narwhals-dev/narwhals/actions/runs/14091955981/job/39470550165?pr=2294

Haven't figured out why yet, but the .pivot(...) here is throwing for Series.dt

narwhals/utils/generate_backend_completeness.py

Lines 103 to 111 in 697bb2c

	def render_table_and_write_to_output(
	results: list[pl.DataFrame], # pyright: ignore[reportRedeclaration]
	title: str,
	output_filename: str,
	) -> None:
	results: pl.DataFrame = (
	pl.concat(results)
	.with_columns(supported=pl.lit(":white_check_mark:"))
	.pivot(on="Backend", values="supported", index=["Method"])

First guess would be the extra properties for PandasLike in

narwhals/narwhals/_pandas_like/utils.py

Lines 813 to 824 in 7daa8f6

	class PandasLikeSeriesNamespace(EagerSeriesNamespace["PandasLikeSeries", Any]):
	@property
	def implementation(self) -> Implementation:
	return self.compliant._implementation
	@property
	def backend_version(self) -> tuple[int, ...]:
	return self.compliant._backend_version
	@property
	def version(self) -> Version:
	return self.compliant._version

Update

Resolved in (3ff77a0)

[MarcoGorelli]

thanks @dangotbanned !