[Enh]: Add `DType.{from,to}_python`

[FBruzzesi]

Polars has parse_py_type_into_dtype. It could be handy to have the same

[dangotbanned]

I'd thought about this before, but figured it might've been out of scope.

It could be nice to have these:

# NOTE: Implemented on each subclass
nw.DType.from_python
nw.DType.to_python

nw.Schema.from_python
nw.Schema.to_python

• https://docs.pola.rs/api/python/stable/reference/api/polars.datatypes.DataType.html
• https://github.com/pola-rs/polars/blob/py-1.25.2/py-polars/polars/schema.py#L202-L218

[dangotbanned]

@FBruzzesi I forgot to mention, I've got part of the from_python side in (#2572) for inferring the DType in lit(...) like polars does:

Impl

narwhals/narwhals/_plan/common.py

Lines 323 to 338 in 01f4d26

	def py_to_narwhals_dtype(obj: NonNestedLiteral, version: Version = Version.MAIN) -> DType:
	dtypes = version.dtypes
	mapping: dict[type[NonNestedLiteral], type[DType]] = {
	int: dtypes.Int64,
	float: dtypes.Float64,
	str: dtypes.String,
	bool: dtypes.Boolean,
	dt.datetime: dtypes.Datetime,
	dt.date: dtypes.Date,
	dt.time: dtypes.Time,
	dt.timedelta: dtypes.Duration,
	bytes: dtypes.Binary,
	Decimal: dtypes.Decimal,
	type(None): dtypes.Unknown,
	}
	return mapping.get(type(obj), dtypes.Unknown)()

Usage

narwhals/narwhals/_plan/common.py

Lines 281 to 294 in 01f4d26

	_NON_NESTED_LITERAL_TPS = (
	int,
	float,
	str,
	dt.date,
	dt.time,
	dt.timedelta,
	bytes,
	Decimal,
	)
	def is_non_nested_literal(obj: Any) -> TypeIs[NonNestedLiteral]:
	return obj is None or isinstance(obj, _NON_NESTED_LITERAL_TPS)

narwhals/narwhals/_plan/demo.py

Lines 56 to 66 in 01f4d26

	def lit(
	value: NonNestedLiteral | DummySeries, dtype: DType | type[DType] | None = None
	) -> DummyExpr:
	if isinstance(value, DummySeries):
	return SeriesLiteral(value=value).to_literal().to_narwhals()
	if not is_non_nested_literal(value):
	msg = f"{type(value).__name__!r} is not supported in `nw.lit`, got: {value!r}."
	raise TypeError(msg)
	if dtype is None or not isinstance(dtype, DType):
	dtype = py_to_narwhals_dtype(value, Version.MAIN)
	return ScalarLiteral(value=value, dtype=dtype).to_literal().to_narwhals()

[FBruzzesi]

Thanks @dangotbanned - would it be worth it to start to prioritize this issue and work on it separately of #2572?

[dangotbanned]

Thanks @dangotbanned - would it be worth it to start to prioritize this issue and work on it separately of #2572?

Yeah @FBruzzesi definitely!

I would recommend landing (#2626) first though, since we'd be able to expand the definition of IntoDType to include some python types as well.
pl.datatypes._parse.parse_into_dtype would then be similar to what we'd use to resolve IntoDType -> DType.

I think doing (#2626) -> (#2264) would leave us in a nice place for (#2486) 🎉
I'd started some very early work on that a while back, but needing to handle the nested case everywhere was getting repetitive.
I imagine we'd end up with some utilities/error handling for that throughout the work on the other issues 🙏

[FBruzzesi]

Hey @dangotbanned - reviving this conversation as I was implementing something similar recently.

Regarding the to_python, I wonder what a complex type should map to. i.e.

nw.List(nw.Int64).to_python() would be list (which is a type) or list[int] (which is a generic alias)?

[dangotbanned]

Hey @FBruzzesi!

IIRC, polars will always return a bare python type[...], from to_python.
But GenericAlias is supported in the other direction (from_python).

Personally, I'd recommend leaving the latter part of that out - and consider it if anyone asks for it?
It definitely can be done but the complexity is waaaaaaaay higher than the benefit IMO 😅