Skip to content
Merged
Show file tree
Hide file tree
Changes from 23 commits
Commits
Show all changes
156 commits
Select commit Hold shift + click to select a range
ff661ae
chore: Add `CompliantExpr.first`
dangotbanned May 10, 2025
1b77bd7
feat: "Implement" `PolarsExpr.First`
dangotbanned May 10, 2025
e84cba3
feat: Add `EagerExpr.first`
dangotbanned May 10, 2025
25ef241
chore: Repeat for `*Series`
dangotbanned May 10, 2025
78822aa
feat: Add `(Arrow|PandasLike)Series.first()`
dangotbanned May 10, 2025
4075c50
chore: Mark `LazyExpr.first` as `not_implemented` for now
dangotbanned May 10, 2025
45f24b9
feat: Add `SparkLikeExpr.first`
dangotbanned May 10, 2025
4041dd1
feat: Add `DuckDBExpr.first`
dangotbanned May 10, 2025
bb9912d
feat: Add `DaskExpr.first`
dangotbanned May 10, 2025
6a53aa1
revert: 4075c50f2496ab9908b25dc15e240650bc686dc0
dangotbanned May 10, 2025
4efc939
feat: Add `nw.Series.first`
dangotbanned May 10, 2025
fc149c1
test: Add `Series.first` tests
dangotbanned May 10, 2025
7489e61
fix: I guess the stubs were wrong then?
dangotbanned May 10, 2025
d2719a4
fix: Handle the out-of-bounds case
dangotbanned May 10, 2025
0af11db
fix: `polars` backcompat
dangotbanned May 10, 2025
afe20f0
docs: Add `Series.first`
dangotbanned May 10, 2025
6c0bd6f
lol version typo
dangotbanned May 10, 2025
e0fdf78
cov
dangotbanned May 10, 2025
aa7c510
chore: Add `nw.Expr.first`
dangotbanned May 11, 2025
4fdc0aa
Merge remote-tracking branch 'upstream/main' into expr-first
dangotbanned May 11, 2025
bd4ab89
feat: Maybe `SparkLike` requires `order_by`?
dangotbanned May 11, 2025
9f7f5a9
test: Try out eager backends
dangotbanned May 11, 2025
ddb50d2
Merge branch 'main' into expr-first
dangotbanned May 11, 2025
7146f60
test: Add mostly broken lazy tests 😒
dangotbanned May 11, 2025
8c24e6e
feat: `duckdb` support?
dangotbanned May 11, 2025
54a4cb4
test: Update xfails
dangotbanned May 11, 2025
63e0459
fix: Use `head(1)` in `DaskExpr`
dangotbanned May 11, 2025
9493aad
ignore cov
dangotbanned May 11, 2025
88535a4
Apply suggestion
dangotbanned May 11, 2025
77ae9c0
test: Remove dask `xfail`
dangotbanned May 11, 2025
c1a6173
revert: Remove `dask` implementation
dangotbanned May 11, 2025
3c4ff9b
refactor(typing): Use `PythonLiteral` for `Series` return
dangotbanned May 11, 2025
696e35d
Merge branch 'main' into expr-first
dangotbanned May 12, 2025
b2866d2
Merge branch 'main' into expr-first
dangotbanned May 12, 2025
cd002f3
test: Add `test_group_by_agg_first`
dangotbanned May 12, 2025
1458530
feat(DRAFT): Start trying `pyarrow` `agg(first())`
dangotbanned May 12, 2025
962ebcd
fix: Maybe `pyarrow` support?
dangotbanned May 12, 2025
5d310bc
refactor: Add `ArrowGroupBy._configure_agg`
dangotbanned May 12, 2025
a417341
fix: Add `pyarrow` compat for `first`
dangotbanned May 12, 2025
354da1a
fix: Don't support below `14` ever
dangotbanned May 12, 2025
0cea41b
test: Add some `None` cases
dangotbanned May 12, 2025
5229096
feat(DRAFT): Partial support for `pandas`
dangotbanned May 12, 2025
8d3aaec
docs: Tidy error and comments
dangotbanned May 12, 2025
a62e3ef
Merge branch 'main' into expr-first
dangotbanned May 12, 2025
9c36285
Merge remote-tracking branch 'upstream/main' into expr-first
dangotbanned May 13, 2025
ad8e3f7
test: xfail `ibis`
dangotbanned May 13, 2025
628f71e
feat: Add `IbisExpr.first`
dangotbanned May 13, 2025
deacc71
test: Don't xfail for `pandas<1.0.0`
dangotbanned May 13, 2025
5c52ee4
Merge branch 'main' into expr-first
dangotbanned May 14, 2025
eec2a4f
Merge branch 'main' into expr-first
dangotbanned May 16, 2025
e003bab
Merge branch 'main' into expr-first
dangotbanned May 16, 2025
fb2dc1c
Merge remote-tracking branch 'upstream/main' into expr-first
dangotbanned May 18, 2025
211673b
Merge remote-tracking branch 'upstream/main' into expr-first
dangotbanned Jun 3, 2025
652615f
fix: Use reverted `partition_by`, `_sort`
dangotbanned Jun 13, 2025
68fdfe8
Merge remote-tracking branch 'upstream/main' into expr-first
dangotbanned Jun 13, 2025
ecaca9a
fix: Update `DuckDBExpr.first`
dangotbanned Jun 15, 2025
ea30f26
fix: Update `IbisExpr.first`
dangotbanned Jun 15, 2025
12987ee
fix: Update `SparkLikeExpr.first`
dangotbanned Jun 15, 2025
7d70a42
Merge remote-tracking branch 'upstream/main' into expr-first
dangotbanned Jun 15, 2025
5446095
test: Update `pandas` xfail
dangotbanned Jun 15, 2025
b927340
Merge branch 'main' into expr-first
dangotbanned Jun 20, 2025
f62c085
test: Don't xfail for pandas `1.1.3<=...<1.1.5`
dangotbanned Jun 20, 2025
45d20c8
Merge branch 'main' into expr-first
dangotbanned Jun 21, 2025
72ab185
Merge remote-tracking branch 'upstream/main' into expr-first
dangotbanned Jun 29, 2025
e72b115
fix: Upgrade `DuckDBExpr.first` again
dangotbanned Jun 29, 2025
fae137c
Merge remote-tracking branch 'upstream/main' into expr-first
dangotbanned Jul 19, 2025
cb363be
test(DRAFT): Let's start trying to fix pandas
dangotbanned Jul 19, 2025
bc80a5f
try `pandas>=2.2.1` path
dangotbanned Jul 19, 2025
14051fa
allow very old pandas that worked?
dangotbanned Jul 19, 2025
3d42dcf
test: xfail `pandas[pyarrow]`, `modin[pyarrow]`
dangotbanned Jul 19, 2025
934d09e
Apply suggestion narwhals/_polars/series.py
dangotbanned Jul 20, 2025
3fbf6f2
Merge remote-tracking branch 'upstream/main' into expr-first
dangotbanned Jul 20, 2025
801a7a8
docs: Be more explicit on WIP `pandas`
dangotbanned Jul 20, 2025
47bfaba
docs: Link to long explanation
dangotbanned Jul 20, 2025
4618d01
revert: remove lazy support
dangotbanned Jul 20, 2025
1998ad2
Merge remote-tracking branch 'upstream/main' into expr-first
dangotbanned Jul 20, 2025
570cdaf
Merge remote-tracking branch 'upstream/main' into expr-first
dangotbanned Jul 20, 2025
d561027
Merge remote-tracking branch 'upstream/main' into expr-first
dangotbanned Jul 21, 2025
b77d2b3
try `nth` for `>=1.1.5; <2.0.0`
dangotbanned Jul 21, 2025
2b0bc16
Is this fixed?
dangotbanned Jul 21, 2025
abbb4b7
cov
dangotbanned Jul 21, 2025
ccfe532
feat: Add `(Expr|Series).last`
dangotbanned Jul 21, 2025
dd1f89e
test: Add `last_test.py`
dangotbanned Jul 22, 2025
54b3188
test: Add `test_group_by_agg_last`
dangotbanned Jul 22, 2025
5f9ff6f
fix: Add missing `PandasLikeGroupBy._REMAP_AGGS` entry
dangotbanned Jul 22, 2025
4000b25
test: Repeat `@single_cases` pattern for `first`
dangotbanned Jul 22, 2025
1c62ce2
docs: Examples for `Expr.(first|last)`
dangotbanned Jul 22, 2025
64fdf10
Merge remote-tracking branch 'upstream/main' into expr-first
dangotbanned Jul 22, 2025
063e5d0
Remove `modin` todo
dangotbanned Jul 22, 2025
2e4f260
Merge branch 'main' into expr-first
dangotbanned Jul 23, 2025
65e6804
clean up and doc `pandas`
dangotbanned Jul 23, 2025
22fae20
feat: Warn on new pandas apply path
dangotbanned Jul 23, 2025
60624b9
Merge remote-tracking branch 'upstream/main' into expr-first
dangotbanned Jul 23, 2025
d66fddc
cov
dangotbanned Jul 23, 2025
5e444a5
always use `apply` for `cudf` 😒
dangotbanned Jul 24, 2025
e1a9bc3
Merge remote-tracking branch 'upstream/main' into expr-first
dangotbanned Jul 25, 2025
0cbe33d
Merge branch 'main' into expr-first
dangotbanned Jul 26, 2025
2960736
Merge remote-tracking branch 'upstream/main' into expr-first
dangotbanned Jul 28, 2025
4ede6b2
merge main
FBruzzesi Jul 29, 2025
2ae4245
special path for orderable_aggregation in over
FBruzzesi Jul 29, 2025
b8066c4
expand on comments
FBruzzesi Jul 29, 2025
2dae6ef
assign metadata in arrow
FBruzzesi Jul 29, 2025
3aa52dc
Merge branch 'main' into expr-first
FBruzzesi Aug 2, 2025
7c578c7
Merge branch 'main' into expr-first
dangotbanned Aug 5, 2025
30bad0e
Merge branch 'main' into expr-first
dangotbanned Aug 7, 2025
d269d56
Merge branch 'main' into expr-first
dangotbanned Aug 7, 2025
c0e37aa
Merge remote-tracking branch 'upstream/main' into expr-first
dangotbanned Aug 8, 2025
6f5c05b
Merge remote-tracking branch 'upstream/main' into expr-first
dangotbanned Aug 12, 2025
20be193
Merge remote-tracking branch 'upstream/main' into expr-first
dangotbanned Aug 13, 2025
abd027a
Merge remote-tracking branch 'upstream/main' into expr-first
dangotbanned Aug 14, 2025
1fd9fd3
Merge branch 'main' into expr-first
dangotbanned Aug 15, 2025
94d6b19
Merge remote-tracking branch 'upstream/main' into expr-first
dangotbanned Aug 17, 2025
849a6d9
Merge branch 'main' into expr-first
dangotbanned Aug 18, 2025
476c63e
Merge remote-tracking branch 'upstream/main' into expr-first
dangotbanned Aug 18, 2025
c169104
Merge remote-tracking branch 'upstream/main' into expr-first
dangotbanned Aug 19, 2025
d77fcd1
Merge branch 'main' into expr-first
dangotbanned Aug 19, 2025
1f38bde
Merge branch 'main' into expr-first
dangotbanned Aug 19, 2025
3c63726
Merge branch 'main' into expr-first
dangotbanned Aug 20, 2025
6d7b09b
Merge remote-tracking branch 'upstream/main' into expr-first
dangotbanned Aug 21, 2025
3b6301f
Merge branch 'main' into expr-first
dangotbanned Aug 23, 2025
bfc55c7
Merge remote-tracking branch 'upstream/main' into expr-first
dangotbanned Aug 23, 2025
f47ef14
docs: Remove *Returns* from `Expr` version
dangotbanned Aug 23, 2025
b32db75
Merge remote-tracking branch 'upstream/main' into expr-first
dangotbanned Aug 25, 2025
f22a497
Merge branch 'main' into expr-first
dangotbanned Aug 25, 2025
b5fe1ba
Merge remote-tracking branch 'upstream/main' into expr-first
dangotbanned Aug 28, 2025
0f301e7
Merge branch 'main' into expr-first
dangotbanned Aug 30, 2025
16a2762
Merge branch 'main' into expr-first
dangotbanned Sep 3, 2025
09dca76
Merge branch 'main' into expr-first
dangotbanned Sep 5, 2025
ffe7e24
Merge remote-tracking branch 'upstream/main' into expr-first
dangotbanned Sep 13, 2025
0fb0455
chore(typing): fix incompatible override
dangotbanned Sep 13, 2025
6d63ea6
simplify grouped first/last#
MarcoGorelli Oct 2, 2025
7b00310
simplify test, remove unnecessary over(order_by)
MarcoGorelli Oct 2, 2025
016abc9
combine tests
MarcoGorelli Oct 2, 2025
29d6cb7
combine tests
MarcoGorelli Oct 2, 2025
3b91e23
duckdb fix#
MarcoGorelli Oct 2, 2025
c87935d
sort out ibis
MarcoGorelli Oct 2, 2025
0393dfe
dask
MarcoGorelli Oct 2, 2025
466c922
add note to docs
MarcoGorelli Oct 2, 2025
4266e4b
remove unnecessary code
MarcoGorelli Oct 2, 2025
555098b
pyarrow
MarcoGorelli Oct 2, 2025
36e38e0
fixup
MarcoGorelli Oct 2, 2025
42d2cd6
typing
MarcoGorelli Oct 2, 2025
63f012a
dask
MarcoGorelli Oct 2, 2025
c4ac043
test and support `diff().sum().over(order_by=...)`
MarcoGorelli Oct 3, 2025
8739b6a
cross-pandas version compat
MarcoGorelli Oct 3, 2025
ff22604
make test more unusual
MarcoGorelli Oct 3, 2025
d9c4a1b
fix another pyarrow issue
MarcoGorelli Oct 3, 2025
03b7969
catch more warnings for modin
MarcoGorelli Oct 3, 2025
d01a398
factor out sql_expression, link to feature request
MarcoGorelli Oct 3, 2025
18c0861
combine first and last blocks
MarcoGorelli Oct 3, 2025
948d96d
remove more unneeded
MarcoGorelli Oct 3, 2025
8810d03
less special-casing
MarcoGorelli Oct 3, 2025
843549f
simplify further
MarcoGorelli Oct 3, 2025
d7be792
Merge remote-tracking branch 'upstream/main' into expr-first
MarcoGorelli Oct 3, 2025
363490d
typing
MarcoGorelli Oct 3, 2025
c25d649
use repeat_by instead of lit for polars
MarcoGorelli Oct 3, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/api-reference/expr.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@
- ewm_mean
- fill_null
- filter
- first
- gather_every
- head
- clip
Expand Down
1 change: 1 addition & 0 deletions docs/api-reference/series.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@
- ewm_mean
- fill_null
- filter
- first
- gather_every
- head
- hist
Expand Down
4 changes: 4 additions & 0 deletions narwhals/_arrow/series.py
Original file line number Diff line number Diff line change
Expand Up @@ -315,6 +315,10 @@ def filter(self, predicate: ArrowSeries | list[bool | None]) -> Self:
other_native = predicate
return self._with_native(self.native.filter(other_native))

def first(self, *, _return_py_scalar: bool = True) -> Any:
result = self.native[0] if len(self.native) else None
return maybe_extract_py_scalar(result, _return_py_scalar)

def mean(self, *, _return_py_scalar: bool = True) -> float:
return maybe_extract_py_scalar(pc.mean(self.native), _return_py_scalar)

Expand Down
4 changes: 4 additions & 0 deletions narwhals/_compliant/expr.py
Original file line number Diff line number Diff line change
Expand Up @@ -160,6 +160,7 @@ def cum_max(self, *, reverse: bool) -> Self: ...
def cum_prod(self, *, reverse: bool) -> Self: ...
def is_in(self, other: Any) -> Self: ...
def sort(self, *, descending: bool, nulls_last: bool) -> Self: ...
def first(self) -> Self: ...
def rank(self, method: RankMethod, *, descending: bool) -> Self: ...
def replace_strict(
self,
Expand Down Expand Up @@ -851,6 +852,9 @@ def func(df: EagerDataFrameT) -> Sequence[EagerSeriesT]:
context=self,
)

def first(self) -> Self:
return self._reuse_series("first", returns_scalar=True)

@property
def cat(self) -> EagerExprCatNamespace[Self]:
return EagerExprCatNamespace(self)
Expand Down
1 change: 1 addition & 0 deletions narwhals/_compliant/series.py
Original file line number Diff line number Diff line change
Expand Up @@ -177,6 +177,7 @@ def fill_null(
limit: int | None,
) -> Self: ...
def filter(self, predicate: Any) -> Self: ...
def first(self) -> Any: ...
def gather_every(self, n: int, offset: int) -> Self: ...
@unstable
def hist(
Expand Down
6 changes: 6 additions & 0 deletions narwhals/_dask/expr.py
Original file line number Diff line number Diff line change
Expand Up @@ -663,6 +663,12 @@ def is_finite(self) -> Self:

return self._with_callable(da.isfinite, "is_finite")

def first(self) -> Self:
def fn(_input: dx.Series) -> dx.Series:
return _input[0].to_series()

return self._with_callable(fn, "first")

@property
def str(self) -> DaskExprStringNamespace:
return DaskExprStringNamespace(self)
Expand Down
6 changes: 6 additions & 0 deletions narwhals/_duckdb/expr.py
Original file line number Diff line number Diff line change
Expand Up @@ -408,6 +408,12 @@ def _clip_both(
_clip_both, lower_bound=lower_bound, upper_bound=upper_bound
)

def first(self) -> Self:
def fn(_input: duckdb.Expression) -> duckdb.Expression:
return FunctionExpression("first", _input)

return self._with_callable(fn)

def sum(self) -> Self:
return self._with_callable(lambda _input: FunctionExpression("sum", _input))

Expand Down
3 changes: 3 additions & 0 deletions narwhals/_pandas_like/series.py
Original file line number Diff line number Diff line change
Expand Up @@ -381,6 +381,9 @@ def filter(self, predicate: Any) -> PandasLikeSeries:
other_native = predicate
return self._with_native(self.native.loc[other_native]).alias(self.name)

def first(self) -> Any:
return self.native.iloc[0] if len(self.native) else None

def __eq__(self, other: object) -> PandasLikeSeries: # type: ignore[override]
ser, other = align_and_extract_native(self, other)
return self._with_native(ser == other).alias(self.name)
Expand Down
1 change: 1 addition & 0 deletions narwhals/_polars/expr.py
Original file line number Diff line number Diff line change
Expand Up @@ -281,6 +281,7 @@ def struct(self) -> PolarsExprStructNamespace:
diff: Method[Self]
drop_nulls: Method[Self]
fill_null: Method[Self]
first: Method[Self]
gather_every: Method[Self]
head: Method[Self]
is_finite: Method[Self]
Expand Down
7 changes: 7 additions & 0 deletions narwhals/_polars/series.py
Original file line number Diff line number Diff line change
Expand Up @@ -611,6 +611,13 @@ def hist( # noqa: C901, PLR0912
def to_polars(self) -> pl.Series:
return self.native

def first(self) -> Any:
if self._backend_version >= (1, 10):
return self.native.first()
elif len(self): # pragma: no cover
return self.native.item(0)
return None # pragma: no cover

@property
def dt(self) -> PolarsSeriesDateTimeNamespace:
return PolarsSeriesDateTimeNamespace(self)
Expand Down
8 changes: 8 additions & 0 deletions narwhals/_spark_like/expr.py
Original file line number Diff line number Diff line change
Expand Up @@ -560,6 +560,14 @@ def _clip_both(
_clip_both, lower_bound=lower_bound, upper_bound=upper_bound
)

def first(self) -> Self:
def fn(inputs: WindowInputs) -> Column:
return self._F.first(inputs.expr, ignorenulls=False).over(
self.partition_by(inputs).orderBy(*self._sort(inputs))
)

return self._with_window_function(fn)

def is_finite(self) -> Self:
def _is_finite(_input: Column) -> Column:
# A value is finite if it's not NaN, and not infinite, while NULLs should be
Expand Down
10 changes: 10 additions & 0 deletions narwhals/expr.py
Original file line number Diff line number Diff line change
Expand Up @@ -1965,6 +1965,16 @@ def clip(
),
)

def first(self) -> Self:
"""Get the first value.

Returns:
A new expression.
"""
return self._with_orderable_aggregation(
lambda plx: self._to_compliant_expr(plx).first()
)

def mode(self) -> Self:
r"""Compute the most occurring value(s).

Expand Down
19 changes: 19 additions & 0 deletions narwhals/series.py
Original file line number Diff line number Diff line change
Expand Up @@ -801,6 +801,25 @@ def clip(
)
)

def first(self) -> Any:
"""Get the first element of the Series.

Returns:
A scalar value or `None` if the Series is empty.

Examples:
>>> import polars as pl
>>> import narwhals as nw
>>>
>>> s_native = pl.Series([1, 2, 3])
>>> s_nw = nw.from_native(s_native, series_only=True)
>>> s_nw.first()
1
>>> s_nw.filter(s_nw > 5).first() is None
True
Comment on lines +900 to +909
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't like the None example, but this was the only way I saw to get a repr 😞

I think it's important to have an example for that case though - since pandas and pyarrow would raise an index error normally

"""
return self._compliant_series.first()

def is_in(self, other: Any) -> Self:
"""Check if the elements of this Series are in the other sequence.

Expand Down
68 changes: 68 additions & 0 deletions tests/expr_and_series/first_test.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
from __future__ import annotations

from typing import TYPE_CHECKING
from typing import Mapping
from typing import Sequence

import pytest

import narwhals as nw
from tests.utils import assert_equal_data

if TYPE_CHECKING:
from narwhals.typing import PythonLiteral
from tests.utils import ConstructorEager

data = {
"a": [8, 2, 1, None],
"b": [58, 5, 6, 12],
"c": [2.5, 1.0, 3.0, 0.9],
"d": [2, 1, 4, 3],
}


@pytest.mark.parametrize(("col", "expected"), [("a", 8), ("b", 58), ("c", 2.5)])
def test_first_series(
constructor_eager: ConstructorEager, col: str, expected: PythonLiteral
) -> None:
series = nw.from_native(constructor_eager(data), eager_only=True)[col]
result = series.first()
assert_equal_data({col: [result]}, {col: [expected]})


def test_first_series_empty(constructor_eager: ConstructorEager) -> None:
series = nw.from_native(constructor_eager(data), eager_only=True)["a"]
series = series.filter(series > 50)
result = series.first()
assert result is None


@pytest.mark.parametrize(("col", "expected"), [("a", 8), ("b", 58), ("c", 2.5)])
def test_first_expr_eager(
constructor_eager: ConstructorEager, col: str, expected: PythonLiteral
) -> None:
df = nw.from_native(constructor_eager(data))
expr = nw.col(col).first()
result = df.select(expr)
assert_equal_data(result, {col: [expected]})


@pytest.mark.parametrize(
"expected",
[{"a": [8], "c": [2.5]}, {"d": [2], "b": [58]}, {"c": [2.5], "a": [8], "d": [2]}],
)
def test_first_expr_eager_expand(
constructor_eager: ConstructorEager, expected: Mapping[str, Sequence[PythonLiteral]]
) -> None:
df = nw.from_native(constructor_eager(data))
expr = nw.col(expected).first()
result = df.select(expr)
assert_equal_data(result, expected)


def test_first_expr_eager_expand_sort(constructor_eager: ConstructorEager) -> None:
df = nw.from_native(constructor_eager(data))
expr = nw.col("d", "a", "b", "c").first()
result = df.sort("d").select(expr)
expected = {"d": [1], "a": [2], "b": [5], "c": [1.0]}
assert_equal_data(result, expected)
Loading