-
Notifications
You must be signed in to change notification settings - Fork 210
feat: Added struct namespace with field method.
#2146
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. Weβll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
MarcoGorelli
merged 44 commits into
narwhals-dev:main
from
okube-ai:feat/struct_namespace
Mar 9, 2025
Merged
Changes from 38 commits
Commits
Show all changes
44 commits
Select commit
Hold shift + click to select a range
0d18f96
Update
osoucy 496cbd0
Update
osoucy 9592277
Update
osoucy aaebbbe
Update
osoucy 273f331
Update
osoucy 2d95d3d
Update
osoucy decba02
Update
osoucy f89f7d3
Merge branch 'main' into feat/struct_namespace
osoucy e75da43
Addressed PR comment
osoucy ef8b863
Update docs/api-completeness/expr_struct.md
osoucy fa802a4
Merge remote-tracking branch 'origin/feat/struct_namespace' into featβ¦
osoucy 5e04a46
Addressed PR comment
osoucy 47345f5
Addressed PR comment
osoucy 9517b91
Merge branch 'main' into feat/struct_namespace
osoucy e8623e2
Addressed PR comment
osoucy 2d4ce6b
Addressed PR comment
osoucy 825d29e
Addressed PR comment
osoucy 6f867ef
Addressed PR comment
osoucy e54663c
Addressed PR comment
osoucy 8fe86ae
Addressed PR comment
osoucy 37dc7fc
Addressed PR comment
osoucy 104d40d
Addressed PR comment
osoucy 9ead00a
Merge branch 'main' into feat/struct_namespace
osoucy 9e2a24d
Merge branch 'main' into feat/struct_namespace
osoucy 20ec4eb
Merge branch 'main' into feat/struct_namespace
osoucy ecfd180
Merge branch 'main' into feat/struct_namespace
osoucy 6c5597b
PR Comments
osoucy 99a78e3
Merge branch 'main' into feat/struct_namespace
osoucy 105d3f9
Update docs/api-completeness/series_struct.md
osoucy d21bdc2
PR Comments
osoucy 9db179a
Merge branch 'main' into feat/struct_namespace
osoucy 865bedd
Merge branch 'main' into feat/struct_namespace
osoucy 3ad7ae9
PR Comments
osoucy 7e5d813
PR Comments
osoucy c5cf993
PR Comments
osoucy 4d36295
PR Comments
osoucy 8701e43
Merge branch 'main' into feat/struct_namespace
osoucy 4ed080c
PR Comments
osoucy 195088c
PR Comments
osoucy 71df89d
Merge branch 'main' into feat/struct_namespace
osoucy ebe4d6b
remove unnecessary aliases
MarcoGorelli 59fb567
remove unnecessary aliases
MarcoGorelli 736599e
extra test
MarcoGorelli f71e677
remove outdated
MarcoGorelli File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| # `narwhals.Expr.struct` | ||
|
|
||
| ::: narwhals.expr.ExprStructNamespace | ||
| handler: python | ||
| options: | ||
| members: | ||
| - field | ||
| show_source: false | ||
| show_bases: false |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| # `narwhals.Series.struct` | ||
|
|
||
| ::: narwhals.series.SeriesStructNamespace | ||
| handler: python | ||
| options: | ||
| members: | ||
| - field | ||
| show_source: false | ||
| show_bases: false |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| from __future__ import annotations | ||
|
|
||
| from typing import TYPE_CHECKING | ||
|
|
||
| from narwhals._expression_parsing import reuse_series_namespace_implementation | ||
|
|
||
| if TYPE_CHECKING: | ||
| from typing_extensions import Self | ||
|
|
||
| from narwhals._arrow.expr import ArrowExpr | ||
|
|
||
|
|
||
| class ArrowExprStructNamespace: | ||
| def __init__(self: Self, expr: ArrowExpr) -> None: | ||
| self._compliant_expr = expr | ||
|
|
||
| def field(self: Self, name: str) -> ArrowExpr: | ||
| return reuse_series_namespace_implementation( | ||
| self._compliant_expr, | ||
| "struct", | ||
| "field", | ||
| name=name, | ||
| ).alias(name) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| from __future__ import annotations | ||
|
|
||
| from typing import TYPE_CHECKING | ||
|
|
||
| import pyarrow.compute as pc | ||
|
|
||
| if TYPE_CHECKING: | ||
| from typing_extensions import Self | ||
|
|
||
| from narwhals._arrow.series import ArrowSeries | ||
|
|
||
|
|
||
| class ArrowSeriesStructNamespace: | ||
| def __init__(self: Self, series: ArrowSeries) -> None: | ||
| self._compliant_series: ArrowSeries = series | ||
|
|
||
| def field(self: Self, name: str) -> ArrowSeries: | ||
| self._compliant_series._name = name | ||
| return self._compliant_series._from_native_series( | ||
| pc.struct_field(self._compliant_series._native_series, name), | ||
| ) | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,25 @@ | ||
| from __future__ import annotations | ||
|
|
||
| from typing import TYPE_CHECKING | ||
|
|
||
| from duckdb import FunctionExpression | ||
|
|
||
| from narwhals._duckdb.utils import lit | ||
|
|
||
| if TYPE_CHECKING: | ||
| from typing_extensions import Self | ||
|
|
||
| from narwhals._duckdb.expr import DuckDBExpr | ||
|
|
||
|
|
||
| class DuckDBExprStructNamespace: | ||
| def __init__(self: Self, expr: DuckDBExpr) -> None: | ||
| self._compliant_expr = expr | ||
|
|
||
| def field(self: Self, name: str) -> DuckDBExpr: | ||
| return self._compliant_expr._from_call( | ||
| lambda _input: FunctionExpression("struct_extract", _input, lit(name)).alias( | ||
| name | ||
| ), | ||
| "field", | ||
| ).alias(name) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -217,9 +217,12 @@ def reuse_series_namespace_implementation( | |
| expr: expression object. | ||
| series_namespace: The Series namespace (e.g. `dt`, `cat`, `str`, `list`, `name`) | ||
| attr: name of method. | ||
| evaluate_output_names: Output names function. | ||
| alias_output_names: Alias output names function. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. outdated |
||
| kwargs: keyword arguments to pass to function. | ||
| """ | ||
| plx = expr.__narwhals_namespace__() | ||
|
|
||
| return plx._create_expr_from_callable( # type: ignore[return-value] | ||
| lambda df: [ | ||
| getattr(getattr(series, series_namespace), attr)(**kwargs) | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| from __future__ import annotations | ||
|
|
||
| from typing import TYPE_CHECKING | ||
|
|
||
| from narwhals._expression_parsing import reuse_series_namespace_implementation | ||
|
|
||
| if TYPE_CHECKING: | ||
| from typing_extensions import Self | ||
|
|
||
| from narwhals._pandas_like.expr import PandasLikeExpr | ||
|
|
||
|
|
||
| class PandasLikeExprStructNamespace: | ||
| def __init__(self: Self, expr: PandasLikeExpr) -> None: | ||
| self._compliant_expr = expr | ||
|
|
||
| def field(self, name: str) -> PandasLikeExpr: | ||
| return reuse_series_namespace_implementation( | ||
| self._compliant_expr, | ||
| "struct", | ||
| "field", | ||
| name=name, | ||
| ).alias(name) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -16,6 +16,9 @@ | |
|
|
||
| class PandasLikeSeriesListNamespace: | ||
| def __init__(self: Self, series: PandasLikeSeries) -> None: | ||
| if not hasattr(series._native_series, "list"): | ||
| msg = "Series must be of PyArrow List type to support struct namespace." | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. typo |
||
| raise TypeError(msg) | ||
| self._compliant_series = series | ||
|
|
||
| def len(self: Self) -> PandasLikeSeries: | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| from __future__ import annotations | ||
|
|
||
| from typing import TYPE_CHECKING | ||
|
|
||
| if TYPE_CHECKING: | ||
| from typing_extensions import Self | ||
|
|
||
| from narwhals._pandas_like.series import PandasLikeSeries | ||
|
|
||
|
|
||
| class PandasLikeSeriesStructNamespace: | ||
| def __init__(self: Self, series: PandasLikeSeries) -> None: | ||
| if not hasattr(series._native_series, "struct"): | ||
| msg = "Series must be of PyArrow Struct type to support struct namespace." | ||
| raise TypeError(msg) | ||
| self._compliant_series = series | ||
|
|
||
| def field(self: Self, name: str) -> PandasLikeSeries: | ||
| return self._compliant_series._from_native_series( | ||
| self._compliant_series._native_series.struct.field(name).rename(name) | ||
| ) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
you can't mutate
self._compliant_series, you'll needaliashere tooThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For the
Arrow*stuff I'd suggest using.compliantand.native_Stores(Native|Compliant)Β #2130They weren't available when you started the PR @osoucy
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some more context #2130 (comment)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this is fine as a follow-up
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
At least the current implementation
avoids mutating the compliant series. Maybe I can look into @dangotbanned in a future PR?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
All good @osoucy π
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So we are good to merge? :)