Skip to content

refactor: Simplify (Arrow|PandasLike)Namespace.concat#2381

Merged
dangotbanned merged 10 commits intomainfrom
simpl-ArrowNamespace.concat
Apr 14, 2025
Merged

refactor: Simplify (Arrow|PandasLike)Namespace.concat#2381
dangotbanned merged 10 commits intomainfrom
simpl-ArrowNamespace.concat

Conversation

@dangotbanned
Copy link
Member

What type of PR is this? (check all applicable)

  • 💾 Refactor
  • ✨ Feature
  • 🐛 Bug Fix
  • 🔧 Optimization
  • 📝 Documentation
  • ✅ Test
  • 🐳 Other

Related issues

Checklist

  • Code follows style guide (ruff)
  • Tests added
  • Documented the changes

If you have comments or can explain your changes, please do so below

@dangotbanned dangotbanned added internal pyarrow Issue is related to pyarrow backend labels Apr 12, 2025
def _concat_horizontal(
self, dfs: Sequence[NativeFrameT | Any], /
) -> NativeFrameT: ...
def _concat_vertical(self, dfs: Sequence[NativeFrameT], /) -> NativeFrameT: ...
Copy link
Member Author

@dangotbanned dangotbanned Apr 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

We could pretty easily support how="vertical" for NativeSeriesT

That is the only ConcatMethod that polars allows for Series

  • pyarrow would use pa.concat_arrays
  • (pandas|polars) just use their concat function again

I'm not in a rush to add it, but this PR certainly makes that next step quite simple 😅

@dangotbanned dangotbanned changed the title refactor: Simplify ArrowNamespace.concat refactor: Simplify (Arrow|PandasLike)Namespace.concat Apr 13, 2025
@dangotbanned dangotbanned added the pandas-like Issue is related to pandas-like backends label Apr 13, 2025
@dangotbanned dangotbanned marked this pull request as ready for review April 13, 2025 12:48
Copy link
Member

@FBruzzesi FBruzzesi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @dangotbanned - it certainly simplifies the logic and set it up for mooore (comment)

SwiftmoTwitchGIF

@dangotbanned dangotbanned merged commit 78f27af into main Apr 14, 2025
28 of 29 checks passed
@dangotbanned dangotbanned deleted the simpl-ArrowNamespace.concat branch April 14, 2025 21:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

internal pandas-like Issue is related to pandas-like backends pyarrow Issue is related to pyarrow backend

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants