Skip to content

Conversation

@coastalwhite
Copy link
Collaborator

@coastalwhite coastalwhite commented Jun 28, 2025

Note

TLDR

Selectors are reimplemented to be more reliable and easier to maintain and expand in the future. This brings many improvements, such as being able to chain .name expressions, all selectors being able to be combined and several new datatype selectors.

This basically makes the expression expansion into one function and moves the items that are not expansion (e.g. name.keep, name.map, struct.field[int]) into the DSL to IR conversion. This makes both a lot cleaner and simpler. Expressions containing multiple selectors will now generally always follow the rule (n, n) -> n or (1, n) -> n.

This also introduces:

  • cs.empty()
  • cs.list()
  • cs.array()
  • cs.struct()
  • cs.enum()
  • name.keep and name.{map, prefix, suffix, ...} can now be used multiple times in succession.

Fixes #10782.
Fixes #10594.
Fixes #21174.
Fixes #21164.
Fixes #14059.
Fixes #13858.
Fixes #12850.
Fixes #8282.
Fixes #19023.
Fixes #19384.
Fixes #21352.
Fixes #22200.
Fixes #11067.
Fixes #11425.
Fixes #22955.

@github-actions github-actions bot added internal An internal refactor or improvement python Related to Python Polars rust Related to Rust Polars labels Jun 28, 2025
@coastalwhite coastalwhite force-pushed the refactor/selector-try-2 branch from 0a37716 to abb9003 Compare July 7, 2025 09:10
@coastalwhite coastalwhite changed the title refactor: Make Selector concrete DSL part refactor: Make Selector a concrete part of the DSL Jul 7, 2025
This basically makes the expression expansion into one function and moves the
items that are not expansion (e.g. `name.keep`, `name.map`,
`struct.field[int]`) into the DSL to IR conversion. This makes both a lot
cleaner and simpler.

This also introduces:
- `cs.empty()`
- `cs.list()`
- `cs.array()`
- `cs.struct()`
- `cs.enum()`
- `name.keep` and `name.{map, prefix, suffix, ...}` can now be used multiple
  times in succession.

TODO:
- [ ] Gate the expr_expansion behind actually having stuff that needs to get expanded.
- [ ] Docs for new functions
- [ ] Clean Up
@coastalwhite coastalwhite merged commit 8a961cd into pola-rs:main Jul 8, 2025
32 checks passed
@coastalwhite coastalwhite deleted the refactor/selector-try-2 branch July 8, 2025 07:35
jqnatividad added a commit to dathere/qsv that referenced this pull request Jul 9, 2025
@coastalwhite coastalwhite added the highlight Highlight this PR in the changelog label Jul 24, 2025
jqnatividad added a commit to dathere/qsv that referenced this pull request Aug 3, 2025
as Selector is now a concrete part of the DSL

see pola-rs/polars#23351
schmidma added a commit to schmidma/great-tables that referenced this pull request Sep 1, 2025
Polars replaced `_selector_proxy_` with a public `Selector` class
(pola-rs/polars#23351). This caused type
checking imports to fail. This updates the type annotation import and
alias accordingly.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

highlight Highlight this PR in the changelog internal An internal refactor or improvement python Related to Python Polars rust Related to Rust Polars

Projects

None yet

1 participant