Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Speed up .dt accessor by preserving Index objects. #7796

Merged
merged 9 commits into from
May 16, 2023
Merged

Speed up .dt accessor by preserving Index objects. #7796

merged 9 commits into from
May 16, 2023

Conversation

dcherian
Copy link
Contributor

  • Closes #xxxx
  • Tests added
  • User visible changes (including notable bug fixes) are documented in whats-new.rst
  • New functions/methods are listed in api.rst

@dcherian dcherian force-pushed the speedup-dt-accesor branch from 981c4c4 to 842bff1 Compare April 30, 2023 03:42
headtr1ck
headtr1ck reviewed Apr 30, 2023
View reviewed changes
Copy link
Collaborator

@headtr1ck headtr1ck left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add a benchmark for this?

@dcherian dcherian added the run-benchmark Run the ASV benchmark workflow label May 2, 2023
dcherian and others added 3 commits May 5, 2023 21:15
@dcherian
Merge branch 'main' into speedup-dt-accesor
993ad42 
* main:
  Introduce Grouper objects internally (pydata#7561)
  [skip-ci] Add cftime groupby, resample benchmarks (pydata#7795)
  Fix groupby binary ops when grouped array is subset relative to other (pydata#7798)
  adjust the deprecation policy for python (pydata#7793)
  [pre-commit.ci] pre-commit autoupdate (pydata#7803)
  Allow the label run-upstream to run upstream CI (pydata#7787)
  Update asv links in contributing guide (pydata#7801)
  Implement DataArray.to_dask_dataframe() (pydata#7635)
  `ds.to_dict` with data as arrays, not lists (pydata#7739)
  Add lshift and rshift operators (pydata#7741)
  Use canonical name for set_horizonalalignment over alias set_ha (pydata#7786)
  Remove pandas<2 pin (pydata#7785)
  [pre-commit.ci] pre-commit autoupdate (pydata#7783)
@dcherian
Add whats-new
0d88bac
@Illviljan
Merge branch 'main' into speedup-dt-accesor
3ef0a63
@dcherian dcherian requested a review from headtr1ck May 8, 2023 18:01
@dcherian dcherian added the plan to merge Final call for comments label May 10, 2023
headtr1ck
headtr1ck approved these changes May 12, 2023
View reviewed changes
Copy link
Collaborator

@headtr1ck headtr1ck left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't see too crazy speedups, but in general this looks good.

@dcherian
Revert "Fix mypy"
1eb7414 
This reverts commit fb0b073.
@dcherian
Copy link
Contributor Author

dcherian commented May 12, 2023 •
edited
Loading

Yes, 10-30% on my machine:

       before           after         ratio
     [91f14c9b]       [1eb74149]
     <v2023.04.2>       <speedup-dt-accesor>
-     1.17±0.04ms      1.02±0.04ms     0.87  accessors.DateTimeAccessor.time_dayofyear('standard')
-     1.25±0.07ms         976±30μs     0.78  accessors.DateTimeAccessor.time_year('standard')
-      3.90±0.1ms      2.68±0.05ms     0.69  accessors.DateTimeAccessor.time_year('noleap')
-     4.75±0.07ms      3.25±0.04ms     0.68  accessors.DateTimeAccessor.time_dayofyear('noleap')

@dcherian dcherian enabled auto-merge (squash) May 16, 2023 17:35
@dcherian dcherian merged commit c8c6a4b into pydata:main May 16, 2023
@dcherian dcherian mentioned this pull request Jun 15, 2023
Open
19 tasks
@idantene idantene mentioned this pull request Jun 19, 2023
Closed
4 tasks
dstansby pushed a commit to dstansby/xarray that referenced this pull request Jun 28, 2023
@dcherian @Illviljan @dstansby
Speed up .dt accessor by preserving Index objects. (pydata#7796)
3a76d0a 
* Speed up .dt accessor by preserving Index objects.

* Switch to _replace

* Add benchmark

* Add whats-new

* Shallow copy

* Fix mypy

* Revert "Fix mypy"

This reverts commit fb0b073.

---------

Co-authored-by: Illviljan <[email protected]>
@dcherian dcherian deleted the speedup-dt-accesor branch November 6, 2023 04:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@headtr1ck headtr1ck headtr1ck approved these changes

Assignees
No one assigned
Labels
plan to merge Final call for comments run-benchmark Run the ASV benchmark workflow topic-groupby topic-performance
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@dcherian @headtr1ck @Illviljan