Skip to content

Pandas 2.0 update #3322

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 10 commits into from
Apr 13, 2023
Merged

Pandas 2.0 update #3322

merged 10 commits into from
Apr 13, 2023

Conversation

@jeffkinnison
Copy link
Contributor

@jeffkinnison jeffkinnison commented Apr 4, 2023 •
edited
Loading

Updates to make Ludwig compatible with Pandas 2.0. The current changes include:

  • Remove experiment_impact_tracker as a dependency
  • Update LudwigProfiler to replicate experiment_impact_tracker CPU/GPU attribute collection
  • df.iteritems() -> df.items()
  • df.append(df2) -> df_engine.concat([df, df2])
  • Support for writing xls files was removed. This only seems to impact one integration test.

@github-actions
Copy link

github-actions bot commented Apr 5, 2023 •
edited
Loading

Unit Test Results

  6 files  +  2    6 suites  +2   1h 1m 56s ⏱️ + 22m 14s
33 tests ±  0  30 ✔️ ±  0  3 💤 ±0  0 ❌ ±0 
99 runs  +33  90 ✔️ +30  9 💤 +3  0 ❌ ±0 

Results for commit b04c261. ± Comparison against base commit d68d96e.

♻️ This comment has been updated with latest results.

@jeffkinnison jeffkinnison requested a review from abidwael April 5, 2023 18:09
justinxzhao
justinxzhao approved these changes Apr 10, 2023
View reviewed changes
Copy link
Contributor

@justinxzhao justinxzhao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sweet! Just one comment, otherwise LGTM.

ludwig/benchmarking/profiler.py Outdated
def get_gpu_info():
"""Gathers general hardware information about an nvidia GPU.
:param args:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Remove empty docs.

If this is copied from the other repo, please reference it as a comment i.e.
Adapted from <link>, with additional adjustments for compatibility with Pandas 2.0.

ludwig/benchmarking/profiler.py
from ludwig.globals import LUDWIG_VERSION
from ludwig.utils.data_utils import save_json

# disabling print because the following imports are verbose
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice cleanup.

@jeffkinnison
Copy link
Contributor Author

The changes listed above should cover the 2.0 update, and as far as I can tell we should still be able to support 1.x because df.items and pd.concat have been in the API since at least 1.0.

requirements.txt does allow for versions all the way back to 0.25, though, and concat doesn't exist that far back. Do we want to bump the minimum version to 1.0?

@justinxzhao

@jeffkinnison jeffkinnison changed the title [Draft] Pandas 2.0 update Pandas 2.0 update Apr 10, 2023
@jeffkinnison jeffkinnison marked this pull request as ready for review April 10, 2023 16:15
@justinxzhao
Copy link
Contributor

The changes listed above should cover the 2.0 update, and as far as I can tell we should still be able to support 1.x because df.items and pd.concat have been in the API since at least 1.0.

requirements.txt does allow for versions all the way back to 0.25, though, and concat doesn't exist that far back. Do we want to bump the minimum version to 1.0?

@justinxzhao

Bumping the min version to 1.0 SGTM!

@jeffkinnison jeffkinnison merged commit b8f7f3a into master Apr 13, 2023
@jeffkinnison jeffkinnison deleted the pandas-2.0-update branch April 13, 2023 20:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@justinxzhao justinxzhao justinxzhao approved these changes

@abidwael abidwael Awaiting requested review from abidwael

@tgaddair tgaddair Awaiting requested review from tgaddair tgaddair is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@jeffkinnison @justinxzhao