Skip to content

fix: Bump pandas to 2.1.4 for python 3.12 #34999

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
rusackas merged 2 commits into from
Sep 19, 2025
Merged

fix: Bump pandas to 2.1.4 for python 3.12 #34999

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
69b464c
Bump pandas to 2.1.4 for python 3.12
rad-pat Sep 3, 2025
d8f3f1f
Override get_view_names in dataloader for PRESTO dialect
rad-pat Sep 8, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ dependencies = [
"packaging",
# --------------------------
# pandas and related (wanting pandas[performance] without numba as it's 100+MB and not needed)
"pandas[excel]>=2.0.3, <2.1",
"pandas[excel]>=2.0.3, <2.2",
"bottleneck", # recommended performance dependency for pandas, see https://pandas.pydata.org/docs/getting_started/install.html#performance-dependencies-recommended
# --------------------------
"parsedatetime",
Expand Down
3 changes: 2 additions & 1 deletion requirements/base.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -158,6 +158,7 @@ greenlet==3.1.1
# via
# apache-superset (pyproject.toml)
# shillelagh
# sqlalchemy
gunicorn==23.0.0
# via apache-superset (pyproject.toml)
h11==0.16.0
Expand Down Expand Up @@ -264,7 +265,7 @@ packaging==25.0
# limits
# marshmallow
# shillelagh
pandas==2.0.3
pandas==2.1.4
# via apache-superset (pyproject.toml)
paramiko==3.5.1
# via
Expand Down
3 changes: 2 additions & 1 deletion requirements/development.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -327,6 +327,7 @@ greenlet==3.1.1
# apache-superset
# gevent
# shillelagh
# sqlalchemy
grpcio==1.71.0
# via
# apache-superset
Expand Down Expand Up @@ -532,7 +533,7 @@ packaging==25.0
# pytest
# shillelagh
# sqlalchemy-bigquery
pandas==2.0.3
pandas==2.1.4
# via
# -c requirements/base-constraint.txt
# apache-superset
Expand Down
2 changes: 1 addition & 1 deletion superset/commands/database/uploaders/excel_reader.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ def file_to_dataframe(self, file: FileStorage) -> pd.DataFrame:
"na_values": self._options.get("null_values")
if self._options.get("null_values") # None if an empty list
else None,
"parse_dates": self._options.get("column_dates"),
"parse_dates": self._options.get("column_dates") or False,
Copy link
Member

@sadpandajoe sadpandajoe Sep 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this backwards compatible? I'm guessing this was done for the upper end of pandas updates

Copy link
Contributor Author

@rad-pat rad-pat Sep 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was done because sending None was doing something incorrectly in later Pandas. I can't remember exactly what now, but I traced it through to a change in Pandas. False is the default param value and has been for as far back as I can see in Pandas docs, so probably should be what is sent in the case no "column_dates" are provided. Hmmm, maybe should be self._options.get("column_dates") if self._options.get("column_dates") is not None else False What do you think?

"skiprows": self._options.get("skip_rows", 0),
"sheet_name": self._options.get("sheet_name", 0),
"nrows": self._options.get("rows_to_read"),
Expand Down
2 changes: 2 additions & 0 deletions tests/conftest.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,8 @@ def data_loader(
pandas_loader_configuration: PandasLoaderConfigurations,
table_to_df_convertor: TableToDfConvertor,
) -> DataLoader:
if example_db_engine.dialect.name == PRESTO:
example_db_engine.dialect.get_view_names = Mock(return_value=[])
return PandasDataLoader(
example_db_engine, pandas_loader_configuration, table_to_df_convertor
)
Expand Down
Loading