Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DataFrame Editor background coloring does not support pandas.Int*Dtype dtypes #21222

Closed
8 of 10 tasks
lamm45 opened this issue Aug 6, 2023 · 1 comment · Fixed by #21295
Closed
8 of 10 tasks

DataFrame Editor background coloring does not support pandas.Int*Dtype dtypes #21222

lamm45 opened this issue Aug 6, 2023 · 1 comment · Fixed by #21295

Comments

@lamm45
Copy link

lamm45 commented Aug 6, 2023

Issue Report Checklist

  • Searched the issues page for similar reports
  • Read the relevant sections of the Spyder Troubleshooting Guide and followed its advice
  • Reproduced the issue after updating with conda update spyder (or pip, if not using Anaconda)
  • Could not reproduce inside jupyter qtconsole (if console-related)
  • Tried basic troubleshooting (if a bug/error)
    • Restarted Spyder
    • Reset preferences with spyder --reset
    • Reinstalled the latest version of Anaconda
    • Tried the other applicable steps from the Troubleshooting Guide
  • Completed the Problem Description, Steps to Reproduce and Version sections below

Problem Description

Introduction

Pandas nowadays has nullable types for integers etc. so that missing values can be represented without conversion to floats. An example is pandas.Int64Dtype described here: https://pandas.pydata.org/docs/user_guide/integer_na.html Although the concept of nullable types is relatively new and mostly non-default, it can be very useful. Moreover, some methods of pandas.StringDtype return nullable integers, and the documentation specifically recommends using pandas.StringDtype for text data.

Problem

The DataFrame Editor in Spyder appears to lack background coloring support for pandas nullable integer data types.

What steps reproduce the problem?

In the IPython console, type the following:

import pandas as pd

vals = [1, 2, 3, 4, 5]
vals_na = [1, 2, 3, None, 5]

df = pd.DataFrame({
    'old': pd.Series(vals),
    'old_na': pd.Series(vals_na),
    'new': pd.Series(vals, dtype='Int64'),
    'new_na': pd.Series(vals_na, dtype='Int64')
})

This produces a DataFrame with the following dtypes (as printed by df.dtypes; here Int64 means pandas.Int64Dtype):

old         int64
old_na    float64
new         Int64
new_na      Int64

Next, double-click df in the Variable Explorer.

What is the expected output? What do you see instead?

In the DataFrame Editor, the following is shown. In the first two columns, the data which uses traditional dtypes is correctly colored, whereas in the last two columns, the data which uses the new nullable integer type is not colored.
Screenshot_20230806_161952

Versions

  • Spyder version: 5.4.4 (conda)
  • Python version: 3.11.4 64-bit
  • Qt version: 5.15.8
  • PyQt5 version: 5.15.9
  • Operating System: Linux-5.4.0-153-generic-x86_64-with-glibc2.31

Dependencies

# Mandatory:
atomicwrites >=1.2.0                                                        :  1.4.1 (OK)
chardet >=2.0.0                                                             :  5.1.0 (OK)
cloudpickle >=0.5.0                                                         :  2.2.1 (OK)
cookiecutter >=1.6.0                                                        :  2.3.0 (OK)
diff_match_patch >=20181111                                                 :  20230430 (OK)
intervaltree >=3.0.2                                                        :  3.1.0 (OK)
IPython >=7.31.1,<9.0.0,!=8.8.0,!=8.9.0,!=8.10.0,!=8.11.0,!=8.12.0,!=8.12.1 :  8.14.0 (OK)
jedi >=0.17.2,<0.19.0                                                       :  0.18.2 (OK)
jellyfish >=0.7                                                             :  1.0.0 (OK)
jsonschema >=3.2.0                                                          :  4.18.6 (OK)
keyring >=17.0.0                                                            :  24.2.0 (OK)
nbconvert >=4.0                                                             :  7.7.3 (OK)
numpydoc >=0.6.0                                                            :  1.5.0 (OK)
parso >=0.7.0,<0.9.0                                                        :  0.8.3 (OK)
pexpect >=4.4.0                                                             :  4.8.0 (OK)
pickleshare >=0.4                                                           :  0.7.5 (OK)
psutil >=5.3                                                                :  5.9.5 (OK)
pygments >=2.0                                                              :  2.15.1 (OK)
pylint >=2.5.0,<3.0                                                         :  2.17.5 (OK)
pylint_venv >=3.0.2                                                         :  3.0.2 (OK)
pyls_spyder >=0.4.0                                                         :  0.4.0 (OK)
pylsp >=1.7.4,<1.8.0                                                        :  1.7.4 (OK)
pylsp_black >=1.2.0,<3.0.0                                                  :  1.3.0 (OK)
qdarkstyle >=3.0.2,<3.2.0                                                   :  3.1 (OK)
qstylizer >=0.2.2                                                           :  0.2.2 (OK)
qtawesome >=1.2.1                                                           :  1.2.3 (OK)
qtconsole >=5.4.2,<5.5.0                                                    :  5.4.3 (OK)
qtpy >=2.1.0                                                                :  2.3.1 (OK)
rtree >=0.9.7                                                               :  1.0.1 (OK)
setuptools >=49.6.0                                                         :  68.0.0 (OK)
sphinx >=0.6.6                                                              :  7.1.2 (OK)
spyder_kernels >=2.4.4,<2.5.0                                               :  2.4.4 (OK)
textdistance >=4.2.0                                                        :  4.5.0 (OK)
three_merge >=0.1.1                                                         :  0.1.1 (OK)
watchdog >=0.10.3                                                           :  3.0.0 (OK)
xdg >=0.26                                                                  :  0.28 (OK)
zmq >=22.1.0                                                                :  25.1.0 (OK)

# Optional:
cython >=0.21                                                               :  3.0.0 (OK)
matplotlib >=3.0.0                                                          :  3.7.2 (OK)
numpy >=1.7                                                                 :  1.25.1 (OK)
pandas >=1.1.1                                                              :  2.0.3 (OK)
scipy >=0.17.0                                                              :  1.11.1 (OK)
sympy >=0.7.3                                                               :  1.12 (OK)
@jitseniesen
Copy link
Member

This makes sense to me and should be quite easy to incorporate. The only potential issue is that pandas describes these nullable integer types as experimental, so some details may change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants