Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Code completion for Pandas is not working #21660

Open
10 tasks done
PhilipYip1988 opened this issue Dec 30, 2023 · 10 comments
Open
10 tasks done

Code completion for Pandas is not working #21660

PhilipYip1988 opened this issue Dec 30, 2023 · 10 comments

Comments

@PhilipYip1988
Copy link

Issue Report Checklist

  • Searched the issues page for similar reports
  • Read the relevant sections of the Spyder Troubleshooting Guide and followed its advice
  • Reproduced the issue after updating with conda update spyder (or pip, if not using Anaconda)
  • Could not reproduce inside jupyter qtconsole (if console-related)
  • Tried basic troubleshooting (if a bug/error)
    • Restarted Spyder
    • Reset preferences with spyder --reset
    • Reinstalled the latest version of Anaconda
    • Tried the other applicable steps from the Troubleshooting Guide
  • Completed the Problem Description, Steps to Reproduce and Version sections below

Problem Description

pandas identifiers for Series and DataFrame don't display properly in the script editor. docstring for a class in a standard module does not display when module and class name are the same e.g. datetime.datetime or time.time

What steps reproduce the problem?

  1. Create Script file with a pandas dataframe. For example:
import numpy as np
import pandas as pd
x = np.array([0, 1, 2, 3, 4, 5])
y = 2 * x
df = pd.DataFrame({'x': x, 'y': y})
import datetime
import time
  1. Then examine identifiers or docstring in the script editor in response to:

a. np. # display correctly
b. x. # display correctly
c. pd. # display correctly
d. df. # display correctly Spyder 5, don't display Spyder 6
e. pd.Series. # display correctly Spyder 5, don't display Spyder 6
f. pd.DataFrame. # display correctly Spyder 5, don't display Spyder 6
g. df.x. # Series accessed from DataFrame as an attribute, don't display Spyder 5 or 6
h. df['x']. # Series accessed from DataFrame as an index, don't display Spyder 5 or 6
i. datetime. # Identifiers display correctly Spyder 5 or 6
j. datetime.timedelta # Identifiers display correctly Spyder 5 or 6
k. datetime.timedelta() # Docstring displays correctly Spyder 5 or 6
l. datetime.datetime. # identifiers display correctly Spyder 5 or 6
m. datetime.datetime() # Docstring does not display Spyder 5 or 6
n. time. # identifiers display correctly Spyder 5 or 6
o. time.sleep() # Docstring displays correctly Spyder 5 or 6
p. time.time() # Docstring does not display Spyder 5 or 6
q. time.time_ns() # Docstring does not display Spyder 5 or 6

What is the expected output? What do you see instead?

d, e and f should work in Spyder 6 like Spyder 5.

g and h should display identifiers of e.

m, p and q should display docstring.

Paste Traceback/Error Below (if applicable)

PASTE TRACEBACK HERE

Versions

Spyder 5

  • Spyder version: 5.4.3 (conda)
  • Python version: 3.11.5 64-bit
  • Qt version: 5.15.2
  • PyQt5 version: 5.15.10
  • Operating System: Windows 10

Spyder 6

  • Spyder version: 6.0.0a3 (conda)
  • Python version: 3.11.7 64-bit
  • Qt version: 5.15.8
  • PyQt5 version: 5.15.9
  • Operating System: Windows-10-10.0.22631-SP0

Dependencies

Spyder 5

# Mandatory:
atomicwrites >=1.2.0                             :  1.4.0 (OK)
chardet >=2.0.0                                  :  4.0.0 (OK)
cloudpickle >=0.5.0                              :  2.2.1 (OK)
cookiecutter >=1.6.0                             :  2.5.0 (OK)
diff_match_patch >=20181111                      :  20200713 (OK)
intervaltree >=3.0.2                             :  3.1.0 (OK)
IPython >=7.31.1,<9.0.0,!=8.8.0,!=8.9.0,!=8.10.0 :  8.15.0 (OK)
jedi >=0.17.2,<0.19.0                            :  0.18.1 (OK)
jellyfish >=0.7                                  :  1.0.1 (OK)
jsonschema >=3.2.0                               :  4.19.2 (OK)
keyring >=17.0.0                                 :  23.13.1 (OK)
nbconvert >=4.0                                  :  7.10.0 (OK)
numpydoc >=0.6.0                                 :  1.5.0 (OK)
paramiko >=2.4.0                                 :  2.8.1 (OK)
parso >=0.7.0,<0.9.0                             :  0.8.3 (OK)
pexpect >=4.4.0                                  :  4.8.0 (OK)
pickleshare >=0.4                                :  0.7.5 (OK)
psutil >=5.3                                     :  5.9.0 (OK)
pygments >=2.0                                   :  2.15.1 (OK)
pylint >=2.5.0,<3.0                              :  2.16.2 (OK)
pylint_venv >=2.1.1                              :  2.3.0 (OK)
pyls_spyder >=0.4.0                              :  0.4.0 (OK)
pylsp >=1.7.2,<1.8.0                             :  1.7.2 (OK)
pylsp_black >=1.2.0                              :  1.2.1 (OK)
qdarkstyle >=3.0.2,<3.2.0                        :  3.0.2 (OK)
qstylizer >=0.2.2                                :  0.2.2 (OK)
qtawesome >=1.2.1                                :  1.2.2 (OK)
qtconsole >=5.4.2,<5.5.0                         :  5.4.2 (OK)
qtpy >=2.1.0                                     :  2.4.1 (OK)
rtree >=0.9.7                                    :  1.0.1 (OK)
setuptools >=49.6.0                              :  68.0.0 (OK)
sphinx >=0.6.6                                   :  5.0.2 (OK)
spyder_kernels >=2.4.3,<2.5.0                    :  2.4.4 (OK)
textdistance >=4.2.0                             :  4.2.1 (OK)
three_merge >=0.1.1                              :  0.1.1 (OK)
watchdog >=0.10.3                                :  2.1.6 (OK)
zmq >=22.1.0                                     :  25.1.0 (OK)

# Optional:
cython >=0.21                                    :  None (NOK)
matplotlib >=3.0.0                               :  3.8.0 (OK)
numpy >=1.7                                      :  1.26.2 (OK)
pandas >=1.1.1                                   :  2.1.1 (OK)
scipy >=0.17.0                                   :  1.11.4 (OK)
sympy >=0.7.3                                    :  1.12 (OK)

Spyder 6

# Mandatory:
atomicwrites >=1.2.0              :  1.4.1 (OK)
chardet >=2.0.0                   :  5.2.0 (OK)
cloudpickle >=0.5.0               :  3.0.0 (OK)
cookiecutter >=1.6.0              :  2.5.0 (OK)
diff_match_patch >=20181111       :  20230430 (OK)
intervaltree >=3.0.2              :  3.1.0 (OK)
IPython >=8.13.0,<9.0.0,!=8.17.1  :  8.18.1 (OK)
jedi >=0.17.2,<0.20.0             :  0.19.1 (OK)
jellyfish >=0.7                   :  1.0.3 (OK)
jsonschema >=3.2.0                :  4.20.0 (OK)
keyring >=17.0.0                  :  24.3.0 (OK)
nbconvert >=4.0                   :  7.13.1 (OK)
numpydoc >=0.6.0                  :  1.6.0 (OK)
paramiko >=2.4.0                  :  3.4.0 (OK)
parso >=0.7.0,<0.9.0              :  0.8.3 (OK)
pexpect >=4.4.0                   :  4.8.0 (OK)
pickleshare >=0.4                 :  0.7.5 (OK)
psutil >=5.3                      :  5.9.7 (OK)
pygments >=2.0                    :  2.17.2 (OK)
pylint >=2.5.0,<3.1               :  2.17.7 (OK)
pylint_venv >=3.0.2               :  3.0.3 (OK)
pyls_spyder >=0.4.0               :  0.4.0 (OK)
pylsp >=1.9.0,<1.10.0             :  1.9.0 (OK)
pylsp_black >=1.2.0,<3.0.0        :  2.0.0 (OK)
pyuca >=1.2                       :  1.2 (OK)
qdarkstyle >=3.2.0,<3.3.0         :  3.2.3 (OK)
qstylizer >=0.2.2                 :  0.2.2 (OK)
qtawesome >=1.3.0                 :  1.3.0 (OK)
qtconsole >=5.5.0,<5.6.0          :  5.5.1 (OK)
qtpy >=2.4.0                      :  2.4.1 (OK)
rtree >=0.9.7                     :  1.1.0 (OK)
setuptools >=49.6.0               :  68.2.2 (OK)
sphinx >=0.6.6                    :  7.2.6 (OK)
spyder_kernels >=3.0.0b3,<3.0.0b4 :  3.0.0b3 (OK)
superqt >=0.6.1,<1.0.0            :  0.6.1 (OK)
textdistance >=4.2.0              :  4.5.0 (OK)
three_merge >=0.1.1               :  0.1.1 (OK)
watchdog >=0.10.3                 :  3.0.0 (OK)
zmq >=22.1.0                      :  25.1.2 (OK)

# Optional:
cython >=0.21                     :  3.0.7 (OK)
matplotlib >=3.0.0                :  3.8.2 (OK)
numpy >=1.7                       :  1.26.2 (OK)
pandas >=1.1.1                    :  2.1.4 (OK)
scipy >=0.17.0                    :  1.11.4 (OK)
sympy >=0.7.3                     :  1.12 (OK)

@ccordoba12 ccordoba12 changed the title pandas identifiers in script editor Code completion for Pandas is not working Jan 4, 2024
@ccordoba12
Copy link
Member

Hey @PhilipYip1988, thanks for reporting. I can't reproduce this on Linux.

@dalthviz, can you try it on Windows? Thanks!

@dalthviz
Copy link
Member

Although I'm not sure if I'm testing this correctly (when you say docstrings and identifiers @PhilipYip1988 you mean like the completion options that are retrieved and also that for the highlighted completion option its documentation is shown, right?). Checking locally I got the same behavior regardless of testing with Spyder 6 from conda and Spyder 5.5.0 for all the cases. However, indeed there are cases when Spyder does not provide any docstring/completion from the Editor like the ones mentioned over the OP, maybe some regression regarding completion happened between Spyder 5.4.3 and 5.5.0?

@PhilipYip1988
Copy link
Author

Here are some screenshots, no identifiers show for pd:

pandas identifiers

Like they do for numpy:

numpy identifiers

Also for the docstring, datetime.time shows:

datetime time docstring

Also for the docstring, datetime.datetime does not show:

datetime datetime docstring

@xieyiqun884886
Copy link

I would like to sincerely ask whether there is a solution for this.

@ccordoba12
Copy link
Member

We don't know why this is happening, sorry.

@ccordoba12 ccordoba12 modified the milestones: v6.0alpha5, v6.0beta1 Mar 12, 2024
@PhilipYip1988
Copy link
Author

PhilipYip1988 commented May 9, 2024

Retested with Spyder 6.0.0a5.

datetime

  • datetime. # works as expected
![1](https://github.com/spyder-ide/spyder/assets/56493395/f9fcee1c-9420-4e46-9cd4-46f6d365b924)
  • datetime.timedelta # works as expected
![2](https://github.com/spyder-ide/spyder/assets/56493395/34396e9f-d6b0-4974-85d8-a49b4c7f5dc5)
  • datetime.timedelta() # works as expected
![3](https://github.com/spyder-ide/spyder/assets/56493395/3f083982-8ff0-4d07-b804-7327e1e573d4)
  • datetime.datetime. # works as expected
![4](https://github.com/spyder-ide/spyder/assets/56493395/c368412a-71cf-4f7e-9d15-c631ab86eb06)
  • datetime.datetime() # Docstring does not display when input with open parenthesis

5

time

  • time. # works as expected
![6](https://github.com/spyder-ide/spyder/assets/56493395/9f31feb2-b2c2-4b1d-8cda-ab45970eeefe)
  • time.sleep # works as expected
![7](https://github.com/spyder-ide/spyder/assets/56493395/aeb3bf8a-69af-49ad-b789-374b2667824c)
  • time.sleep() # works as expected
![8](https://github.com/spyder-ide/spyder/assets/56493395/df0f7a6e-1b05-4c4d-9c2b-b66915ad5f4e)
  • time.time # works as expected
![9](https://github.com/spyder-ide/spyder/assets/56493395/a0527a62-e328-4f18-a547-2a5b52c82266)
  • time.time() # works as expected
![10](https://github.com/spyder-ide/spyder/assets/56493395/782789fb-47ef-4d10-a0d4-1de68956dc0c)
  • time.time_ns # works as expected
![11](https://github.com/spyder-ide/spyder/assets/56493395/a7b18762-c25d-4325-b4d5-e35d098e9060)
  • time.time_ns() # Docstring does not display when input with open parenthesis

12

numpy

  • np. # works as expected
![13](https://github.com/spyder-ide/spyder/assets/56493395/8c3e84e4-9aa5-4c44-8244-581dde5d1d2e)
  • np.array # works as expected
![14](https://github.com/spyder-ide/spyder/assets/56493395/77350b02-6ea2-4cac-b041-82906d54d084)
  • np.array() # works as expected
![15](https://github.com/spyder-ide/spyder/assets/56493395/5cdc0679-4239-4edc-bc83-a46ed551da69)
  • x. # works as expected
![16](https://github.com/spyder-ide/spyder/assets/56493395/c2eb2adf-1308-4180-b551-9edbe20d548c)
  • x.sum # works as expected
![17](https://github.com/spyder-ide/spyder/assets/56493395/2f85f2fb-7191-4cb8-b1d5-2a52e1e3ea64)
  • x.sum() #works as expected
![18](https://github.com/spyder-ide/spyder/assets/56493395/bc5599f6-1a25-43f5-916a-8beb9bef592a)

pandas

  • pd. # works as expected
![19](https://github.com/spyder-ide/spyder/assets/56493395/be34e028-81ff-4fd2-9e21-51f88367ed86)
  • pd.Series # works as expected
![20](https://github.com/spyder-ide/spyder/assets/56493395/0c4b6049-47bf-4915-a153-74267dcd5c1e)
  • pd.Series() # works as expected
![21](https://github.com/spyder-ide/spyder/assets/56493395/c8cff894-88bc-4a7f-b877-151b0bdb39e4)
  • pd.Series. # no Series identifiers display

22

  • pd.DataFrame # works as expected
![23](https://github.com/spyder-ide/spyder/assets/56493395/6fe5e4d7-63df-4683-a354-de450610a0ce)
  • pd.DataFrame() # works as expected
![24](https://github.com/spyder-ide/spyder/assets/56493395/9765fdcf-ac2a-4422-bfca-6479bb1d8190)
  • pd.DataFrame. # no DataFrame identifiers display

25

  • df. # no DataFrame identifiers display

26

  • df.x. # no Series identifiers display (attribute access)

27

  • df['x']. # no Series identifiers display (key access)

28

@ccordoba12
Copy link
Member

ccordoba12 commented May 9, 2024

@PhilipYip1988, thanks for the very detailed update! I collapsed the screenshots of those things that are working and left the ones that are not, so we can focus on solving them.

@ccordoba12 ccordoba12 modified the milestones: v6.0beta1, v6.0beta2 May 10, 2024
@ccordoba12 ccordoba12 modified the milestones: v6.0beta2, v6.0beta3 Jun 17, 2024
@ccordoba12 ccordoba12 removed their assignment Jul 7, 2024
@ccordoba12
Copy link
Member

@dalthviz, please take care of this one.

@ccordoba12 ccordoba12 modified the milestones: v6.0beta3, v6.0rc1 Jul 16, 2024
@dalthviz
Copy link
Member

dalthviz commented Jul 30, 2024

Note: Seems like the majority of cases listed here are not working as expected due to a change over the upper limit of jedi done at 60ae3c6 (so it affects Spyder >= 5.5.0).

When checking things with jedi 0.18.2 and latest master things work except for:

  • Cases like df.x. and df['x']: From the Console seems like this works, which thinking about it, seems like something that we have discussed previously and would imply to somehow connect the completions from the Console to the ones from the Editor (to have such completions you would need to actually execute the code, right @ccordoba12 ?)

  • Cases like time.time_ns(): I think the docstring/signature tooltip is not being displayed when you input parenthesis since those methods/functions don't have arguments in their signature. Basically, the current way Spyder handles this is by showing the tooltip when typing the parenthesis only if the function/method has arguments. Still, if you want to see the tooltip you can hover over the definition and it should appear regardless of the function/method having args/kwargs or not. As an example of the handling of something with args/kwargs vs something without them:
    completion_jedi_0182
    Should we change this behavior @ccordoba12 ?

@ccordoba12
Copy link
Member

Note: Seems like the majority of cases listed here are not working as expected due to a change over the upper limit of jedi

Thanks for digging into this @dalthviz! Let's leave it for 6.0.1 then because it's not so simple to fix (you'll need to run git bisect on the Jedi repo between 0.18.2 and the latest release to find exactly what introduced the problem and report that upstream).

Cases like df.x. and df['x']: From the Console seems like this works, which thinking about it, seems like something that we have discussed previously and would imply to somehow connect the completions from the Console to the ones from the Editor (to have such completions you would need to actually execute the code, right @ccordoba12 ?)

Correct, that's the only way to fix that. But that will work only if you've run your code first.

Should we change this behavior @ccordoba12 ?

Yeah, I think that's a good idea and you can implement it for 6.0.0 because it doesn't sound too hard to fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants