Improved data science debugging tools

[jamesstidard]

Hi,

I've moved from using PyCharm as my daily driver to VSCode and there is one quite prominent feature that I miss. PyCharm has visualisations for libraries like pandas.

This is extremely useful for debugging these kinda multi-dimensional and often large structures. It would be great to see some kind of tool like this in VSCode.

Additionally, library's like numpy, pandas, and xarray, provide pretty multi-line repl and str representations. However, these are also not very usable in VSCode at the moment.

For example when in debug mode and using the Debug Console, newlines are printed as a literal "\n". This makes the repr output (that might have been a nice stop-gap solution to the feature suggested above) also not a particularly great alternative.

Debug Console Print

IPython

Would also be happy to hear about maybe any plugins that could help address this or how other's use VSCode Debugger + Python Data Science libs.

Thanks.

[rchiodo]

@DonJayamanne this is actually a debugging request. At least the second part. First part is too, but could be handled if we shared our visualizers with the real debugger.

@jamesstidard your first request for viewing data frames isn't currently available in the debugger, but will soon be available in the 'Python Interactive' window.

[jamesstidard]

Hi @rchiodo,

Would the Python Interactive method you're referencing require Jupyter Notebook? I'm aware of it but haven't really used it much as most of my programming is application type rather then a kinda script style usage of these libraries - if that makes sense.

Thanks

[rchiodo]

Yes @jamesstidard it requires jupyter to be installed in your python environment. It does not require running notebooks or doing datascience though. We just use jupyter to run a console that we interact with.

And to be clear, the first feature you asked for isn't even ready in the 'Python Interactive' window yet. It's currently being implemented. About halfway done. You can try it out with this setting here:

[rchiodo]

This is the current state:

You can see variables active in the console, but you can't yet go into more detail like you wanted to do for the dataframe you have in your PyCharm screenshot yet. That should be available next week or so.

Oh and I forgot to mention you need our insider's build to try it out. Download the VSIX and install it from the ... menu in the extensions tab.

[jamesstidard]

@rchiodo Fantastic, this looks promising. Thanks for taking the time to reply, screenshots and all. 🥇 I'll check it out.

[jamesstidard]

@rchiodo I've played around a little with the interactive window. Is there plans (I don't know if this is possible) to attach it to the currently running python debug session so I can see those variables?

Maybe this is the intended use and it's meant to do that. I could have just got a beta build that had some bugs.

In your screenshot you seem to be able to see myvar, but I can't see if that was created in that python interactive session or was pulled from a active debug session.

[rchiodo]

@jamesstidard The plan is to (in priority order)

• Allow debugging 'cells' or an entire file from the interactive window.
• Support these same data viewers during normal debugging

So sort of? There's no attach your current debug session and then have it move over to this new window. You start the session in this new window, or you use the same tools in the debugger.

[jamesstidard]

Ah yeah, would be no reason to attach it to an existing window if the debugger had its own. OK @rchiodo, that sounds great and like it'll make my development workflow a lot nicer. Thanks for taking the time to answer my questions - really appreciate it.

Just to stay in the loop, are there any github issues/tickets I could watch to track the progress of this? I had a but off a search but came up empty. Maybe just tracked internally at the moment?

Anyway, thanks again.

[rchiodo]

Sure no problem. Your feedback helps us a ton, so we love getting it. :)

Here's the internal item that is currently working on implementing debugging in the Python Interactive window:
microsoft/vscode-python#3671

It's kinda cryptic, but it's essentially talking about ideas on how we go about implementing the debugging support when talking to Jupyter.

[rchiodo]

The second item, these same windows in the base debugger, I don't believe there is one. This issue we're chatting on would probably work.

[mcdevitts]

@rchiodo I'm absolutely loving the variable viewer (especially the ndarray view). Are you guys adding a default colormap for the DataViewer (example: https://docs.spyder-ide.org/_images/variable_explorer_array_2D_resize.png)?

[rchiodo]

@mcdevitts I don't think we have plans for the color view yet. Feel free to suggest it.

We're looking into adding extra data like:

• mean
• median
  -variance
  -std deviation

[loftusa]

@rchiodo I've noticed that the default colors for text are really hard to see when you're using a dark theme. Are there any plans to change text colors based on theme?

[rchiodo]

@loftusa they should be already. Can you post a screenshot of it not working?

[rchiodo]

Oh and what version of the extension do you have? The latest should be picking up themes for the dataviewer and your code.

[mcdevitts]

@rchiodo Should I just create another "issue" with this suggestion?

[rchiodo]

You mean the color map suggestion? Yes please do.

[loftusa]

@rchiodo sure. Black on gray in to_latex(), the blue is really hard to see with the dark background, etc.

[loftusa]

And I have version 2019.4.2

[rchiodo]

That's a strange version. The shipped version should be 2019.4.12954. The insiders is 2019.5.13046-dev (at least as of me writing this).

What's the theme you're using?

[loftusa]

Actually, "About Visual Studio Code" in Code>About Visual Studio Code (I'm on a mac) gives Version 1.33.1 (1.33.1), which is also not 2019.4.12954.

I'm using One Dark Pro.

[rchiodo]

Sorry that's the VS code version. Our extension is separate from that. Its version is shown in the extensions tab:

Here's what I get with One Dark Pro:

[loftusa]

Oh, woops.

2019.4.12954

[loftusa]

Hmm. Here's what I get:

[rchiodo]

If you try one of the default themes, do the colors come out okay? Might be a mac specific custom theme problem.

[loftusa]

Nope, even worse, using Dark (Visual Studio):

[rchiodo]

Hmm. Okay I just tried on a Mac too and it works fine.

You said you're VS Code version is 1.33.1? Can you try updating that too? I don't think they changed themes in the last 2 versions (I'm on 1.33.3), but not sure.

[rchiodo]

No sorry, take that back. The mac is 1.33.1.

[rchiodo]

So something is crashing in our theme parsing. You're getting the 'default light theme' colors. Can you go to Help | Developer Tools, click on Console, right click, save as, and upload the file here?

[loftusa]

Yep:
-1556905426260.log

[rchiodo]

Very weird. It thinks your theme id is this:

console.ts:134 [Extension Host] Python Extension: Searching for token colors ...
console.ts:134 [Extension Host] Python Extension: Attempting search for colors ...
console.ts:134 [Extension Host] Python Extension: Loading colors from /Applications/Visual Studio Code.app/Contents/Resources/app/extensions/theme-defaults/themes/light_plus.json ...
console.ts:134 [Extension Host] Python Extension: Using colors to generate CSS ...

Can you try renaming your .vscode/settings.json file? Something weird is going on. Do you have 'ignoreVscodeTheme' set? Perhaps that's not working.

[loftusa]

Oh. Yep, definitely had 'ignoreVscodeTheme' set. That'd be it.

[rchiodo]

Yeah that must be it. That's been fixed in our insider's build.

I can repro your problem with our released build.

[loftusa]

Awesome, nice and efficient! Thanks for the help :)

[rchiodo]

No problem :)

Here's the bug for the ignoreVscodeTheme:
https://github.com/microsoft/vscode-python/issues/5480

[AeneasZhu]

How long will the python interactive debugger be available the extension? Will it be ok on June update?

[rchiodo]

Sorry but not sure. It's still under progress. You can track this issue if you want to know when it finishes:
https://github.com/microsoft/vscode-python/issues/3876

Please feel free to upvote issue 3876.

[AeneasZhu]

Sorry but not sure. It's still under progress. You can track this issue if you want to know when it finishes:
#1283

Please feel free to upvote issue 3876.

Ok, it seems a long time to wait. Thank you for all you guys have done. You made me enjoy python programming.

[YoniChechik]

@rchiodo - love your detailed explanations! Just wanted to say that I'm also one of the people that are waiting for the debug break-point option in interactive mode, it will boost the data-science abilities in python-vscode and for sure bring more people to use it!

[stevenlis]

@rchiodo I think we use the same theme One Dark Pro and after the 2019.8.29288 update, the input window no longer matches with the theme. Is this intended? or maybe I missed something in the setting?

[rchiodo]

This is intended. We've done usability studies around using the interactive window and people are missing the input window. The background was changed to be the peek color in the hopes of making it stand out more.

There's a setting to turn it off:

[stevenlis]

@rchiodo It really grows on me but it is still at the bottom of the windows. I remember you mentioned in an issue that it bothers you as well. I think that might be why people are not using it. Fairly speaking, I don't know where it should go.

[loftusa]

@rchiodo I actually really love that new feature. Makes the interactive window feel a lot nicer. +1

[DavidKutu]

Dupe of #1283