Return scope metadata on ScopesRequest

[gfszr]

Scopes returned by the ScopesRequest are returned without source and startLine, which are optional values according to Scope's schema:

        "source": {
            "description": "Optional source for this scope.",
            "type": "Source"
        },
        "line": {
            "type": "integer",
            "description": "Optional start line of the range covered by this scope."
        },
        "column": {
            "type": "integer",
            "description": "Optional start column of the range covered by this scope."
        },
        "endLine": {
            "type": "integer",
            "description": "Optional end line of the range covered by this scope."
        },
        "endColumn": {
            "type": "integer",
            "description": "Optional end column of the range covered by this scope."
        }

Currently, debugpy doesn't fill any of these values, which are in practice extremely useful when the UI wants to present scope variables (Such as locals) in a pretty way over the definition and source code.

[gfszr]

@fabioz How tough is it implementing this? As far as I'm aware, internally pydev is aware of a start and an end of a frame (And even knowing the location of a variable's definition).

[fabioz]

Well, the scope request only gives locals and globals hardcoded right now without even going further to query the frame.

i.e.: https://github.com/microsoft/debugpy/blob/v1.6.3/src/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_process_net_command_json.py#L984

It shouldn't be hard to provide the local variables scope source/lines (there are utilities for getting the frame and then getting the code and querying for lines/source should be reasonably straightforward, although some care needs to be done to translate paths to the client according to path mappings).

Something as:

thread_id = py_db.suspended_frames_manager.get_thread_id_for_variable_reference(frame_id)

if thread_id is not None:
    frame = py_db.find_frame(thread_id, frame_id)
    if frame is not None:
        frame.f_code  # Query frame.f_code for lines (using dis.findlinestarts?)
        filename = pydevd_file_utils.map_file_to_client(frame.f_code.co_filename)

The debugger doesn't create a separate scope for arguments (so, if you're thinking about the scope for the arguments that'd be harder as that scope would need to be created first, but for locals it should be straightforward).

@gfszr Are you interested in providing a PR?

Note that as this is all in pydevd, the best place would probably be creating a PR in https://github.com/fabioz/PyDev.Debugger (and this would need some tests in the tests_python.test_debugger_json module too) -- after it's merged with its tests there I can take care of updating debugpy accordingly.

[gfszr]

@fabioz Thanks for the quick reply! 😄 For the following days I don't have a lot of time to take care of it, but I have traversed the code a was bit and was curious as to how hard it can be to implement it. I'm also interested to see what turns out of microsoft/debug-adapter-protocol#343 which is even more specific, and if it's even easier to implement rather than scopes, then it'd provide an even more accurate result.

[fabioz]

Regarding that other request, I'm not sure... it seems clients usually can usually know about that and should be able to make that mapping without additional debugger support.

I believe that's even possible already. See: https://stackoverflow.com/questions/62337163/visual-studio-code-how-to-display-variable-values-inline-during-debugging

[gfszr]

It is not that easy for a client to do so without some deep understanding of the code (Thus necessarily requiring an LSP at least, or the client to be much more aware), whilst my motivation providing that information to debug clients which aren't necessarily VSCode. The current support in VSCode seems a bit odd behaving as well:

In that topic, do you happen to know how PyCharm, which also uses pydev, deduce that information if not via pydev?

[fabioz]

In that topic, do you happen to know how PyCharm, which also uses pydev, deduce that information if not via pydev?

They use the client knowledge to get info on the scope (and then just query the debugger to get the value).

[gfszr]

@fabioz Thanks!
In any way, I do believe that a debug adapter should be able to supply that information, same as it is able to supply the scopes' source range, but we've gone far from this feature request's original purpose, so I'll leave it at that 👍