WIP: share the stack with JuliaInterpreter and add support for breakpoints

[KristofferC]

No description provided.

[timholy]

The alternative, of having JuliaInterpreter use stack everywhere, is also something I can consider. The hard part is that our way of handling Compiled() would presumably have to change.

[KristofferC]

In some situations, not having the top-level frame be "special" (and just have it be the top frame in the stack) makes things a bit easier.

However, I would think a linked list of frames for the stack might be an even easier structure to handle, since the random access of a Vector is seldom useful. Might be too intrusive to change at this point though.

[KristofferC]

For example here, just being able to store original_frame = stack.frame and then restore stack.frame = original_frame would be nice, instead of having to both store the frame and the stack to restore the original state.

[timholy]

However, I would think a linked list of frames for the stack might be an even easier structure to handle, since the random access of a Vector is seldom useful. Might be too intrusive to change at this point though.

It's worth considering; you have a good point with regards to the cleanliness of the API.

If I'm thinking about this clearly, the caller would have to hold the "root," meaning the top-level caller; otherwise if an error gets thrown deeper in the stack, there is no way to propagate the state back to the caller. Would you have to traverse the whole list to do anything? Or would each recursive call pass only the immediate parent? I guess you must be intending a doubly-linked list? Otherwise you have to choose between performance (avoiding the need for routine whole-stack traversal) and the ability to print the whole stack for debugging purposes (e.g., those two commented lines at the top of _step_expr!).

A related question: if we go that route, how do you signal intent for the behavior of recursive calls? Currently we can pass stack or Compiled() as an argument. Now that I think about it, it is a little strange: stack should be holding the parents of frame, but in fact it affects the allocation of the child-frames.

I'm willing to help implement this, but let's discuss a little more first.

[KristofferC]

I guess you must be intending a doubly-linked list?

Yes, I was thinking double linked.

I'm willing to help implement this, but let's discuss a little more first.

Yes, getting this right is important since it will shape the API and the convenience in using the interpreter significantly.

An argument against the linked list is that it gives more power to the frame that it doesn't really need. If you see a function f(::JuliaStackFrame) you know it won't change the stack. However, the fact that almost all functions have a signature like f!(stack, frame) makes this argument slightly weaker.

[timholy]

An argument against the linked list is that it gives more power to the frame that it doesn't really need. If you see a function f(::JuliaStackFrame) you know it won't change the stack. However, the fact that almost all functions have a signature like f!(stack, frame) makes this argument slightly weaker.

If you wanted that clarity, a design like this:

struct Node{D}
    head::Union{Node{D},Nothing}
    tail::Union{Node{D},Nothing}
    data::D
end

would allow you to pass node.data instead of the entire node.

In addition to frame and stack, we have a couple of auxiliary variables to consider:

• pc: pass as an argument or stash in the frame data more often? I suspect @Keno created new frames rather than updating the pc because he wanted the ability to re-run the same frame? (Essentially making frames immutable.)
• istoplevel
• something to indicate the desired form of recursion (interpreted or compiled). This might be more than a Bool or simple flag; the exec! argument of evaluate_call! is essentially a mechanism for detailed control of recursion. We might consider whether we can pass a function argument, recurse, and have it handle all aspects of recursion.