Send InterruptExceptions to "foreground" tasks first?

[ssfrr]

here @stevengj expressed a desire for Ctrl-C to preferentially kill the task currently executing user code. I've also been thinking about this lately so I figured it might warrant a separate issue.

AFAIK there's currently no concept of "foreground" and "background" tasks - would it make sense to be able to tag tasks as one or the other do control how Ctrl-C gets handled? One way I'd think would be for the REPL/IJulia/Juno/etc. to tag their launched tasks as foreground, and when the user presses Ctrl-C, the forground tasks would always be front in line to receive them. Continuing to press Ctrl-C when there are no foreground tasks would start sending InterruptExceptions to non-foreground tasks.

My common use-case e.g. opening an audio stream object, which launches a background task to handle the streaming with the low-level driver. If the user accidentally puts their code into an infinite loop, or something that they want to kill, pressing Ctrl-C will just as likely kill the audio task as stop their code. Similarly with packages like Makie.jl, which launch a background event loop.

[timholy]

Here's a simple example where current behavior is buggy (expanded from https://julialang.slack.com/archives/C67910KEH/p1587641199017700, which will disappear soon):

using FileWatching

# Block via `f` on "condition" `c`. Here we'll use `f=watch_file` and `c=filename`
# because it's easy to send a notification by `touch`ing the file, but
# wait/Condition would presumably show the same behavior.
function block(f, c)
    while true
        println("block")
        try
            f(c)
        catch err
            @show err
            throw(err)
        end
    end
end

filename = "/tmp/dummy.txt"
open(filename, "w") do io
    println(io, "dummy")
end

@async block(watch_file, filename)

And now:

julia> readline()
^CERROR: InterruptException:
Stacktrace:
 [1] poptaskref(::Base.InvasiveLinkedListSynchronized{Task}) at ./task.jl:702
 [2] wait at ./task.jl:709 [inlined]
 [3] wait(::Base.GenericCondition{Base.Threads.SpinLock}) at ./condition.jl:106
 [4] readuntil(::Base.TTY, ::UInt8; keep::Bool) at ./stream.jl:901
 [5] readline(::Base.TTY; keep::Bool) at ./io.jl:454
 [6] readline(::Base.TTY) at ./io.jl:454 (repeats 2 times)
 [7] top-level scope at REPL[2]:1

All's well with our block task, as can be verified by touch /tmp/dummy.txt from a separate shell:

julia> block

Now let's call readline() again, but this time touch the file before we hit Ctrl-C:

julia> readline()
block
^Cerr = InterruptException()
ERROR: InterruptException:
Stacktrace:
 [1] try_yieldto(::typeof(Base.ensure_rescheduled), ::Base.RefValue{Task}) at ./task.jl:654
 [2] wait at ./task.jl:710 [inlined]
 [3] wait(::Base.GenericCondition{Base.Threads.SpinLock}) at ./condition.jl:106
 [4] readuntil(::Base.TTY, ::UInt8; keep::Bool) at ./stream.jl:901
 [5] readline(::Base.TTY; keep::Bool) at ./io.jl:454
 [6] readline(::Base.TTY) at ./io.jl:454 (repeats 2 times)
 [7] top-level scope at REPL[2]:1

We successfully broke out of the readline, but the err = InterruptException() indicates that block saw it too. Sure enough, if we touch /tmp/dummy.txt we get no response, indicating that the block Task has ended.

No problem, you say, instead of throw(err) indiscriminately, let's check first to see if it's an InterruptException:

            if err isa InterruptException
                println("no problem!")
            else
                throw(err)
            end

But then you get a different problem: it's impossible to exit the readline():

julia> readline()
block
^Cerr = InterruptException()
no problem!
block
^Cerr = InterruptException()
no problem!
block
^Cerr = InterruptException()
no problem!
block
^Cerr = InterruptException()
no problem!
block
^Cerr = InterruptException()
no problem!
block

Indeed, to quit my Julia session I need to kill the process.

The "obvious" workaround, modify the exception-handling to

            if err isa InterruptException
                println("no problem!")
                @async block(f, c)
            end
            throw(err)

to my surprise doesn't work (you still can't interrupt the readline). The only workaround I've found is much more laborious:

using FileWatching

const rescheduler = Channel()

# Block via `f` on "condition" `c`. Here we'll use `f=watch_file` and `c=filename`
# because it's easy to send a notification by `touch`ing the file, but
# wait/Condition would presumably show the same behavior.
function block(f, c)
    while true
        println("block")
        try
            f(c)
        catch err
            @show err
            if err isa InterruptException
                put!(rescheduler, (f, c))
                println("no problem!")
            end
            throw(err)
        end
    end
end

function reschedule()
    while true
        f, c = take!(rescheduler)
        block(f, c)
    end
end

filename = "/tmp/dummy.txt"
open(filename, "w") do io
    println(io, "dummy")
end

@async block(watch_file, filename)
@async reschedule()

This variant seems to work as desired, but it's obviously much more of a pain than it should be.

This is the root of timholy/Revise.jl#459.

[timholy]

Actually, even this workaround only works once (or at least, I can find combinations of actions that cause the reschedule Task to quit). disable_sigint doesn't help either, because it blocks the interrupt of readline when it works; if you're persistent enough the timeout overcomes this, but it interrupt block.

[timholy]

For interactive sessions, this seems to be a more effective workaround:

using FileWatching

# Block via `f` on "condition" `c`. Here we'll use `f=watch_file` and `c=filename`
# because it's easy to send a notification by `touch`ing the file, but
# wait/Condition would presumably show the same behavior.
function block(f, c)
    while true
        println("block")
        try
            f(c)
        catch e
            @show e
            if isa(e, InterruptException) &&
                    isdefined(Base, :active_repl_backend) &&
                    Base.active_repl_backend.backend_task.state === :runnable &&
                    isempty(Base.Workqueue) &&
                    Base.active_repl_backend.in_eval
                println("let's handle this")
                @async Base.throwto(Base.active_repl_backend.backend_task, e)
                println("handled")
            else
                println("unsatisfied")
                throw(e)
            end
        end
        println("looping")
    end
end

filename = "/tmp/dummy.txt"
open(filename, "w") do io
    println(io, "dummy")
end

@async block(watch_file, filename)

# readline()

I don't understand why I need the @async in the throwto call.

[vtjnash]

See also #14032

[ViralBShah]

Is this relevant to #35524?

[timholy]

This isn't specific to multithreading, but to the extent that #35524 isn't either, yes, they are probably related. Ctrl-C seems fine for "simple" single-threaded code but the problems start to arise once you have multiple Tasks ("green threads").

[tkf]

I think implementing structured concurrency #33248 in Julia would be a nice principled way to solve this. It is one of the success stories of structured concurrency: Control-C handling in Python and Trio — njs blog