Making produce/consume bi-directional

[andrewcooke]

I'd like to suggest a change to produce and consume. I don't understand Julia's tasks in any depth, so this may not be practical, but I have used the code below and it does appear to work as expected.

The change is motivated by python's co-routines, which only have the equivalent of produce and consume (there's no yieldto), yet seem to be sufficiently general for many tasks.

The extension is pretty simple - it extends consume so that it can take an argument that is "returned" to the task being consumed, and it extends produce so that value is returned inside the task. This makes it possible to have a bi-directional "conversation" between producer and consumer. It remains asymmetric (one half calls produce, the other consume), so it's not as elegant as yieldto (which remains, of course) in some respects, but it allows the programmer to get a bit more general without needing new concepts.

Here is the implementation I have been using:

function produce2(v)
    ct = current_task()
    q = ct.consumers
    if isa(q,Condition)
        # make a task waiting for us runnable again
        notify1(q)
    end
    r = yieldto(ct.last, v)
    ct.parent = ct.last  # always exit to last consumer
    r
end
produce2(v...) = produce2(v)

function consume2(P::Task, args...)
    while !(P.runnable || P.done)
        if P.consumers === nothing
            P.consumers = Condition()
        end
        wait(P.consumers)
    end
    ct = current_task()
    prev = ct.last
    ct.runnable = false
    v = yieldto(P, args...)
    ct.last = prev
    ct.runnable = true
    if P.done
        q = P.consumers
        if !is(q, nothing)
            notify(q, P.result)
        end
    end
    v
end

which has only a handful of lines modified from the original (adding r in produce and args in consume).

Note that this does not break existing code (as far as I can tell) because currently no return is expected from produce, and consume doesn't accept additional parameters.

Finally two examples. First, some simple Julia code that allows a cipher to be implemented as a coroutine - it receives a character of plaintext and returns the encrypted value. https://github.com/andrewcooke/Stupid.jl/blob/master/src/Cipher.jl#L112

Second, a blog post on how this is useful in Python when simulating protocols between actors http://acooke.org/cute/UsingCorou0.html

Thanks,
Andrew

[JeffBezanson]

Yes, I'm fine with this, since as you point out it is simply allowing a value where there was nothing before.

[JeffBezanson]

I remember this came up on the mailing list --- thanks for making an issue, otherwise it is easy to forget these things.

[andrewcooke]

oh, thanks! i actually forked a branch to add this, but got lost tying to understand tests...

[stevengj]

Needs documentation; see also #4857