RemoteRefs refer to any type of AbstractChannel

[amitmurthy]

With this PR:

• RemoteRefs refer to an AbstractChannel
• A concrete implementation of AbstractChannel, implements take!, fetch, put!, isready and wait
• New constructor RemoteRef(f::Function, pid=myid()) where f() must return an AbstractChannel, is used to construct a remote channel on worker pid.
• The current RemoteRef constructors create a remote Channel{Any}(1) which mimics the current behavior.

The larger and deeper issue of redoing distributed gc in #12042 can be deferred to 0.5

We don't break any existing code and this is a minimal change that enables us to have remote channels of specified type and length in 0.4 itself. And more importantly unifies RemoteRefs and Channels.

cc @JeffBezanson, @jakebolewski , @ViralBShah, @malmaud

[amitmurthy]

Have added docs and tests. Please refer to the manual changes and added example for a more detailed explanation of this PR.

[amitmurthy]

Feedback? Bikeshedding names?

[malmaud]

How hard would it be in this design to used shared memory for implementing remote references if the workers are all on the same node?

[amitmurthy]

Well, the usual considerations in using shmem for IPC apply.

A ShmemChannel <: AbstractChannel would need to

• map shmem segments on all workers in its constructors
• limit itself to either dealing with bitstypes or support non-bitstypes objects in their serialized form
• use a portable locking/signalling library for co-ordinating between processes

[amitmurthy]

Will merge in a day or two.

[amitmurthy]

Have rebased and removed changes to doc/stdlib/parallel.rst. Will merge once CI is green.

[amitmurthy]

Error in test/profile.jl seems unrelated to this PR - https://gist.github.com/amitmurthy/38e4bac61adc76b7fbdb

restarting job.

[tkelman]

Have rebased and removed changes to doc/stdlib/parallel.rst. Will merge once CI is green.

I think those should be re-added, but now in helpdb

[tkelman]

strange parallel segfault on osx travis a few builds after this was merged https://travis-ci.org/JuliaLang/julia/jobs/73898628, might be unrelated though?

[amitmurthy]

Hmmm, the PR was Julia code only and seg faults usually point to a bug elsewhere - I don't know if this PR triggered some other fault elsewhere.

https://gist.github.com/amitmurthy/38e4bac61adc76b7fbdb above was also an invalid free.