Checking a Promise without blocking?

[Hoverbear]

I haven't quite figured out how to create a "barrier" to wait for multiple requests.

Something like this:

    println!("Issuing request_vote.");
    {
        let mut request = raft_client.request_vote_request();
        let mut builder = request.init();
        builder.set_term(0u64);
        builder.set_candidate_id(0u64);
        builder.set_last_log_index(0u64);
        builder.set_last_log_term(0u64);
        let mut promise = request.send();

        println!("Retrieving request_vote response.");
        let response = promise.wait().unwrap();
        let term = response.get_term();
        let vote_granted = response.get_vote_granted();

        println!("
            Term: {}
            voteGranted: {}
        ", term, vote_granted)
    }

Works fine for a single RPC. However because of wait() I'm not sure how to approach waiting on the results of multiple RPC calls without using extra threads. I don't see a sort of has_responded() type functionality. Do you have a suggestion?

[dwrensha]

I think your best option in such cases is to spawn a new thread and synchronize using channels. The RPC system was designed with green threads in mind. Now that libgreen is gone, I think we may need to go back and revisit some assumptions.

[Hoverbear]

It looks like wait() is using recv on anwer_port... It's possible we can use try_recv() to implement a poll() type function.

[Hoverbear]

I got a bit of a draft implementation here but I think, as you discussed, it might be better to rethink things with syncbox or mio.

[Hoverbear]

@dwrensha Any news on this?

[dwrensha]

I'm okay with adding something like your poll() if it helps you get stuff done. Send me a pull request if you want to go that route. :)

Longer term, we clearly do need to rework our concurrency strategy. This is going to be a rather big project, and a lot of the interfaces will change.

I see at least two promising approaches for getting started on this: we could port the KJ async library to Rust, or we could build off of mio/syncbox. I am personally partial to KJ because I'm familiar with it and it has proven to work well for capnproto-c++. A big difference between KJ and mio is that a KJ event loop is single-threaded, so it needs to worry less about atomic operations and synchronization. The flip side is that achieving actual parallelism with KJ can be a bit cumbersome. Tradeoffs abound.

[dwrensha]

It looks like mio::EventLoop is more or less equivalent to kj::EventPort, so I think we should try to use it in any case. What's still unclear to me is whether we should use KJ-style promises, which are single-threaded, or syncbox-style futures, which might not be.

[Hoverbear]

@carllerche have you looked at KJ-style promises?

[carllerche]

I am not super familiar with KJ-style promises, but they are compared to JS promises.

Syncbox promises (which I plan on breaking out to a dedicated lib), are primarily async, but are able to block waiting for the value. They work with one thread or with many (they are thread safe and can be used to send values across threads).

I'm pretty sure that they would support whatever you need, if not let me know what is missing.

[dwrensha]

Update: I've started playing around with porting KJ to Rust. Working name: GJ. https://github.com/dwrensha/gj

Looks like a big difference between GJ promises and Eventual futures is going to be that GJ does not require a Send bound on data that's passed through the then() method.

[carllerche]

It would be fine to remove the default Send bound on eventual. Just bound Future by Send if T: Send. A lot of work has already gone into Eventual, it would be a shame to not be able to reuse it.

[dwrensha]

@carllerche: I see that Future::map() has a signature like this:

pub fn map<F, U>(self, f: F) -> Future<U, E>
        where F: FnOnce(T) -> U + Send + 'static,
              U: Send + 'static

Are you saying that this could be amended to the following?

pub fn map<F, U>(self, f: F) -> Future<U, E>
        where F: FnOnce(T) -> U + 'static,
              U: 'static

This is important for my use case because I think I'll very often want to share Rc data between the callback and the caller.

[carllerche]

So yeah, after thinking about it for a while, I think you are correct in that this is analogous to Rc vs. Arc. To support this, there would need to be Future and NoSendFuture (obviously not a good name). The problem is that there is no good way to define the Async trait to support both of these versions. I think that supporting both Send and non Send versions would be plausible when (if?) HKT lands.

The flip side is that the these futures won't be able to be sent across threads, which is something that I would have hoped to be able to do w/ this.

Hopefully one day we will get HKT and I can update Eventual for this case.

[dwrensha]

This is fixed in the 0.6 release, which uses GJ: https://dwrensha.github.io/capnproto-rust/2016/01/11/async-rpc.html . You can now collect the results with Promise::all().

[Hoverbear]

Woo! Go Drew!

[dwrensha]

@Hoverbear are you still confusing me with @zarvox? :)

[zarvox]

We did meet at a Rust Meetup once. @Hoverbear has sweet business cards. :)

[Hoverbear]

@dwrensha David! Sigh.... Sorry.