[ZEPPELIN-2645] Adding way to register RemoteInterpreterServer's port into InterpreterProcess

[jongyoul]

What is this PR for?

Changing the way to register interpreter to server. interpreter tells server its port of thrift server, and server connect interpreter through this information

What type of PR is it?

[Feature]

Todos

• - And netty session for transferring data

What is the Jira issue?

• https://issues.apache.org/jira/browse/ZEPPELIN-2645

How should this be tested?

1. Set SPARK_HOME and HADOOP_CONF_DIR
2. Set 'yarn-cluster' to spark's master
3. Run a script

Screenshots (if appropriate)

Questions:

• Does the licenses files need update? No
• Is there breaking changes for older versions? No
• Does this needs documentation? No

[zjffdu]

@jongyoul I still don't understand why we can only set up only one server-client via netty. Maybe I miss something here.
e.g. We set up netty server in zeppelin-server and netty client in interpreter process so that interpreter process can send request or push data to zeppelin server. But how does zeppelin server send request to interpreter process ? It seems we still need to set up another pair of netty server/client for that communication channel.

[jongyoul]

@zjffdu Did you see the link I sent you? In that example, server keeps the channel when client bind at the first time. then, server can send any data through that channel when it wants to do it. I'll test it by my local laptop with two processes, and let you know

[zjffdu]

Yeah, I see that example. But it seems server can only send message to client only when it receive data from client (SecureChatServerHandler.java). How does server send request to client proactively ?

[jongyoul]

Server broadcasts a message to all clients.

On Tue, 27 Jun 2017 at 14:36 Jeff Zhang ***@***.***> wrote: Yeah, I see that example. But it seems server can only send message to client only when it receive data from client (SecureChatServerHandler.java). How does server send request to client proactively ? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2410 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADcflqQqO72PZMl3340rQPdyG4QYwkHvks5sIJTrgaJpZM4N51E2> .

-- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net

[zjffdu]

Yeah, I mean Server broadcasts a message to all clients only after it receive message from client. It looks like Method channelRead0 is invoked after it get message from client. There's no api to call this method directly.
e.g. How does zeppelin server send message to interpreter process without receiving request from interpreter process.
https://github.com/netty/netty/blob/4.0/example/src/main/java/io/netty/example/securechat/SecureChatServerHandler.java#L58