[ZEPPELIN-3610] Cluster Raft module design #3183

xunliu · 2018-09-19T09:14:24Z

What is this PR for?

By using the Raft protocol, multiple Zeppelin-Server groups are built into a Zeppelin cluster, the cluster State Machine is maintained through the Raft protocol, and the services in the cluster are agreed upon. The Zeppelin-Server and Zeppelin-Interperter services and processes are stored in the Cluster MetaData. Metadata information;

What type of PR is it?

[Feature]

Todos

add raft algorithm atomix jar
add cluster state machine
add state machine query command
add state machine delete command
add state machine put command
Isolate the netty JAR package introduced by atomix

What is the Jira issue?

https://issues.apache.org/jira/browse/ZEPPELIN-3610

How should this be tested?

CI pass

Screenshots (if appropriate)

Questions:

Does the licenses files need update? No
Is there breaking changes for older versions? No
Does this needs documentation? Yes

zjffdu · 2018-09-20T01:47:09Z

zeppelin-interpreter/pom.xml

+      <version>${atomix.version}</version>
+      <exclusions>
+        <exclusion>
+          <groupId>io.netty</groupId>


Why not use the netty version of io.atomic and instead use another version of netty ?

If don't modify the dependencies, compiling the zengine module will fail, I won't know how to fix it.

[INFO] Zeppelin: Zengine .................................. FAILURE [ 3.343 s]
[INFO] Zeppelin: Server 0.9.0-SNAPSHOT .................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 05:27 min
[INFO] Finished at: 2018-09-20T10:05:32+08:00
[INFO] ------------------------------------------------------------------------
[WARNING] The requested profile "hadoop-2.7" could not be activated because it does not exist.
[WARNING] The requested profile "yarn" could not be activated because it does not exist.
[WARNING] The requested profile "pyspark" could not be activated because it does not exist.
[WARNING] The requested profile "sparkr" could not be activated because it does not exist.
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-enforcer-plugin:1.3.1:enforce (enforce) on project zeppelin-zengine: org.apache.maven.plugins.enforcer.DependencyConvergence failed with message:
[ERROR] Failed while enforcing releasability the error(s) are [
[ERROR] Dependency convergence error for com.google.guava:guava:22.0 paths to dependency are:
[ERROR] +-org.apache.zeppelin:zeppelin-zengine:0.9.0-SNAPSHOT
[ERROR] +-org.apache.zeppelin:zeppelin-interpreter:0.9.0-SNAPSHOT
[ERROR] +-io.atomix:atomix:3.0.0-rc4
[ERROR] +-io.atomix:atomix-utils:3.0.0-rc4
[ERROR] +-com.google.guava:guava:22.0
[ERROR] and
[ERROR] +-org.apache.zeppelin:zeppelin-zengine:0.9.0-SNAPSHOT
[ERROR] +-com.google.guava:guava:20.0
[ERROR] ,
[ERROR] Dependency convergence error for org.apache.commons:commons-lang3:3.7 paths to dependency are:
[ERROR] +-org.apache.zeppelin:zeppelin-zengine:0.9.0-SNAPSHOT
[ERROR] +-org.apache.zeppelin:zeppelin-interpreter:0.9.0-SNAPSHOT
[ERROR] +-io.atomix:atomix:3.0.0-rc4
[ERROR] +-io.atomix:atomix-utils:3.0.0-rc4
[ERROR] +-org.apache.commons:commons-lang3:3.7
[ERROR] and
[ERROR] +-org.apache.zeppelin:zeppelin-zengine:0.9.0-SNAPSHOT
[ERROR] +-org.apache.commons:commons-lang3:3.4
[ERROR] ,
[ERROR] Dependency convergence error for io.netty:netty-handler:4.1.27.Final paths to dependency are:
[ERROR] +-org.apache.zeppelin:zeppelin-zengine:0.9.0-SNAPSHOT
[ERROR] +-org.apache.zeppelin:zeppelin-interpreter:0.9.0-SNAPSHOT
[ERROR] +-io.atomix:atomix:3.0.0-rc4
[ERROR] +-io.atomix:atomix-cluster:3.0.0-rc4
[ERROR] +-io.netty:netty-handler:4.1.27.Final
[ERROR] and
[ERROR] +-org.apache.zeppelin:zeppelin-zengine:0.9.0-SNAPSHOT
[ERROR] +-org.apache.zeppelin:spark-interpreter:0.9.0-SNAPSHOT
[ERROR] +-org.apache.zeppelin:zeppelin-python:0.9.0-SNAPSHOT
[ERROR] +-io.grpc:grpc-netty:1.4.0
[ERROR] +-io.netty:netty-codec-http2:4.1.11.Final
[ERROR] +-io.netty:netty-handler:4.1.11.Final
[ERROR] ,
[ERROR] Dependency convergence error for io.netty:netty-transport:4.1.27.Final paths to dependency are:
[ERROR] +-org.apache.zeppelin:zeppelin-zengine:0.9.0-SNAPSHOT
[ERROR] +-org.apache.zeppelin:zeppelin-interpreter:0.9.0-SNAPSHOT
[ERROR] +-io.atomix:atomix:3.0.0-rc4
[ERROR] +-io.atomix:atomix-cluster:3.0.0-rc4
[ERROR] +-io.netty:netty-transport:4.1.27.Final
[ERROR] and
[ERROR] +-org.apache.zeppelin:zeppelin-zengine:0.9.0-SNAPSHOT
[ERROR] +-org.apache.zeppelin:zeppelin-interpreter:0.9.0-SNAPSHOT
[ERROR] +-io.atomix:atomix:3.0.0-rc4
[ERROR] +-io.atomix:atomix-cluster:3.0.0-rc4
[ERROR] +-io.netty:netty-codec:4.1.27.Final
[ERROR] +-io.netty:netty-transport:4.1.27.Final
[ERROR] +-org.apache.zeppelin:zeppelin-zengine:0.9.0-SNAPSHOT
[ERROR] +-org.apache.zeppelin:zeppelin-interpreter:0.9.0-SNAPSHOT
[ERROR] +-io.atomix:atomix:3.0.0-rc4
[ERROR] +-io.atomix:atomix-cluster:3.0.0-rc4
[ERROR] +-io.netty:netty-transport-native-epoll:4.1.27.Final
[ERROR] +-io.netty:netty-transport:4.1.27.Final
[ERROR] ]
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn -rf :zeppelin-zengine

zjffdu · 2018-09-20T01:47:58Z

zeppelin-interpreter/src/main/java/org/apache/zeppelin/cluster/BroadcastServiceAdapter.java

+import java.util.function.Consumer;
+
+/**
+ * Broadcast Service Adapter


Can you add more doc to explain this class's responsibility ? e.g. How does it communicate with other components.

Can you add more doc about who broadcast message to whom

The comment is added in line 25, So it didn't show up.
The next few places that need to be modified are also the problem.

I mean to be more specific on which node this service should run. Overall there's 2 kinds of nodes. Zeppelin Server nodes and Interpreter Process node. And regarding the broadcast method, who broadcast message to whom

zjffdu · 2018-09-20T01:49:31Z

zeppelin-interpreter/src/main/java/org/apache/zeppelin/cluster/ClusterManager.java

+ * 3. Cluster monitoring
+ */
+public abstract class ClusterManager {
+  private static Logger logger = LoggerFactory.getLogger(ClusterManager.class);


logger --> LOGGER
This is just code convention zeppelin use. Static fields should use uppercase

zjffdu · 2018-09-20T01:50:14Z

zeppelin-interpreter/src/main/java/org/apache/zeppelin/cluster/ClusterManager.java

+  protected LocalRaftProtocolFactory protocolFactory
+      = new LocalRaftProtocolFactory(protocolSerializer);
+  protected List<MessagingService> messagingServices = new ArrayList<>();
+  protected Collection<MemberId> clusterMemberIds = new ArrayList<MemberId>();


Collection --> List (to make code style consistent)

zjffdu · 2018-09-20T01:51:09Z

zeppelin-interpreter/src/main/java/org/apache/zeppelin/cluster/ClusterManager.java

+        }
+      }
+    } catch (UnknownHostException e) {
+      e.printStackTrace();


use LOGGER.error instead of e.printStackTrace

zjffdu · 2018-09-20T01:51:32Z

zeppelin-interpreter/src/main/java/org/apache/zeppelin/cluster/ClusterManager.java

+      e.printStackTrace();
+    } catch (SocketException e) {
+      e.printStackTrace();
+      return;


return is not necessary for constructor.

zjffdu · 2018-09-20T01:52:25Z

zeppelin-interpreter/src/main/java/org/apache/zeppelin/cluster/ClusterManager.java

+        raftClient.close().get(3, TimeUnit.SECONDS);
+      }
+    } catch (InterruptedException e) {
+      e.printStackTrace();


zjffdu · 2018-09-20T01:52:30Z

zeppelin-interpreter/src/main/java/org/apache/zeppelin/cluster/ClusterManager.java

+    } catch (InterruptedException e) {
+      e.printStackTrace();
+    } catch (ExecutionException e) {
+      e.printStackTrace();


zjffdu · 2018-09-20T01:52:35Z

zeppelin-interpreter/src/main/java/org/apache/zeppelin/cluster/ClusterManager.java

+    } catch (ExecutionException e) {
+      e.printStackTrace();
+    } catch (TimeoutException e) {
+      e.printStackTrace();


zjffdu · 2018-09-20T01:53:27Z

zeppelin-interpreter/src/main/java/org/apache/zeppelin/cluster/ClusterPrimitiveType.java

+import io.atomix.primitive.service.ServiceConfig;
+
+/**
+ * Cluster primitive type


Need more doc

zjffdu · 2018-09-20T01:53:36Z

zeppelin-interpreter/src/main/java/org/apache/zeppelin/cluster/ClusterStateMachine.java

+import java.util.Map;
+
+/**
+ * Cluster State Machine


Need more doc

zjffdu · 2018-09-20T01:53:54Z

...c/main/java/org/apache/zeppelin/cluster/listener/ZeppelinClusterMembershipEventListener.java

+import org.slf4j.LoggerFactory;
+
+/**
+ * Zeppelin ClusterMembershipEventListener


Need more doc

zjffdu · 2018-09-20T01:57:18Z

...-interpreter/src/main/java/org/apache/zeppelin/cluster/protocol/LocalRaftClientProtocol.java

+import java.util.function.Function;
+
+/**
+ * Cluster Raft client protocol.


Need more docs.

zjffdu · 2018-09-26T03:32:01Z

jdbc/src/test/java/org/apache/zeppelin/jdbc/JDBCInterpreterTest.java


+  @Test
+  public void testColumnAliasQuery() throws IOException {
+    Properties properties = new Properties();


Something must be wrong, this is not supposed to be in your commit. It is someone else's commit

I checked the code I submitted, I did not modify this, is it caused by Git's bug?

By using the Raft protocol, multiple Zeppelin-Server groups are built into a Zeppelin cluster, the cluster State Machine is maintained through the Raft protocol, and the services in the cluster are agreed upon. The Zeppelin-Server and Zeppelin-Interperter services and processes are stored in the Cluster MetaData. Metadata information; [Feature] * [x] add raft algorithm atomix jar * [x] add cluster state machine * [x] add state machine query command * [x] add state machine delete command * [x] add state machine put command * [x] Isolate the netty JAR package introduced by atomix * https://issues.apache.org/jira/browse/ZEPPELIN-3610 CI pass * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? Yes

xunliu · 2018-09-26T08:03:08Z

Has passed the CI

zjffdu · 2018-09-26T08:51:41Z

zeppelin-interpreter-parent/pom.xml

+        <!--<configuration>-->
+          <!--<skip>true</skip>-->
+        <!--</configuration>-->
+      <!--</plugin>-->


Could you uncomment this ? It is my mistake in my PR.

I have not modified this pom.xml.

This is due to my PR which you merged

zjffdu · 2018-09-27T02:37:20Z

zeppelin-interpreter/src/main/java/org/apache/zeppelin/cluster/ClusterManager.java

+ * 1. RaftClient as the raft client
+ * 2. Threading to provide retry after cluster metadata submission failure
+ * 3. Cluster monitoring
+ */


Where does this class run ?

ClusterManager.java is the base class for cluster management.
My next PR will submit ClusterManagerClient.java and ClusterManagerServer.java that are extends from ClusterManager.java.

zjffdu · 2018-09-27T23:29:16Z

LGTM, will merge if no more comments

zjffdu reviewed Sep 20, 2018

View reviewed changes

zjffdu reviewed Sep 26, 2018

View reviewed changes

xunliu force-pushed the ZEPPELIN-3610 branch from 335dbfb to f25005f Compare September 26, 2018 04:53

xunliu force-pushed the ZEPPELIN-3610 branch from f25005f to 97f17ac Compare September 26, 2018 04:57

zjffdu and others added 2 commits September 26, 2018 14:31

fix test (#2)

d268ee7

add more doc to explain this class's responsibility.

af728bd

zjffdu reviewed Sep 26, 2018

View reviewed changes

zjffdu reviewed Sep 27, 2018

View reviewed changes

uncomment maven-dependency-plugin.

c899b15

asfgit closed this in f28e963 Sep 28, 2018

[ZEPPELIN-3610] Cluster Raft module design #3183

[ZEPPELIN-3610] Cluster Raft module design #3183

Uh oh!

Conversation

xunliu commented Sep 19, 2018

What is this PR for?

What type of PR is it?

Todos

What is the Jira issue?

How should this be tested?

Screenshots (if appropriate)

Questions:

Uh oh!

Choose a reason for hiding this comment

Uh oh!

xunliu Sep 20, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zjffdu Sep 20, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zjffdu Sep 20, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

xunliu Sep 20, 2018 •

edited

Loading

zjffdu Sep 20, 2018 •

edited

Loading

zjffdu Sep 20, 2018 •

edited

Loading

zjffdu Sep 27, 2018 •

edited

Loading