feat(dgraph): making all internal communications with tls configured

[aman-bansal]

Fixes DGRAPH-2505, DGRAPH-2506 and DGRAPH-546. All internal communication can now be TLS encrypted.
Three flags have been exposed tls_internal_port_enabled, tls_cert and tls_key to enable internal communication.

---

This change is 

Docs Preview:

[codelingo]

LGTM. 12 rules errored during the review.

[codelingo]

LGTM. 12 rules errored during the review.

[codelingo]

LGTM. 25 rules errored during the review.

[codelingo]

LGTM. 30 rules errored during the review.

[pawanrawal]

Also, cherry-pick this onto the release branch as this has new flags that we don't want in master.

Reviewed 9 of 59 files at r1, 93 of 105 files at r2.
Reviewable status: all files reviewed, 11 unresolved discussions (waiting on @aman-bansal, @manishrjain, @parasssh, and @vvbalaji-dgraph)

---

conn/pool.go, line 149 at r2 (raw file):

	}

	tlsConfig, err := x.GenerateClientTLSConfig(conf)

Can generate it at startup and then pass it here.nn

---

dgraph/cmd/live/run.go, line 547 at r2 (raw file):

		x.Checkf(tlsErr, "Unable to generate TLS Cert Pool")
	} else {
		helperConfig, err := x.LoadNodeTLSClientHelperConfig(conf)

these 4 lines can go in one func and be reused everywhere

---

dgraph/cmd/zero/raft.go, line 216 at r2 (raw file):

	// Create a connection to this server.
	go conn.GetPools().Connect(member.Addr, opts.tlsClientConfig)

can this be part of the node struct

---

dgraph/cmd/zero/run.go, line 56 at r2 (raw file):

	rebalanceInterval time.Duration

	//tls client config which will be used by zeros to connect internally

// tls...

---

tlstest/mtls_internal/multi_group/mtls_internal_multi_group_test.go, line 36 at r2 (raw file):

		txn := client.NewTxn()
		_, err := txn.Mutate(ctx, &api.Mutation{SetJson: []byte(setJSON)})

send mutation to one alpha and query the other two alphas

[aman-bansal]

Reviewable status: 79 of 103 files reviewed, 11 unresolved discussions (waiting on @manishrjain, @parasssh, @pawanrawal, and @vvbalaji-dgraph)

---

conn/pool.go, line 149 at r2 (raw file):

Previously, pawanrawal (Pawan Rawal) wrote…

Can generate it at startup and then pass it here.nn

Done.

---

dgraph/cmd/bulk/run.go, line 92 at r1 (raw file):

Previously, parasssh wrote…

ISnt this name confusing? This dir contains the bulk loader client's certs, no?

Done.

---

dgraph/cmd/live/run.go, line 547 at r2 (raw file):

Previously, pawanrawal (Pawan Rawal) wrote…

these 4 lines can go in one func and be reused everywhere

Done.

---

dgraph/cmd/zero/raft.go, line 216 at r2 (raw file):

Previously, pawanrawal (Pawan Rawal) wrote…

can this be part of the node struct

Done.

---

dgraph/cmd/zero/run.go, line 125 at r1 (raw file):

Previously, aman-bansal (aman bansal) wrote…

No this won't change. cmux will only control zero HTTP port.

Done.

---

dgraph/cmd/zero/run.go, line 56 at r2 (raw file):

Previously, pawanrawal (Pawan Rawal) wrote…

// tls...

Done.

---

dgraph/cmd/zero/zero.go, line 231 at r1 (raw file):

Previously, parasssh wrote…

remove vertical spacing.

Done.

---

dgraph/cmd/zero/zero.go, line 236 at r1 (raw file):

Previously, aman-bansal (aman bansal) wrote…

Yes this was for my understanding to see which connections are made. I will remove these

Done.

---

x/tls_helper.go, line 63 at r1 (raw file):

Previously, parasssh wrote…

we dont need to differentiate client and server names. They can be the same thing.

Done.

---

x/tls_helper.go, line 256 at r1 (raw file):

Previously, parasssh wrote…

spelling ... tzhe

Done.

---

tlstest/mtls_internal/multi_group/mtls_internal_multi_group_test.go, line 36 at r2 (raw file):

Previously, pawanrawal (Pawan Rawal) wrote…

send mutation to one alpha and query the other two alphas

Done.

[aman-bansal]

Reviewable status: 61 of 111 files reviewed, 13 unresolved discussions (waiting on @manishrjain, @parasssh, @pawanrawal, and @vvbalaji-dgraph)

---

tlstest/mtls_internal/acl/docker-compose.yml, line 48 at r4 (raw file):

--tls_dir /dgraph-tls

---

tlstest/mtls_internal/loader/loader_test.go, line 74 at r4 (raw file):

"--tls_enable_inter_node=true"

tls_internal_port_enabled

[pawanrawal]

with some comments

Reviewed 12 of 105 files at r2, 47 of 51 files at r3, 4 of 4 files at r4.
Reviewable status: all files reviewed, 11 unresolved discussions (waiting on @aman-bansal, @manishrjain, @parasssh, and @vvbalaji-dgraph)

---

dgraph/cmd/alpha/run.go, line 171 at r3 (raw file):

	flag.String("tls_client_auth", "VERIFYIFGIVEN", "Enable TLS client authentication")
	flag.Bool("tls_enable_inter_node", false, "(optional) enable inter node TLS encryption between cluster nodes.")
	flag.String("tls_cert", "", "(optional) The Cert file provided for the nodes to connect with the cluster.")

connect as a client with other nodes in the cluster.

Similarly below

---

dgraph/cmd/alpha/run.go, line 671 at r3 (raw file):

		TLSClientConfig:      tlsConf,
		TLSDir:               Alpha.Conf.GetString("tls_dir"),
		TLSInterNodeEnabled: Alpha.Conf.GetBool("tls_enable_inter_node"),

Is gofmt run?

---

tlstest/mtls_internal/ha_6_node/ha_test.go, line 138 at r4 (raw file):

	dgConn, err := grpc.Dial(":" + port, grpc.WithTransportCredentials(credentials.NewTLS(tlsConf)))
	require.NoError(t, err)
	time.Sleep(time.Second * 6)

Why is this sleep required?

[aman-bansal]

Reviewable status: 98 of 111 files reviewed, 9 unresolved discussions (waiting on @manishrjain, @parasssh, @pawanrawal, and @vvbalaji-dgraph)

---

dgraph/cmd/alpha/run.go, line 171 at r3 (raw file):

Previously, pawanrawal (Pawan Rawal) wrote…

connect as a client with other nodes in the cluster.

Similarly below

Done.

---

dgraph/cmd/alpha/run.go, line 671 at r3 (raw file):

Previously, pawanrawal (Pawan Rawal) wrote…

Is gofmt run?

Done.

---

tlstest/mtls_internal/ha_6_node/ha_test.go, line 138 at r4 (raw file):

Previously, pawanrawal (Pawan Rawal) wrote…

Why is this sleep required?

this is Done. Actually I write the test case in different way initially that requires wait for the cluster to setup. changed it to docker compose setup later. I missed this refactoring.