HDDS-2766. security/SecuringDataNodes.md #1175

iamabug · 2020-07-08T01:43:07Z

What changes were proposed in this pull request?

translation to doc https://hadoop.apache.org/ozone/docs/0.5.0-beta/security/securingdatanodes.html

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-2766

How was this patch tested?

hugo server

hadoop-hdds/docs/content/security/SecuringDatanodes.zh.md

iamabug · 2020-07-08T01:55:47Z

hadoop-hdds/docs/content/security/SecuringDatanodes.zh.md

+
+
+<h3>手动颁发<span class="badge badge-primary">开发中</span></h3>
+如果 datanode 是新加入的并且没有 keytab，那么证书请求需要等待管理员的批准。换句话说，信任关系链由集群管理员建立。


Original doc says the web of trust, but according to my understanding, it is called chain of trust in PKI.

A typo in origin doc maybe, 'band newshould bebrand new` ? If yes, should I fix it in this PR or open a new one ?

this should be "brand new" you can fix it in this one along with the CN doc.

"chain of trust" is better, pls fix it in both EN and CN doc.

那么证书请求需要等待管理员的批准（手动批准功能尚未完全支持）

iamabug · 2020-07-08T08:23:12Z

@xiaoyuyao @cxorm Could you help review this ? Thanks. BTW, I cannot find request review button on this page.

xiaoyuyao · 2020-07-08T16:32:33Z

@xiaoyuyao @cxorm Could you help review this ? Thanks. BTW, I cannot find request review button on this page.

You can do that on the top right, "reviewers" button. The dropdown list allows you to select reviewers.

xiaoyuyao · 2020-07-08T17:34:00Z

hadoop-hdds/docs/content/security/SecuringDatanodes.zh.md

+-->
+
+
+过去，Hadoop 中 datanode 的安全机制是通过在节点上创建 Keytab 文件实现的，而 Ozone 改用 datanode 证书，在安全的 Ozone 集群中，datanode 不再需要 Kerberos。


Can we rephrase:
"过去，Hadoop 中 datanode 的安全机制是通过在节点上创建 Keytab 文件实现的，而 Ozone 改用 datanode 证书，在安全的 Ozone 集群中，datanode 不再需要 Kerberos。"
=>
"Hadoop 中 datanode 的安全机制是通过给每个节点创建 Keytab 文件实现的。 Ozone 的datanode安全机制不依赖Kerberos，而是改用 datanode 证书。

xiaoyuyao · 2020-07-08T17:49:52Z

hadoop-hdds/docs/content/security/SecuringDatanodes.zh.md

+--------|--------------
+dfs.datanode.kerberos.principal| datanode 的服务主体名 <br/> 比如：dn/[email protected]
+dfs.datanode.keytab.file| datanode 进程所使用的 keytab 文件
+hdds.datanode.http.kerberos.principal| datanode http 服务器的服务主体名


hdds.datanode.http.kerberos.principal=>hdds.datanode.http.auth.kerberos.principal

Is this a committed change ? I notice that it is still hdds.datanode.http.kerberos.principal in master branch.

@iamabug Yes. It is in master
https://github.com/apache/hadoop-ozone/blob/master/hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/HddsConfigKeys.java#L249-L251

Thanks for the link and commit. @smengcl

xiaoyuyao · 2020-07-08T17:50:12Z

hadoop-hdds/docs/content/security/SecuringDatanodes.zh.md

+dfs.datanode.kerberos.principal| datanode 的服务主体名 <br/> 比如：dn/[email protected]
+dfs.datanode.keytab.file| datanode 进程所使用的 keytab 文件
+hdds.datanode.http.kerberos.principal| datanode http 服务器的服务主体名
+hdds.datanode.http.kerberos.keytab| datanode http 服务器所使用的 keytab 文件


hdds.datanode.http.kerberos.keytab=>hdds.datanode.http.auth.kerberos.keytab

datanode http 服务器所使用的 keytab 文件=>datanode http 服务器的服务主体登陆所使用的 keytab 文件

xiaoyuyao · 2020-07-08T17:52:57Z

hadoop-hdds/docs/content/security/SecuringDatanodes.zh.md

+
+## 如何安全化 datanode
+
+在 Ozone 中，当 datanode 启动并发现 SCM 的地址之后，datanode 要做的第一件事就是创建私钥并向 SCM 发送证书请求。


datanode 要做的第一件事就是创建私钥并向 SCM 发送证书请求-》datanode 首先创建私钥并向 SCM 发送证书请求。

xiaoyuyao · 2020-07-08T17:57:32Z

hadoop-hdds/docs/content/security/SecuringDatanodes.zh.md

+如果你通过 Kubernetes 这样的容器编排软件运行 Ozone，Kubernetes 需要为 datanode 创建一次性 token，用于在启动阶段证明 datanode 容器的身份。（这个特性也正在开发中。）
+
+
+证书颁发后，datanode 的安全就得到了保障，并且 OM 可以颁发块 token。如果 datanode 没有证书或者 SCM 的根证书，datanode 会自动进行注册，下载 SCM 的根证书，并获取自己的证书。


The original document has a typo here "down load" should be "download"

xiaoyuyao

LGTM overall, a few comments added inline.

iamabug · 2020-07-09T13:27:23Z

@xiaoyuyao @cxorm Could you help review this ? Thanks. BTW, I cannot find request review button on this page.

You can do that on the top right, "reviewers" button. The dropdown list allows you to select reviewers.

I used to have that, but now it looks like this. Reviewers is just plain text, I don't think there is a place to click.

hdds.datanode.http.kerberos.principal -> hdds.datanode.http.auth.kerberos.principal hdds.datanode.http.kerberos.keytab -> hdds.datanode.http.auth.kerberos.keytab

iamabug · 2020-07-09T23:01:33Z

/retest

cxorm · 2020-09-22T12:53:53Z

Thanks @iamabug for the work,
Overall looks great to me (including the fixes),
I would committed it If all CI checks passed.

cxorm · 2020-09-23T07:10:46Z

Thanks @iamabug for the contribution.
Thanks @xiaoyuyao and @smengcl for reviewing this PR,
and thanks @adoroszlai for adding the label.

* master: HDDS-4102. Normalize Keypath for lookupKey. (apache#1328) HDDS-4263. ReplicatiomManager shouldn't consider origin node Id for CLOSED containers. (apache#1438) HDDS-4282. Improve the emptyDir syntax (apache#1450) HDDS-4194. Create a script to check AWS S3 compatibility (apache#1383) HDDS-4270. Add more reusable byteman scripts to debug ofs/o3fs performance (apache#1443) HDDS-2660. Create insight point for datanode container protocol (apache#1272) HDDS-3297. Enable TestOzoneClientKeyGenerator. (apache#1442) HDDS-4324. Add important comment to ListVolumes logic (apache#1417) HDDS-4236. Move "Om*Codec.java" to new project hadoop-ozone/interface-storage (apache#1424) HDDS-4254. Bucket space: add usedBytes and update it when create and delete key. (apache#1431) HDDS-2766. security/SecuringDataNodes.md (apache#1175) HDDS-4206. Attempt pipeline creation more frequently in acceptance tests (apache#1389) HDDS-4233. Interrupted exeception printed out from DatanodeStateMachine (apache#1416) HDDS-3947: Sort DNs for client when the key is a file for #getFileStatus #listStatus APIs (apache#1385) HDDS-3102. ozone getconf command should use the GenericCli parent class (apache#1410) HDDS-3981. Add more debug level log to XceiverClientGrpc for debug purpose (apache#1214) HDDS-4255. Remove unused Ant and Jdiff dependency versions (apache#1433) HDDS-4247. Fixed log4j usage in some places (apache#1426) HDDS-4241. Support HADOOP_TOKEN_FILE_LOCATION for Ozone token CLI. (apache#1422)

* HDDS-4122-remove-code-consolidation: (21 commits) Restore files that had deduplicated code from master Revert other delete request/response files back to their original states on master HDDS-4102. Normalize Keypath for lookupKey. (apache#1328) HDDS-4263. ReplicatiomManager shouldn't consider origin node Id for CLOSED containers. (apache#1438) HDDS-4282. Improve the emptyDir syntax (apache#1450) HDDS-4194. Create a script to check AWS S3 compatibility (apache#1383) HDDS-4270. Add more reusable byteman scripts to debug ofs/o3fs performance (apache#1443) HDDS-2660. Create insight point for datanode container protocol (apache#1272) HDDS-3297. Enable TestOzoneClientKeyGenerator. (apache#1442) HDDS-4324. Add important comment to ListVolumes logic (apache#1417) HDDS-4236. Move "Om*Codec.java" to new project hadoop-ozone/interface-storage (apache#1424) HDDS-4254. Bucket space: add usedBytes and update it when create and delete key. (apache#1431) HDDS-2766. security/SecuringDataNodes.md (apache#1175) HDDS-4206. Attempt pipeline creation more frequently in acceptance tests (apache#1389) HDDS-4233. Interrupted exeception printed out from DatanodeStateMachine (apache#1416) HDDS-3947: Sort DNs for client when the key is a file for #getFileStatus #listStatus APIs (apache#1385) HDDS-3102. ozone getconf command should use the GenericCli parent class (apache#1410) HDDS-3981. Add more debug level log to XceiverClientGrpc for debug purpose (apache#1214) HDDS-4255. Remove unused Ant and Jdiff dependency versions (apache#1433) HDDS-4247. Fixed log4j usage in some places (apache#1426) ...

ChenSammi · 2020-11-03T07:42:29Z

Hi @iamabug , Can I get your email address? you can sent it to [email protected].

iamabug · 2020-11-04T11:34:51Z

Hi @iamabug , Can I get your email address? you can sent it to [email protected].

Hi @ChenSammi , my email is [email protected]

HDDS-2766. security/SecuringDataNodes.md

b7d17fd

iamabug commented Jul 8, 2020

View reviewed changes

hadoop-hdds/docs/content/security/SecuringDatanodes.zh.md Show resolved Hide resolved

iamabug commented Jul 8, 2020

View reviewed changes

xiaoyuyao reviewed Jul 8, 2020

View reviewed changes

HDDS-2766. security/SecuringDataNodes.md

592bf67

smengcl added 2 commits July 9, 2020 14:41

Update SecuringDatanodes.zh.md

a01b991

hdds.datanode.http.kerberos.principal -> hdds.datanode.http.auth.kerberos.principal hdds.datanode.http.kerberos.keytab -> hdds.datanode.http.auth.kerberos.keytab

Update SecuringDatanodes.md

c13475a

hdds.datanode.http.kerberos.principal -> hdds.datanode.http.auth.kerberos.principal hdds.datanode.http.kerberos.keytab -> hdds.datanode.http.auth.kerberos.keytab

smengcl requested a review from cxorm July 9, 2020 21:44

empty commit to retest build

b7b69cd

adoroszlai added the documentation Improvements or additions to documentation label Jul 10, 2020

cxorm removed their request for review July 22, 2020 06:11

xiaoyuyao and others added 2 commits September 20, 2020 10:15

Merge branch 'master' into HDDS-2766

673cf00

trigger new CI check

c7ea4cb

cxorm merged commit 0eceb4c into apache:master Sep 23, 2020



		<h3>手动颁发<span class="badge badge-primary">开发中</span></h3>
		如果 datanode 是新加入的并且没有 keytab，那么证书请求需要等待管理员的批准。换句话说，信任关系链由集群管理员建立。

		-->


		过去，Hadoop 中 datanode 的安全机制是通过在节点上创建 Keytab 文件实现的，而 Ozone 改用 datanode 证书，在安全的 Ozone 集群中，datanode 不再需要 Kerberos。


		## 如何安全化 datanode

		在 Ozone 中，当 datanode 启动并发现 SCM 的地址之后，datanode 要做的第一件事就是创建私钥并向 SCM 发送证书请求。

		如果你通过 Kubernetes 这样的容器编排软件运行 Ozone，Kubernetes 需要为 datanode 创建一次性 token，用于在启动阶段证明 datanode 容器的身份。（这个特性也正在开发中。）


		证书颁发后，datanode 的安全就得到了保障，并且 OM 可以颁发块 token。如果 datanode 没有证书或者 SCM 的根证书，datanode 会自动进行注册，下载 SCM 的根证书，并获取自己的证书。

HDDS-2766. security/SecuringDataNodes.md #1175

HDDS-2766. security/SecuringDataNodes.md #1175

Uh oh!

Conversation

iamabug commented Jul 8, 2020

What changes were proposed in this pull request?

What is the link to the Apache JIRA

How was this patch tested?

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

iamabug commented Jul 8, 2020

Uh oh!

xiaoyuyao commented Jul 8, 2020

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

smengcl Jul 9, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

xiaoyuyao left a comment

Choose a reason for hiding this comment

Uh oh!

iamabug commented Jul 9, 2020

Uh oh!

iamabug commented Jul 9, 2020

Uh oh!

cxorm commented Sep 22, 2020

Uh oh!

cxorm commented Sep 23, 2020

Uh oh!

ChenSammi commented Nov 3, 2020

Uh oh!

iamabug commented Nov 4, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

smengcl Jul 9, 2020 •

edited

Loading