-
Notifications
You must be signed in to change notification settings - Fork 588
HDDS-2766. security/SecuringDataNodes.md #1175
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
7 commits
Select commit
Hold shift + click to select a range
b7d17fd
HDDS-2766. security/SecuringDataNodes.md
592bf67
HDDS-2766. security/SecuringDataNodes.md
a01b991
Update SecuringDatanodes.zh.md
smengcl c13475a
Update SecuringDatanodes.md
smengcl b7b69cd
empty commit to retest build
web-flow 673cf00
Merge branch 'master' into HDDS-2766
xiaoyuyao c7ea4cb
trigger new CI check
cxorm File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -27,8 +27,8 @@ icon: th | |
|
|
||
|
|
||
| Datanodes under Hadoop is traditionally secured by creating a Keytab file on | ||
| the data nodes. With Ozone, we have moved away to using data node | ||
| certificates. That is, Kerberos on data nodes is not needed in case of a | ||
| the datanodes. With Ozone, we have moved away to using datanode | ||
| certificates. That is, Kerberos on datanodes is not needed in case of a | ||
| secure Ozone cluster. | ||
|
|
||
| However, we support the legacy Kerberos based Authentication to make it easy | ||
|
|
@@ -39,38 +39,38 @@ Property|Description | |
| --------|-------------- | ||
| dfs.datanode.kerberos.principal|The datanode service principal. <br/> e.g. dn/[email protected] | ||
| dfs.datanode.keytab.file| The keytab file used by datanode daemon to login as its service principal. | ||
| hdds.datanode.http.kerberos.principal| Datanode http server service principal. | ||
| hdds.datanode.http.kerberos.keytab| The keytab file used by datanode http server to login as its service principal. | ||
| hdds.datanode.http.auth.kerberos.principal| Datanode http server service principal. | ||
| hdds.datanode.http.auth.kerberos.keytab| The keytab file used by datanode http server to login as its service principal. | ||
|
|
||
|
|
||
| ## How a data node becomes secure. | ||
| ## How a datanode becomes secure. | ||
|
|
||
| Under Ozone, when a data node boots up and discovers SCM's address, the first | ||
| thing that data node does is to create a private key and send a certificate | ||
| Under Ozone, when a datanode boots up and discovers SCM's address, the first | ||
| thing that datanode does is to create a private key and send a certificate | ||
| request to the SCM. | ||
|
|
||
| <h3>Certificate Approval via Kerberos <span class="badge badge-secondary">Current Model</span></h3> | ||
| SCM has a built-in CA, and SCM has to approve this request. If the data node | ||
| SCM has a built-in CA, and SCM has to approve this request. If the datanode | ||
| already has a Kerberos key tab, then SCM will trust Kerberos credentials and | ||
| issue a certificate automatically. | ||
|
|
||
|
|
||
| <h3>Manual Approval <span class="badge badge-primary">In Progress</span></h3> | ||
| If these are band new data nodes and Kerberos key tabs are not present at the | ||
| data nodes, then this request for the data nodes identity certificate is | ||
| If these are brand new datanodes and Kerberos key tabs are not present at the | ||
| datanodes, then this request for the datanodes identity certificate is | ||
| queued up for approval from the administrator(This is work in progress, | ||
| not committed in Ozone yet). In other words, the web of trust is established | ||
| not committed in Ozone yet). In other words, the chain of trust is established | ||
| by the administrator of the cluster. | ||
|
|
||
| <h3>Automatic Approval <span class="badge badge-secondary">In Progress</span></h3> | ||
| If you running under an container orchestrator like Kubernetes, we rely on | ||
| Kubernetes to create a one-time token that will be given to data node during | ||
| boot time to prove the identity of the data node container (This is also work | ||
| Kubernetes to create a one-time token that will be given to datanode during | ||
| boot time to prove the identity of the datanode container (This is also work | ||
| in progress.) | ||
|
|
||
|
|
||
| Once a certificate is issued, a data node is secure and Ozone manager can | ||
| issue block tokens. If there is no data node certificates or the SCM's root | ||
| certificate is not present in the data node, then data node will register | ||
| itself and down load the SCM's root certificate as well get the certificates | ||
| Once a certificate is issued, a datanode is secure and Ozone manager can | ||
| issue block tokens. If there is no datanode certificates or the SCM's root | ||
| certificate is not present in the datanode, then datanode will register | ||
| itself and download the SCM's root certificate as well get the certificates | ||
| for itself. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,53 @@ | ||
| --- | ||
| title: "安全化 Datanode" | ||
| date: "2019-April-03" | ||
| weight: 2 | ||
| summary: 解释安全化 datanode 的不同模式,包括 Kerberos、证书的手动颁发和自动颁发等。 | ||
| icon: th | ||
| --- | ||
| <!--- | ||
| Licensed to the Apache Software Foundation (ASF) under one or more | ||
| contributor license agreements. See the NOTICE file distributed with | ||
| this work for additional information regarding copyright ownership. | ||
| The ASF licenses this file to You under the Apache License, Version 2.0 | ||
| (the "License"); you may not use this file except in compliance with | ||
| the License. You may obtain a copy of the License at | ||
|
|
||
| http://www.apache.org/licenses/LICENSE-2.0 | ||
|
|
||
| Unless required by applicable law or agreed to in writing, software | ||
| distributed under the License is distributed on an "AS IS" BASIS, | ||
| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| See the License for the specific language governing permissions and | ||
| limitations under the License. | ||
| --> | ||
|
|
||
|
|
||
| Hadoop 中 datanode 的安全机制是通过给每个节点创建 Keytab 文件实现的。Ozone 的 datanode 安全机制不依赖 Kerberos,而是改用 datanode 证书。 | ||
|
|
||
| 但是我们也支持传统的基于 Kerberos 的认证来方便现有用户,用户只需要在 hdfs-site.xml 里配置下面参数即可: | ||
|
|
||
| 参数名|描述 | ||
| --------|-------------- | ||
| dfs.datanode.kerberos.principal| datanode 的服务主体名 <br/> 比如:dn/[email protected] | ||
| dfs.datanode.keytab.file| datanode 进程所使用的 keytab 文件 | ||
| hdds.datanode.http.auth.kerberos.principal| datanode http 服务器的服务主体名 | ||
| hdds.datanode.http.auth.kerberos.keytab| datanode http 服务器的服务主体登录所使用的 keytab 文件 | ||
|
|
||
|
|
||
| ## 如何安全化 datanode | ||
|
|
||
| 在 Ozone 中,当 datanode 启动并发现 SCM 的地址之后,datanode 首先创建私钥并向 SCM 发送证书请求。 | ||
|
|
||
| <h3>通过 Kerberos 颁发证书<span class="badge badge-secondary">当前模型</span></h3> | ||
| SCM 有一个内置的 CA 用来批准证书请求,如果 datanode 已经有一个 Kerberos keytab,SCM 会信任它并自动颁发一个证书。 | ||
|
|
||
|
|
||
| <h3>手动颁发<span class="badge badge-primary">开发中</span></h3> | ||
| 如果 datanode 是新加入的并且没有 keytab,那么证书请求需要等待管理员的批(手动批准功能尚未完全支持)。换句话说,信任关系链由集群管理员建立。 | ||
|
|
||
| <h3>自动颁发 <span class="badge badge-secondary">开发中</span></h3> | ||
| 如果你通过 Kubernetes 这样的容器编排软件运行 Ozone,Kubernetes 需要为 datanode 创建一次性 token,用于在启动阶段证明 datanode 容器的身份。(这个特性也正在开发中。) | ||
|
|
||
|
|
||
| 证书颁发后,datanode 的安全就得到了保障,并且 OM 可以颁发块 token。如果 datanode 没有证书或者 SCM 的根证书,datanode 会自动进行注册,下载 SCM 的根证书,并获取自己的证书。 | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The original document has a typo here "down load" should be "download" |
||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.