Skip to content

operator: remove the generated ssh key machineconfig #356

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
openshift-merge-robot merged 3 commits into from
Mar 28, 2019
Merged

operator: remove the generated ssh key machineconfig #356

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
0b56194
operator: remove the generated ssh key machineconfig
abhinavdahiya Jan 30, 2019
a3cfc48
vendor: drop unused openshift/installer import
abhinavdahiya Mar 26, 2019
a7a3bb0
docs: update instruction on updating ssh keys for pool
abhinavdahiya Jan 30, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 1 addition & 31 deletions Gopkg.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 0 additions & 4 deletions Gopkg.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,10 +87,6 @@ required = [
name = "github.com/openshift/client-go"
branch = "master"

[[constraint]]
branch = "master"
name = "github.com/openshift/installer"

[[constraint]]
name = "github.com/openshift/kubernetes-drain"
revision = "c2e51be1758efa30d71a4d30dc4e2db86b70a4df"
Expand Down
171 changes: 105 additions & 66 deletions docs/Update-SSHKeys.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

By default, the OpenShift 4.0 installer creates a single user named `core` (derived in spirit from CoreOS Container Linux) with optional SSH keys specified at install time.

This operator supports updating the SSH keys of user `core` via a MachineConfig object. The SSH keys are updated for all members of the MachineConfig pool specified in the MachineConfig, for example: all worker nodes.
This controller supports updating the SSH keys of user `core` via a MachineConfig object. The SSH keys are updated for all members of the MachineConfig pool specified in the MachineConfig, for example: all worker nodes.

Please note that RHCOS nodes will be [annotated](https://github.com/openshift/machine-config-operator/blob/master/docs/MachineConfigDaemon.md#annotating-on-ssh-access) when accessed via SSH.

Expand All @@ -18,21 +18,39 @@ Please note that RHCOS nodes will be [annotated](https://github.com/openshift/ma

You will need the following information for the MachineConfig that will be used to update your SSHKeys.

- `machineconfiguration.openshift.io/role:` the MachineConfig that is created will be applied to all nodes with the role specified here. For example: `master` or `worker`

- `name:` each MachineConfig that you create must have a unique name. Do not reuse the same MachineConfig name. MachineConfigs are cumulative and applied in alphabetical/lexicographic order so that the last MachineConfig will be the final one applied. We recommend using a naming scheme that accounts for this such as: `ssh-workers-01`, `ssh-workers-02`, `ssh-master-01`, `ssh-masters-02`, etc...
- `machineconfiguration.openshift.io/role:` the MachineConfig that is updated will be applied to all nodes with the role specified here. For example: `master` or `worker`

- `sshAuthorizedKeys:` you will need one or more public keys to be assigned to user `core`. Multiple SSH Keys should begin on different lines and each be preceded by `-`.

## Example MachineConfig (with 2 SSH Keys added)
```yaml
# example-ssh-update.yaml
## Updating SSH for workers

List all the machineconfigs currently in the cluster.

```console
$ oc get machineconfigs
NAME GENERATEDBYCONTROLLER IGNITIONVERSION CREATED
00-master 4.0.0-alpha.0-116-gdc9b354d-dirty 2.2.0 3m40s
00-worker 4.0.0-alpha.0-116-gdc9b354d-dirty 2.2.0 3m40s
01-master-container-runtime 4.0.0-alpha.0-116-gdc9b354d-dirty 2.2.0 3m40s
01-master-kubelet 4.0.0-alpha.0-116-gdc9b354d-dirty 2.2.0 3m40s
01-worker-container-runtime 4.0.0-alpha.0-116-gdc9b354d-dirty 2.2.0 3m39s
01-worker-kubelet 4.0.0-alpha.0-116-gdc9b354d-dirty 2.2.0 3m39s
99-master-ssh 2.2.0 3m57s
99-worker-ssh 2.2.0 3m56s
rendered-master-753f1b4476475611360ba65b4d0d378d 4.0.0-alpha.0-116-gdc9b354d-dirty 2.2.0 3m36s
rendered-worker-6a77758776c24585f8db809b64569b6f 4.0.0-alpha.0-116-gdc9b354d-dirty 2.2.0 3m36s
```

`99-worker-ssh` contains the current SSHAuthorizedKeys for `worker` pool.

```console
$ oc get machineconfigs 99-worker-ssh -oyaml
apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfig
metadata:
labels:
machineconfiguration.openshift.io/role: worker
name: ssh-workers-01
name: 99-worker-ssh
spec:
config:
passwd:
Expand All @@ -42,85 +60,106 @@ spec:
- ssh-rsa ABC123....
- ssh-ed25519 XYZ7890....
- ecdsa-sha2-nistp256 AAAAE2....
```

```
## Common MachineConfig Pitfalls
- Assuming that the name of the file is the name of the MachineConfig: If you choose to modify one of your existing MachineConfigs, do not forget to change the `metadata: name:` field.
Export the `99-worker-ssh` to edit the `SSHAuthorizedKeys`.

- Thinking that you will retain an old SSH key when you apply an SSH update: New SSH updates completely overwrite existing keys. If you would like to add an additional SSH key and retain the current SSH Key, you must add *both* the old and new SSH keys into the new MachineConfig.
```sh
oc get machineconfigs 99-worker-ssh -oyaml --export > update-ssh-worker.yaml
```

- Updating `user: name`: Do not update the `user: name` field. The only user currently supported is `core` as shown in the above example config.
Update the `sshAuthorizedKeys` for `core` user in `update-ssh-worker.yaml`.

## Applying the MachineConfig
```yaml
# update-ssh-worker.yaml
apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfig
metadata:
labels:
machineconfiguration.openshift.io/role: worker
name: 99-worker-ssh
spec:
config:
passwd:
users:
- name: core
sshAuthorizedKeys:
- ssh rsa ABC123....
- ssh rsa XYZ7890....
```

Now with your MachineConfig yaml file (using the example above), apply the changes.

Now with your new MachineConfig yaml file (using the example above):
```sh
oc create -f example-ssh-update.yaml
oc apply -f update-ssh-worker.yaml
```

You should see the new MachineConfig name appear almost immediately, from our example config:
```sh
oc get machineconfigs

NAME GENERATEDBYCONTROLLER IGNITIONVERSION CREATED
00-master 4.0.0-alpha.0-83-g0a84145f-dirty 2.2.0 87m
00-master-ssh 4.0.0-alpha.0-83-g0a84145f-dirty 2.2.0 87m
00-worker 4.0.0-alpha.0-83-g0a84145f-dirty 2.2.0 87m
00-worker-ssh 4.0.0-alpha.0-83-g0a84145f-dirty 2.2.0 87m
01-master-container-runtime 4.0.0-alpha.0-83-g0a84145f-dirty 2.2.0 87m
01-master-kubelet 4.0.0-alpha.0-83-g0a84145f-dirty 2.2.0 87m
01-worker-container-runtime 4.0.0-alpha.0-83-g0a84145f-dirty 2.2.0 87m
01-worker-kubelet 4.0.0-alpha.0-83-g0a84145f-dirty 2.2.0 87m
99-master-9c65f9fb-41d0-11e9-994d-02360d172130-registries 4.0.0-alpha.0-83-g0a84145f-dirty 2.2.0 84m
99-worker-9c67b221-41d0-11e9-994d-02360d172130-registries 4.0.0-alpha.0-83-g0a84145f-dirty 2.2.0 84m
rendered-master-a1884339a91f02f19898fe9c5929256b 4.0.0-alpha.0-83-g0a84145f-dirty 2.2.0 87m
rendered-worker-777a036887bb25188f47c6d3219eabb9 4.0.0-alpha.0-83-g0a84145f-dirty 2.2.0 2s
rendered-worker-e99b9948bccfd387c71397b31e2c985e 4.0.0-alpha.0-83-g0a84145f-dirty 2.2.0 87m
ssh-workers-01 7s
You should see the new MachineConfig being generated by the controller to roll the changes:

```console
$ oc get machineconfigs
NAME GENERATEDBYCONTROLLER IGNITIONVERSION CREATED
00-master 4.0.0-alpha.0-116-gdc9b354d-dirty 2.2.0 3m40s
00-worker 4.0.0-alpha.0-116-gdc9b354d-dirty 2.2.0 3m40s
01-master-container-runtime 4.0.0-alpha.0-116-gdc9b354d-dirty 2.2.0 3m40s
01-master-kubelet 4.0.0-alpha.0-116-gdc9b354d-dirty 2.2.0 3m40s
01-worker-container-runtime 4.0.0-alpha.0-116-gdc9b354d-dirty 2.2.0 3m39s
01-worker-kubelet 4.0.0-alpha.0-116-gdc9b354d-dirty 2.2.0 3m39s
99-master-ssh 2.2.0 3m57s
99-worker-ssh 2.2.0 3m56s
rendered-master-753f1b4476475611360ba65b4d0d378d 4.0.0-alpha.0-116-gdc9b354d-dirty 2.2.0 3m36s
rendered-worker-6a77758776c24585f8db809b64569b6f 4.0.0-alpha.0-116-gdc9b354d-dirty 2.2.0 3m36s
rendered-worker-21612112efc1dde979f1c73c1d9df168 4.0.0-alpha.0-116-gdc9b354d-dirty 2.2.0 10s
```

You are then able to monitor the MCD logs of a worker or master (whichever the config applied to), which should check the proposed changes and reboot into the new config:

```sh
oc logs -f -n openshift-machine-config-operator machine-config-daemon-<hash>
oc logs -f -n openshift-machine-config-operator machine-config-daemon-<hash>
```

If the update was succesfully applied, you should expect to see lines similar to in these logs:
```sh
I0111 19:59:07.360110 7993 update.go:258] SSH Keys reconcilable
...
I0111 19:59:07.371253 7993 update.go:569] Writing SSHKeys at "/home/core/.ssh"
...
I0111 19:59:07.372208 7993 update.go:613] machine-config-daemon initiating reboot: Node will reboot into config worker-96b48815fa067f651fa50541ea6a9b5d

```console
I0111 19:59:07.360110 7993 update.go:258] SSH Keys reconcilable
...
I0111 19:59:07.371253 7993 update.go:569] Writing SSHKeys at "/home/core/.ssh"
...
I0111 19:59:07.372208 7993 update.go:613] machine-config-daemon initiating reboot: Node will reboot into config worker-96b48815fa067f651fa50541ea6a9b5d
```

After the node reboots, expect to see the daemons for the node specified restarted:

```sh
oc get pods -n openshift-machine-config-operator

NAME READY STATUS RESTARTS AGE
machine-config-controller-68f5989588-2cfvq 1/1 Running 0 1h
machine-config-daemon-58d6c 1/1 Running 0 1h
machine-config-daemon-c7jkk 1/1 Running 1 1h
machine-config-daemon-ddsnp 1/1 Running 1 1h
machine-config-daemon-kx49n 1/1 Running 1 1h
machine-config-daemon-q8d7j 1/1 Running 0 1h
machine-config-daemon-w68t9 1/1 Running 0 1h
machine-config-operator-769967ddf5-9blb8 1/1 Running 0 1h
machine-config-server-7gckv 1/1 Running 0 1h
machine-config-server-98cpz 1/1 Running 0 1h
machine-config-server-pzj68 1/1 Running 0 1h
```console
$ oc get pods -n openshift-machine-config-operator

NAME READY STATUS RESTARTS AGE
machine-config-controller-68f5989588-2cfvq 1/1 Running 0 1h
machine-config-daemon-58d6c 1/1 Running 0 1h
machine-config-daemon-c7jkk 1/1 Running 1 1h
machine-config-daemon-ddsnp 1/1 Running 1 1h
machine-config-daemon-kx49n 1/1 Running 1 1h
machine-config-daemon-q8d7j 1/1 Running 0 1h
machine-config-daemon-w68t9 1/1 Running 0 1h
machine-config-operator-769967ddf5-9blb8 1/1 Running 0 1h
machine-config-server-7gckv 1/1 Running 0 1h
machine-config-server-98cpz 1/1 Running 0 1h
machine-config-server-pzj68 1/1 Running 0 1h
```

If we check the same daemon's logs, we should now see similar lines in the output:

```sh
oc logs -f -n openshift-machine-config-operator machine-config-daemon-<same-hash>

...
I0111 20:00:15.755052 6900 daemon.go:497] Completing pending config worker-52df682dc5cb3976b063ef3f197ead5e
...
I0111 20:00:15.769349 6900 update.go:613] machine-config-daemon: completed update for config worker-52df682dc5cb3976b063ef3f197ead5e
...
I0111 20:00:15.778909 6900 daemon.go:503] In desired config worker-52df682dc5cb3976b063ef3f197ead5e
```console
$ oc logs -f -n openshift-machine-config-operator machine-config-daemon-<same-hash>

...
I0111 20:00:15.755052 6900 daemon.go:497] Completing pending config worker-52df682dc5cb3976b063ef3f197ead5e
...
I0111 20:00:15.769349 6900 update.go:613] machine-config-daemon: completed update for config worker-52df682dc5cb3976b063ef3f197ead5e
...
I0111 20:00:15.778909 6900 daemon.go:503] In desired config worker-52df682dc5cb3976b063ef3f197ead5e
```

## Common Pitfalls

- Updating `user: name`: Do not update the `user: name` field. The only user currently supported is `core` as shown in the above example config.
1 change: 0 additions & 1 deletion lib/resourcemerge/machineconfig.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,6 @@ func ensureControllerConfigSpec(modified *bool, existing *mcfgv1.ControllerConfi
setStringIfSet(modified, &existing.CloudProviderConfig, required.CloudProviderConfig)
setStringIfSet(modified, &existing.Platform, required.Platform)
setStringIfSet(modified, &existing.EtcdDiscoveryDomain, required.EtcdDiscoveryDomain)
setStringIfSet(modified, &existing.SSHKey, required.SSHKey)
setStringIfSet(modified, &existing.OSImageURL, required.OSImageURL)

setBytesIfSet(modified, &existing.EtcdCAData, required.EtcdCAData)
Expand Down
3 changes: 0 additions & 3 deletions pkg/apis/machineconfiguration.openshift.io/v1/types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -125,9 +125,6 @@ type ControllerConfigSpec struct {
// on all machines.
PullSecret *corev1.ObjectReference `json:"pullSecret,omitempty"`

// Public SSH
SSHKey string `json:"sshKey"`

// Images is map of images that are used by the controller.
Images map[string]string `json:"images"`

Expand Down
24 changes: 0 additions & 24 deletions pkg/controller/template/render.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -95,14 +95,7 @@ func GenerateMachineConfigsForRole(config *RenderConfig, role string, path strin
return nil, fmt.Errorf("failed to read dir %q: %v", path, err)
}

sshMachineConfigForRole, err := generateSSHConfig(config.SSHKey, role)
if err != nil {
return nil, err
}

cfgs := []*mcfgv1.MachineConfig{}
cfgs = append(cfgs, sshMachineConfigForRole)

for _, info := range infos {
if !info.IsDir() {
glog.Infof("ignoring non-directory path %q", info.Name())
Expand All @@ -120,23 +113,6 @@ func GenerateMachineConfigsForRole(config *RenderConfig, role string, path strin
return cfgs, nil
}

// for each role a machine config is created containing the sshauthorized keys to allow for ssh access
// ex: role = worker -> machine config "00-worker-ssh" created containing user core and ssh key
func generateSSHConfig(sshKey string, role string) (*mcfgv1.MachineConfig, error) {
// Using cttypes.Config and then ctc.config.Covert allows us to create a properly structured
// validated Ignition Config (instead of creating one by hand) that is then added to a
// MachineConfig.
var tempctCfg cttypes.Config
tempUser := cttypes.User{Name: "core", SSHAuthorizedKeys: []string{sshKey}}
tempctCfg.Passwd.Users = append(tempctCfg.Passwd.Users, tempUser)
tempIgnConfig, rep := ctconfig.Convert(tempctCfg, "", nil)
if rep.IsFatal() {
return nil, fmt.Errorf("failed to convert config to Ignition config %s", rep)
}
sshConfigName := "00-" + role + "-ssh"
return MachineConfigFromIgnConfig(role, sshConfigName, &tempIgnConfig), nil
}

func platformFromControllerConfigSpec(ic *mcfgv1.ControllerConfigSpec) (string, error) {
switch ic.Platform {
case "":
Expand Down
Loading