Added post-install script to apply mcos

[e-minguez]

Once the storage VLAN is clear (see #4) the mco object can be hosted in
assets/post-install/ as well

[phoracek]

The bridge should have vlan_filtering enabled and set trunk on the default NIC port. I'm not sure whether it is possible to do that via ifcfg or we would need to have a custom systemd script on top of it, calling ip link ....

[e-minguez]

There are a couple of handy variables here such as BRIDGE_PORT_VLANS and BRIDGING_OPTS: vlan_filtering=

I've modified the commit to only include the post-install script that will create the mcos until we figure out how to do the vlan filtering and such. Any help @phoracek ? Thanks!

[phoracek]

@e-minguez sounds great! Will open an issue to track the VLAN part separately.

[russellb]

What does mcos mean?

[russellb]

What needs to happen in these 30 seconds? There's a wait right after this. Will that one fail if this wait doesn't happen? What does it fix?

[e-minguez]

If the oc wait is performed just after applying the machine configs, it finishes as the machine config operator takes a while to notice the changes and create the rendered machine config. 30 seconds is just random, but definitely in my tests I needed to wait a few seconds.

[e-minguez]

What does mcos mean?

machine config objects (I've modified the comment to be more explicit)

[e-minguez]

Actually, I'm not sure about this PR as it seems that we are about to be able to add manifests at installation time #30 #35

[russellb]

Actually, I'm not sure about this PR as it seems that we are about to be able to add manifests at installation time #30 #35

Don't wait on that, as we'll intentionally limit the install-time manifest mangling as much as possible

[rdoxenham]

@russellb @e-minguez - don't count on adding MCO's at boot time, especially if they involve configuring interfaces that DHCP...

level=debug msg="Still waiting for the cluster to initialize: Cluster operator machine-config is reporting a failure: Failed to resync 4.2.0-0.ci-2019-08-21-102306-rhte.0 because: timed out waiting for the condition during syncRequiredMachineConfigPools: pool master has not progressed to latest configuration: configuration status for pool master is empty: pool is degraded because nodes fail with \"3 nodes are reporting degraded status on sync\": \"Node master-0 is reporting: \\\"unexpected on-disk state validating against rendered-master-cc6f4aca4fbe45fc61faacf77b7057de\\\", Node master-2 is reporting: \\\"machineconfig.machineconfiguration.openshift.io \\\\\\\"rendered-master-cc6f4aca4fbe45fc61faacf77b7057de\\\\\\\" not found\\\", Node master-1 is reporting: \\\"machineconfig.machineconfiguration.openshift.io \\\\\\\"rendered-master-cc6f4aca4fbe45fc61faacf77b7057de\\\\\\\" not found\\\"\", retrying"

I was trying to setup ens4 to be part of a brext bridge via an MCO, but upon DHCP it overrides the config I want to lay down, whereas it successfully creates the brext bridge-

$ ssh core@master-0 cat /etc/sysconfig/network-scripts/ifcfg-ens4
# Generated by dracut initrd
NAME="ens4"
DEVICE="ens4"
ONBOOT=yes
NETBOOT=yes
UUID="293f27a8-2247-44f9-a26d-5fa665d8658b"
IPV6INIT=yes
BOOTPROTO=dhcp

$ ssh core@master-0 cat /etc/sysconfig/network-scripts/ifcfg-brext
DEVICE=brext
NAME=brext
TYPE=Bridge
ONBOOT=yes
NM_CONTROLLED=yes
BOOTPROTO=dhcp

For reference, he's my MCO object-

$ cat dev-scripts/assets/generated/99_master-brext.yaml
apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfig
metadata:
  labels:
    machineconfiguration.openshift.io/role: master
  name: 99-master-brext
spec:
  config:
    ignition:
      version: 2.2.0
    storage:
      files:
      - contents:
          source: data:text/plain;charset=utf-8;base64,REVWSUNFPWVuczQKQlJJREdFPWJyZXh0Ck9OQk9PVD15ZXMKTk1fQ09OVFJPTExFRD15ZXMKQk9PVFBST1RPPW5vbmUK
          verification: {}
        filesystem: root
        mode: 420
        path: /etc/sysconfig/network-scripts/ifcfg-ens4
      - contents:
          source: data:text/plain;charset=utf-8;base64,REVWSUNFPWJyZXh0Ck5BTUU9YnJleHQKVFlQRT1CcmlkZ2UKT05CT09UPXllcwpOTV9DT05UUk9MTEVEPXllcwpCT09UUFJPVE89ZGhjcAo=
          verification: {}
        filesystem: root
        mode: 420
        path: /etc/sysconfig/network-scripts/ifcfg-brext

[rdoxenham]

To follow this up, I managed to get this working by adding a dispatcher.d script-

$ cat work/dev-scripts/assets/files/etc/NetworkManager/dispatcher.d/98-brextscript
#!/usr/bin/env bash

IF=$(ip r | grep default | grep -Po '(?<=dev )(\S+)')
if $(grep -q "dracut" /etc/sysconfig/network-scripts/ifcfg-$IF); then
echo brext acting on $IF
cat <<EOF > /etc/sysconfig/network-scripts/ifcfg-$IF
DEVICE=$IF
BRIDGE=brext
ONBOOT=yes
NM_CONTROLLED=yes
BOOTPROTO=none
EOF
systemctl restart NetworkManager
nmcli conn down $IF
fi

[e-minguez]

I've mixed this PR with #33 in order to create the bridge as a function in the post-install script instead doing it just for CNV

[e-minguez]

Ping @rlopez133 @sreichar as it looks like you own the repo now :)