Single Node deployment with bootstrap-in-place #4482
Conversation
openshift-ci-robot
added
the
do-not-merge/hold
eranco74
force-pushed
the
bootstrap-in-place
branch
2 times, most recently
from
4d679af to
3669ed0
Compare
|
Relevant cluster-bootstrap pr is: openshift/cluster-bootstrap#46 |
eranco74
force-pushed
the
bootstrap-in-place
branch
from
3669ed0 to
8802086
Compare
openshift-ci-robot
added
the
needs-rebase
eranco74
force-pushed
the
bootstrap-in-place
branch
2 times, most recently
from
4dc4c9d to
d2db289
Compare
eranco74
force-pushed
the
bootstrap-in-place
branch
from
d2db289 to
d7e88ad
Compare
openshift-ci-robot
removed
the
needs-rebase
eranco74
force-pushed
the
bootstrap-in-place
branch
2 times, most recently
from
5d1a8a1 to
b3eda08
Compare
|
@eranco74: The following tests failed, say
Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here. |
|
/assign |
staebler
left a comment
staebler
left a comment
There was a problem hiding this comment.
I don't have any major qualms with this approach.
eranco74
force-pushed
the
bootstrap-in-place
branch
4 times, most recently
from
4080605 to
f21bc87
Compare
eranco74
force-pushed
the
bootstrap-in-place
branch
3 times, most recently
from
74d33e0 to
3088a01
Compare
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: dhellmann, staebler The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
openshift-ci-robot
added
the
approved
|
booooom!!!!!
…On Thu, Feb 11, 2021, 23:48 OpenShift CI Robot ***@***.***> wrote:
[APPROVALNOTIFIER] This PR is *APPROVED*
This pull-request has been approved by: *dhellmann
<#4482 (review)>*,
*staebler
<#4482 (review)>*
The full list of commands accepted by this bot can be found here
<https://go.k8s.io/bot-commands?repo=openshift%2Finstaller>.
The pull request process is described here
<https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process>
Needs approval from an approver in each of these files:
- OWNERS <https://github.com/openshift/installer/blob/master/OWNERS>
[staebler]
Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#4482 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABLPP4YXQ3GUOTBDXWNOG23S6RGCPANCNFSM4U4DZE6A>
.
|
|
/lgtm |
|
/retest |
1 similar comment
|
/retest |
|
/retest Please review the full test history for this PR and help us cut down flakes. |
|
/lgtm cancel |
Added new asset for SingleNodeBootstrapInPlace Updated bootkube to set BOOTSTRAP_INPLACE variable and evaluate it at run time. This change allow someone looking at the script to follow the shell logic for single-node vs. multi-node deployment while debugging. When creating single-node-ignition-config we now validate that the install-config contain configuration for bootstrapInPlace. Added install-to-disk serivce that will complete the installation by writing the OS to the desired instnallation disk and reboot the node. Signed-off-by: Eran Cohen <[email protected]>
eranco74
force-pushed
the
bootstrap-in-place
branch
from
161c4b9 to
8e4a408
Compare
|
/lgtm |
|
/retest Please review the full test history for this PR and help us cut down flakes. |
1 similar comment
|
/retest Please review the full test history for this PR and help us cut down flakes. |
🕺 |
|
/retest |
|
/test e2e-aws |
|
/retest Please review the full test history for this PR and help us cut down flakes. |
2 similar comments
|
/retest Please review the full test history for this PR and help us cut down flakes. |
|
/retest Please review the full test history for this PR and help us cut down flakes. |
|
@eranco74: The following tests failed, say
Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here. |
11 participants
single-node deployment with bootstrap-in-place
Add new
create single-node-ignition-configcommand to the installer to createbootstrap-in-place-for-live-iso.ignIgnition config.This new target will not output master.ign and worker.ign.
This Ignition config will have a different
bootkube.shfrom thedefault bootstrap Ignition. In addition to the standard rendering
logic, the modified script will:
cluster-bootstrapwithout required pods by setting--required-pods=''cluster-bootstrapwith the--bootstrap-in-placeoption.config, the control plane static pod manifests, the required
kubernetes resources, and the bootstrap etcd database snapshot to
create a new Ignition config for the host.
This service will execute
coreos-installer installto write the rhcos imageand the rendered master Ignition to disk and reboot the node.
Log gathering on bootstrap-in-place.
Support collecting logs from bootstrap-in-place cluster with, for example,
openshift-install gather bootstrap --bootstrap 192.168.126.10 --master 192.168.126.10 --key id_rsaPre-pivot - gather bootstrap only
The command works as usual before the pivot occurs because the
/usr/local/bin/installer-gather.shscript is present and works as expected without any changes.Post-pivot - gather bootstrap & master logs
Gathering bootstrap logs
Before reboot, bootstrap will gather from itself using
/usr/local/bin/installer-gather.sh. This script also tries to gather from the masters in the cluster, but since there are none, it will only gather about itself.After the gathering is complete, the bundle is added to the master ignition - thus making the bootstrap logs available from the master after reboot.
Gathering master logs
Typically, in non-BiP scenarios, masters logs are gathered using the bootstrap node as a proxy by remotely running, via
ssh, the/usr/local/bin/installer-gather.shscript present on the bootstrap node. That script in turn detects all masters (or receives a list of masters via the--mastercommand-line arguments), then for each master itscps a second script called/usr/local/bin/installer-masters-gather.shto that master. It then runs that script remotely, on the master, usingssh. When the script is finished running, itscps the resulting files back to the bootstrap node, adding them to the bootstrap log bundle, with each master logs appearing in its own named-folder inside thecontrol-planedirectory of the bootstrap node log bundle.In the BiP scenario, we created a new script called
installer-master-bootstrap-in-place-gather.sh. This script is copied to the master (using ignition) to the same location where the bootstrap node usually hasinstaller-gather.sh. i.e. this script masquerades asinstaller-gather.sh. We also copy the/usr/local/bin/installer-masters-gather.sh.The
installer-master-bootstrap-in-place-gather.sh, masquerading asinstaller-gather.sh, gets remotely called by theopenshift-install gathercommand that believes its collecting logs from a bootstrap node. The script however behaves slightly differently - instead of collecting bootstrap logs and then remotely running/usr/local/bin/installer-masters-gather.shon all master nodes, it collects the bootstrap logs from the bundle copied to via the master-ignition, and it collect master logs by running/usr/local/bin/installer-masters-gather.shdirectly on itself instead of remotely on other masters (which don't exist. All master IP addresses passed to it are completely ignored). The final archiving of all the logs into the home directory is done exactly in the same manner as/usr/local/bin/installer-gather.sh.The end result is a log bundle that looks pretty much the same as non-bootstrap-in-place scenarios.
Small caveat
As you can see -
openshift-install gather bootstrap --bootstrap 192.168.126.10 --master 192.168.126.10 --key id_rsa- the command requires you to specify at least one master and one bootstrap node. In our case, you just pass the single node IP as both master and bootstrap. This causes a small "bug" when the node is pre-pivot - it collects both bootstrap logs on itself but also "remotely" collects master logs from itself via SSH. This causes a duplication of log files when the gather command is ran pre-pivot.