-
Notifications
You must be signed in to change notification settings - Fork 1.9k
OSDOCS-4347: Account Information: installing-aws-localzone created from cloudformation #54535
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
e714459 to
000b196
Compare
|
🤖 Updated build preview is available at: Build log: https://circleci.com/gh/ocpdocs-previewbot/openshift-docs/7076 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is supposed to be the public subnets, right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
that's correct.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry, you must include the private subnet here too. Take a look in my suggestion below.
606403b to
0ae8b8a
Compare
|
@mtulio, will you PTAL? @yunjiang29, will you please review this late change? Marco said that you provided the review for his article about this installation method, and we'd like to have the downstream version available at GA if at all possible.
|
cb1bca0 to
f9ae1b5
Compare
f9ae1b5 to
4f731f8
Compare
mtulio
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@GroceryBoyJr @kalexand-rh :
Thanks for providing the documentation. This is my initial review. Overall looks good, I am focusing on removing the sections that we don't need for this guide. I will take a second look tomorrow focusing on the Local Zone details.
Addressing your questions:
- I am not sure what to do about the manifest files for the workers. The existing UPI method has the user removing all the manifests for the control plane and compute nodes and creating the all of the nodes as separate stacks in AWS. However, Marco has a new manifest file for the workers. Should we remove the existing worker manifests and just use Marcos? Do we remove the instruction to create separate worker stacks?
I answered in detail in the specific line. We must remove the part that we are removing the machineset manifests. The installer must place those manifests to MAPI to create it. (default IPI)
| ClusterInfraId: | ||
| Type: String | ||
| Default: "unmanaged" | ||
| Description: ClusterInfraId used to tag required resources |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I need to review the unmanaged initial value for ClusterInfraId. The InfraID is a chicken-and-egg problem where we need the VPC created to place the subnets in install-config to get the generated InfraId. In the original article, I place an option to patch this. Let's keep this thread open until I review if it is a must or not.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let us know the best approach!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The documentation of "Requirements for using your VPC" when installing a cluster in existing VPC, says The VPC must not use the kubernetes.io/cluster/.*: owned tag.. This is in consensus what we are doing here. You can skip this comment.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ClusterInfraId is not required while creating VPC like what original VPC cloudformation template does, tag kubernetes.io/cluster/<infra-id>: shared will be added by installer automatically
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@mtulio @yunjiang29 Does this mean I should delete the four lines of "ClusterInfraId:" or leave them alone?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@GroceryBoyJr you can remove this parameter, please read my comment below on the default value that should be defined when removing this.
https://github.com/openshift/openshift-docs/pull/54535/files#r1070098476
cc @yunjiang29
| PublicRouteTableId: | ||
| Description: Public Route table ID | ||
| Value: !Ref PublicRouteTable |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Those lines are the difference from the original VPC cloudformation template.
4f731f8 to
e05694c
Compare
e05694c to
4f731f8
Compare
98d091d to
4b050ce
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is one of the most important pieces of documentation, this will be the unique part when installing in existing VPC, and there should have the subnetIDs for CloudFormation for VPC. So I would improve this example to keep closer to the real values.
Let me take some suggestions of what could be, feel free to adapt:
| subnets: <1> | |
| - subnet-1 | |
| - subnet-2 | |
| - subnet-3 | |
| ---- | |
| <1> Add the `subnets` section and specify the `PublicSubnetIds` from the output of the CloudFormation template for the VPC. | |
| subnets: <1> | |
| - publicSubnetId-1 | |
| - publicSubnetId-2 | |
| - publicSubnetId-3 | |
| - privateSubnetId-1 | |
| - privateSubnetId-2 | |
| - privateSubnetId-3 | |
| ---- | |
| <1> Add the `subnets` section and specify the `PrivateSubnetIds` and `PublicSubnetIds` from the outputs of the CloudFormation template for the VPC. You should not include the Local Zone subnets here. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| . Create the machineset for the nodes in your local zone. | |
| .. Export a variable to define the instance type for your control plane machines by running the following command: | |
| . Create the MachineSet manifests for the worker nodes in your Local Zone. | |
| .. Export a variable to define the instance type for the worker machine to deploy on the Local Zone subnet by running the following command: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've updated the step text but retained the styling of "machine set" to match the repo guidelines on API objects.
c5ff9d4 to
9d7b298
Compare
mjpytlak
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@kalexand-rh Just a handful of minor fixes. Well done.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| * You read the link:https://aws.amazon.com/about-aws/global-infrastructure/localzones/features/[Features] for each AWS Local Zones location | |
| * You read the link:https://aws.amazon.com/about-aws/global-infrastructure/localzones/features/[Features] for each AWS Local Zones location. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not a now thing, but let's be sure to coordinate with QE on reference a managed list, similar to that of line 37.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@kalexand-rh thanks for putting the hardcode here. According previous discussion, I tested c5d.2xlarge and t3.xlarge and get passed.
Now we have two options:
- Add
c5d.2xlargeandt3.xlargeto the hardcode. (note,m5.2xlargeis NOT applicable ) - After auto-include list on installer side added, then update this doc to include them.
- 4.13 Add tested x86_64 instance types for AWS Local Zone installer#6774
- 4.12 (not created), will create after PR 6774 get merged
Both are acceptable for me.
cc @mtulio
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi @yunjiang29 , adding only c5d.2xlarge and t3.xlarge, we will exclude:
- all Local Zones available on the regions eu-*, ap-*, me-*, sa-*[us-east-1-{bue,scl}-*], all outside US
- will allow only "t3*" instances on the Los Angeles zones (the first announced Local Zone location with the newest resources, and currently the unique including two zones). t3 is also not recommended for production workloads
If we could add/test m5.2xlarge or c5.2xlarge we could cover all of those Local Zones in the mentioned regions, increasing the coverage to +14 zones in/outside US.
I just exported a new matrix of EC2 offering across those regions/zones: https://gist.github.com/mtulio/c98aa15128a7becb06a372f00d824c42
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@mtulio m5.2xlarge and c5.2xlarge can be added to the list, they have been tested and get passed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is a statement that typically appears in UPI doc. Is the use of installer-provisioned infrastructure intentional?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes. This is IPI not UPI. However, it does use some CF templates to make the VPC and subnets. Unlike the UPI methods, the installation program still stands up all the machines. The VPC and subnet data is added to a custom Kubernetes manifest for the workers to get it done. I think the note still holds.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
General comment for this PR, as I see this approach elsewhere. We typically reference GitHub source for CloudFormation templates [1]. Is an update a day-2 item?
[1] Example: https://raw.githubusercontent.com/openshift/installer/release-4.12/upi/aws/cloudformation/01_vpc.yaml
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes. @mtulio, do you have a separate task to track getting these added to the /installer repo? I can add a subtask to the docs follow-ups story, if you'd like.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@katherinedube I opened a PR last year to minimal changes on the existing CloudFormation templates on installer, but it wasn't reviewed.
The Phase-1 of Local Zones development should be required and has installer changes. I will add it to the same PR as it is part of the same solution:
As the PR of the installer for Phase-1 is taking a long to review, feel free to share the needed to create a separate PR for that. (as we are delivering this now, I think it could be a good idea)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@mjpytlak, do you have a suggestion for who we should tag on the installer team about this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Did you intend the break?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No, I copied so many files for this and did not pay attention to whether or not the editor or the file was doing the breaking. (I mean, I can grant myself an exception, if you'd like. ;) )
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Line break.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| . Confirm that the template components exist: | |
| . Confirm that the template components exist by running the following command: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I shall fix it, but I blame this. We codified that style guidance after, and I missed it in the copied file.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| . Launch the CloudFormation template to create a stack of AWS resources that represent the VPC: | |
| . Launch the CloudFormation template to create a stack of AWS resources that represent the VPC by running the following command: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| . Confirm that the template components exist: | |
| . Confirm that the template components exist by running the following command: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| . Change to the directory that contains the {product-title} installation program and generate the Kubernetes manifests for the cluster: | |
| . Change to the directory that contains the {product-title} installation program and generate the Kubernetes manifests for the cluster by running the following command: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| The steps for performing an installer-provisioned infrastructure installation are provided as an example only. Installing a cluster with VPC you provide requires knowledge of the cloud provider and the installation process of {product-title}. The CloudFormation templates are provided to assist in completing these steps or to help model your own. You are also free to create the required resources through other methods; the templates are just an example. | |
| The steps for performing an installer-provisioned infrastructure installation are provided as an example only. Installing a cluster with a VPC you provide requires knowledge of the cloud provider and the installation process of {product-title}. The CloudFormation templates are provided to assist in completing these steps or to help model your own. You are also free to create the required resources through other methods; the templates are just an example. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This link redirects to https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-welcome.html and best I can tell, this page does not have steps to install the AWS CLI.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm going to use https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html. This link is used in five other places in the repo.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| `${REGION}-wl1-LID-wlz-[1-9]`:: Available Wavelength zones | |
| `${REGION}-wl1-LID-wlz-[1-9]`:: Available Wavelength zones. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| <3> The VPC created resources will belong to. | |
| <3> The VPC ID that the subnet will be created. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@mtulio, do you mean something like "The VPC ID in which the cluster's subnet will be created?"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@kalexand-rh the subnet is part(belongs to) of VPC, so I think something like this could be better: "The VPC ID in which the Local Zone's subnet will be created".
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When removing the unused parameter ClusterInfraId[1], we should set the value of this tag to unmanned.
This should something we need to ship in this version, the in-tree cloud-provider uses those tags to discover the subnets to create the default ingress controller. If the Local Zones subnets is discovered by the ingress operator on the install time, it's expected to the installation to fail. We got this issue on the begging on the research and we are working with upstream to address it on the better way, but the legacy code seems to be untouchable.
Please take a look at [1][2][3].
| - Key: !Join [ "", [ "kubernetes.io/cluster/", !Ref ClusterInfraId ] ] | |
| - Key: !Join [ "", [ "kubernetes.io/cluster/unmanaged" ] ] |
ClusterInfraId:
Type: String
Default: "unmanaged"
Description: ClusterInfraId used to tag required resources
[1] tracking the discover bug https://issues.redhat.com/browse/OCPBUGSM-46513
[2] tracking card for ALB Operator requiring to set this tag https://issues.redhat.com/browse/OCPBUGSM-46513
[3] Upstream discussion: https://kubernetes.slack.com/archives/C718BPBQ8/p1672850778127529
mtulio
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@kalexand-rh @GroceryBoyJr @yunjiang29 LGTM for me. Thanks so much!!! 🎉
/lgtm
57bc8c2 to
af05380
Compare
|
/lgtm |
17a3f39 to
d038a09
Compare
|
New changes are detected. LGTM label has been removed. |
|
@yunjiang29, because Marco agrees that we have addressed your feedback and has validated updates, I am going to merge this PR so that localization can start work on it. If you have additional feedback, please leave it on this PR, and we will incorporate it immediately. |
|
/cherrypick enterprise-4.12 |
|
@kalexand-rh: new pull request created: #54697 DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
|
@kalexand-rh one comment about structures, it looks like the current title level/structure are incorrect: I think we can update to the similar one as Installing a cluster on AWS into an existing VPC chapter:
The final structure would be: cc @mtulio |
|
@mtulio @yunjiang29 I am taking over the final comments for @kalexand-rh and have opened a new PR to address the most recent feedback. [1] @mtulio You can find the latest update for #54535 (comment) in the new PR. [1] #54727 |
mtulio
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@mjpytlak @kalexand-rh suggested changes for #54727
| + | ||
| [source,terminal] | ||
| ---- | ||
| $ aws cloudformation create-stack --stack-name <name> \ <1> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Suggest to change to:
$ aws cloudformation create-stack --stack-name <subnet_stack_name> \ <1>
[1] Section: Creating the Kubernetes manifest files
Procedure.2: Create the machine set manifests for the worker nodes in your Local Zone.
Step: Store the subnet ID as a local variable by running the following command:
| + | ||
| [source,terminal] | ||
| ---- | ||
| $ aws cloudformation describe-stacks --stack-name <name> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Suggest change to to keep consistency on[1]:
$ aws cloudformation describe-stacks --stack-name <subnet_stack_name>
[1] Section: Creating the Kubernetes manifest files
Procedure.2: Create the machine set manifests for the worker nodes in your Local Zone.
Step: Store the subnet ID as a local variable by running the following command:
Status: In Development
Problem: This PR adds a new assembly documenting AWS Local Zone installation Account Information
Built using the assembly installing-aws-user-infra.adoc
Version(s): 4.12
Issue:
AWS Local Zones - Phase 0 IPI - Document options to create compute nodes on Day-0
https://issues.redhat.com/browse/OSDOCS-4347
Link to docs preview:
https://54535--docspreview.netlify.app/openshift-enterprise/latest/installing/installing_aws/installing-aws-localzone.html
Additional information:
Marcos Entenza Garcia PM. QE: Jianli Wei, SME: Marco Braga