Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@

## Sign the CLA

Kubernetes projects require that you sign a Contributor License Agreement (CLA) before we can accept your pull requests. Please see https://git.k8s.io/community/CLA.md for more info
Kubernetes projects require that you sign a Contributor License Agreement (CLA) before we can accept your pull requests. Please see https://git.k8s.io/community/CLA.md for more info

### Contributing A Patch

Expand Down
10 changes: 1 addition & 9 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -197,7 +197,6 @@ generate-examples: clean-examples ## Generate examples configurations to run a c
.PHONY: docker-build
docker-build: ## Build the docker image for controller-manager
docker build --pull --build-arg ARCH=$(ARCH) . -t $(CONTROLLER_IMG)-$(ARCH):$(TAG)
MANIFEST_IMG=$(CONTROLLER_IMG)-$(ARCH) MANIFEST_TAG=$(TAG) $(MAKE) set-manifest-image

.PHONY: docker-push
docker-push: ## Push the docker image
Expand Down Expand Up @@ -226,12 +225,6 @@ docker-push-manifest: ## Push the fat manifest docker image.
docker manifest create --amend $(CONTROLLER_IMG):$(TAG) $(shell echo $(ALL_ARCH) | sed -e "s~[^ ]*~$(CONTROLLER_IMG)\-&:$(TAG)~g")
@for arch in $(ALL_ARCH); do docker manifest annotate --arch $${arch} ${CONTROLLER_IMG}:${TAG} ${CONTROLLER_IMG}-$${arch}:${TAG}; done
docker manifest push --purge $(CONTROLLER_IMG):$(TAG)
MANIFEST_IMG=$(CONTROLLER_IMG) MANIFEST_TAG=$(TAG) $(MAKE) set-manifest-image

.PHONY: set-manifest-image
set-manifest-image:
$(info Updating kustomize image patch file for manager resource)
sed -i'' -e 's@image: .*@image: '"${MANIFEST_IMG}:$(MANIFEST_TAG)"'@' ./config/default/manager_image_patch.yaml

## --------------------------------------
## Release
Expand All @@ -252,8 +245,7 @@ release: clean-release ## Builds and push container images using the latest git
REGISTRY=$(STAGING_REGISTRY) TAG=$(RELEASE_TAG) \
$(MAKE) docker-build-all docker-push-all
# Set the manifest image to the production bucket.
MANIFEST_IMG=$(PROD_REGISTRY)/$(IMAGE_NAME) MANIFEST_TAG=$(RELEASE_TAG) \
$(MAKE) set-manifest-image
MANIFEST_IMG=$(PROD_REGISTRY)/$(IMAGE_NAME) MANIFEST_TAG=$(RELEASE_TAG)
$(MAKE) release-manifests

.PHONY: release-manifests
Expand Down
234 changes: 95 additions & 139 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,192 +1,148 @@
<!-- START doctoc generated TOC please keep comment here to allow auto update -->
<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
**Table of Contents** *generated with [DocToc](https://github.com/thlorenz/doctoc)*
# Kubernetes Cluster API Provider OpenStack

- [Kubernetes cluster-api-provider-openstack Project](#kubernetes-cluster-api-provider-openstack-project)
- [Community, discussion, contribution, and support](#community-discussion-contribution-and-support)
- [Code of conduct](#code-of-conduct)
- [Compatibility with Cluster API, Kubernetes and OpenStack Versions](#compatibility-with-cluster-api-kubernetes-and-openstack-versions)
- [Getting Started](#getting-started)
- [Prerequisites](#prerequisites)
- [Cluster Creation](#cluster-creation)
- [Managed OpenStack Security Groups](#managed-openstack-security-groups)
- [Interacting with your cluster](#interacting-with-your-cluster)
- [Cluster Deletion](#cluster-deletion)
- [Trouble shooting](#trouble-shooting)
- [Contributing](#contributing)
[![Go Report Card](https://goreportcard.com/badge/kubernetes-sigs/cluster-api-provider-openstack)](https://goreportcard.com/report/kubernetes-sigs/cluster-api-provider-openstack)

<!-- END doctoc generated TOC please keep comment here to allow auto update -->
------

# Kubernetes cluster-api-provider-openstack Project
Kubernetes-native declarative infrastructure for OpenStack.

This repository hosts a concrete implementation of an OpenStack provider for the [cluster-api project](https://github.com/kubernetes-sigs/cluster-api).
## What is the Cluster API Provider OpenStack

## Community, discussion, contribution, and support
The [Cluster API][cluster_api] brings
declarative, Kubernetes-style APIs to cluster creation, configuration and
management.

Learn how to engage with the Kubernetes community on the [community page](http://kubernetes.io/community/).
The API itself is shared across multiple cloud providers allowing for true OpenStack
hybrid deployments of Kubernetes. It is built atop the lessons learned from
previous cluster managers such as [kops][kops] and
[kubicorn][kubicorn].

You can reach the maintainers of this project at:

- [#cluster-api-openstack on Kubernetes Slack](https://kubernetes.slack.com/messages/cluster-api-openstack)
- [SIG-Cluster-Lifecycle Mailing List](https://groups.google.com/forum/#!forum/kubernetes-sig-cluster-lifecycle)
## Launching a Kubernetes cluster on OpenStack

### Code of conduct
Check out the [getting started guide](./docs/getting-started.md) for launching a cluster on OpenStack.

Participation in the Kubernetes community is governed by the [Kubernetes Code of Conduct](code-of-conduct.md).
## Features

- Native Kubernetes manifests and API
- Choice of Linux distribution (as long as a current cloud-init is available)
- Support for single and multi-node control plane clusters
- Deploy clusters with and without LBaaS available
- Support for security groups
- Doesn't use SSH for bootstrapping nodes

------

## Compatibility with Cluster API, Kubernetes and OpenStack Versions
## Compatibility with Cluster API and Kubernetes Versions

This provider's versions are compatible with the following versions of Cluster API:

||Cluster API v1alpha1 (v0.1)|
|-|-|
|OpenStack Provider v1alpha1 (ea309e7f)|✓|
||Cluster API v1alpha1 (v0.1)|Cluster API v1alpha2 (v0.2)|
|-|-|-|
|OpenStack Provider v1alpha1 (release-0.1 branch)|✓||
|OpenStack Provider v1alpha2 (v0.2)||✓|

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we say master instead of v0.2? or v0.2 branch?

@sbueringer sbueringer Sep 24, 2019

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because I wanted to release 0.2.0 based on v1alpha2 and we shouldn't branch away a release-0.2 before v1alpha3 I would like to keep 0.2 (see also: https://github.com/kubernetes-sigs/cluster-api-provider-aws#compatibility-with-cluster-api-and-kubernetes-versions (their list is not up-to-date))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, then why we need v0.2 ? as master actually works fine and we can have release-0.2 when v1alpha3 out..

@sbueringer sbueringer Sep 25, 2019

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0.2 stands for the tags 0.2.0, 0.2.1, etc.. I think it's more precise to say CAPO 0.2 then something on the master branch. And it's the same as CAPA is doing. If we don't want to release then master would be the only option. But I think when we're doing releases it's far better to tell our users which release supports what, so they don't have to figure out which release comes from which branch.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0.2 stands for the tags make sense to me.


This provider's versions are able to install and manage the following versions of Kubernetes:

||Kubernetes 1.13.5+|Kubernetes 1.14|Kubernetes 1.15|
||Kubernetes 1.13|Kubernetes 1.14|Kubernetes 1.15|
|-|-|-|-|
|OpenStack Provider v1alpha1 (ea309e7f)|✓|✓|✓|

Kubernetes control plane and Kubelet versions are defined in `spec.versions.controlPlane` and `spec.versions.kubelet` of `cmd/clusterctl/examples/openstack/machines.yaml.template` respectively.
You can generate `cmd/clusterctl/examples/openstack/out/machines.yaml` by running the `generate-yaml.sh` from the template and change the versions if you want.

**NOTE**: Because the user is able to customize any `user-data`, it is also possible to deploy older versions.
But we won't provide any examples or working templates. See [user-data in the examples](https://github.com/kubernetes-sigs/cluster-api-provider-openstack/tree/master/cmd/clusterctl/examples/openstack/provider-component/user-data).
|OpenStack Provider v1alpha1 (release-0.1 branch)|✓|✓|✓|
|OpenStack Provider v1alpha2 (v0.2)|||✓|

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

master branch or v0.2 branch will be better? v0.2 seems a tag..

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above 0.2 stands for all our 0.2.x releases (which will be tags)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto..


This provider's versions are able to install kubernetes to the following versions of OpenStack:
This provider's versions are able to install Kubernetes to the following versions of OpenStack:

||OpenStack Pike|OpenStack Queens|OpenStack Rocky|OpenStack Stein|
|-|-|-|-|-|
|OpenStack Provider v1alpha1 (ea309e7f)|✓|✓|✓|✓|
|OpenStack Provider v1alpha1 (release-0.1 branch)|✓|✓|✓|✓|
|OpenStack Provider v1alpha2 (v0.2)|+|✓|+|✓|

Key:
* `✓` tested
* `+` should work, but we weren't able to test it

Each version of Cluster API for OpenStack will attempt to support two Kubernetes versions.

**NOTE:** As the versioning for this project is tied to the versioning of Cluster API, future modifications to this
policy may be made to more closely align with other providers in the Cluster API ecosystem.
policy may be made to more closely aligned with other providers in the Cluster API ecosystem.

------

## Getting Started

### Notice
Currently `cluster-api-provider-openstack` project is evolving into `cluster-api v1alpha2`, please use `release-0.1` branch for `cluster-api v1alpha1` development as it provides function workable code and configurations.

For more information, please refer to [v1alpha2](https://github.com/kubernetes-sigs/cluster-api-provider-openstack/issues/380)

### Prerequisites
## Operating system images

1. Install `kustomize` v3.1.0+ (see [here](https://github.com/kubernetes-sigs/kustomize/releases).
2. You can use either VM, container or existing Kubernetes cluster act as bootstrap cluster.
- If you want to use VM, install [minikube](https://kubernetes.io/docs/tasks/tools/install-minikube/), version 0.30.0 or greater.
- If you want to use container, install [kind](https://github.com/kubernetes-sigs/kind#installation-and-usage).
- If you want to use existing Kubernetes cluster, prepare your kubeconfig.
3. Install a [driver](https://github.com/kubernetes/minikube/blob/master/docs/drivers.md) **if you are using Minikube**. For Linux, we recommend kvm2. For MacOS, we recommend VirtualBox.
4. An appropriately configured [Go development environment](https://golang.org/doc/install)
5. Build the `clusterctl` tool and make it available in your `PATH`
Note: Cluster API Provider OpenStack relies on a few prerequisites which have to be already
installed in the used operating system images, e.g. a container runtime, kubelet, kubeadm,.. .
Reference images can be found in [kubernetes-sigs/image-builder](https://github.com/kubernetes-sigs/image-builder/tree/master/images/capi). If it isn't possible to pre-install those
prerequisites in the image, you can always deploy and execute some custom scripts
through the [KubeadmConfig](https://github.com/kubernetes-sigs/cluster-api-bootstrap-provider-kubeadm).

```bash
git clone https://github.com/kubernetes-sigs/cluster-api ${GOPATH}/src/sigs.k8s.io/cluster-api
cd ${GOPATH}/src/sigs.k8s.io/cluster-api/
make clusterctl
```

### Cluster Creation

1. Create the YAML files if needed. You can use the `examples/generate.sh` script as documented [here](examples/README.md).

2. Create a cluster:
- If you are using Minikube:

```bash
clusterctl create cluster \
--bootstrap-type minikube --bootstrap-flags kubernetes-version=v1.15.0 \
-c examples/_out/cluster.yaml \
-m examples/_out/machines.yaml \
-p examples/_out/provider-components.yaml \
-a examples/addons.yaml
```

To choose a specific Minikube driver, please use the `--bootstrap-flags vm-driver=xxx` command line parameter. For example to use the kvm2 driver with clusterctl you would add `--bootstrap-flags vm-driver=kvm2`, for linux, if you haven't installed any driver, you can add `--bootstrap-flags vm-driver=none`.

- If you are using Kind:

```bash
clusterctl create cluster \
--bootstrap-type kind --bootstrap-cluster-cleanup=false \
-c examples/_out/cluster.yaml \
-m examples/_out/machines.yaml \
-p examples/_out/provider-components.yaml \
-a examples/addons.yaml
# Alternatively
make create-cluster
```
------

- If you are using an existing Kubernetes cluster:
## Documentation

```bash
clusterctl create cluster \
--bootstrap-cluster-kubeconfig ~/.kube/config \
-c examples/_out/cluster.yaml \
-m examples/_out/machines.yaml \
-p examples/_out/provider-components.yaml \
-a examples/addons.yaml
```
Documentation is in the `/docs` directory

### Interacting with your cluster
## Getting involved and contributing

Once you have created a cluster, you can interact with the cluster and machine resources using kubectl:
Are you interested in contributing to cluster-api-provider-openstack? We, the
maintainers and community, would love your suggestions, contributions, and help!
Also, the maintainers can be contacted at any time to learn more about how to get
involved via the [cluster-api-openstack channel on Kubernetes Slack][slack] or the
[SIG-Cluster-Lifecycle Mailing List](https://groups.google.com/forum/#!forum/kubernetes-sig-cluster-lifecycle).

```bash
kubectl --kubeconfig=kubeconfig get clusters
kubectl --kubeconfig=kubeconfig get machines
kubectl --kubeconfig=kubeconfig get machines -o yaml
```
In the interest of getting more new people involved we try to tag issues with
[`good first issue`][good_first_issue].
These are typically issues that have smaller scope but are good ways to start
to get acquainted with the codebase.

### Cluster Deletion
We also encourage ALL active community participants to act as if they are
maintainers, even if you don't have "official" write permissions. This is a
community effort, we are here to serve the Kubernetes community. If you have an
active interest and you want to get involved, you have real power! Don't assume
that the only people who can get things done around here are the "maintainers".

Use following command to delete a cluster and all resources it created.
```bash
clusterctl delete cluster --cluster <cluster-name> --bootstrap-type kind --kubeconfig kubeconfig --provider-components examples/_out/provider-components.yaml
```
We also would love to add more "official" maintainers, so show us what you can
do!

Or you can manually delete all resources that were created as part of
your openstack Cluster API Kubernetes cluster.
This repository uses the Kubernetes bots. See a full list of the commands [here][prow].
Please also refer to the [Contribution Guide](CONTRIBUTING.md) and the [Development Guide](docs/development.md) for this project.

1. Delete all of the node Machines in the cluster. Make sure to wait for the
corresponding Nodes to be deleted before moving onto the next step. After this
step, the master node will be the only remaining node.
## Code of conduct

```bash
kubectl --kubeconfig=kubeconfig delete machines -l set=node
kubectl --kubeconfig=kubeconfig get nodes
```
Participation in the Kubernetes community is governed by the [Kubernetes Code of Conduct](code-of-conduct.md).

2. Delete the master machine.
```bash
kubectl --kubeconfig=kubeconfig delete machines -l set=master
```
## Github issues

3. (optional) Delete the load balancer in your OpenStack cloud if you created them.
### Bugs

4. Delete the kubeconfig file that were created for your cluster.
If you think you have found a bug please follow the instructions below.

```bash
rm kubeconfig
```
- Please spend a small amount of time giving due diligence to the issue tracker. Your issue might be a duplicate.
- Get the logs from the cluster controllers. Please paste this into your issue.
- Open a [new issue][new_issue].
- Remember that users might be searching for your issue in the future, so please give it a meaningful title to help others.
- Feel free to reach out to the Cluster API community on the [Kubernetes Slack][slack].

5. Delete the ssh keypair that were created for your cluster machine.
### Tracking new features

```bash
rm -rf $HOME/.ssh/openstack_tmp*
```
We also use the issue tracker to track features. If you have an idea for a feature, or think you can help Cluster API Provider OpenStack become even more awesome follow the steps below.

### Troubleshooting
- Open a [new issue][new_issue].
- Remember that users might be searching for your issue in the future, so please
give it a meaningful title to help others.
- Clearly define the use case, using concrete examples.
- Some of our larger features will require some design. If you would like to
include a technical design for your feature, please include it in the issue.
- After the new feature is well understood, and the design agreed upon, we can
start coding the feature. We would love for you to code it. So please open
up a **WIP** *(work in progress)* pull request, and happy coding.

Please refer to [Trouble shooting documentation](docs/trouble_shooting.md) for further info.

## Contributing
<!-- References -->

Please refer to the [Contribution Guide](CONTRIBUTING.md) and [Development Guide](docs/development.md) for this project.
[cluster_api]: https://github.com/kubernetes-sigs/cluster-api
[kops]: https://github.com/kubernetes/kops
[kubicorn]: http://kubicorn.io/
[slack]: https://kubernetes.slack.com/messages/cluster-api-openstack
[good_first_issue]: https://github.com/kubernetes-sigs/cluster-api-provider-openstack/issues?q=is%3Aissue+is%3Aopen+sort%3Aupdated-desc+label%3A%22good+first+issue%22
[prow]: https://go.k8s.io/bot-commands
[new_issue]: https://github.com/kubernetes-sigs/cluster-api-provider-openstack/issues/new
47 changes: 43 additions & 4 deletions RELEASE.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,48 @@
# Release Process

The Kubernetes cluster-api-provider-openstack is released on an as-needed basis. The process is as follows:
# Releasing

## Output

### Expected artifacts

1. A container image of the `cluster-api-provider-openstack` controller manager

### Artifact locations

1. The container image is found in the registry `us.gcr.io/k8s-artifacts-prod/capi-openstack/` with an image
name of `capi-openstack-controller` and a tag that matches the release version. For
example, in the `v0.2.0` release, the container image location is
`us.gcr.io/k8s-artifacts-prod/capi-openstack/capi-openstack-controller:v0.2.0`


## Process

For version v0.x.y:

1. An issue is proposing a new release with a changelog since the last release
1. All [OWNERS](OWNERS) must LGTM this release
1. An OWNER runs `git tag -s $VERSION` and inserts the changelog and pushes the tag with `git push $VERSION`
1. Create an annotated tag `git tag -a v0.x.y -m v0.x.y`
1. To use your GPG signature when pushing the tag, use `git tag -s [...]` instead
1. Push the tag to the GitHub repository `git push origin v0.x.y`
1. Note: `origin` should be the name of the remote pointing to `github.com/kubernetes-sigs/cluster-api-provider-openstack`
1. Run `make release` to build artifacts and push the images to the staging bucket
1. Follow the [Image Promotion process](https://github.com/kubernetes/k8s.io/tree/master/k8s.gcr.io#image-promoter) to promote the image from the staging repo to `us.gcr.io/k8s-artifacts-prod/capi-openstack`
1. Create a release (with the above mentioned release notes) in GitHub based on the tag created above
1. The release issue is closed
1. An announcement email is sent to `kubernetes-dev@googlegroups.com` with the subject `[ANNOUNCE] cluster-api-provider-openstack $VERSION is released`

<!-- TODO add link to image promote PR after the first release -->

### Permissions

Releasing requires a particular set of permissions.

* Push access to the staging gcr bucket ([kubernetes/k8s.io/k8s.gcr.io/k8s-staging-capi-openstack/OWNERS](https://github.com/kubernetes/k8s.io/blob/master/k8s.gcr.io/k8s-staging-capi-openstack/OWNERS)
* Tag push access to the GitHub repository ([kubernetes/org/config/kubernetes-sigs/sig-cluster-lifecycle/teams.yaml](https://github.com/kubernetes/org/blob/master/config/kubernetes-sigs/sig-cluster-lifecycle/teams.yaml#L136-L137))
* GitHub release creation access ([kubernetes/org/config/kubernetes-sigs/sig-cluster-lifecycle/teams.yaml](https://github.com/kubernetes/org/blob/master/config/kubernetes-sigs/sig-cluster-lifecycle/teams.yaml#L136-L137))

## Staging

There is a post-submit Prow job running after each commit on master which pushes a new image to the staging repo (`gcr.io/k8s-staging-capi-openstack/capi-openstack-controller:latest`). Following configuration is involved:
* staging gcr bucket: [kubernetes/k8s.io/k8s.gcr.io/k8s-staging-capi-openstack/manifest.yaml](https://github.com/kubernetes/k8s.io/blob/master/k8s.gcr.io/k8s-staging-capi-openstack/manifest.yaml)
* post-submit `post-capi-openstack-push-images` Prow job: [kubernetes/test-infra/config/jobs/image-pushing/k8s-staging-capi-openstack.yaml](https://github.com/kubernetes/test-infra/blob/master/config/jobs/image-pushing/k8s-staging-capi-openstack.yaml)) (corresponding dashboard is located at [https://testgrid.k8s.io/sig-cluster-lifecycle-image-pushes#post-capi-openstack-push-images](https://testgrid.k8s.io/sig-cluster-lifecycle-image-pushes#post-capi-openstack-push-images))
* Google Cloud Build configuration which is used by the Prow job: [kubernetes-sigs/cluster-api-provider-openstack/cloudbuild.yaml](https://github.com/kubernetes-sigs/cluster-api-provider-openstack/blob/master/cloudbuild.yaml)
Loading