Getting Started Guide for Windows Support #1771

jbhurat · 2016-11-22T20:53:41Z

This PR is to add documentation for getting started guide for supporting Windows Server Containers on Kubernetes.
The work to support Windows Server Container is tracked by kubernetes/enhancements#116 and the code changes are already merged and tagged for Release 1.5

This change is

errordeveloper · 2016-11-23T14:42:39Z

Have you tried rendering this? I mostly wonder if it will get added to the site navigation automatically, or you need to do something special? Also, I have very little idea about anything Windows-specific, so I think it needs a different assignee...

errordeveloper · 2016-11-23T14:43:35Z

ping @jaredbhatti

jbhurat · 2016-11-23T20:12:53Z

@errordeveloper I am not sure if it will get added to the site navigation automatically. @jaredbhatti might be able to help here

erictune · 2016-11-24T15:17:22Z

docs/getting-started-guides/windows/README.md

@@ -0,0 +1,163 @@
+# Running Windows Server Containers using Kubernetes
+Microsoft, in collaboration with Docker, is introducing a new feature called Windows Server Containers with Windows Server 2016, which enables Docker containers to run on Windows. We have enhanced Kubernetes to support Windows Server Containers (and Windows Server 2016 as a container host node).


Kubernetes documentation typically does not highlight the work of a specific company, group or individual. (A blog post is a great place for giving credit.)

With that in mind, I'd suggest rewording the first two sentences to something like:

"Kubernetes supports nodes running Microsoft Windows Server 2016. Docker version x.xx is required . Pods are comprised of Windows Server Containers."

You can definitely link from various terms (e.g. Windows Server Containers) to product pages on the Microsoft and Docker websites.

I'd also suggest replacing "we" everywhere in this doc.

Updated as requested

erictune · 2016-11-24T15:19:27Z

docs/getting-started-guides/windows/README.md

+
+Windows Server and .NET still account for a significant portion of the workloads in the enterprise, and with this work, customers will be able to run Windows-based and .NET-based applications inside containers on Kubernetes, making Kubernetes the first fully functional cross-platform cluster manager. Services that span Windows-based and Linux-based containers are now a reality.
+
+Windows Server Containers will be supported on Kubernetes as an alpha feature. Kubernetes control plane (API Server, Scheduler, Controller Manager etc) continue to run on Linux, while the kubelet and kube-proxy can be run on Windows Server.


Is this expected to work with Kubernetes 1.5.0? If so, then say "are" rather than "will be". If not supported in 1.5.0, then we should consider whether this belongs in the 1.5.0 docs.

Updated as requested

erictune · 2016-11-24T15:28:07Z

docs/getting-started-guides/windows/README.md

+The above networking approach is already supported on Linux using a bridge interface, which essentially creates a private network local to the node. Similar to the Windows side, routes to all other pod CIDRs must be created in order to send packets via the “public” NIC.
+
+### Windows
+To support networking requirements, we will need the following configuration on each Kubernetes Windows Server node:


Replace "To support networking requirements, we will need the following configuration on each Kubernetes Windows Server node" with "Each Window Server node should have the following configuration".

Updated as requested

erictune

Thanks for writing this detailed documentation! A couple of comments.

jbhurat · 2016-11-26T00:11:15Z

Thanks for the review @erictune . I have updated the documentation as requested

jbhurat · 2016-11-28T18:31:33Z

@erictune Can you please review the changes and let me know if your concerns are addressed

devin-donnelly · 2016-12-01T21:39:33Z

docs/getting-started-guides/windows/README.md

@@ -0,0 +1,159 @@
+# Running Windows Server Containers using Kubernetes
+Windows Server Containers are supported on Kubernetes as an alpha feature. Kubernetes control plane (API Server, Scheduler, Controller Manager etc) will continue to run on Linux, while the kubelet and kube-proxy can be run on Windows Server.


Reference a specific version; it helps the documentation from being rendered obsolete.

"Kubernetes version 1.5 introduces support for Windows Server Containers. In version 1.5, the Kubernetes control plane (API Server, Scheduler, Controller Manager, etc) continue to run on Linux, while the kubelet and kube-proxy can be run on Windows Server.

Note: Windows Server Containers on Kubernetes is an Alpha feature in Kubernetes 1.5."

Thanks for the review @devin-donnelly. I am making the suggested changes

devin-donnelly · 2016-12-01T21:40:01Z

docs/getting-started-guides/windows/README.md

+Windows Server Containers are supported on Kubernetes as an alpha feature. Kubernetes control plane (API Server, Scheduler, Controller Manager etc) will continue to run on Linux, while the kubelet and kube-proxy can be run on Windows Server.
+
+## Prerequisites
+With the alpha release, Windows Server Containers for Kubernetes is supported using the following:


"With the alpha release," -> "In Kubernetes version 1.5,"

Thanks for the review @devin-donnelly. I am making the suggested changes

devin-donnelly · 2016-12-01T21:46:52Z

docs/getting-started-guides/windows/README.md

+The following diagram illustrates the Windows Server networking setup for Kubernetes Setup
+![Windows Setup](windows-setup.png)
+
+## Setup


Three successive nested headings is indicative of major structural problems.

Better structure:

Setting up Windows Server Containers on Kubernetes

To run Windows Server Containers on Kubernetes, you'll need to set up both your host machines and the Kubernetes node components for Windows. The setup requirements for Windows Server Containers vary depending on whether or not you're using Windows or Linux Hosts.

Windows Host Setup

Steps

Linux Host Setup (editorial note: Why? I thought the whole point of this document was how to do Windows node setups).

Steps

Component Setup

Requirements

The reason I have 3 successive nested headings is because Setup have 3 types Host, Component and Route. Host and Route instructions are different for Windows and Linux Hosts and Components setup have instructions for building kubelet and kube-proxy for Windows. So as each Setup type have 2 more child elements I thought it is easier to follow if it is nested headings.

Setting up Windows Server Containers on Kubernetes

To run Windows Server Containers on Kubernetes, you'll need to set up both your host machines and the Kubernetes node components for Windows and setup Routes for Pod communication on different nodes

Host Setup

Windows Host Setup

Windows Server container host running Windows Server 2016 and Docker v1.12. Follow the setup instructions outlined by this blog post: https://msdn.microsoft.com/en-us/virtualization/windowscontainers/quick_start/quick_start_windows_server

DNS support for Windows recently got merged to docker master and is currently not supported in a stable docker release. To use DNS build docker from master or download the binary from Docker master

Pull the apprenda/pause image from https://hub.docker.com/r/apprenda/pause

RRAS (Routing) Windows feature enabled

Linux Host Setup

Linux hosts should be setup according to their respective distro documentation and the requirements of the Kubernetes version you will be using.

CNI network plugin installed.

Component Setup

Requirements

Git, Go 1.7.1+

make (if using Linux or MacOS)

Important notes and other dependencies are listed here

kubelet
To build the kubelet, run:

cd $GOPATH/src/k8s.io/kubernetes

Build kubelet

Linux/MacOS: KUBE_BUILD_PLATFORMS=windows/amd64 make WHAT=cmd/kubelet

Windows: go build cmd/kubelet/kubelet.go

kube-proxy
To build kube-proxy, run:

cd $GOPATH/src/k8s.io/kubernetes

Build kube-proxy

Linux/MacOS: KUBE_BUILD_PLATFORMS=windows/amd64 make WHAT=cmd/kube-proxy

Windows: go build cmd/kube-proxy/proxy.go

Route Setup

The below example setup assumes one Linux and two Windows Server 2016 nodes and a cluster CIDR 192.168.0.0/16

Hostname Routable IP address Pod CIDR

Lin01 <IP of Lin01 host> 192.168.0.0/24

Win01 <IP of Win01 host> 192.168.1.0/24

Win02 <IP of Win02 host> 192.168.2.0/24

Lin01

ip route add 192.168.1.0/24 via <IP of Win01 host> ip route add 192.168.2.0/24 via <IP of Win02 host>

Is this acceptable resolution

Yes, that's good.

devin-donnelly · 2016-12-01T21:47:41Z

docs/getting-started-guides/windows/README.md

+
+#### kubelet
+
+In order to build the *kubelet*, run:


"In order" not necessary.

Thanks for the review @devin-donnelly. I am making the suggested changes

devin-donnelly · 2016-12-01T21:47:55Z

docs/getting-started-guides/windows/README.md

+Linux hosts should be setup according to their respective distro documentation and the requirements of the Kubernetes version you will be using. Linux hosts also require to have CNI Setup.
+
+### Component Setup
+In order to build the work node components, the *kubelet* and *kube-proxy* for Windows, the following needs to be installed on the host


"In order" not necessary.

Thanks for the review @devin-donnelly. I am making the suggested changes

devin-donnelly · 2016-12-01T21:52:56Z

docs/getting-started-guides/windows/README.md

+## Linux
+Use your preferred method to start Kubernetes cluster on Linux. Please note that Cluster CIDR might need to be updated.
+## Windows
+### kubelet


Nested headings. Replace "### kubelet" with "To start kubelet on your Windows node:"

I am accepting the suggestions and making the required changes

devin-donnelly · 2016-12-01T21:53:25Z

docs/getting-started-guides/windows/README.md

+2. Run *kubelet* executable using the below command
+`kubelet.exe --hostname-override=<ip address/hostname of the windows node>  --pod-infra-container-image="apprenda/pause" --resolv-conf="" --api_servers=<api server location>`
+
+### kube-proxy


Replace "### kube-proxy" with "To start kube-proxy on your Windows node:"

I am accepting the suggestions and making the required changes

devin-donnelly · 2016-12-01T21:54:45Z

docs/getting-started-guides/windows/README.md

+2. Run *kube-proxy* executable using the below command
+`.\proxy.exe --v=3 --proxy-mode=userspace --hostname-override=<ip address/hostname of the windows node> --master=<api server location> --bind-address=<ip address of the windows node>`
+
+## Scheduling PODs on Windows


"Pod" should not be all-caps. It's not an acronym.

I am accepting the suggestions and making the required changes

devin-donnelly · 2016-12-01T21:55:12Z

docs/getting-started-guides/windows/README.md

+`.\proxy.exe --v=3 --proxy-mode=userspace --hostname-override=<ip address/hostname of the windows node> --master=<api server location> --bind-address=<ip address of the windows node>`
+
+## Scheduling PODs on Windows
+As Kubernetes control plane currently runs on Linux, the resulting cluster will have both Linux and Windows nodes. TO schedule PODs on Windows, `nodeSelector` constraint has to be set with label `beta.kubernetes.io/os` having value `windows` as shown in the example below,


Neither "TO" nor "POD" should be all-caps.

I am accepting the suggestions and making the required changes

devin-donnelly · 2016-12-01T21:56:48Z

docs/getting-started-guides/windows/README.md

+`.\proxy.exe --v=3 --proxy-mode=userspace --hostname-override=<ip address/hostname of the windows node> --master=<api server location> --bind-address=<ip address of the windows node>`
+
+## Scheduling PODs on Windows
+As Kubernetes control plane currently runs on Linux, the resulting cluster will have both Linux and Windows nodes. TO schedule PODs on Windows, `nodeSelector` constraint has to be set with label `beta.kubernetes.io/os` having value `windows` as shown in the example below,


Suggested rephrase for clarity: "Because your cluster has both Linux and Windows nodes, you must explictly set the nodeSelector constraint to be able to schedule Pods to Windows nodes. You must set nodeSelector with the label beta.kubernetes.io/os to the value windows; see the following example:

I am accepting the suggestions and making the required changes

jbhurat · 2016-12-02T19:17:44Z

Thanks for your review @devin-donnelly. I have made the suggested chances.

devin-donnelly · 2016-12-02T21:06:39Z

Awesome, @jbhurat . I'll give this a docs LGTM; I'll let @erictune handle tech LGTM, and then we can get this merged.

erictune · 2016-12-02T22:07:19Z

Thanks for your patience on my review. This looks great. Excited to see this new way to use Kubernetes and Windows!

LGTM.

Adding Getting Started Guide for Windows Support

57a99a8

k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Nov 22, 2016

k8s-github-robot assigned errordeveloper Nov 22, 2016

Added a sample POD spec and added kubenet requirements

2818da2

erictune reviewed Nov 24, 2016

View reviewed changes

erictune suggested changes Nov 24, 2016

View reviewed changes

Updated based on review comments

c6b2604

devin-donnelly assigned erictune Dec 1, 2016

devin-donnelly added Needs Tech Review cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. and removed cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Dec 1, 2016

devin-donnelly suggested changes Dec 1, 2016

View reviewed changes

devin-donnelly added the Docs Review: Open Issues label Dec 1, 2016

devin-donnelly approved these changes Dec 2, 2016

View reviewed changes

jbhurat added 2 commits December 2, 2016 14:12

Cosmetic changes as per review

de30aba

Changes as per review

77fd224

devin-donnelly added Docs LGTM and removed Docs Review: Open Issues labels Dec 2, 2016

erictune added Tech Review LGTM and removed Needs Tech Review labels Dec 2, 2016

erictune approved these changes Dec 2, 2016

View reviewed changes

devin-donnelly merged commit 9eaaa14 into kubernetes:release-1.5 Dec 2, 2016

		@@ -0,0 +1,163 @@
		# Running Windows Server Containers using Kubernetes
		Microsoft, in collaboration with Docker, is introducing a new feature called Windows Server Containers with Windows Server 2016, which enables Docker containers to run on Windows. We have enhanced Kubernetes to support Windows Server Containers (and Windows Server 2016 as a container host node).


		Windows Server and .NET still account for a significant portion of the workloads in the enterprise, and with this work, customers will be able to run Windows-based and .NET-based applications inside containers on Kubernetes, making Kubernetes the first fully functional cross-platform cluster manager. Services that span Windows-based and Linux-based containers are now a reality.

		Windows Server Containers will be supported on Kubernetes as an alpha feature. Kubernetes control plane (API Server, Scheduler, Controller Manager etc) continue to run on Linux, while the kubelet and kube-proxy can be run on Windows Server.

		@@ -0,0 +1,159 @@
		# Running Windows Server Containers using Kubernetes
		Windows Server Containers are supported on Kubernetes as an alpha feature. Kubernetes control plane (API Server, Scheduler, Controller Manager etc) will continue to run on Linux, while the kubelet and kube-proxy can be run on Windows Server.

Hostname	Routable IP address	Pod CIDR
Lin01	`<IP of Lin01 host>`	192.168.0.0/24
Win01	`<IP of Win01 host>`	192.168.1.0/24
Win02	`<IP of Win02 host>`	192.168.2.0/24

Getting Started Guide for Windows Support #1771

Getting Started Guide for Windows Support #1771

Conversation

jbhurat commented Nov 22, 2016 • edited by thockin Loading

errordeveloper commented Nov 23, 2016

errordeveloper commented Nov 23, 2016

jbhurat commented Nov 23, 2016

erictune Nov 24, 2016 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

erictune left a comment

Choose a reason for hiding this comment

jbhurat commented Nov 26, 2016

jbhurat commented Nov 28, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Setting up Windows Server Containers on Kubernetes

Windows Host Setup

Linux Host Setup (editorial note: Why? I thought the whole point of this document was how to do Windows node setups).

Component Setup

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Setting up Windows Server Containers on Kubernetes

Host Setup

Component Setup

Route Setup

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jbhurat commented Dec 2, 2016

devin-donnelly commented Dec 2, 2016

erictune commented Dec 2, 2016

jbhurat commented Nov 22, 2016 •

edited by thockin

Loading

erictune Nov 24, 2016 •

edited

Loading