Skip to content

Explain use of pod os field #35439

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

ravisantoshgudimetla
Copy link
Contributor

Identify OS authoritatively during pod admission time using pod.OS field

@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. area/blog Issues or PRs related to the Kubernetes Blog subproject labels Jul 27, 2022
@k8s-ci-robot k8s-ci-robot added language/en Issues or PRs related to English language sig/docs Categorizes an issue or PR as relevant to SIG Docs. labels Jul 27, 2022
@netlify
Copy link

netlify bot commented Jul 27, 2022

Pull request preview available for checking

Built without sensitive environment variables

Name Link
🔨 Latest commit 266d6c3
🔍 Latest deploy log https://app.netlify.com/sites/kubernetes-io-main-staging/deploys/63eebba2e5bf40000807dcd1
😎 Deploy Preview https://deploy-preview-35439--kubernetes-io-main-staging.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site settings.

@sftim
Copy link
Contributor

sftim commented Jul 27, 2022

/retitle [WIP] Explain use of pod os field

@k8s-ci-robot k8s-ci-robot changed the title pod os field [WIP] Explain use of pod os field Jul 27, 2022
@k8s-ci-robot k8s-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jul 27, 2022
@k8s-ci-robot k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Aug 12, 2022
@katcosgrove
Copy link
Contributor

Hi from the Comms team! Just a reminder that the Ready to Review deadline for feature blogs is Tuesday, August 16. You will also be assigned a publication date post-release. Is there anything we can do to help you right now?

onto the container runtime, it'd be better to identify the OS early in the life-cycle of pod and use the
identification mechanism consistently across all kubernetes components including `kubelet`.
With that in mind, `OS` field has been added to the pod spec in 1.23 release of kubernetes as an
alpha feature and it graduated to stable in 1.25.`PodSecurity` admission plugin has been updated to use the `OS` field.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
alpha feature and it graduated to stable in 1.25.`PodSecurity` admission plugin has been updated to use the `OS` field.
alpha feature and it graduated to stable in 1.25. In addition, the `PodSecurity` admission plugin has been updated to use the `OS` field.

Or something like that

Mabye a link to the admission plugin?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The field is os. In the Golang code the capitalization is different, but to a Kubernetes end user the field name is all-lowercase.


## What problems does this solve?

Today, the `kube-apiserver` and `kubelet` while admitting pods have no notion of the OS on which the pod can run.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Today, the `kube-apiserver` and `kubelet` while admitting pods have no notion of the OS on which the pod can run.
Today, at pod admission time the `kube-apiserver` and `kubelet` have no notion of the OS on which the pod can run.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Today, the `kube-apiserver` and `kubelet` while admitting pods have no notion of the OS on which the pod can run.
In the first releases of Kubernetes the `kube-apiserver` did not have record any detail
about which operating system a Pod should run on. Initially, Kubernetes only supported Linux
nodes; later, Kubernetes added support for Windows nodes (stable since Kubernetes v1.14).
You could use [labels](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/)
to give the scheduler a hint about where to place your Pod, but the actual Pod API didn't track that
OS at all.

Copy link
Contributor

@sftim sftim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some early feedback.

/hold
as valid a publication date isn't assigned yet

@@ -0,0 +1,50 @@
# Identify Pod Operating System authoritatively during pod admission time
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# Identify Pod Operating System authoritatively during pod admission time

layout: blog
title: "Identifying pod operating system authoritatively during pod admission time"
date: 2022-07-27
slug: Identifying pod operating system authoritatively during pod admission time
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
slug: Identifying pod operating system authoritatively during pod admission time
slug: pod-os-field-explained


**Authors:** Ravi Gudimetla (Apple)

This blog describes how to identify pod's operating system authoritatively using the `OS` field in the pod spec and how it
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
This blog describes how to identify pod's operating system authoritatively using the `OS` field in the pod spec and how it
This article describes how to identify pod's operating system authoritatively using the `os` field in the pod spec, and how that


## What problems does this solve?

Today, the `kube-apiserver` and `kubelet` while admitting pods have no notion of the OS on which the pod can run.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Today, the `kube-apiserver` and `kubelet` while admitting pods have no notion of the OS on which the pod can run.
In the first releases of Kubernetes the `kube-apiserver` did not have record any detail
about which operating system a Pod should run on. Initially, Kubernetes only supported Linux
nodes; later, Kubernetes added support for Windows nodes (stable since Kubernetes v1.14).
You could use [labels](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/)
to give the scheduler a hint about where to place your Pod, but the actual Pod API didn't track that
OS at all.

## What problems does this solve?

Today, the `kube-apiserver` and `kubelet` while admitting pods have no notion of the OS on which the pod can run.
As a result, the end user of the kubernetes cluster can set Linux specific security constraints onto Windows pods or vice-versa.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
As a result, the end user of the kubernetes cluster can set Linux specific security constraints onto Windows pods or vice-versa.
As a result, the end user of a Kubernetes cluster can specify Linux specific security constraints onto Windows pods, or vice-versa.



## How does it work?
A new field called `OS` has been added to the pod spec. Every object in kubernetes is validated before it
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
A new field called `OS` has been added to the pod spec. Every object in kubernetes is validated before it
Pods now have an optional `.spec.os` field (and this is a stable feature, available in every cluster running
Kubernetes v1.25 or later). Every object in Kubernetes is validated before it

Comment on lines 31 to 32
gets persisted to etcd during API admission time, including pod object. Using the `OS` field in pod spec,
we introduced new validation which forbids Linux specific constraints to be set on Windows pods and vice-versa.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
gets persisted to etcd during API admission time, including pod object. Using the `OS` field in pod spec,
we introduced new validation which forbids Linux specific constraints to be set on Windows pods and vice-versa.
gets persisted to etcd during API admission time, including pod object. Kubernetes v1.25 also introduced
new validation which forbids you from setting Linux specific constraints on Windows pods and vice-versa.
``

- AllowPrivilegeEscalation
- Capabilities
- SeccompProfile
when `pod.Spec.OS.Name` is set to `Windows`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
when `pod.Spec.OS.Name` is set to `Windows`
when you set `.spec.os.name` for a Pod to `Windows`.



## How do I use it?
By setting `pod.Spec.OS.Name=Windows` or `pod.Spec.OS.Name=Linux`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please consider providing an example manifest for a Pod.

Comment on lines 45 to 50
KEP: https://github.com/kubernetes/enhancements/issues/2802
API Changes: https://github.com/kubernetes/kubernetes/pull/104693
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once https://kubernetes.io/docs/concepts/workloads/pods/ mentions the new field (see #35590), please link there.

@k8s-ci-robot k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Aug 14, 2022
@katcosgrove
Copy link
Contributor

Hi there! Your assigned publication date is September 5. Thank you!

@sftim
Copy link
Contributor

sftim commented Aug 22, 2022

@ravisantoshgudimetla it'd be great to get this PR ready for review. Would you like help with that?

---
layout: blog
title: "Identifying pod operating system authoritatively during pod admission time"
date: 2022-07-27
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
date: 2022-07-27
date: 2022-09-05

Also please rename the file to 2022-09-05-pod-os-field.md

@reylejano
Copy link
Member

Hi @ravisantoshgudimetla , this blog post is scheduled to be published next week on Sept 5.
Please have this PR at a "ready-state" (remove [WIP] from the title) and take a look at comments and suggestions

@sftim
Copy link
Contributor

sftim commented Sep 2, 2022

(if this is ready for review, please edit the PR title or add a comment to make it clear that this work should move forward)

Copy link
Contributor

@sftim sftim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. Some feedback that I recommend applying.

@sftim
Copy link
Contributor

sftim commented Sep 4, 2022

@katcosgrove let's postpone this one a week - does that work?

@katcosgrove
Copy link
Contributor

@sftim We can do September 12!

@sftim
Copy link
Contributor

sftim commented Sep 26, 2022

@ravisantoshgudimetla we're happy to have a version of this article for the blog, but it has missed the timings for post-release blogs. Even though the timing didn't work out, I'd like to make use of the work that has gone in.

Let's aim for a new date. How about the 19th of October? Does that work.

You need to make two changes: one to the filename, and another to the date in the front matter.
Could you also recommend someone who can review the content for technical accuracy? We need to review this before we publish it, and that review needs to cover the technical side.

---
layout: blog
title: "Identifying Pod Operating System Authoritatively During Pod Admission Time"
date: 2022-09-20
Copy link
Member

@reylejano reylejano Oct 4, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The proposed published date has passed, the filename also needs to be changed

Suggested change
date: 2022-09-20
date: 2022-10-19

@reylejano
Copy link
Member

Hi @kubernetes/sig-windows-leads, can we get a technical review for this blog
Thank you

@marosset
Copy link
Contributor

marosset commented Oct 4, 2022

/sig windows

@k8s-ci-robot k8s-ci-robot added the sig/windows Categorizes an issue or PR as relevant to SIG Windows. label Oct 4, 2022

In the first releases of Kubernetes the `kube-apiserver` did not record any detail about which operating system a Pod should run on. Initially, Kubernetes only supported Linux
nodes; later, Kubernetes added support for Windows nodes (stable since Kubernetes v1.14).
You could use [labels](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should just say there are many different ways to give the scheduler a hint?
One of the more common ways is to use a nodeSelector.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll reword it to While Kubernetes has multiple ways to hint scheduler about the node on which pod needs to run, a more common way is to use nodeSelector, WDYT?

@marosset
Copy link
Contributor

marosset commented Oct 4, 2022

One of the big motivating factors of having this pod os field is so that windows pods can still be scheduled with Restricted policies which require things like "containers must drop ALL capabilities".
This is called out https://kubernetes.io/docs/concepts/security/pod-security-standards/#restricted

I think it would be good to highlight this in the blog post.

@sftim
Copy link
Contributor

sftim commented Dec 19, 2022

@ravisantoshgudimetla would you be willing to to revise this in light of #35439 (comment) ?

@tengqm
Copy link
Contributor

tengqm commented Jan 3, 2023

@ravisantoshgudimetla Ready to give this another stab?

@nitishfy
Copy link
Member

@ravisantoshgudimetla Ping!

@claudiubelu
Copy link

Ping @ravisantoshgudimetla . If you're not available, we can take over this PR and address the necessary changes.

@ravisantoshgudimetla
Copy link
Contributor Author

ravisantoshgudimetla commented Feb 15, 2023

Hey all, I was out for the past few weeks, I can address the changes and will update this PR this week.

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign onlydole for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ravisantoshgudimetla
Copy link
Contributor Author

One of the big motivating factors of having this pod os field is so that windows pods can still be scheduled with Restricted policies which require things like "containers must drop ALL capabilities".
This is called out https://kubernetes.io/docs/concepts/security/pod-security-standards/#restricted
I think it would be good to highlight this in the blog post.

I have added it here - https://github.com/kubernetes/website/pull/35439/files#diff-f2f020ab42d6a660ddb4835f70a43f8a64f9f50204e7ab49d0f275d76af4345cR31. PTAL

@sftim
Copy link
Contributor

sftim commented Feb 23, 2023

@ravisantoshgudimetla how about a new publication date - perhaps the 14th of March 2023? If you would like to aim for that, we can see if we can get reviews to happen on the blog team side.

@natalisucks
Copy link
Contributor

Adding another friendly ping here @ravisantoshgudimetla 🙂

@sftim Lets choose a new publication date – perhaps a couple of months or so away from today? That way Ravi has enough time to address feedback and the blog team have enough time in parallel to review.

@marosset
Copy link
Contributor

@ravisantoshgudimetla - can you make me a collaborator on your k/webite fork and then I can help push this through?

@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

  • Mark this PR as fresh with /remove-lifecycle stale
  • Close this PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jun 21, 2023
@divya-mohan0209
Copy link
Contributor

/remove-lifeycycle stale

@divya-mohan0209
Copy link
Contributor

@ravisantoshgudimetla : Please advise if you'd like to continue work on this PR and also, let us know of a realistic target date for publication. Additionally, it'd be great if you could grant collaborator access to @marosset as requested in the above comments. This has been in the works for a year and I'm sure you'd appreciate that we'd like to see this merged sooner rather than later.

@divya-mohan0209 divya-mohan0209 removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jul 13, 2023
@divya-mohan0209
Copy link
Contributor

Hey @ravisantoshgudimetla , thank you for all your hard work on this! However, since there has been no activity on addressing the feedback despite repeated reminders we'll be closing the PR. Please feel free to reopen the PR whenever you have the bandwidth to work together so that we can get this blog published.
/close

@k8s-ci-robot
Copy link
Contributor

@divya-mohan0209: Closed this PR.

In response to this:

Hey @ravisantoshgudimetla , thank you for all your hard work on this! However, since there has been no activity on addressing the feedback despite repeated reminders we'll be closing the PR. Please feel free to reopen the PR whenever you have the bandwidth to work together so that we can get this blog published.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/blog Issues or PRs related to the Kubernetes Blog subproject cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. language/en Issues or PRs related to English language sig/docs Categorizes an issue or PR as relevant to SIG Docs. sig/windows Categorizes an issue or PR as relevant to SIG Windows. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.