AppArmor support

[timstclair]

Description

Add AppArmor support to Kubernetes. Initial support should include the ability to specify an AppArmor profile for a container or pod in the API, and have that profile applied by the container runtime.

Progress Tracker

• Before Alpha
  • Design Approval
    • Design Proposal. This goes under docs/proposals. Doing a proposal as a PR allows line-by-line commenting from community, and creates the basis for later design documentation. Paste link to merged design proposal here: AppArmor design proposal kubernetes#29168
    • Initial API review (if API). Maybe same PR as design doc. AppArmor design proposal kubernetes#29168
      • Any code that changes an API (/pkg/apis/...)
      • cc @kubernetes/api
  • Write (code + tests + docs) then get them merged. Add AppArmor validation logic kubernetes#29812 Implement AppArmor Kubelet support kubernetes#30118 Validate AppArmor annotations in the API server kubernetes#30722 AppArmor PodSecurityPolicy support kubernetes#30183 Increase the AppArmor pod stop timeout to match the start timeout kubernetes#31314 Add AppArmor feature gate kubernetes#31473 [AppArmor] Promote AppArmor annotations to beta kubernetes#31471 Include security options in the container created event kubernetes#31557 AppArmor was flipped to beta, update feature gate kubernetes#31625 Append "AppArmor enabled" to the Node ready condition message kubernetes#31659
    • Code needs to be disabled by default. Verified by code OWNERS
      AppArmor is enabled by default, but gated by a feature-gate: Add AppArmor feature gate kubernetes#31473
    • Minimal testing
    • Minimal docs - AppArmor documentation website#1147
      • cc @kubernetes/docs on docs PR
      • cc @kubernetes/feature-reviewers on this issue to get approval before checking this off
      • New apis: Glossary Section Item in the docs repo: kubernetes/kubernetes.github.io
    • Update release notes
• Before Beta [AppArmor] Promote AppArmor annotations to beta kubernetes#31471
  • Testing is sufficient for beta
  • User docs with tutorials - AppArmor documentation website#1147
    • Updated walkthrough / tutorial in the docs repo: kubernetes/kubernetes.github.io
    • cc @kubernetes/docs on docs PR
    • cc @kubernetes/feature-reviewers on this issue to get approval before checking this off
  • Thorough API review
    • cc @kubernetes/api
• Before Stable
  • KEPS
    • KEP-24: Promote Apparmor to GA #3298
    • KEP-24: Promote AppArmor to GA #4417
  • PRs
    • AppArmor fields API kubernetes#123435
  • Soak, load testing
  • detailed user docs and examples
    • cc @kubernetes/docs
    • cc @kubernetes/feature-reviewers on this issue to get approval before checking this off

FEATURE_STATUS is used for feature tracking and to be updated by @kubernetes/feature-reviewers.
FEATURE_STATUS: BETA

More advice:

Design

• Once you get LGTM from a @kubernetes/feature-reviewers member, you can check this checkbox, and the reviewer will apply the "design-complete" label.

Coding

• Use as many PRs as you need. Write tests in the same or different PRs, as is convenient for you.
• As each PR is merged, add a comment to this issue referencing the PRs. Code goes in the https://github.com/kubernetes/kubernetes repository,
  and sometimes https://github.com/kubernetes/contrib, or other repos.
• When you are done with the code, apply the "code-complete" label.
• When the feature has user docs, please add a comment mentioning @kubernetes/feature-reviewers and they will
  check that the code matches the proposed feature and design, and that everything is done, and that there is adequate
  testing. They won't do detailed code review: that already happened when your PRs were reviewed.
  When that is done, you can check this box and the reviewer will apply the "code-complete" label.

Docs

• Write user docs and get them merged in.
• User docs go into https://github.com/kubernetes/kubernetes.github.io.
• When the feature has user docs, please add a comment mentioning @kubernetes/docs.
• When you get LGTM, you can check this checkbox, and the reviewer will apply the "docs-complete" label.

[timstclair]

Original issue here: kubernetes/kubernetes#22159

[janetkuo]

@timstclair it looks like the docs PR number is outdated. Please update the PR number and check the docs box once it's done

[timstclair]

Fixed. Thanks @janetkuo !

[timstclair]

Docs kubernetes/website#1147 - @kubernetes/docs

[devin-donnelly]

Is there an issue? I merged this one in last week.

On Sep 21, 2016 1:30 PM, "Tim St. Clair" [email protected] wrote:

Docs kubernetes/website#1147
kubernetes/website#1147 -
@kubernetes/docs https://github.com/orgs/kubernetes/teams/docs

—
You are receiving this because you are on a team that was mentioned.
Reply to this email directly, view it on GitHub
#24 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ARmNwOTArylXQHoAoz2lMTsKhg9luaTYks5qsZPlgaJpZM4JMBOR
.

[timstclair]

No, I was just following the instructions at the bottom of the issue, which I hadn't done before...

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or @fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten
/remove-lifecycle stale

[fejta-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

[liggitt]

/remove-lifecycle rotten

[justaugustus]

@tallclair @liggitt
Any plans for this in 1.11?

If so, can you please ensure the feature is up-to-date with the appropriate:

• Description
• Milestone
• Assignee(s)
• Labels:
  • stage/{alpha,beta,stable}
  • sig/*
  • kind/feature

cc @idvoretskyi

[SergeyKanzhelev]

/label lead-opted-in

[prianna]

Hello @tallclair and @vinayakankugoyal 👋, Enhancements team here.

Just checking in as we approach enhancements freeze on 02:00 UTC Friday 14th June 2024 / 19:00 PDT Thursday 13th June 2024.

This enhancement is targeting stage stable for v1.31 (correct me if otherwise)

Here's where this enhancement currently stands:

• KEP readme using the latest template has been merged into the k/enhancements repo.
• KEP status is marked as implementable for latest-milestone: v1.31. KEPs targeting stable will need to be marked as implemented after code PRs are merged and the feature gates are removed.
• KEP readme has up-to-date graduation criteria
• KEP has submitted a production readiness review request for approval and has a reviewer assigned.
• KEP has a production readiness review that has been completed and merged into k/enhancements. (For more information on the PRR process, check here).

With all the KEP requirements in place and merged into k/enhancements, this enhancement is all good for the upcoming enhancements freeze. 🚀

The status of this enhancement is marked as tracked for enhancement freeze. Please keep the issue description up-to-date with appropriate stages as well. Thank you!

[hacktivist123]

Hello @timstclair 👋, 1.31 Docs Shadow here.
Does this enhancement work planned for 1.31 require any new docs or modification to existing docs?

If so, please follows the steps here to open a PR against dev-1.31 branch in the k/website repo. This PR can be just a placeholder at this time and must be created before Thursday June 27, 2024 18:00 PDT.

Also, take a look at Documenting for a release to get yourself familiarised with the docs requirement for the release.

Thank you!

[mbianchidev]

Hey hey @tallclair @vinayakankugoyal

👋 from the v1.31 Communications Team!

We'd love for you to opt in to write a feature blog about your enhancement!
Some reasons why you might want to write a blog for this feature include (but are not limited to) if this introduces breaking changes, is important to our users, or has been in progress for a long time and is graduating.

To opt in, let us know and open a Feature Blog placeholder PR against the website repository by 3rd July, 2024. For more information about writing a blog see the blog contribution guidelines.

Note: In your placeholder PR, use XX characters for the blog date in the front matter and file name. We will work with you on updating the PR with the publication date once we have a final number of feature blogs for this release.

[Princesso]

Hello @timstclair 👋, 1.31 Docs Shadow here. Does this enhancement work planned for 1.31 require any new docs or modification to existing docs?

If so, please follows the steps here to open a PR against dev-1.31 branch in the k/website repo. This PR can be just a placeholder at this time and must be created before Thursday June 27, 2024 18:00 PDT.

Also, take a look at Documenting for a release to get yourself familiarised with the docs requirement for the release.

Thank you!

Hello @tallclair @vinayakankugoyal, @timstclair,
Just a reminder to open a placeholder PR against the dev-1.31 branch in the k/website repo, if this enhancement work requires new additions or modifications to existing docs. The deadline for this is a week away at Thursday, June 27, 2024, 18:00 PDT.

[mbianchidev]

Hey hey @tallclair @vinayakankugoyal

👋 from the v1.31 Communications Team!

We'd love for you to opt in to write a feature blog about your enhancement! Some reasons why you might want to write a blog for this feature include (but are not limited to) if this introduces breaking changes, is important to our users, or has been in progress for a long time and is graduating.

To opt in, let us know and open a Feature Blog placeholder PR against the website repository by 3rd July, 2024. For more information about writing a blog see the blog contribution guidelines.

Note: In your placeholder PR, use XX characters for the blog date in the front matter and file name. We will work with you on updating the PR with the publication date once we have a final number of feature blogs for this release.

Reminder of the 3rd of July deadline!
It's totally fine to also opt out if you don't think that writing a blog is useful for our users or if you don't have time (in that case team comms can also help you out 👀 )

[natalisucks]

Hello @timstclair 👋, 1.31 Docs Shadow here. Does this enhancement work planned for 1.31 require any new docs or modification to existing docs?
If so, please follows the steps here to open a PR against dev-1.31 branch in the k/website repo. This PR can be just a placeholder at this time and must be created before Thursday June 27, 2024 18:00 PDT.
Also, take a look at Documenting for a release to get yourself familiarised with the docs requirement for the release.
Thank you!

Hello @tallclair @vinayakankugoyal, @timstclair, Just a reminder to open a placeholder PR against the dev-1.31 branch in the k/website repo, if this enhancement work requires new additions or modifications to existing docs. The deadline for this is a week away at Thursday, June 27, 2024, 18:00 PDT.

Howdy @timstclair and @vinayakankugoyal, SIG Docs co-chair here 👋 I wanted to add another reminder about the docs deadline for this enhancement: updating feature gates qualifies as requiring documentation as far as Release Docs is concerned, so please check out @Princesso's reminder above to ensure you meet the deadline today

[vinayakankugoyal]

@natalisucks - I added a draft PR for this feature. Thanks!

[sftim]

A question, is the annotation apparmor.security.beta.kubernetes.io/defaultProfileName deprecated? If so, what's the new mechanism?

[vinayakankugoyal]

We have a new field.containers[*].securityContext.appArmorProfile.type:

[sftim]

OK, cool. The new thing is documented; we also need to document the deprecated thing (and tell people to switch to the new thing). We leave deprecated annotations documented forever.

[prianna]

Hey again @vinayakankugoyal and @tallclair 👋 Enhancements team here,

Just checking in as we approach code freeze at 02:00 UTC Wednesday 24th July 2024 / 19:00 PDT Tuesday 23rd July 2024.

Here's where this enhancement currently stands:

• All PRs to the Kubernetes repo that are related to your enhancement are linked in the above issue description (for tracking purposes).
• All PR/s are ready to be merged (they have approved and lgtm labels applied) by the code freeze deadline. This includes tests.

For this enhancement, it looks like the following PRs are open and need to be merged before code freeze:

• KEP-24: Update AppArmor feature gates to GA stage. kubernetes#125257

Please let me know if there are other PRs in k/k we should be tracking for this KEP, and please update your enhancement description with the requisite PRs.

If you anticipate missing code freeze, you can file an exception request in advance. As always, we are here to help if any questions come up. Thanks!

[sreeram-venkitesh]

@vinayakankugoyal Please let us know if kubernetes/kubernetes#125257 is the only PR that we need to track for this KEP.

[vinayakankugoyal]

Hi @sreeram-venkitesh. Yes kubernetes/kubernetes#125257 is the only PR associated with this KEP.

[sreeram-venkitesh]

Awesome, marking the KEP as tracked for code freeze 🎉 Thanks @vinayakankugoyal!

[tjons]

@tallclair in preparation for the next release, now that this KEP has been implemented, would you kindly update the KEP status to implemented and then close this out, unless there's remaining work to conduct in the 1.32 release cycle?

[haircommander]

I moved this for Not for release because I don't see any work we'll be doing here for 1.32 as it's already GA

[tjons]

Hello 👋 1.32 Enhancements Lead here,

I'm closing milestone 1.31 now,
If you have more work on this enhancement to complete in v1.32, please follow the instructions here to opt in the enhancement and make sure the lead-opted-in label is set so it can get added to the tracking board and finally add /milestone v1.32. Thanks!

/milestone clear

[tjons]

Inadvertently added this to the 1.32 tracking board

/remove-label lead-opted-in

[tallclair]

Remaining work, copied from the KEP (https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/24-apparmor/README.md#removing-annotation-support)

• Phase 1 (v1.30): AppArmor field support merged (AppArmor fields API kubernetes#123435)
  • Sync annotations & fields on Pod create (version skew strategy described above)
  • Warn on annotation use, if field isn't set
  • Kubelet copies static pod annotations to fields
• Phase 2 (v1.34):
  • API server stops copying fields to annotations
  • Warn on ALL annotation use
  • Risk: policy controllers that don't consider field values
• Phase 3 (v1.36): End state
  • API server stops copying annotations to fields
  • Kubelet stops copying annotations to fields for static pods
  • Validation that annotations & fields match persists indefinitely
  • Risk: workloads that haven't migrated