Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve the multi-platform compability #288

Closed
luxas opened this issue Apr 30, 2017 · 61 comments
Closed

Improve the multi-platform compability #288

luxas opened this issue Apr 30, 2017 · 61 comments
Assignees
Labels
kind/feature Categorizes issue or PR as related to a new feature. sig/release Categorizes an issue or PR as relevant to SIG Release. stage/beta Denotes an issue tracking an enhancement targeted for Beta status tracked/no Denotes an enhancement issue is NOT actively being tracked by the Release Team

Comments

@luxas
Copy link
Member

luxas commented Apr 30, 2017

Kubernetes on multiple platforms

  • One-line feature description (can be used as a release note): Kubernetes should work on the platforms the community expects it to work on. Automated CI e2e tests should be run for all supported architectures. It should be possible to run clusters with nodes of mixed architectures.
  • Primary contact (assignee): @luxas @mkumatag @ixdy
  • Responsible SIGs: no formal sig / sig-release will probably be the closest
    • Mostly this is about making our release/test tooling work on multiple platforms and providing a deployment solution that works everywhere: kubeadm
  • Design proposal link (community repo): https://github.com/kubernetes/community/blob/master/contributors/design-proposals/multi-platform.md
  • Reviewer(s) - (for LGTM) recommend having 2+ reviewers (at least one from code-area OWNERS file) agreed to review. Reviewers from multiple companies preferred: @ixdy @luxas @vishh
  • Approver (likely from SIG/area to which feature belongs): @ixdy @luxas @vishh @thockin @brendandburns
  • Feature target (which target equals to which milestone):
    • Alpha release target (x.y): v1.3/v1.4
    • Beta release target (x.y): v1.12?
    • Stable release target (x.y): TBD

It's very hard to estimate alpha/beta/stable levels of this feature as there is no clear graduation path.
Anyway, I'd consider this feature "done" when:

  • We have automated CI e2e Conformance tests running for arm, arm64, ppc64le, s390x and windows continously. Currently we're lacking the hardware support. I'm in touch with CNCF to fix this eventually. Also the e2e suite should be
  • Manifest lists are pushed to the same image tag is usable on any platform without having to substitute -ARCH in order to get the right docker image downloaded.
  • All Kubernetes core and Incubated projects push artifacts for all supported platforms
  • QEMU inside of the multiarch containers isn't needed anymore. When building docker images for a foreign architecture, we're using QEMU for the syscall emulation, which currently requires the QEMU binary to be in the resulting image. However, with a new 4.8 kernel it can be done without any extra addition to the resulting image.

KubeCon video where I'm talking about this: https://www.youtube.com/watch?v=ZdzKQwMjg2w

@mkumatag
Copy link
Member

mkumatag commented May 2, 2017

Sounds good to me..

@idvoretskyi idvoretskyi added the stage/beta Denotes an issue tracking an enhancement targeted for Beta status label May 3, 2017
@idvoretskyi idvoretskyi added this to the v1.7 milestone May 3, 2017
@idvoretskyi idvoretskyi added the help wanted Denotes an issue that needs help from a contributor. Must meet "help wanted" guidelines. label May 3, 2017
@idvoretskyi
Copy link
Member

Labeled as "help wanted" - SIG has to be defined.

@calebamiles calebamiles added the sig/cluster-lifecycle Categorizes an issue or PR as relevant to SIG Cluster Lifecycle. label May 4, 2017
@luxas
Copy link
Member Author

luxas commented May 4, 2017

@idvoretskyi As soon as SIG Release is formally formed as a group, I expect the work to belong to that SIG. Also crosses some borders with the cluster lifecycle SIG.

@idvoretskyi
Copy link
Member

@calebamiles @pwittrock is SIG-release going to work on this feature?

@luxas
Copy link
Member Author

luxas commented May 5, 2017

@idvoretskyi To clarify, so far I and a couple of other persons have been hacking on this with no formal SIG owning it.
This is basically a thing that touches sig-cluster-lifecycle, sig-testing, sig-release and sig-windows all at the same time.

However, I'll talk to @ixdy @vishh @thockin and sig-release where we should formally place this

@idvoretskyi
Copy link
Member

@luxas it's amazing to see this work in progress!

At the same time, the feature has to have one or multiple SIG's, that are formally responsible for its implementation.

@luxas luxas modified the milestones: next-milestone, v1.7 May 25, 2017
@idvoretskyi idvoretskyi added the sig/release Categorizes an issue or PR as relevant to SIG Release. label Jun 1, 2017
@idvoretskyi
Copy link
Member

@luxas FYI - I've created a sig/release label and labeled this issue with it.

@luxas
Copy link
Member Author

luxas commented Jun 2, 2017

@idvoretskyi Perfect, waited for that label!
As you saw, I removed it from the v1.7 as it won't make it due to external dependencies (gcr.io, etc.).

@luxas
Copy link
Member Author

luxas commented Jun 2, 2017

I also removed it from Action Required as no actions has to be made for this, targeting v1.8 now

@luxas
Copy link
Member Author

luxas commented Jul 26, 2017

This depends on the GCR team adding support for manifest lists. Adding to the v1.8 milestone for now.
I'm not sure the dependency will be resolved before code freeze (I have waited a year for this to happen already), but in case it does we should track this.

@luxas luxas modified the milestones: 1.8, next-milestone Jul 26, 2017
@detiber
Copy link
Member

detiber commented Jul 26, 2017

Is GCR support for manifest list images an absolute requirement for this? Could docker hub or another public registry that supports manifest list images (if there are any) be used until GCR supports manifest list images?

@luxas
Copy link
Member Author

luxas commented Sep 1, 2017

Is GCR support for manifest list images an absolute requirement for this?

Yes, as gcr.io is the repo we're using for kubernetes release artifacts.

Could docker hub or another public registry that supports manifest list images (if there are any) be used until GCR supports manifest list images?

No unfortunately not; as that would be a huge, backwards-incompatible change and would result in lots of other complications.

I'm told the GCR team is still working on this 😄; but due to that it is not ready yet, this stays in alpha :(
On the good side; significant improvements have been made during the cycle to make sure all Conformance tests (an other requirement for beta) are now passing on the other platforms 👍

@luxas luxas added stage/alpha Denotes an issue tracking an enhancement targeted for Alpha status and removed stage/beta Denotes an issue tracking an enhancement targeted for Beta status labels Sep 1, 2017
@kacole2
Copy link

kacole2 commented Oct 8, 2018

@luxas @mkumatag @ixdy I'm following up from @claurence's post if there any plans for this to graduate in 1.13?

@neolit123
Copy link
Member

given the shortened 1.13 release cycle, i would say that only some work for missing e2e tests could be done in 1.13, maybe CNI could see improvements, but that's a 3rd party dependency.

RE: incubated projects: IMHO, a separate KEP should be sent for this and we should omit the incubated project demand from this feature as outlined here.

@kacole2
Copy link

kacole2 commented Oct 8, 2018

/milestone clear

@k8s-ci-robot k8s-ci-robot removed this from the v1.12 milestone Oct 8, 2018
@kacole2 kacole2 added tracked/no Denotes an enhancement issue is NOT actively being tracked by the Release Team and removed tracked/yes Denotes an enhancement issue is actively being tracked by the Release Team labels Oct 8, 2018
@timothysc
Copy link
Member

At this point, this really should not be owned by sig-cluster-lifecycle but should be passed onto @kubernetes/sig-release-feature-requests or the k8s-infra-team.

@timothysc timothysc removed the sig/cluster-lifecycle Categorizes an issue or PR as relevant to SIG Cluster Lifecycle. label Oct 9, 2018
@dims
Copy link
Member

dims commented Oct 9, 2018

@timothysc sig-testing may be another option as the main thing lacking right now is automated CI e2e Conformance tests running for arm, arm64, ppc64le, s390x and windows (well, we do have ppc64le, but others are missing)

@timothysc
Copy link
Member

@dims I don't think it should be on sig-testings shoulders to manage things like s390x when they don't have the gear to debug. Ideally, I think the consumers should supply signal via federated testing.

@timothysc
Copy link
Member

/unassign @timothysc

@fejta-bot
Copy link

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jan 7, 2019
@vielmetti
Copy link

@dims - is there a concise description of the hardware and software needs for "federated testing" as described by @timothysc above?

I see this has gone stale again, but I don't think the need has passed for it!

@dims
Copy link
Member

dims commented Jan 8, 2019

@vielmetti yes!. what he meant was that interested parties should set up their own CI using whatever tools they deem fit BUT publish the results to the community test grid, so any consumers or say the release team can check if any recent changes have broken functionality in other platforms.

Here's a HOWTO post results to the community test grid:
https://github.com/kubernetes/test-infra/tree/master/testgrid/conformance

@fejta-bot
Copy link

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

@neolit123
Copy link
Member

/lifecycle frozen

@k8s-ci-robot k8s-ci-robot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Feb 7, 2019
@neolit123
Copy link
Member

/lifecycle frozen

@k8s-ci-robot k8s-ci-robot added lifecycle/frozen Indicates that an issue or PR should not be auto-closed due to staleness. and removed lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. labels Feb 7, 2019
@fejta-bot
Copy link

Enhancement issues opened in kubernetes/enhancements should never be marked as frozen.
Enhancement Owners can ensure that enhancements stay fresh by consistently updating their states across release cycles.

/remove-lifecycle frozen

@k8s-ci-robot k8s-ci-robot removed the lifecycle/frozen Indicates that an issue or PR should not be auto-closed due to staleness. label Feb 7, 2019
@kacole2
Copy link

kacole2 commented Apr 12, 2019

Hello @neolit123 @luxas , I'm the Enhancement Lead for 1.15. Is this feature going to be graduating alpha/beta/stable stages in 1.15? Please let me know so it can be tracked properly and added to the spreadsheet. A KEP will need to be merged before this can be included in 1.15 as well.

Once coding begins, please list all relevant k/k PRs in this issue so they can be tracked properly.

@dims
Copy link
Member

dims commented Apr 16, 2019

/close

we have separate issues for different platforms already logged. let's close this one out

@k8s-ci-robot
Copy link
Contributor

@dims: Closing this issue.

In response to this:

/close

we have separate issues for different platforms already logged. let's close this one out

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@neolit123
Copy link
Member

@dims thanks for taking the action.

we have separate issues for different platforms already logged. let's close this one out

+1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/feature Categorizes issue or PR as related to a new feature. sig/release Categorizes an issue or PR as relevant to SIG Release. stage/beta Denotes an issue tracking an enhancement targeted for Beta status tracked/no Denotes an enhancement issue is NOT actively being tracked by the Release Team
Projects
None yet
Development

No branches or pull requests