Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The test-infra architecture diagram is out of date #13063

Open
spiffxp opened this issue Jun 17, 2019 · 41 comments
Open

The test-infra architecture diagram is out of date #13063

spiffxp opened this issue Jun 17, 2019 · 41 comments
Assignees
Labels
help wanted Denotes an issue that needs help from a contributor. Must meet "help wanted" guidelines. kind/bug Categorizes issue or PR as related to a bug. kind/documentation Categorizes issue or PR as related to documentation. lifecycle/frozen Indicates that an issue or PR should not be auto-closed due to staleness. priority/important-longterm Important over the long term, but may not be staffed and/or may need multiple releases to complete. sig/testing Categorizes an issue or PR as relevant to SIG Testing.
Milestone

Comments

@spiffxp
Copy link
Member

spiffxp commented Jun 17, 2019

/kind documentation
/kind bug

I visited https://github.com/spiffxp/test-infra/blob/write-some-docs/docs/architecture.svg and saw an architecture that is wildly out of date, eg:

  • has mungegithub
  • has tot
  • missing boskos
  • missing spyglass

I count this as a bug since this diagram is one of the first links in our repo, and it's definitely misleading.

@k8s-ci-robot k8s-ci-robot added kind/documentation Categorizes issue or PR as related to documentation. kind/bug Categorizes issue or PR as related to a bug. labels Jun 17, 2019
@stevekuznetsov
Copy link
Contributor

:yikes:

@spiffxp
Copy link
Member Author

spiffxp commented Jun 25, 2019

While this bug is still open, I recommend checking out the SIG Testing Intro presented a KubeCon NA 2018, find video and slides at https://github.com/kubernetes/community/tree/master/sig-testing#presentations. The final diagram is a bit dense so I tried to walk through how we got to the rube goldberg thing we have going on now.

@spiffxp
Copy link
Member Author

spiffxp commented Aug 1, 2019

/sig testing
/priority important-longterm

@k8s-ci-robot k8s-ci-robot added sig/testing Categorizes an issue or PR as relevant to SIG Testing. priority/important-longterm Important over the long term, but may not be staffed and/or may need multiple releases to complete. labels Aug 1, 2019
@fejta-bot
Copy link

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Oct 30, 2019
@fejta-bot
Copy link

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

@k8s-ci-robot k8s-ci-robot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Nov 29, 2019
@fejta-bot
Copy link

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

@k8s-ci-robot
Copy link
Contributor

@fejta-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@spiffxp
Copy link
Member Author

spiffxp commented Jan 1, 2020

/reopen
/remove-lifecycle rotten
/help-wanted

@k8s-ci-robot
Copy link
Contributor

@spiffxp: Reopened this issue.

In response to this:

/reopen
/remove-lifecycle rotten
/help-wanted

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot reopened this Jan 1, 2020
@k8s-ci-robot k8s-ci-robot removed the lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. label Jan 1, 2020
@fejta-bot
Copy link

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Mar 31, 2020
@spiffxp
Copy link
Member Author

spiffxp commented Apr 1, 2020

/remove-lifecycle stale
/help

@k8s-ci-robot
Copy link
Contributor

@spiffxp:
This request has been marked as needing help from a contributor.

Please ensure the request meets the requirements listed here.

If this request no longer meets these requirements, the label can be removed
by commenting with the /remove-help command.

In response to this:

/remove-lifecycle stale
/help

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added help wanted Denotes an issue that needs help from a contributor. Must meet "help wanted" guidelines. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Apr 1, 2020
@hasheddan
Copy link
Contributor

@spiffxp I'd like to help out here both in creating a new diagram and documenting other parts of the repo. For instance, I was recently looking at the kubekins-e2e image because it is so prevalent across jobs and there is pretty minimal documentation on it and other Docker images used. I wonder what would be a good format for a sort of "Intro to test-infra" guide. This may be outside the scope of this issue, but I wanted to see if you had thoughts on the matter?

@fejta-bot
Copy link

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jul 3, 2020
@fejta-bot
Copy link

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

@k8s-ci-robot k8s-ci-robot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Aug 2, 2020
@k8s-ci-robot k8s-ci-robot added the lifecycle/frozen Indicates that an issue or PR should not be auto-closed due to staleness. label Feb 9, 2021
@k8s-ci-robot k8s-ci-robot added this to the v1.21 milestone Feb 9, 2021
@hh
Copy link
Member

hh commented Mar 1, 2021

/assign @BobyMCbobs

@spiffxp
Copy link
Member Author

spiffxp commented Mar 3, 2021

Was presented at k8s-infra meeting today. Thank you for picking this up

Good followup: depict the clusters, projects and orgs involved. Service cluster, build clusters, Google.com, kubernetes.io, etc

@pacoxu
Copy link
Member

pacoxu commented Mar 12, 2021

Kubernetes Test Infrastructure (Last updated 2020-03-04)

https://raw.githubusercontent.com/kubernetes/test-infra/master/docs/architecture.svg

2020 -> 2021 😄

@BenTheElder
Copy link
Member

BenTheElder commented Jun 25, 2021

this was updated pretty recently, should the follow-up be a follow-up or should we keep tracking here?
if the latter we should update the milestone

@listx
Copy link
Contributor

listx commented Jul 15, 2021

Velodrome is still in the SVG.

@spiffxp
Copy link
Member Author

spiffxp commented Jul 15, 2021

/milestone v1.23
What I discussed during the most recent meeting is:

  1. there are likely still some stale things (good catch @listx)
  2. being able to draw boxes around what lives within google.com and what lives within kubernetes.io would be helpful for wg-k8s-infra

I'll try to clarify 1 since that's a little too vague to be actionable right now

@k8s-ci-robot k8s-ci-robot modified the milestones: v1.21, v1.23 Jul 15, 2021
@spiffxp
Copy link
Member Author

spiffxp commented Aug 30, 2021

I've taken an alternate crack at this over in kubernetes/k8s.io#2635

It doesn't get a much into the mechanics of Prow, but more all of the other things that prow interacts with

@spiffxp
Copy link
Member Author

spiffxp commented Aug 30, 2021

/assign
I'm not removing help as I could still use plenty of help in suggesting edits and better tools to use to document this stuff

@spiffxp
Copy link
Member Author

spiffxp commented Sep 13, 2021

I browsed through https://xosh.org/text-to-diagram/ to see if I could find a non-proprietary tool that could accomplish the same sort of thing I did with kubernetes/k8s.io#2635

@spiffxp
Copy link
Member Author

spiffxp commented Oct 13, 2021

cc @hh

@MeNsaaH
Copy link

MeNsaaH commented Oct 29, 2021

@spiffxp have we considered https://github.com/mingrammer/diagrams? Looks great.

@ameukam
Copy link
Member

ameukam commented Feb 17, 2022

@MeNsaaH
Copy link

MeNsaaH commented Feb 17, 2022

This is heavily bundled with GCP. What of AWS workloads?

@BenTheElder
Copy link
Member

This is heavily bundled with GCP. What of AWS workloads?

test-infra runs on GCP (due to https://www.cncf.io/announcements/2020/12/17/cloud-native-computing-foundation-receives-renewed-3-million-cloud-credit-grant-from-google-cloud/), FWIW.

When it isn't reasonable to DIY infra all the way down, we use off the shelf offerings sometimes. Sometimes not often enough 🙃

spiffxp is out for a while right now, I think mostly it would be good to have a rendered result like we have now.

@BenTheElder BenTheElder modified the milestones: v1.23, someday Apr 19, 2022
@vatsalparekh
Copy link

if this is still relevant, would like to be a part
/assign

@listx
Copy link
Contributor

listx commented Oct 22, 2022

At least for the Prow components, there is an architecture diagram at https://docs.prow.k8s.io/docs/overview/architecture/.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Denotes an issue that needs help from a contributor. Must meet "help wanted" guidelines. kind/bug Categorizes issue or PR as related to a bug. kind/documentation Categorizes issue or PR as related to documentation. lifecycle/frozen Indicates that an issue or PR should not be auto-closed due to staleness. priority/important-longterm Important over the long term, but may not be staffed and/or may need multiple releases to complete. sig/testing Categorizes an issue or PR as relevant to SIG Testing.
Projects
None yet
Development

No branches or pull requests

16 participants