[Suggestion] Define process for due diligence recommendations by the SIG at project requests

[TheFoxAtWork]

Description: Currently, the SIG performs assessments of projects at various stages in the CNCF. We receive requests to provide recommendations on the project as part of the due diligence document. This is not clearly captured in the repository as to:

• a co-chair must provide concurrence to the recommendation
• due diligence recommendations should include any substantive concerns, and the overall recommendation
• who may document the concerns and overall recommendation

Impact: Provide clear expectations of who is responsible and how it is performed to better enable succession of co-chairs, TLs, and assessment leads

Scope: collaboration to define what this actually looks like today, confirm a "template" for the co-chair/reviewer to complete. Define any additional process items that need covers (Separate ticket by the project lead requesting with link to TOC issue? etc.)

CC; @lumjjb @pragashj @ultrasaurus

[stale]

This issue has been automatically marked as inactive because it has not had recent activity.

[PushkarJ]

✋🏼 can work on this.

[PushkarJ]

I went through this proposal again today. Couple of comments or requests for pointers:

1. Do we have any prior art in the form of a document that describes a process that works well for this SIG or others?
2. Do we have examples of deliverables that can be considered "due diligence documents" that we have we written in the past for one or more projects ?

[stale]

This issue has been automatically marked as inactive because it has not had recent activity.

[lumjjb]

There were recent changes in the DD process of a project, where it is now guided by the TOC sponsor, so the general flow of what happens is not quite defined yet, I think cloud custodian is the first project that is going through this right now.

[PushkarJ]

@lumjjb thanks for the update. Should we wait until security assessment on cloud custodian goes through and then document the process in terms of roles and responsibilities as described in the issue after the assessment is completed?

[lumjjb]

yea that sounds like a good course of action!

[TheFoxAtWork]

This should be placed on hold until we have the next one that comes to us.

[stale]

This issue has been automatically marked as inactive because it has not had recent activity.

[TheFoxAtWork]

pinging this for action.

[stale]

This issue has been automatically marked as inactive because it has not had recent activity.

[PushkarJ]

@lumjjb with #307 complete, are we ready to revisit this ?

[anvega]

@TheFoxAtWork There might have been context at the time of the issue which is currently missing. Can you help clarify what the original intent was? My read was there are, from time to time, extraordinary asks from projects to the tag in their due diligence as they progress maturity stages other than assessments. The other interpretation is that after an assessment there are recommendations formulated that require following up on. Which one is right?

[TheFoxAtWork]

Its a little of both and then some.

At the time this issue was filed, the TAG was asked to provide Recommendations to the TOC as part of a project's Due Diligence for moving levels (incubation applications new DD and graduation application DD refresh).

When these requests occurred, the TAG lacked any basis by which a recommendation could be made from:

1. Does a recommendation come from a joint-assessment by the TAG?
2. Does a recommendation come from a review of the Security Audit for the project
3. Does it require verification that the fundamental security processes described by the project support secure development and secure design patterns?
4. Is a presentation by the project sufficient?
5. Is it simpler than this?
6. Is it a combination thereof depending on what has been completed to date?

With a large Caveat that the TAG cannot create or apply more stringent criteria for moving levels than what is already documented by the TOC, i.e. the TAG cannot withhold a graduation DD recommendation from being provided because a project hasn't undergone a joint-assessment by the TAG even though it has a received a Security Audit ("The TAG does
not recommend the project for graduation because no joint-review was performed").

Rather, the TAG can state that "no recommendation may be provided at this time due to lack of review by the TAG on the project", and may recommend one of several opportunities for review to occur (such as those defined above). I.e. "We recommend the project's Security Audit be reviewed, please submit an issue here, or that the project employ a Security Buddy to review the described process for secure development practices, please submit an issue here."

Further, it was unclear who was responsible for preparing the recommendation. Was it to be done by the lead security reviewer at the end of a joint-review? Is it done by a Security Buddy supporting an ad hoc request for support?
After the recommendation is prepared (or the statement of no recommendation can be provided at this time due to lack of review), who is responsible within the TAG Leadership for providing final sign-off on the statement?

In its simplest, it was about the "recommendation" statement by the TAG on DD docs. At its most complex/complete, it covered what forms the basis of content by which the TAG may arrive at a recommendation to the TOC for a DD.

[anvega]

Chairs and TLs remain a relatively small group that acts as a collective. While in the future we may have assessment leads that are not one of these two, it has historically been one of the two. The summary of the assessment doesn't get checked in without collaboration with the assessment facilitator and the signoff of two reviewers in this group which guarantees input and approval from a subset of us all.

In every instance where recommendations have been produced, one appointed representative from the group relays in writing or in front of the TOC what the whole group deliberated and agreed upon. There hasn't been any historical contention in recommendations as guidance is always given a positive framing. Examples include by studying scenarios of attack vectors previously not considered, checking assessment assets into the project repo, expanding the security team, and for the CNCF to conduct third party audits, elevate awareness of the project, investing in education programs, doing usability surveys, etc.

I think we can keep this fairly fluid the way it is without being overly procedural or lawyerly besides the common sense of not issuing recommendations that are based on individual opinions and not the collective of the group, or perhaps you can suggest some text as to what you envision this should look like instead if the current approach is deemed inadequate by the TOC.

[mnm678]

New guidance from the TOC on assessments addresses this discussion. Closing as completed.