Security Advisories Feature Requests & Improvements

[JLLeitschuh]

I've been working with GHSA since the feature was originally released. I've attempted to capture the collection of issues and features I've had over the past several years with GitHub Security Advisories (GHSA):

User Context: I'm primarily using GHSA as a security researcher, however, I've also been on the receiving end of reports (when I worked for Gradle). As such, my perspective more heavily comes from that of the party attempting to report security vulnerabilities, less the party on the receiving end, but I do have both perspectives.

Note to viewers outside of GitHub or GitHub Stars the slack links point to internal slack conversations with the GitHub Stars slack channel and aren't viewable by the public.

'User Pain Priority' is purely subjective based upon my own experience using GHSA. More stars, more painfully needed feature.

Preamble

Vulnerability handling does not end with disclosure. Vulnerabilities evolve after disclosure. They sometimes get worse. New information is discovered about the vulnerability. Threat actors begin abusing these vulnerabilities. This point plays into several of the feature requests below, so I figured that I'd open with this.

Feature Requests

Allow Security Researchers to Open GHSA without Requiring a Maintainer to be Involved

User Story: As a security researcher, I'd like to be able to open a GHSA against any OSS GitHub repository, without needing the maintainers to open the GHSA for me.

User Pain Priority: ⭐ ⭐ ⭐ ⭐ ⭐

Making contact with the maintainer can be one of most complex and annoying parts of vulnerability disclosure. As a security researcher, I've been begging for this feature since GHSA was released.

I understand that there's an argument about attempting to limit maintainer spam. The GitHub team doesn't want to create a private, potentially, information-restricted channel where maintainer harassment can be conducted. These concerns could be potentially mitigated by creating a list of known & trusted security researchers that have the permission to take advantage of this feature.

Allow Repository privileges below "Admin" to open GHSA

User Story: As a repository maintainer with 'write' level permissions on a repository, I should be able to create a GHSA against my repository.

User Pain Priority: ⭐ ⭐

According to the docs you still need to have the 'Admin' level permissions to create a GHSA against a repository.

https://docs.github.com/en/code-security/repository-security-advisories/creating-a-repository-security-advisory

IMHO, this permission is way too high. Having worked for a company that likes to keep the number of accounts with 'Admin' level permissions rather light (as it should be for a good security posture), requiring the 'Admin' permission level for simply creating a GHSA seems like a high bar to clear.

I'd argue that the permission of 'Write' should be enough to grant the permission to open a GHSA. If you have the permission to merge a PR, you should be able to open a GHSA and kick off a discussion with a researcher or your team about a sensitive vulnerability easily.

Discoverability of an individual GH users open/disclosed/closed GHSA

User Story: As a GitHub User who has been a part of several GHSA (either as a reporter or as a maintainer), I should be able to easily discover these GHSA across all organizations and repositories in a single GitHub UI.

User Pain Priority: ⭐ ⭐ ⭐ ⭐

As a security researcher, or maintainer, who has been involved with several GitHub Security Advisories, there is no way to "discover" these advisories in a simplified list or search via the GitHub API. User interfaces already exist for finding all open/closed issues and pull requests, however the same does not exist for GitHub Security Advisories.

• Pull Requests - https://github.com/pulls?q=is%3Apr+is%3Aopen+author%3A%40me+archived%3Afalse+sort%3Aupdated-desc
• Issues - https://github.com/issues?q=is%3Aissue+is%3Aopen+author%3A%40me+archived%3Afalse+sort%3Aupdated-desc

This can make it easy to lose open/in-progress GitHub Security Advisories. The only way to reliably make these advisories 'pop' to the users attention is via the notification feed, either finding the advisory in your feed, or by sending a message so others are reminded about the GHSA.

Personally, I use a hacky notification search to keep track of all my GitHub Security Advisories, but this is imperfect and doesn't contain advisories for notifications (idk why some are missing but they are): is:repository-advisory is:done is:read is:unread

A user interface to find all of the existing GitHub Security Advisories your user account has been a part of (ie. open/closed/disclosed) is sorely needed by the security research community.

Slack Conversation: https://github-partners.slack.com/archives/C01MF7QQK3P/p1614987770032300?thread_ts=1614987743.032200&cid=C01MF7QQK3P

GHSA Comments are Locked for EVERYONE After Disclosure

User Story: As a maintainer or reporter, I should be able to continue a discussion about a vulnerability after it has been published.

User Pain Priority: ⭐ ⭐ ⭐

See above note about 'vulnerability handling does not end with disclosure'.

When a GHSA is disclosed, no further discussion can happen on that GHSA amongst the maintainers/reporters. The rationale behind this has never been fully communicated to me, and it's baffling that this hasn't been fixed.

If GHSA truly want to be the location where maintainers disclose vulnerabilities, they should also be the way that the maintainers can keep their users updated as new information is disclosed about these vulnerabilities.

Keeping a dialogue/conversation channel open with the researcher through this is important. Sometimes the researcher finds new information that needs to be shared privately.

Currently, because GHSA are locked, to achieve this, the researcher must re-engage with the maintainers again instead of using the existing communication channel that had already existed prior to disclosure. From experience, this has been a cause for severe annoyance on my behalf as getting in touch with the maintainers via a secure channel is sometimes a significant amount of work. Having to spend this time again, is very annoying.

Ideally, additional comments should support the ability to either be private (exclusively to the the GHSA Collaborators & Publishers), or to make these additional comments public.

Slack Discussion: https://github-partners.slack.com/archives/C013VPD24BW/p1611951569123500

Discussion/Comment Support on GHSA

User Story: As a GitHub user, a GHSA should also allow me to comment on the GHSA to provide new/updated information or ask questions/clarification about the vulnerability.

User Pain Priority: ⭐ ⭐

See above note about 'vulnerability handling does not end with disclosure'.

New information about vulnerabilities can come from a variety of sources. Additionally, the discussion around a vulnerability can evolve and potentially uncover additional details or impacted conditions that were not originally understood by group fixing/disclosing the vulnerability.

A perfect example of this came regarding my disclosure of a vulnerability in Google Guava. When performing the initial disclosure, we assumed that the vulnerability only impacted unix like systems, but then the community brought it to our attention that the ramifications of the vulnerability could also adversely impact the Android ecosystem. This lead to a bunch of research by the community to capture this information and further refine the advice provided.

Google Guava Disclosure Issue Thread: google/guava#4011

Slack Discussion: https://github-partners.slack.com/archives/C013VPD24BW/p1611951569123500

GHSA Should Display Change History

User Story: As a visitor to the GitHub site looking at a GHSA, I should be able to easily see the edit history on that GHSA.

User Pain Priority: ⭐

As stated above, after a vulnerability is disclosed, it may evolve. New information may need to be communicated about the vulnerability.

ISO/IEC 29147:2018 Information technology — Security techniques — Vulnerability disclosure states the following:

9.5.12 Revision history
This section should contain the date when the advisory was first published. It can contain a modification history if the advisory is subsequently updated.

GitHub is fundamentally built on Git and diffs are commonly rendered in places like the wiki's and even issue comments (you can view edit history). This history should also be viewable on advisories as well. Being able to see how an advisory has changed over time, with new information being disclosed is valuable to the defender community.

Additionally, the ability to subscribe to information changes on a GHSA could be very valuable as well.

Slack Discussion

• https://github-partners.slack.com/archives/C013VPD24BW/p1613509427397700?thread_ts=1611951569.123500&cid=C013VPD24BW
• https://github-partners.slack.com/archives/C013VPD24BW/p1597868226043000

Display Inline Code Blocks Consistently across GHSA

User Story: As a visitor to the GitHub site looking at a GHSA, I should be able to see inline code blocks no matter what variant of the GHSA I am viewing (ie. advisory feed, from repository, or from external repository referencing another repository).

User Pain Priority: ⭐ ⭐ ⭐

These two GHSA are from the same source but render differently:

• Source GHSA: https://github.com/eclipse/jetty.project/security/advisories/GHSA-g3wg-6mcf-8jj6#:~:text=directory%0AtmpDir.mkdirs()%3B-,jetty.project/jetty%2Dwebapp/src/main/java/org/eclipse/jetty/webapp/WebInfConfiguration.java,-Lines%20511%20to
  
• GitHub 'Advisories' version: https://github.com/advisories/GHSA-g3wg-6mcf-8jj6#:~:text=eclipse/jetty.project%401b59672/jetty%2Dwebapp/src/main/java/org/eclipse/jetty/webapp/WebInfConfiguration.java%23L511%2DL518
  

Additionally, when GHSA are cross referencing another repository, the inline-code blocks don't render (for example):

• GHSA-f4jh-ww96-9h9j

Inline code block links should all render the same across GHSA publication locations.

Slack Discussion: https://github-partners.slack.com/archives/C013VPD24BW/p1642787947206400

[jhutchings1]

Thanks for this feedback thread, @JLLeitschuh !

[mkurz]

When a user watches only "Security alerts" for a repository, shouldn't that user get a notification from GitHub as soon a Security Advisory is published for that repo? Right now no notification will be send at all.

[RaphaelKimmig]

I'd really love to see that as well. Some projects only publish their security advisories on github and I'd love to be notified here when they do.

[LiroyvH]

Thank you for opening this. I notably agree that "User Story: As a repository maintainer with 'write' level permissions on a repository, I should be able to create a GHSA against my repository." is a major problem. In fact, the damn for Admin level permissions on a repository before being able to open a GHSA is a critical blocker towards effectively using the feature and severely limits the amount of people you can reasonably give this access to.

Currently, GitHub has an all-or-nothing approach to the Security Advisories. This means that if you want multiple people to be able to open GHSA's, you must give them all Admin access to the repository. That's quite the extreme and technically introduces a security vulnerability by having to assign deep permissions to achieve something that ought to be very simple. It should indeed be moved to the "write" group, at the very least the "maintain" group, but ideally you can simply per repository/team assign the ability to open GHSA's without requiring further permissions.

Moreover, there is the ability to assign "security managers" in organisations, but that role is again an all-or-nothing role. It demands that you give the user you assign this role to to get read access to ALL of the repositories; including those of other projects owned by the same organisation or other private repositories the user should not have access to.

Currently, this all-or-nothing approach is far too aggressive and leads to an unworkable situation.
Ideally, GitHub allows everyone to make custom permission profiles without having to drop $3K/year/seat, but in the interim this Security Advisories permission requirements really need to be revised.

[KateCatlin]

Hi @JLLeitschuh et al,

First of all, really appreciate the feedback. For the last couple of quarters, we've been doubling down on user interviews, synthesis of feedback, and setting aside time to knock out UX issues large and small. In order to prioritize, we need every data point we can get, and this is an excellent and thoroughly written source of data.

I'm going to walk through each of the issues I see voiced above one by one and let you know GitHub's game plan on each.

Allow Security Researchers to Open GHSA without Requiring a Maintainer to be Involved

I see you've proposed a solution to this, but it feels like the real need you're voicing is resolving this problem: "Making contact with the maintainer can be one of most complex and annoying parts of vulnerability disclosure."

If so, this is something we are working on addressing with some upcoming new features. User research and initial design are happening for that now.

Allow Repository privileges below "Admin" to open GHSA

This is an issue that's coming up in many of my user conversations and is something that requires change. I'm not committed to exactly what that change is at the moment, but I've got a process started for it and will be exploring many options.

Discoverability of an individual GH users open/disclosed/closed GHSA

This is definitely an interesting idea as we explore how to make our platform the primary place for researchers to operate. I'll document this ask so it's in our system.

GHSA Comments are Locked for EVERYONE After Disclosure

@JLLeitschuh is the need on this one resolved by community contributions? In that case, you'd open a contribution for change, the maintainer is alerted if interested, and you can have an open back-and-forth on the PR about this issue. That said, it will change on the global advisory level instead of the repo-level, but that's the one most people view statistically.

Let me know your thoughts on whether this one is partially or fully resolved.

GHSA Should Display Change History

This one I believe is resolve, at least at the global advisory level. If the advisory has changed at all (by our doing or a community contribution), you can click the link below the community contribution link to see all changes:

Display Inline Code Blocks Consistently across GHSA

Creating a ticket to look into this one, feels like something we could fix. Thanks for raising, this was not on my radar.

When a user watches only "Security alerts" for a repository, shouldn't that user get a notification from GitHub as soon a Security Advisory is published for that repo? Right now no notification will be send at all.

This one is from @mkurz, I'll create an issue for this and circle back if there are problems with fixing it. That aligns with my expectation of that feature (as a new-ish PM here) as well!

Thanks again to everyone for chiming in, please keep the feedback coming– it's how we grow.

Kate Catin
Senior Product Manager
Advisory Database Team

[carogalvin]

Hi @mkurz , I've inherited this product area from Kate. Improvements to notifications are not currently on our roadmap.

[mkurz]

@carogalvin Thanks for your reply! Do you know if following is still the case?

The way it's implemented now is that if you are an admin or collaborator and you check the box, you will receive notifications about Dependabot alerts affecting your repo. If you uncheck the box, you'll stop receiving those alerts. If you're not an admin, it appears to do nothing whether you check it or not.

If so, the "Security alerts" notification option is totally misleading and IMHO still broken. Actually what does anyone expect watching a repo (which you are not a member of nor associated with in any kind) for "Security alerts"? I would expect to get notified about any new Security Advisories that get published for that repo. E.g. if you would watch "Security alerts" for our playframework repo (which you are not associated with) you would get notified as soon as we publish something new under https://github.com/playframework/playframework/security/advisories.
I think if you keep the "Security alerts" notification option as it is and just also send out notifications to all users (no matter if admin or collaborator, so also for "normal" users, which are not even members of a repo) that are subscribed to it when new advisories get published, that would be much much more useful. What do you think? Can this be discussed?
Is this something you could make happen? From outside this does not seem like a big thing to implement 😉

[StayPirate]

I second @mkurz on that. I felt victim of this misleading UI button since Jun 2021:

Sorry for stepping in, but I just want to add that I noticed the same issue almost 1y ago and I mentioned it nextcloud/security-advisories#25 (reply in thread). My understanding was that I was able to get notifications every time a new public SA was added, but that wasn't the case and that checkbox seemed to be broken since its introduction.

Please consider the proposed solution from @mkurz:

I think if you keep the "Security alerts" notification option as it is and just also send out notifications to all users (no matter if admin or collaborator, so also for "normal" users, which are not even members of a repo) that are subscribed to it when new advisories get published, that would be much much more useful.

IMO, allowing users to opt-in to be notified about security advisories published on repositories they are interested in provides an additional channel to react to mitigation steps quickly.

[mkurz]

@carogalvin I would be availabe for a call in case you want to talk about this improvement (actually bug fix IMHO)

[mkurz]

@carogalvin Let me add, the reason we would love to have this feature work correctly is that right now we have a security mailing list which we use for security alerts concerning Play framework, like announcing critical bugs, CVE's etc.: https://groups.google.com/g/play-framework-security As you can see this is just a simple google group which users can join with their email address. We want to move away from that, so what we think about is to just tell those people to sign up to GitHub and to subscribe the GitHub "Security alerts" on our repo and they will continue receiving emails (=GitHub notifications) about CVE's (=advisories from that repo).
Right now, if people subscribe to this "Security alerts", nothing happens, they will not receive any notifications.... 😢

[StayPirate]

Hi @KateCatlin, glad to see how active you and your team are among the GH community :).
I'd like to share my feedback on GHSA with you. Like many others I'm a big consumer of RSS/Atom/Json feeds, and GH is amazing for that: I can just add .atom to many URLs to easily get a feed for that page.

For instance:

• to follow a user activities: https://github.com/staypirate.atom
• software releases: https://github.com/StayPirate/sieve-susede/releases.atom
• tags: https://github.com/StayPirate/sieve-susede/tags.atom
• commits: https://github.com/StayPirate/sieve-susede/commits/master.atom
• etc.

Unfortunately, that's not true for Security Advisories. And IMO this is quite an important resource that should be easily monitored in multiple ways. Let's take the Nextcloud project as an example, they stop maintaining their own SA RSS-feed to switch to GHSA instead (https://github.com/nextcloud/security-advisories/security/advisories), but now I can no longer ingest these data into my scripts anymore. Everyday more and more open source project start using GHSA, which is great, but the risk is that security analysts may lose visibility on SA.
It would be amazing if I could just append the .atom extension to the URL in order to get the Atom feed, like: https://github.com/nextcloud/security-advisories/security/advisories.atom.

I work in the security team of a widely used downstream Linux distribution, and being able to get these information in a non-human readable format is very important for us, RSS/Atom/Json would be a good fit. I'm sure many other users will benefit from this as well.
There is a related discussion at #7517 (comment).

[KateCatlin]

Hey @StayPirate! Thanks for reaching out. This was a great read and highly constructive feedback. Can we set up a time to chat? I haven't dug into the RSS/atom feed user journeys yet and I'd like to learn more. We'd be happy to share a gift card as a thank you for your time: https://calendly.com/security-advisories-ux-calls/25min

[dwisiswant0]

Hi, why my security advisory credit didn't update? It just showed 1 credit, but when I click-thru, it has 2 there.

[KateCatlin]

Hi @dwisiswant0, can you link me to where you're seeing this?

[dwisiswant0]

Right in the highlight of my profile, it says "1", when I click to view my credits — it shows more than one.

[KateCatlin]

Looking into it...

[lieser]

Feature Requests: pull request created in the temporary private fork for the GHSA should still be visible after it is merged and published.

We recently worked for the first time with the GHSA and there surprised that we could no longer view the merged and closed pull request created in the temporary private fork.
I think it is unfortunate that the PR description and the discussions are lost. Especially since we planned to continue to work on the closed PR after the GHSA is published.

[KateCatlin]

Hey @lieser can you tell me more about why this is important to you? Was there something specific in the PR discussion that you wanted to go back to and reassess?

[lieser]

The biggest problem here was that this loose of information came as a surprise for us. If we had known it before we published the GHSA we could have prepared for it.
So IMO the first and most important thing here is to properly make people aware of this. If there is already a warning about this somewhere and we just overlooked it, maybe consider making it more visible.

Was there something specific in the PR discussion that you wanted to go back to and reassess?

For the PR in the GHSA that got merged:
Especially the PR description, but sometimes also the comments and discussions can help better understand why something was changed. So keeping them can help others better understand the change, but it can also be useful for oneself if e.g. a year later one has to again understand why a change was done in a certain way.

For the PR in the GHSA that we closed:
There we started the work on fixing this not just for the latest release (this was done in the merged PR), but also for the next major version still under development.
The PR hat a bigger description and we also already did some code review with discussions there, but the PR was not yet finished. But we also did not want the work here to delay releasing a fix for the latest release version.
So we closed the PR, published the GHSA, and then wanted to continue the work on the closed PR. Either by recreating it from the information in the closed PR (by coping the description and open discussions) or by simply reopen it if possible.

In both cases we could have worked around it if we know about that all this information is lost. Although not the best experience IMO it still would have been OK.
And in our case we also got lucky because I still had the closed PR in an open an cached tab there I copied the description and discussions out of after a coworker noticed he could no longer see the closed PR. But if not we would have had to rewrite the longer PR description and hope that we still remember the open discussion.

[ThiefMaster]

IMHO it would be nice if there was an option to migrate the PR to the public repo after publishing the advisory. My project's changelog references the PRs for all changes, but in case of a security fix I have no public PR which I could reference there...

[lieser]

Feature Requests: optionally allow workflows to run in the private fork for the GHSA.

Although probably a good idea to disable the workflows by default, it would be nice if one can still run the workflows if explicitly requested.
Then we there recently working on a GHSA, for us it would have been fine if the runners provided by GitHub would have seen the fixed code before the GHSA got published. And it would have been helpful to easily run all the test and code analyzes we have in the CI.

The best thing would be if each job in a workflow could be independently marked as trusted to run in the the GHSA.

[KateCatlin]

Coming [waves hands vaguely] soon :)

The best thing would be if each job in a workflow could be independently marked as trusted to run in the the GHSA.

Can you tell me more about this part? Why do you want that? And would you still use Actions in GHSAs without it?

[lieser]

Why do you want [to exclude certain jobs from running in the GHSA]?

There may be some jobs that also publish the code somewhere public and are run for all branches not just master. E.g. some external service to visualize the coverage data. This are jobs most would probably not want to run for GHSAs, even if they want to enable the workflows in general.

I could imagine that it is already possible to write some exclude rule that would achieve this. So maybe just writing a helpful example in the documentation for GHSA on how one could exclude a certain job would already be enough here.

I think both the following would work:

1. Allow enabling all jobs in general for the GHSA somewhere and then needing to somehow excluding certain jobs from running.
2. Having to explicitly enable each job by marking it as trusted.

(1) is probably less work in most cases. The advantage of (2) would be to make it a little harder to enable problematic jobs by accident in the GHSA. But don't now if that is worth the additional effort.

And would you still use Actions in GHSAs without it?

Yes I would still use them, and try to come up with a way to ensure certain jobs are not run.

[glye]

Nearly 2 years later, are we any closer to be able to run CI workflows on GHSA PRs?

I understand the risk in letting 3rd party CI providers see our security PRs. But the risks in not allowing it are greater. It's sadly a common scenario: We prepare a security fix, merge it, and only then does CI run and we see the fix breaks some crucial use case. So we have to revert the fix and scramble to make a new one asap. Meanwhile the vulnerability is publicly known or possible to figure out for the so inclined, while our customers are unprotected. GitHub's policy is increasing vulnerability! Please fix this.

github/roadmap#627 was just closed as "not planned".

[gtjoseph]

I guess we're on our own.

[glye]

GitHub continues to harm security with their policy. They just "provided more details" on github/roadmap#627 but all they actually said was "We are currently focusing on other features. This has been removed from the roadmap for now, and we will revisit it once we can provide a more accurate delivery estimate."

No explanation for why they don't consider this important enough to work on. The feature as described wouldn't even solve the issue, since it says "Developers who open a temporary private fork to fix a security issue will be able to use most GitHub Actions workflows in that private fork as normal. However, because these repositories often have untrusted external collaborators, workflows will be unable to use organization secrets."

Without giving CI vendors some org secrets (access tokens to code) we can't run CI. This "security" is doubly meaningless since the CI vendors of course have access to the code in regular PRs anyway. The CI vendors depend on our trust and will lose their customers if they aren't trustworthy. Trusting those CI vendors is fundamentally no different from trusting GitHub itself. Bad actors and security flaws can exist in either one.

GitHub is a great service, but this situation illustrates the problem with relying on cloud vendors: They make the decisions, not us. One can get the impression that GitHub is exploiting its role as gatekeeper to squeeze external CI vendors out, in order to sell more of GH's own services. I hope that's not true, and that they just haven't understood the problem yet. Either way, it harms security.

[feanil]

Feature Request: Be able to give more people access to all advisories in a repo.

Right now only the org security team or repo admin can see all the security advisories on a repo.

Either allow those with write access to see advisories similar to what's happening with dependabot alerts. Or be able to grant permission to advisories to specific people or teams like you currently do with access to alerts:

[KateCatlin]

Hey @feanil this is an active conversation for us! We're very interested in updating our permissions model. Can we take some time to chat and learn more about your use case? You can book time with me here: https://calendly.com/security-advisories-ux-calls/30min?month=2023-01

[ScriptAutomate]

Related feedback, to extend the discussions that have happened here:

• GitHub Discussion: Extend APIs related to GitHub Security Advisories and their Temporary Private Forks
• GitHub Discussion: Is there a way to run GitHub Actions against a temporary private fork created from a Security Advisory?
• GitHub Discussion: Deleting a temporary private fork of a repository after addressing a security advisory

[gtjoseph]

The Asterisk team just did its first security release using Security Advisories back in December and it was neither fun nor pretty. The reasons have been mentioned before but let me give you a bit of history so you know why they're important.

Asterisk has been around for 24 years so we've got lots of history, not just in terms of commits but also baggage. We moved our entire delivery pipeline from self hosted Gerrit/Jenkins to Github only this past May. We're currently maintaining 4 active release branches some of which receive new features but all of which receive bug fixes and security fixes (it's one of those "baggage" things we can't change). This means that a typical bug fix starts its life in the master branch and gets cherry-picked to release branches. Since Github doesn't have the ability to cherry-pick and we don't want to burden our contributors or staff with having to manage multiple PRs for the same change, we have workflows that pre-test a PR against all release branches by testing for a clean cherry-pick and running our over 600 test functional testsuite. When the PR is approved, it's merged into the master branch and automatically cherry-picked to the other branches. Can you tell where I'm going with this? :)

OK so fast forward to mid-December. We had 4 security advisories, each with their own private fork, to go into 4 releases. Now bear in mind, our normal release process consists of filling out a workflow input form consisting of 1 string field for the new version and a few checkboxes. It took us all day to just get to the point where we could fill out that form. Why?...

• No workflow in private forks. Not being able to check that a PR will even cherry-pick cleanly to all branches, let alone pass the functional tests before making it public, makes the private forks as currently implemented useless for us.
• Not being able to "copy" a PR from the private fork to the parent repo is also a bit of a pain.
• No easy way to update the private fork from the parent repo.

Here's what we had to do...

• Locally create a new clone of the private forks' parent repo.
• Use gh pr checkout to checkout the PR's commit from the 4 private forks into separate branches in the clone.
• Manually test each commit branch against the release branches.
• Use gh pr create in each commit branch to create 4 new PRs in the parent repo's master branch.
• Let the existing PR workflow do its thing and cherry-pick to the release branches.
• When everything merged, attempt to publish the advisories, which failed because even though the commits actually merged into parent repo, you still have to press the "Merge" button before the "Publish" button becomes active.
• Pressed the "Merge" button for the first advisory and was dismayed to find that it created a no-op merge commit which had to be deleted (we don't use merge commits).
• Discovered that if you delete the private fork, the "Publish" button becomes active without needing to "merge".
• Deleted the other 3 private forks and published the advisories.
• Finally, kicked off the release workflows.

In hindsight, we probably could have created 1 PR that had the 4 advisory commits in it instead of 1 PR for each commit but we didn't think of it at the time.

What we're going to have to do going forward with the Security Advisory feature in its present state....

• Create a manually kicked off workflow that, given a ghsa_id, creates a new repo with that name, makes it private, adds the public repo as a remote, fetches the branches, and finally kicks off a workflow in the private repo that adds the advisory reporter as a collaborator and adds a comment to the advisory with the name of the private repo.
• Create a workflow for the private repo that periodically updates the local branches from the parent repo.
• Use the normal PR workflow in the private repo to test any PRs, then, when approved, cherry-pick them to the appropriate release branches (still in the private repo).
• Create a workflow to be manually kicked off in the private repo that, at release time, will take the commits that were previously cherry-picked and push them directly to the release branches in the public repo, bypassing PRs and workflows.

Not fun.

[gtjoseph]

@KateCatlin Any progress on this issue? Just for reference, hare's an example of what we have to go through to create a usable private "fork"...

CopyRepoToPrivate.yml.txt

[KateCatlin]

Thanks for reaching back out @gtjoseph. I know this is frustrating to hear, but this work is gated behind a large and highly-important project that we're focusing all resources on for now, and it will be some time before we're able to get to this CI support work. It is still on our minds and our roadmap though ❤️

[edemaine]

@gtjoseph Thanks for outlining the workarounds you made. I just went through 4 security advisories with 4 private forks, too, on the KaTeX project. I ran into similar issues because we have CI that automatically releases upon push to main, and I wanted all four security patches to be in a single release. Normally I'd merge PRs into a temporary release branch, but this doesn't seem to be possible: the private-fork PRs could target a different branch, but only branches in the temporary fork, not any newly created branches in the parent repository. 😦

Inspired by your workaround, I did the following slightly different steps:

• git remote add and git fetch for each private fork
• git merge --squash the private-fork branches corresponding to the PRs (--squash because that's our merge policy, also not an option with the private-fork GitHub interface sadly)
• git push these commits all at once to main (triggering release)
• Close each PR (didn't have to fully delete the fork, as you did)
• Publish the report

I also find it particularly weird that the private forks get deleted when publishing the report. As a result, all private discussions and review around the PRs is lost. This seems like a major flaw in the current system. I can understand wanting to archive the private forks, but why do they need to be deleted? Overall it would seem much cleaner to have private PRs on the main repository...

[aelnosu]

This comment is here so I will get notification about this thread.

[gtjoseph]

We've now run into into another issue trying to work around the limitations of private repos created by GHSAs... Since we can't use them, we've been creating our own private repos to run CI. Unfortunately, organizations only get 3000 minutes of Actions runtime per month across all private repos. We got caught by this limit because time spent waiting for docker images to download from GitHub's own registry is included in the limit and there was a GitHub infrastructure issue causing delays of up to 45 minutes for images that normally take 10 seconds to download.