[RFC]: Enable stale bot for issues and PRs

[russellb]

Motivation.

There are currently >400 open PRs and ~1.7k open issues. Many of these are not active and could be closed. This problem is often more challenging in a very active and popular project like vLLM. GitHub provides an action to help automate this process.

https://github.com/actions/stale

By adopting this action, those with the time and interest in helping with issue triage or PR reviews will have an easier time ensuring their time is used most effectively.

Proposed Change.

I propose adopting actions/stale to automate the process of pruning the issue and PR queues of items with no activity.

A sample policy would be something like this:

• Add the stale label and leave a comment after an issue or PR has not had any activity within 90 days.
• Close an issue and add another comment if an issue or PR continues to have no activity for an addition 30 days.

The times are configurable. An interested party can leave a comment to reset the stale timer. Configuring a label that will tell the bot to ignore that issue or PR (like keep-open or something) is also possible.

If there's interest, I am happy to do the PR for it. Agreeing on the policy is the harder and more important part.

Feedback Period.

No response

CC List.

No response

Any Other Things.

No response

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[russellb]

a draft PR showing sample configuration is here: #9436

[simon-mo]

This sounds good. Two comment:

• I created a keep-open label. However, not everyone has the ability to attach label. Can this be done by some other way?
• We don't have to close PRs that are draft. Maybe draft PR can be a good way to avoid getting stale and closed.

[hmellor]

I also support this, in the past I have spent many hours closing 100s of issues but it's not sustainable.

I'm happy to be a reviewer on the PR to dial in the behaviour.

[hmellor]

actions/stale has an argument to exempt draft PRs from being marked as stale (and therefore closed) https://github.com/actions/stale?tab=readme-ov-file#exempt-draft-pr

There is also an argument to exempt specific labels like keep-open https://github.com/actions/stale?tab=readme-ov-file#exempt-pr-labels

[russellb]

This sounds good. Two comment:

• I created a keep-open label. However, not everyone has the ability to attach label. Can this be done by some other way?
• We don't have to close PRs that are draft. Maybe draft PR can be a good way to avoid getting stale and closed.

Drafts and issues with the keep-open label are now ignored in the proposed config. Thanks, @hmellor!

I have mixed feelings on access to apply the keep-open label. I propose for now having a github workflow that will allow anyone to use a "/keep-open" command in a comment to add the label. That could get abused, so I wouldn't necessarily advertise it since really we'd rather issues and PRs actually stay active. We can always remove that ability if it is abused, though. I'll include that in the draft PR.

[russellb]

As an interesting data point, here is the expected impact of the proposed config based on closing things after 120 days of inactivity:

• Number of issues not updated in the last 120 days: 495

$ gh issue list --repo vllm-project/vllm --limit 2000 --state open --json number,updatedAt --jq '[.[] | select((now -
 (.updatedAt | fromdateiso8601)) > (120 * 86400))] | length'
495

• Number of PRs not updated in the last 120 days: 106

$ gh pr list --repo vllm-project/vllm --limit 2000 --state open --json number,updatedAt --jq '[.[] | select((now - (.
updatedAt | fromdateiso8601)) > (120 * 86400))] | length'
106

[hmellor]

I agree that a single comment is might make it a little too easy to keep an issue open forever.

In the PR you have the workflow running on every comment, might that get expensive given the volume of comments received in this repo?

@simon-mo would it be so bad if the only people able to add the keep-open label had triage permissions and above?

[hmellor]

Actually, how about instead of automatically adding keep-open on "unstale" we add an intermediate label such as unstale (which would be removed if the issue went stale again).

Then people with triage and above can filter by this label and decide on a case by case basis?

[russellb]

I like restricting it to those with triage access.

unstale to indicate that "this was once stale, but then was updated again" also seems helpful for finding things that have been dragging on for a long time.

It's probably worth adding a page to the docs that covers all of this process info! I'll add that when the dust settles.