Skip to content

Conversation

@sampaiodiego
Copy link
Member

@sampaiodiego sampaiodiego commented Jun 2, 2020

Proposed changes

I'm proposing 3 different automations to manage our issues:

  • Close issues opened without using one of our pre-defined issue templates
  • Close stale issues labeled as stat: need more info after 2 weeks
  • Close stale issues without any comments after 67 days

The stale issue handling are done via a GitHub action called stale and the other one via Close Issue app.

The main idea behind this is to get rid of very old and not relevant issues so we can focus on the important ones.

Once we enabled this, this is the issues that would be immediately affected:

  • 1459 issues marked as stale to be closed after 7 days if nobody adds a comment - link to preview
  • 23 issues marked with "no response" to be closed after 4 days if nobody adds a comment - link to preview

Issue(s)

How to test or reproduce

Screenshots

Types of changes

  • Bugfix (non-breaking change which fixes an issue)
  • Improvement (non-breaking change which improves a current function)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Hotfix (a major bugfix that has to be merged asap)
  • Documentation Update (if none of the other choices apply)

Checklist

  • I have read the CONTRIBUTING doc
  • I have signed the CLA
  • Lint and unit tests pass locally with my changes
  • I have added tests that prove my fix is effective or that my feature works (if applicable)
  • I have added necessary documentation (if applicable)
  • Any dependent changes have been merged and published in downstream modules

Changelog

Further comments

This is how they are intended to work:

Stale issues (without interaction)

  • Bot will search for issues without new comments in the last 60 days that doesn't have any of the following labels:

    • Epic
    • Feature: Planned
    • sla
    • sponsored
    • stat: waiting PR merge
    • Triaged
    • subj: security
  • For issues matching the criteria the bot will add the label stat: stale and will add a comment to explain that the issue is now marked as stale.

    • If the issue receives a new comment or any other update after being marked as stale, it will automatically be removed from stale and have it's label removed.
  • Issues with the label stat: stale that didn't receive new comments or updates after 7 days will be automatically closed.

  • New comments after the issue is closed will not open the issue again (not supported by the bot). What we could do in this (and is supported by the bot) is to add a new comment saying what the issue owner can do, it could be something like "please test with most recent release and open a new issue if still happening".

No response issues

  • Bot will search for issues with label stat: need more info without new comments or updates in the last 10 days

  • For issues matching the criteria the bot will add the label stat: no response and will add a comment to explain that the issue is now marked as stale.

    • If the issue receives a new comment or any other update after being marked as stale, it will automatically be removed from stale and have it's label removed.
  • Issues with the label stat: no response that didn't receive new comments or updates after 4 days will be automatically closed.

Since someone have interacted with the issue before the bot actually close the issue, if the owner replies after it is closed, it's responsibility to the user that add the label stat: need more info to evaluate the reply and open issue again.

@geekgonecrazy
Copy link
Contributor

maybe 60 days of no comment so almost in half? If no comment in 60 days would be hard to imagine that its really an issue. Can we add a way for them to re-open?

K8s for example on theirs it will close and then tell them a command they can use to re-open it if its still an issue

@sampaiodiego
Copy link
Member Author

sampaiodiego commented Jun 2, 2020

maybe 60 days of no comment so almost in half? If no comment in 60 days would be hard to imagine that its really an issue. Can we add a way for them to re-open?

K8s for example on theirs it will close and then tell them a command they can use to re-open it if its still an issue

the default value was 60 to add a comment and mark as stale, I changed to 90 but I regretted of this while opening the PR 😛
I didn't found anything about re-opening the issue, I'll try to find something on the source code.

edit: this github action doesn't have a logic to re-open issues.. we could possibly add another integration to re-open issues 🤔

@sampaiodiego
Copy link
Member Author

this is a sneak peek of what issues would be immediately affected:

  • 1308 issues marked as stale and closed after 14 days if no comments - link to preview
  • if we change to 60 days of inactivity it would be 1459 issues marked as stale - link to preview
  • 23 issues marked with "no response" and closed after 4 days if no comments - link to preview

@sampaiodiego
Copy link
Member Author

I have changed to 60+7 days for stale issues 😉

@engelgabriel
Copy link
Member

engelgabriel commented Jun 9, 2020

Close stale issues without any comments after 67 days, even if they are really a bug? I think we should only close stale issues if they are marked as stat: need more info. Am I missing something?

Copy link
Member

@engelgabriel engelgabriel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The time limits and the rules are too aggressive for an initial version.

with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
days-before-stale: 10
days-before-close: 4
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think those numbers are far too short. If we are moving from having no automation to having those date limits, we should start being A LOT more flexible with the dates, and gradually decrease the time limits as we fine-tune the process.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that's after adding the label stat: need more info. Do you think it should be more than 14 days @engelgabriel ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think two weeks is too aggressive to the close an issue AFTER an interaction.. and again, it will receive a new comment from the bot after 10 days in case the owner misses the notification from first interaction.. it will be closed after 4 days after the second interaction..

I'm not saying those are the right numbers. I'm just explaining exactly what will happen.. also I found other repos using the same params and found them reasonable. do you have something else in mind?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IF.. I said... IF an user opened an issue and we answered/asked in time...7 days is enough time to interact again... I mean, the user should be more interested than us to solve his problem and probably he is waiting for some reply...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it should be at least longer than the Brazilian mandatory vacations days duration.

@sampaiodiego
Copy link
Member Author

Close stale issues without any comments after 67 days, even if they are really a bug? I think we should only close stale issues if they are not marked as stat: need more info. Am I missing something?

if they're really a bug (we have verified it as a bug) it will receive a label from us to identify that.. the bot will not touch those issues. I'll describe a bit better what labels are exempt by the bot.

@makibras
Copy link

makibras commented Jun 9, 2020

+1 for re-open possibility
+1 for distinguishing really non-interacted issues ("stale") and relevant, yet not recently interacted with issues ("bugs", etc.)

for this we need a consistency with applying tags. would be good to add that to any guideline how the tags are applied.
also: though rare, if the bot is circumvented by giving your own issue a "+1"-comment to indicate activity. Will we close those manually if we come across this or do we count as activity?

@sampaiodiego
Copy link
Member Author

for this we need a consistency with applying tags. would be good to add that to any guideline how the tags are applied.

yes, everything will be added to our handbook. I hope this also empowers everyone to take more care of issues.

@sampaiodiego
Copy link
Member Author

this is a human readable explanation of how the automation is proposed to work:

Stale issues (without interaction)

  • Bot will search for issues without new comments in the last 60 days that doesn't have any of the following labels:

    • Epic
    • Feature: Planned
    • sla
    • sponsored
    • stat: waiting PR merge
    • Triaged
    • subj: security
  • For issues matching the criteria the bot will add the label stat: stale and will add a comment to explain that the issue is now marked as stale.

    • If the issue receives a new comment or any other update after being marked as stale, it will automatically be removed from stale and have it's label removed.
  • Issues with the label stat: stale that didn't receive new comments or updates after 7 days will be automatically closed.

  • New comments after the issue is closed will not open the issue again (not supported by the bot). What we could do in this (and is supported by the bot) is to add a new comment saying what the issue owner can do, it could be something like "please test with most recent release and open a new issue if still happening".

No response issues

  • Bot will search for issues with label stat: need more info without new comments or updates in the last 10 days

  • For issues matching the criteria the bot will add the label stat: no response and will add a comment to explain that the issue is now marked as stale.

    • If the issue receives a new comment or any other update after being marked as stale, it will automatically be removed from stale and have it's label removed.
  • Issues with the label stat: no response that didn't receive new comments or updates after 4 days will be automatically closed.

Since someone have interacted with the issue before the bot actually close the issue, if the owner replies after it is closed, it's responsibility to the user that add the label stat: need more info to evaluate the reply and open issue again.

@engelgabriel
Copy link
Member

Let's implement the above, but rather than closing automatically at this point, lets create a label like "BOT WILL CLOSE" and we can manually evaluate and close. If we are happy with the process, we can change the adding this label fro closing it really.

@sampaiodiego
Copy link
Member Author

Let's implement the above, but rather than closing automatically at this point, lets create a label like "BOT WILL CLOSE" and we can manually evaluate and close. If we are happy with the process, we can change the adding this label fro closing it really.

@engelgabriel I'm sry, but who will gonna check 1459 issues that will be tagged for close 67 days after we implement this? this is why I'm proposing this, to have an automation in place so we don't need to check all of them manually. if we were willing to do it manually we wouldn't have this many at this point..

I have also posted links to have a preview of the issues the bot will touch/close, so you can check it right now if any issue should not be closed by the bot and we can adjust the filters.

@rodrigok
Copy link
Member

Sounds like a good plan for me, and since the issues will have those 2 new labels for both cases it will be easy to find and reopen if we see any problem with the process.

@rodrigok
Copy link
Member

Sorry, I missed the last two comments when I did my reply.

The Diego's plan sounds good for me, I don't think we should just add the label and check later, lets allow the bot close the issues and we review later if people start complaining, as I said it's easy to revert since there will be the labels indicating the bot action.

Most of the open source repositories are using this approach, I don't see any reason to not follow the flow and let us focus more on the important ones.

@rodrigok
Copy link
Member

@sampaiodiego is it possible to check and not close if the issue is in a milestone or referenced by a pull request?

@sampaiodiego
Copy link
Member Author

@sampaiodiego is it possible to check and not close if the issue is in a milestone or referenced by a pull request?

@rodrigok no, unfortunately 😞 I've found this issue asking for this exact same thing actions/stale#20

I agree it would be good to have this ability, but looking into the labels we already use, I can see Feature: Planned, stat: waiting PR merge and Triaged which I guess have the same purpose (and they are already ignored on the bot filter).

we could run a mass update before adding the bot to make sure those labels are applied in those conditions and also include this step to the guideline I'll create.

@rodrigok
Copy link
Member

@sampaiodiego is it possible to check and not close if the issue is in a milestone or referenced by a pull request?

@rodrigok no, unfortunately 😞 I've found this issue asking for this exact same thing actions/stale#20

I agree it would be good to have this ability, but looking into the labels we already use, I can see Feature: Planned, stat: waiting PR merge and Triaged which I guess have the same purpose (and they are already ignored on the bot filter).

we could run a mass update before adding the bot to make sure those labels are applied in those conditions and also include this step to the guideline I'll create.

I think we can use GitHub ci to keep those labels. Then we can base the bot only on the labels without problem.

@sampaiodiego
Copy link
Member Author

I think we can use GitHub ci to keep those labels. Then we can base the bot only on the labels without problem.

that works too =)

@sampaiodiego
Copy link
Member Author

another App I crossed upon today https://github.com/apps/lock

@sampaiodiego
Copy link
Member Author

Is there anything missing?

@geekgonecrazy
Copy link
Contributor

I’d say at some point we gotta just try it. So long as we aren’t being too aggressive, we can adjust and or get rid of it if it doesn’t work.

@rodrigok rodrigok merged commit 9852963 into develop Jun 18, 2020
@rodrigok rodrigok deleted the add-apps-control-issues branch June 18, 2020 23:14
ggazzo added a commit that referenced this pull request Jun 19, 2020
…into feat/new-threads

* 'feat/new-threads' of github.com:RocketChat/Rocket.Chat:
  Add Apps to control GitHub issues (#17807)
@Sing-Li
Copy link
Member

Sing-Li commented Jun 20, 2020

If the issue receives a new comment or any other update after being marked as stale, it will automatically be removed from stale and have it's label removed.

@sampaiodiego When and how will this happen?

It seems with issues such as this one, the stale label is not removed after new comments:

#17100

@rodrigok
Copy link
Member

@Sing-Li if you check the issue again the label was removed by the bot. It's not right away since the bot runs once a day.

@sampaiodiego sampaiodiego mentioned this pull request Jun 30, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants