A proposal for bug and issue handling, and roadmap management

[dannon]

This is a still-rough proposal for how we might use GitHub issues for bug and project development goal tracking, replacing the galaxy-development board completely.

We've tried several different approaches over the course of the project and two common problems this approach attempts to address are:

• The issue graveyard -- where, once off the first 'page' of issues or below the 'fold' in Trello, things sometimes do not resurface and are lost to history
• Difficulty clearly presenting and maintaining a high level project roadmap and associated meta-issues.

I think with good (and enforced!) labeling behavior and the rules surrounding issues/prs and milestones we can make some nice improvements.

Please comment and submit any changes, all opinions welcome. I will probably wake up and fix grammar in the morning.

[dannon]

Forgot to mention migration strategy. I don't think we should auto-migrate everything (partially because then we'd only have to triage it again, anyway, under these rules).

I'd propose letting issues trickle over, letting people spend time on it as they'd like over the course of a couple weeks(?). We'd set a firm deadline, after which we'd sit down as a team and do a mass-triage (which is hopefully motivation to do it in the allotted time).

[nsoranzo]

Here "issue is resolved" is not clear to me, do you mean that the issue has been addressed with a merged PR and we ask the submitter to check if the issue is resolved for him?

[dannon]

Yeah, my intent was that it indicated the issue needed some extra verification prior to close. That might not be a useful indicator for issues, though, and I'd expect many issues to go directly from WIP to closed (especially when using the GitHub auto-close like "resolves #ISSUENUMBER".

[martenson]

For the milestones specification and focus I vote huge 👍 As I see it, more important than where to keep the record is to actually sit down in the beginning of the cycle and plan ahead which features will be implemented and what bugs fixed in the next release(s).

The issue tracking format/place/tags/triage I am sceptical about as I expect it to end the same way as BB/Trello ended. I see no easy way out when digging through hundreds of issues/cards/ideas. We might get minor improvements like better search and tags, however we lose e.g. votes and history.

I would much rather see us to start working on 16.01 milestone breakdown (through GitHub) than to spend time digging through Trello for the sake of migration to GH issues.

[jxtx]

I really like "kind" better than class here.

[jmchilton]

The documentation is part of the code and the code is part of the documentation, this seems like a harmful or arbitrary distinction to me. I think kind/documentation should be dropped - either the documentation is wrong (kind/bug) or the documentation needs to be improved (kind/enhancement).

[hexylena]

agreed, not a useful distinction

[jxtx]

What about tasks that really are purely documentation and don't touch code? Not everything we do is code. Do we manage those issues somewhere else?

[martenson]

@jxtx Leaving them in Trello is an option. Same as 'management', 'main', 'deployment' etc. tasks.

[jmchilton]

@jxtx Documentation in repository or out? I assume you mean out, so like on the wiki.

I think we should have a separate Galaxy project project on github for discussing project wide things via issues and laying out procedures for things that aren't directly related to this repository (like how we manage galaxyproject teams on github for instance). That repository may or may not be the right place to track such issues.

[jxtx]

This comparison doesn't work for me. Of course programs must be written primarily for people to read. That doesn't mean there are not tasks that are primarily writing documentation in nature.

Still feels like a kind to me. I read area as an area of the application impacted, while kind refers to the kind of work that needs to be done. Of course, there are some things that are less clear to me like UI.

[jxtx]

@martenson A goal here, at least for @nekrut and I, is to gradually move everything out of Trello. So, this would start with "Galaxy Development" becoming read-only and moving everything here. Then the rest of our public boards, either here or to another repo.

ToolShed is an interesting case. When does it get its own repository?

[hexylena]

@martenson since my alternative "blue sky" toolshed is unlikely to catch on any time soon... in the short term would you have any interest in doing this with me? Clone galaxy into a separate TS repository, and slowly remove everything but TS code, then work toward adding/removing features/cleaner API, internal refactoring, etc?

[martenson]

@erasche I certainly have interest in that, however the proper prioritization of this is important, let's not dilute this PR though. Created a new issue here: #906

[hexylena]

Agreed, thanks. That was slightly off topic, you're right. Just wanted to respond to @jxtx's comment about separate TS trello and have a plan for that.

[nturaga]

👍 to having the "roadmap" on github.
Wondering about the "status" tag, can all sub-tags have justification or specifically how do we document something that won't fix?
Another tag suggestion is the one we already have i.e "help-wanted".

[nsoranzo]

Maybe add a link to doc/source/project/organization.rst for further details?

[hexylena]

Will add to p4. Update: galaxyproject/p4#2

[dannon]

@jmchilton I agree re: procedures. It'd be a bummer to have to issue a procedures PR and wait a week to add a new area tag, something that I'm sure we'll have to do at least a few times in the first few weeks.

[nsoranzo]

About procedures: I also agree this should not be part of procedures documents.

About kind/documentation: My preference would be area/documentation or drop it.

About ROADMAP.md: I vote for less redundancy, drop it.

About missing things: https://github.com/galaxyproject/galaxy/pull/902/files#r41880224

[martenson]

Does this PR imply/demand/mean that we are going to convert the whole Trello into GH issues?
ping @jxtx

[jmchilton]

Also want to bump this issue also dannon#4. Creating minor and major tags. This would simplify creation of release notes and the minor tag is why I opened #871.

[dannon]

@martenson In #902 (comment), I proposed a general trickle, after which we'd do a triage if it isn't going to work. I think it would be a mistake to just abandon issues that have been reported to Trello, so they have to be moved or resolved where they are (which I think is unlikely that anyone will do).

[dannon]

@jmchilton Can we infer minor/major from 'kind', though? That was my hope, to avoid a release-notes specific tag.

[jmchilton]

@dannon Give the proposed definition - I believe there are certainly major and non-major bug fixes, feature, and enhancements. Also there are both minor and non-minor bug fixes.

[dannon]

Hrmm. Is there a way we can tweak feature/enhancement/bug/refactoring to encompass this, instead of having effectively a fourth required tag set? I was sort of thinking the release notes would just break down along those lines.

1. New features
2. Enhancements to existing features
3. Bug fixes

Wild idea- Can/should we tweak our milestone usage here? Things that would be 'minor' simply aren't associated with a milestone, since we don't really care to indicate them?

[jmchilton]

90% of PRs won't be either major or minor - this is only a new required tag set in the way procedures or not procedures is - or any of the other special tags.

I feel like even in those three categories, we still want to push major stuff to the top and still want to exclude a few select things.

I like explicitly stating minor with a tag better than not associating it with a release - feels more explicit and less like something is falling through the cracks. If the milestone are getting auto assigned though it would take an explicit choice to drop them.

[dannon]

@jmchilton Hrmm, k. Let's try it, see how it works.

[dannon]

Does anyone have any major outstanding concerns about this?

We can certainly continue to tweak the wording as we move forward (we are not classifying this as Procedures for PR purposes, so it'll be easy to do), but I'd like to go ahead and get the basics implemented.

[jmchilton]

@dannon I'm very happy with how this turned out, thanks for putting it together and incorporating suggestions even when you weren't 100% sold. I'll give this a few more hours and then merge it if no one else has.

[martenson]

I raised a question on IRC about how to handle bugs affecting Main. Does a label solution Main affected that would cause the issue not be closed until Main is fixed sound plausible?

Regardless 👍 on this and thanks @dannon