Don't add useless labels which will bother changelog generation

[lunny]

When generating release notes for v1.26, many pull requests haven't been given correct labels so that I have to do many manual work. I think this could be avoid to remove these useless modify labels.

[silverwind]

Can't we just ignore modifies/* labels in that changelog tool? Where is it?

I do find these labels somewhat useful, but not overly so.

[lunny]

Can't we just ignore modifies/* labels in that changelog tool? Where is it?

I do find these labels somewhat useful, but not overly so.

The problem is maintainers will forget to add correct labels once modify labels are added. The changelog tool itself never read modify labels because they cannot be used to generate changelogs.

[silverwind]

Maybe we should just use AI to generate the changelog. Will probably be much more accurate then those labels will ever be because almost no one sets them correctly.

I don't think I can approve removing all these labels, they could be slimmed down considerable, but outright removal is too much imho.

Can you manually fix this up for v1.26 or do you absolutely need it now?

[lunny]

Maybe we should just use AI to generate the changelog. Will probably be much more accurate then those labels will ever be because almost no one sets them correctly.

I don't think I can approve removing all these labels, they could be slimmed down considerable, but outright removal is too much imho.

Can you manually fix this up for v1.26 or do you absolutely need it now?

Could you have some real usages for these labels? I don't know how will you use these labels and how many users/maintainers will use these labels.

[silverwind]

They give me a overview what a PR touches. When commits are pushed, it's often nice validation to see labels being added/removed.

[lunny]

They give me a overview what a PR touches. When commits are pushed, it's often nice validation to see labels being added/removed.

If you are using AI, it can have a summary for which files changed on the PR description which is more useful than these labels.

i.e. What could you know from label modifies/internal? Or modifies/frontend?

[silverwind]

Those 2 are useless, especially internal is ambiguous and should be dropped. modifies/go for example is somewhat useful. I could live with these 4 only:

• modifies/go
• modifies/ts
• modifies/css
• modifies/templates

But no hard blocker I guess. If more people agree to remove all, I'm fine.

[silverwind]

Where is that changelog tool's source code? I think this is a weak attempt at fixing the problem from the wrong angle. The real fix should be in that tool.

[silverwind]

Not acceptable, this is a blind workaround, not a root cause fix (which is in the changelog tool being too eager on labels).

[TheFox0x7]

Why not mandate each PR to have a small markdown file for the changelog?
I don't trust LLM to accurately summarize each PR in history without going off track and making stuff up - and you'd have to look through it either way so doing (even LLM made) change log per PR seems more managable to me.

[silverwind]

I would remove this changelog tool and either:

• adopt a single "notable-change" opt-in label which, when set, will include a change in the changelog. small manual management effort, but guarantees no false-positives
• force authors to write changelogs, imho bad approach, too much extra work for authors we shouldn't expect from authors
• make changelog include a full unfiltered list of commits (i do that on all my repos)
• run an AI at release-time to identify notable user-facing changes and list them in the changelog by commit title, one-time action every 3 months

[silverwind]

Why not mandate each PR to have a small markdown file for the changelog?

This forces extra work onto every PR author that they might not be willing to do, I'm against it. Some authors don't even bother to write a good PR title or descriptions.

I don't trust LLM to accurately summarize each PR in history

I don't mean summarize, I mean use AI just to identify notable user-facing changes and filter the list of commit titles. Same output format as currently.

[TheFox0x7]

make changelog include a full unfiltered list of commits

Yeah this won't work here unless PR descriptions get shortish and get updated before merge. Which isn't always the case from what I seen.

run an AI at release-time to write the changelog for notable changes, one-time action every 3 months

Still requires you to go through the output, unless you're willing to risk non-existing changes being documented. I don't trust LLMs to produce reliable output in that regard. Besides - personally - I don't like reading what they write. It feels... wrong. I can't explain why but there's sometimes an uncanny valley feeling to the text they make.

adopt a single "notable-change" opt-in label which, when set, will include a change in the changelog. small manual management effort, but guarantees no false-positives

It's more for a blog post I think... But it's a good idea regardless.

force authors to write changelogs, imho bad approach, too much extra work for authors we shouldn't expect from authors

why? Who else would know what the change changes? Author is the right person to write what their change does.
Some authors don't bother reading what they wrote in PR description or in the code they generated. I guess we should start enforcing the PR description first.

[silverwind]

I don't like reading what they write

You misunderstand. AI will not be writing any content, it will just do the filtering only, so it's job is just the identification of relevant PRs only.

Some authors don't bother reading

Lazy authors will not bother and contribution count will go down. Many core contributors I would classify as lazy, they often open PRs with "no description" and sloppy titles.

[TheFox0x7]

You misunderstand. AI will not be writing any content, it will just do the filtering only, so it's job is just the identification of relevant PRs only.

I did write it before you expanded on it so yeah, I did. Regardless I don't trust it to do that either. You have no guarantee that it'll catch all PRs that are relevant, nor that it won't include irrelevant ones.
Manual (label or via an author made comment) approach is just reliable and reproducible (and it costs 0 units of any currency to run). If you want you can run LLM on top of that to look if there's something that initially was missed and I have no issue with that being an add-on on top of the "existing" process

[lunny]

Those 2 are useless, especially internal is ambiguous and should be dropped. modifies/go for example is somewhat useful. I could live with these 4 only:

• modifies/go
• modifies/ts
• modifies/css
• modifies/templates

But no hard blocker I guess. If more people agree to remove all, I'm fine.

What do these labels mean to you? Once you find them, what information do they provide, and how would you interpret and act on it?

[lunny]

Not acceptable, this is a blind workaround, not a root cause fix (which is in the changelog tool being too eager on labels).

These labels are not intended for changelog generation. I couldn’t find any meaningful purpose for them, and they only seem to confuse or bother reviewers.

[silverwind]

These labels give me context what subsystems a PR touches, it is marginally helpful imho. I already told that 3 times already.

These labels are not intended for changelog generation

Then ignore them in that tool, I already told you that 3 times already too.

[lunny]

These labels give me context what subsystems a PR touches, it is marginally helpful imho. I already told that 3 times already.

These labels are not intended for changelog generation

Then ignore them in that tool, I already told you that 3 times already too.

Do you mean that when a PR is labeled with a special modifies/xx label, that label determines whether you will review it without even clicking into the changed files? Personally, I don’t usually look at these labels—I make my judgment based on the PR title and description instead.

.

[silverwind]

I'll just remain neutral on this topic. If others don't like these labels I'm ok with removal. They do add some noise which I agree on.

[bircni]

I would also like to keep them

[bircni]

Was there ever a discussion about conventional commits? Would also help a lot I think

[silverwind]

I think labels could be kept but reorganized to be by subsystem purely, interesting subsystems identifyable by path alone may be migration, build, ui. Ideally no prefix.

[silverwind]

Let's remove them for now. @lunny after merge delete all relevant labels on GitHub UI please.

[silverwind]

conventional commits

Not sure. If we generate Changelog via AI, than I'd say no, not needed.

[silverwind]

Looking at the PR list now, it looks bare without the labels, making it impossible to visually categorize PRs. Not even dependency-touching PRs (which require extra scrutiny) are marked. I think it's a step back. I would definitely want to add automatic subsystem labels back e.g. a deps label etc.

[bircni]

Looking at the PR list now, it looks bare without the labels, making it impossible to visually categorize PRs. Not even dependency-touching PRs (which require extra scrutiny) are marked. I think it's a step back. I would definitely want to add automatic subsystem labels back e.g. a deps label etc.

Yes please!!!

[silverwind]

I could make Claude find a label scheme for subsystem-based labels, I'd keep simple with no modifies/ prefix, but if filtering based on that prefix is desired, we can keep it. @lunny please provide your feedback on that first.

[lunny]

Sorry. I’ve never felt the need for those labels, and I’m personally very satisfied with the current system.

My habit is to read every GitHub notification, so I review every PR notification anyway. Because of that, I don’t really need labels to help me filter PRs. I trust the labels that maintainers apply to understand what a PR is for.

What's your requirements to filter PRs via labels?

[silverwind]

I don't really use it for filtering but as a visual classification. Take a look at https://github.com/nodejs/node/pulls, i want to replicate that, e.g. deps, go, js, mig, actions etc labels, very simple and short labels.

BTW you still have not told me where that changelog tool lives. I think it should be trivial to add a label exclusion config from within the repo.

[lunny]

I don't really use it for filtering but as a visual classification. Take a look at nodejs/node/pulls, i want to replicate that, e.g. deps, go, js, mig, actions etc labels, very simple and short labels.

What's the labels meant for them? I will not block the tool if there are enough maintainers need it.

BTW you still have not told me where that changelog tool lives. I think it should be trivial to add a label exclusion config from within the repo.

Sorry. Missed that. It's in https://gitea.com/gitea/changelog

[silverwind]

1. It's less about "need" and more about "nice to have"
2. Ok, I see that changelog tool is stale since 2 years, maybe we should adopt a better system like conventional commits. Better to use something well-thought out instead of such homebrew tools.