add CHANGELOG.md, back to the dawn of time

[h-vetinari]

In #2364, the following came up:

@kolyshkin: Are we doing any kind of changelog somewhere?

@cyphar: I think we discussed that there should probably be a CHANGELOG.md which PR authors should update, but the problem is that we probably don't have the time to back-fill it with all the other pre-1.0 releases.

Well, this is actually pretty easy (in a basic form at least), so I here's a PR. Note that this is limited to detecting and annotating merges to master, so it will not detect direct commits (to the extent that they happened).

The way I produced this:

• git log --oneline --merges v1.0.0-rc10..HEAD >> CHANGELOG_raw.md
git log --oneline --merges v1.0.0-rc9..v1.0.0-rc10 >> CHANGELOG_raw.md
...
git log --oneline --merges v0.0.1..v0.0.2 >> CHANGELOG_raw.md
git log --oneline --merges 6415e8be..v0.0.1 >> CHANGELOG_raw.md
  Splitting this by tag is important because merges of release commits sometimes happen after other merges have happened. Here I also inserted breaks to easily recognise the different blocks.
• Apply a python script that parses the lines and fills in the PR title and author (through the github API)
• A few manual edits (where the extraction failed; add headers for tags; reorder release-PRs correctly).
• Some more manual intervention to deal with the repo-breaks.

The core of the python script is:

PR_API_PATH = 'https://api.github.com/repos/opencontainers/runc/pulls/'

with open('CHANGELOG_raw.md') as f:
    lines = f.readlines()

def add_pr_title(line, auth=None):
    m = re.search(r"^[a-f\d]+ (?:Merge pull request \#(?P<num_gh>\d+).*|[Mm]erge branch 'pr-(?P<num_cyphar>\d+)')", line)
    if m:
        pr_num = m.group('num_gh') if m.group('num_cyphar') is None else m.group('num_cyphar')
        print(f'Fetching title for PR #{pr_num}')
        try:
            r = requests.get(PR_API_PATH + pr_num, auth=auth)
            author = r.json()['user']['login']
            # readlines leaves in the linebreak; add here as well
            line = f'* {r.json()["title"]} (#{pr_num} by @{author})\n'
        except Exception as e:
            print(f'Error fetching title for PR #{pr_num}: {repr(e)}')
            return line
    return line

lines = [add_pr_title(line, auth=auth) for line in lines]

with open('CHANGELOG.md', 'w') as f:
    # lines have their own \n
    f.write(''.join(result))

I'm happy to share the full script and/or the prepared CHANGELOG_raw.md, if someone wants to write their own parser.

[tianon]

(I'm not a maintainer on this repo, just providing my thoughts.)

If it's just the git log output (lightly munged), IMO this doesn't have much value over what we already can get from Git -- do you think it would be feasible to instead get the human-curated release notes that have been attached to each GitHub release?
(https://github.com/opencontainers/runc/releases, maybe minus the "thanks" sections)

It could also include links like https://github.com/opencontainers/runc/releases/tag/v1.0.0-rc7 and v1.0.0-rc9...v1.0.0-rc10 for folks that want to drill down into the low-level details.

[kolyshkin]

should exclude or escape formatting symbols in some way

[cyphar]

I agree with @tianon. When I said that it would be hard to do, I meant producing a useful CHANGELOG.md. Formatting the git log and inserting it into a file really doesn't help anyone get a good overview of what changed in a release. Ideally, the CHANGELOG.md should be clean enough that you can just insert it directly into the release notes (that's what I do for umoci).

A better place to start than the commit log would be the release notes (at least for the last 8 or so releases). For all of those releases, I went through the commit log by hand and picked out all of the major features (explaining what the changes were).

[h-vetinari]

@tianon: If it's just the git log output (lightly munged), IMO this doesn't have much value over what we already can get from Git

I'm not sure if others people will agree, but I think this is separate from the release notes, as in: the release notes would IMO be a distilled version of the changelog, possibly grouped further into features/fixes etc.

do you think it would be feasible to instead get the human-curated release notes that have been attached to each GitHub release?
(https://github.com/opencontainers/runc/releases, maybe minus the "thanks" sections)

I know these of course, but it doesn't give nearly a full picture, IMO.

It could also include links like https://github.com/opencontainers/runc/releases/tag/v1.0.0-rc7 and v1.0.0-rc9...v1.0.0-rc10 for folks that want to drill down into the low-level details.

Sure, absolutely.

[cyphar]

I disagree about the purpose of a changelog (though I do recognise this is a historical disagreement between different camps -- the whole GNU News file is a "changelog" which was more akin to a commit log). IMHO, if you need to know in detail what changed, you have git log and we gain very little by serialising that to a file in the repo.

In addition, the git log is often insufficient to nicely describe (to users) what a new feature actually does -- in my view, that's a minimal requirement of what a changelog should contain. I personally think we should just adopt the style of Keep a Changelog (which is what I use for umoci and it works perfectly fine). Minor patches don't need to be mentioned in a changelog, and major patches deserve a proper description (not just the subject line of a patch).

[h-vetinari]

@cyphar: IMHO, if you need to know in detail what changed, you have git log and we gain very little by serialising that to a file in the repo.

For a maintainer, the difference might be negligible, but the difference for someone who hasn't cloned the repo yet is not trivial.

In addition, the git log is often insufficient to nicely describe (to users) what a new feature actually does [...]

Absolutely, but deciding what is a feature worthy of mentioning is already a non-trivial task (and IMO the difference between a changelog and release notes). But I'm not claiming to have the solution here already, only possibly a basis for forming that. If such a log is considered trivial (again: that's fair enough), then no worries.

[h-vetinari]

@cyphar: IMHO, if you need to know in detail what changed, you have git log and we gain very little by serialising that to a file in the repo.

@h-vetinari: For a maintainer, the difference might be negligible, but the difference for someone who hasn't cloned the repo yet is not trivial.

Some more thoughts: Even for someone who cloned the repo (rather than the likely larger group of people just browsing on Github), git log is IMO not such a big help to understand what happened from one release to the next.

I'd argue that the most granular unit of change that is reasonable for an indepth overview is a PR (not a commit), but that's not easy to get from the log at all. Either you get the full firehose of all individual commits, or the following with git log --oneline --merges :

3b7e32fe Merge pull request #2210 from Zyqsempai/2164-remove-deprecated-systemd-resources
688cf6d4 merge branch 'pr-2223'
0f32b03d merge branch 'pr-2192'
13b1603f Merge pull request #2224 from kolyshkin/systemd-props
81ef5024 Merge pull request #2213 from Zyqsempai/2166-convert-cpu-weight-poperly
e6555cc0 merge branch 'pr-2184'
ff107ee0 merge branch 'pr-2190'
2b5730a5 Merge pull request #2221 from inductor/feature/fix_path_security

Compared to that I think there's added value for having something like the following, and in an easily accessible manner (which would also make it very easy to dive in deeper into specific PRs where necessary/interesting).

* Exchange deprecated systemd resources with the appropriate for cgroupv2 (#2210 by @Zyqsempai)
* Fix the value corresponding to rlimitmap [key] (#2223 by @wanghuaiqing2010)
* Fix MAJ:MIN io.stat parsing order (#2192 by @Zyqsempai)
* Allow to set systemd unit properties via annotations (#2224 by @kolyshkin)
* Added conversion for cpu.weight v2 (#2213 by @Zyqsempai)
* README.md: modify the explanation of make flags (#2184 by @KentaTada)
* Adding Security audit (#2190 by @amye)
* Fix path for security report line (#2221 by @inductor)

Now, release notes fill a different role IMO. They should definitely categorize (features/security/etc.), collect and filter the PRs from the CHANGELOG into a more high-level overview. There are two variants that I see often:

• either the RELEASE_NOTES are completely separate from the CHANGELOG
• or the RELEASE_NOTES are formed directly from the CHANGELOG, leaving all items not covered by the high-level summary under "other changes/fixes".

[h-vetinari]

The larger question is: how should the CHANGELOG be maintained in the future? (because, with some effort, it should be possible to restructure the log here along those lines...)

For example, it's common practice in other projects that PRs need to write their own news item (reviewed along with everything else). If it qualifies for a highlight, even that can already be done within the PR.

This could be filled in in a preformatted file (e.g. docs/changelog/v1.0.1.rst) along the lines of:

### Changes in runc v1.0.1.rst (20yy-mm-dd)

## Security

* [insert updates]

## Features

* [insert updates]

## Spec-Compliance

* [insert updates]

## Fixes

* [insert updates]

## Miscellaneous

* [insert updates]

All the release notes would have to do is add a summary at the top and the thanks at the bottom. This is quite close to what was used for the release notes from rc3 to rc8. Some similar examples: docker, kubernetes, podman, lxd, etc. etc.

[AkihiroSuda]

how should the CHANGELOG be maintained in the future?

We can just maintain them in https://github.com/opencontainers/runc/releases

[h-vetinari]

@h-vetinari: how should the CHANGELOG be maintained in the future?

@AkihiroSuda: We can just maintain them in https://github.com/opencontainers/runc/releases

Of course, but I was talking about the format of the release notes.

[kolyshkin]

Can I suggest

1. Writing the rc11 changelog in the format outlined in add CHANGELOG.md, back to the dawn of time #2368 (comment), put it to CHANGELOG.md, reuse when making a release.
2. Add something like
   "for older releases, see git log" to the end of CHANGELOG.md
3. For every future release, write a changelog before a release, based on PRs merged.
4. For any new PRs, suggest adding "entry for a changelog" in PR description (we can use PR template for that, or just add a recommendation).

[h-vetinari]

@kolyshkin
I think that's a good plan. Depending on the chosen format (and likely some review/feedback), I could try preparing such notes going back a few releases. IIUC, these could retroactively be included in the release notes, independently from whether a CHANGELOG.md gets added.

[h-vetinari]

@AkihiroSuda @cyphar
What about @kolyshkin's suggestion above?

[cyphar]

I've created a more human-friendly changelog from the release notes from 1.0.0 and later releases in #3320. If you'd still like to help back-fill the changelog you can open a PR afterwards to add new entries for old releases (it's fairly easy to back-fill data for old releases since all of the release notes are available in a human-readable form on GitHub).

[h-vetinari]

If you'd still like to help back-fill the changelog you can open a PR afterwards to add new entries for old releases

Would that be welcome? Given the fate of this PR, it feels like the appetite is low, and hence the chance of me wasting my time high...

[cyphar]

The issue with this PR was that it wasn't based on the human-readable release notes. Since we already have human-readable release notes written (in the GitHub release text) it would most just be a reformatting job (that's how I created the entries for 1.0.0 to 1.0.3 and it only took a few minutes each).

But I wouldn't say it's high priority, since adding data for several-year old releases isn't going to be something a lot of people would need. It's up to to you, and if you don't feel you want to spend the time on it that's also perfectly okay. :D