OWD BCD project: Implement a process for updating BCD whenever a new browser release is available

[Elchi3]

Problem statement

When browser vendors release a new version of a browser, compatibility information in the BCD project becomes outdated. Known web platform features might become supported in more browsers or entirely new web platform features might ship in any of the browsers for the first time.

Proposed solutions

To ensure web developers have up-to-date browser support information available as fast as possible and MDN author know about newly available web platform features, the BCD project should introduce a process that is triggered automatically whenever a new browser release is available. It should update existing data points and add new data points for features shipping for the first time in any of the browsers.

We can start by making this work for stable browser releases and expand it to beta and preview versions if desirable.

Task list

Previously, @queengooborg described the process like this:

1. Initial setup: Locally check out BCD, the collector and the collector results repos (preferably adjacent to each other)
2. Run the collector website on the new browser version
3. Submit the results to GitHub using the appropriate export button, then checkout the PR branch while waiting for merge
4. Run the update-bcd script using --browser=<browser_id> --release=<browser_version>
5. Start submitting BCD PRs from the changes made to the BCD local copy

We should investigate how to automate more of this and also look into how we will be notified about new browser releases as the trigger point of this whole process. Thoughts around this are also in mdn/browser-compat-data#1664.

It's OK and desired that there is a human control step in this process. Most of the mechanical work should be automated, though. BCD peers are required to assure quality of the pull requests (as usual).

• Figure out how to get notified about browser releases and update the data in the browsers/ folder
• Automate the process described above as much as possible. Existing features should get updated and entirely new web platform features shipping in a browser for the first time should be added.
• Report about new features of a browser release and about which features have newly become interoperable ("baseline").

Priority assessment

• Effort: Medium. Estimating 8-12 weeks. (a quarter).
• Dependencies: BCD collector, GitHub actions, information/API about new browser releases
• Community enablement: n/a This is infrastructure work that can likely only be worked on by team members. It will take off work from contributors who work on adding and updating data manually for a given release.
• Momentum: We can create momentum by reporting all new features of a browser release to the community.
• Enabling learners: n/a
• Enabling professionals:
• Underrepresented topics / Ethical web: n/a
• Operational necessities: no
• Addressing needs of the web industry: Accurate compatibility info helps reducing web developer's pain with compatibility. Could also look into reporting features that have become interoperable (or "baseline") as we update data for a release.

More information

Open Web Docs (OWD) is a non-profit collective funded by corporate and individual donations.

In order for this project to happen, please consider donating to OWD on https://opencollective.com/open-web-docs.
For more information on sponsorship and membership tiers, see https://openwebdocs.org/membership/

More information is available at https://openwebdocs.org/.
For questions, please reach out to [email protected].

[tomayac]

This is a very important project!

Quick question on step 2. if I may: Where can I find the collector website ?

[Elchi3]

Quick question on step 2. if I may: Where can I find the collector website ?

It currently lives here https://mdn-bcd-collector.gooborg.com

[tomayac]

Wow, thank you! Very helpful!

[lolaodelola]

cc @foolip @jugglinmike @mariestaver @mzgoddard

[lolaodelola]

@queengooborg @Elchi3 This is something the Bocoup team have on our roadmap to tackle, can we please add it to the agenda for discussion at Tuesday's (03/Oct/23) meeting?

[Elchi3]

@lolaodelola Yes, it has been on the agenda already. Would be lovely if you could join!

[foolip]

One important piece here is automating the creation of pull requests, as discussed in openwebdocs/mdn-bcd-collector#726. The most important use case is submitting changes for a single browser release, and a mock invocation of that script might look like:

npm run create-pr --browser chrome --version 116

That pull request should then only include changes that affect Chrome 116, and have a reasonable subject and body, similar to the pull requests that @queengooborg submits using a standalone script.

[foolip]

Open question: is it best to have a PR creation script internally invoke update-bcd with certain filters, or to start from the local changes in the BCD repo and try to summarize them?

[Elchi3]

We achieved a first milestone in this project. @teoli2003 was leading this work! 🎉

Jean-Yves landed 9 pull request in mdn/browser-compat-data to automatically update the browser release information which BCD stores for these 6 BCD browsers:

• Firefox and Firefox Android (using whattrainisitnow.com)
• Chrome, Chrome Android, WebView Android (using chromestatus.com/api)
• Edge (using edgeupdates.microsoft.com/api/products)

More browsers can easily be added in the future and we reached out to WebKit/Safari if they could provide a data source for their release information for us.

All scripts created can be found here: https://github.com/mdn/browser-compat-data/tree/main/scripts/update-browser-releases and the new GitHub workflow lives here: https://github.com/mdn/browser-compat-data/blob/main/.github/workflows/update-browser-releases.yml

The PRs this script creates will look like this: mdn/browser-compat-data#21113.

Now we get to know automatically when browsers released a new version. As a next step we will look into providing more automation for updating the compatibility data itself for those new browser releases.

[foolip]

Very cool, thanks @teoli2003 for driving this!

[foolip]

@Elchi3, given mdn/browser-compat-data#21084 and mdn/browser-compat-data#21083, are you (personally) planning any work to create these pull requests automatically? I'm discussing PR creation with @lolaodelola and @mzgoddard and want to make sure we don't have overlapping plans.

[Elchi3]

are you (personally) planning any work to create these pull requests automatically?

Yes we are woking on this. We are investigating how such automatically created PRs should look like exactly. We've started our research by creating such PRs manually and we're collecting feedback from key people (release drivers, tech writers). I'm happy to sync about implementation and we've been using the #bcd channel on the OWD Slack to coordinate and share our progress and thoughts as well as the weekly BCD calls.

[queengooborg]

Update on how this project is going:

• The Selenium script runs have been integrated into the collector's deployment workflow
• We're still running into some issues with the script that are getting worked out over time
• Some browsers (Safari and mobile browsers) still need to be run manually
  • Most Safari minor versions are not available on CTs, so I've been creating VMs for them
  • For Android browsers, CTs track the Android version, not the browser version, so we have no way of knowing if the browser version is the most recent

[queengooborg]

Update on the project: the Selenium autorun is now functional and live! Results will now be automatically collected and pushed to the results repo on every release!

There are still some browsers and browser versions that require manual runs due to the limitations explained above, but we now have a path forward!

[Elchi3]

I'm closing this project as complete. There is more we can do to improve our tooling/processes and we will continuously try to improve. Over the last few months, we were able to successfully propose lots of updates to BCD. Example PRs:

• Updates for Chrome 120 beta mdn/browser-compat-data#21148
• Updates for Chrome 121 beta mdn/browser-compat-data#21603
• Updates for Chrome 122 beta mdn/browser-compat-data#21998
• Updates for Chrome 123 beta mdn/browser-compat-data#22280
• Updates for Chrome 124 beta mdn/browser-compat-data#22681
• Updates for Chrome 125 beta mdn/browser-compat-data#22924
• Updates for Chrome 126 beta mdn/browser-compat-data#23207
• Updates for Firefox 120 mdn/browser-compat-data#21083
• Updates for Firefox 122 beta mdn/browser-compat-data#21678
• Updates for Firefox 123 beta mdn/browser-compat-data#21984
• Updates for Firefox 124 beta mdn/browser-compat-data#22272
• Updates for Firefox 125 beta mdn/browser-compat-data#22669
• Updates for Firefox 126 beta mdn/browser-compat-data#22854
• Updates for Firefox 127 beta / WebGL drawingBufferColorSpace mdn/browser-compat-data#23198
• Updates for Safari 17.2 beta mdn/browser-compat-data#21267
• Updates for Safari 17.4 beta mdn/browser-compat-data#22101
• Update for Safari 17.4 beta (19618.1.15) mdn/browser-compat-data#22273
• Updates for Safari 17.5 (19618.2.4.11.2) mdn/browser-compat-data#22765

We will continue to produce these. There are at least about 20 more releases this year. About 30-40 releases every year. Your feedback/suggestions can be filed at https://github.com/openwebdocs/mdn-bcd-collector.

Thanks again to funding from the Sovereign Tech Fund that made this work possible.

From our final report:

Deliverable 1: Complete and improve the automation pipeline (completed)

We successfully ran the automation pipeline we developed and improved it further making it a smoother process overall. During the timeframe of round 2 of the challenge, we observed ten browser releases. Each time a new beta version of a browser was released, the automation enabled immediate PR submission to browser-compat-data. We merged these PRs before the browser was released as a stable version to end-users. The data collected through the automated process we developed was even more accurate than browser makers' documentation and status sites in some cases, and we could feed our findings back to these projects.

This process is now well established. Our developed tools will be enormously helpful in getting timely, complete, and fully accurate updates completed for the approximately 30 additional browser versions expected to be released during the remainder of 2024., and for every release for years to come.

[Elchi3]

I would like to thank the following people who helped with this work and/or gave feedback and reviews as part of this project:

Representatives of browser projects: Philip Jägenstedt (Chromium/Google), Rachel Andrew (Chromium/Google), Chris Mills (Chromium/Google), Patrick Brosset (Chromium/Microsoft), Jon Davis (WebKit/Apple), Jen Simmons (WebKit/Apple), Ruth John (Gecko/Mozilla), Brian Smith (Gecko/Mozilla), Hamish Willee (Gecko/Mozilla).

Open Web Docs members and external contributors: Jean-Yves Perrier, Vinyl Da.i'gyu, Lola Odelola, Z Goddard, and Chris Cuellar.