Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider some improvements to WPT dash #663

Open
bkardell opened this issue May 22, 2024 · 1 comment
Open

Consider some improvements to WPT dash #663

bkardell opened this issue May 22, 2024 · 1 comment

Comments

@bkardell
Copy link
Contributor

bkardell commented May 22, 2024

I guess it is within our charter to deal with the BSF chart, but I assume that that also might include things related to it: For example, making that chart itself more understandable by linking an 'understanding this chart' link and linking to (at least what is queryable) somehow that the chart represents. But also, perhaps the idea that we should have other views. I've been thinking about this a lot and I've written a few posts about it. The last one looks at the idea of tracking/bubbling up data related to "universally passing tests" - see https://bkardell.com/blog/WPT-Dashboard.html

I think this could be interesting and helpful for helping us align and accomplish a lot, as well as portray a kind of 'score' that is similar to BSF, but aimed differently: How well are we working together over time? I think that is a pretty interesting and important metric too.

(related to #270)

@foolip
Copy link
Member

foolip commented May 22, 2024

I like the suggestion of a Universally Passing metric. This represents the size of the interoperable web platform, and is something we want to be always growing. In this graph from 2019, it would be the green 4/4 slice:

image

This graph is test-weighted and that's also what we do for Interop. Still, a all(status:pass) shows the tests and subtests that would contribute to a Universally Passing metric.

I'd also like to have simple pass and fail counts for all browsers (products) in wpt.fyi, and the N/4 (or N/3) style metric from my screenshot above. (BSF is the 2/3 slice in such a metric, and can be broken down further by browser. Similarly, the 1/3 slice could be broken down by browser.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants