Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adopt new definition and guideline for experimental status #9933

Merged
merged 6 commits into from
May 11, 2021

Conversation

ddbeck
Copy link
Collaborator

@ddbeck ddbeck commented Apr 19, 2021

This PR:

  • Proposes a new, hopefully clearer definition of experimental in the schema docs, for consumers. This is less an attempt to change the definition, so much as to describe what it actually is. It has an additional benefit of sidestepping standards questions.
  • Proposes a guideline for setting and changing the experimental status, for contributors. This is an attempt to get consensus on how we set the status as PR authors and reviewers.
  • Reformats the other status definitions in the schema docs for symmetry with the new definition.

That said, this is probably not complete. See my line comment on sunsetting experimental features. There's a wrinkle that I don't like and I'm not sure how to resolve it.

Fixes #6905.

@github-actions github-actions bot added docs Issues or pull requests regarding the documentation of this project. schema Isses or pull requests regarding the JSON schema files used in this project. labels Apr 19, 2021
docs/data-guidelines.md Outdated Show resolved Hide resolved
docs/data-guidelines.md Outdated Show resolved Hide resolved
@ddbeck ddbeck force-pushed the 6905-new-experimental-definition branch from 4f9de27 to f34803d Compare April 21, 2021 17:09
@ddbeck ddbeck force-pushed the 6905-new-experimental-definition branch from f34803d to 05429dc Compare April 21, 2021 17:09
docs/data-guidelines.md Show resolved Hide resolved
| An API supported in Chrome and Firefox, without flags or prefixes. | No |
| A CSS property supported in Chrome and Firefox, with the `-webkit-` prefix. | No |
| An HTTP header supported in Chrome and Firefox, behind flags. | Yes |
| A CSS property value supported in Safari, released last week. | Yes |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if the feature is in a W3C recommendation and it just happens that Safari reached the finish line first?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If Safari crosses the finish line first, then it's still experimental until at least one engine catches up.

Part of the goal here is to clearly distinguish between experimental and standard track. Under this revised approach, experimental can be set independently of the standard track value (that is to say, the two values don't depend on each other—you don't even have to know what the standards status is to set the experimental status). This makes it more clear what experimental actually adds to our data set (a marker of consensus, or failing that, stasis) and lets us deal with features that are unchanging, but not standardized.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I get all that. It just seems misapplied here. If a feature is a full W3C recommendation, it's not a matter of if the second browser will implement. It's a matter of when. Knowing that fact is useful to developers. This information can help developers make other vendors aware that they should prioritize something. Here's how this would play out for Chrome.

  1. Safari implements a full recommendation.
  2. Developers see that Chrome hasn't implemented it yet, but they want it.
  3. They do one of three things: star an existing ticket on crbug.com, open a ticket on crbug.com, or they star the feature's entry on chromestatus.com.
  4. Chrome management sees a surge in developer interest and decides to prioritize it over The Obscure API. (Developer interest isn't the only thing we consider, but we do prioritize developer interest if we see a lot of it.)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It just seems misapplied here.

First, a question: do you mean that the guideline and definition as written shouldn't result in this outcome? Or that you'd like to see different text that leads to a different outcome?

Second, what I want from this PR is to encourage the scenario you describe. Ideally, the experimental flag should be a signal to web developers to tell browser vendors and spec authors what they think of an emerging feature, including saying "I want this feature." I want to upstream the new experimental definition to MDN itself too, so this signal is stronger.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You overall heuristic seems misapplied to the case of a completely speced feature with one implementation, where 'speced feature' means one in a full recommendation.

what I want from this PR is to encourage the scenario you describe

The scenarios for a speced and unspeced feature are different.

  • For features in candidate recommendations, or wicg specs, or whatever, the signal vendors need is should the experimentation continue?
  • For features that are full recommendations, the signal vendors need is should we prioritize implementing feature X over feature Y? We have to do both because we agreed they should be part of the platform. Which do we want to do first?

Web developers would treat these differently as well.

  • For truly experimental features, I may not want to use the feature yet, or my employer may not let me use the feature in production. My shop doesn't let me use an open source polyfill. (Yes, that's a thing. I worked in one. There were liability concerns.) My shop's approved framework says they'll support it with a polyfill, but it's not coming for six months. Perhaps I'm overly cautious because I previously embraced an experimental feature and it got dropped. (Remember WebVR?)
  • For full recommendations, the most important concern with using is when will the other vendors catch up and will my framework support it in the meantime.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jpmedley I'm not really following here. How good a spec here is actually very hard to judge, and browser implementers will not trust the status the spec claims for itself, whether that's "Unofficial Draft", "Living Standard", or "Recommendation".

I don't think we should use the spec's own claim about how stable it is as input to the "experimental" setting here, but base it entirely on implementation status, like @ddbeck is suggesting in this guideline.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't follow Joe either. I think this guideline is a lot clearer and I agree with @foolip and @ddbeck that it is good that we don't need to make any assumptions about the status of specs. @jpmedley feel free to start a discussion in an issue if you feel strongly about it.

Let's merge this. There is a lot of useful guideline work in here.

@Elchi3 Elchi3 merged commit c7189d0 into mdn:main May 11, 2021
@ddbeck ddbeck deleted the 6905-new-experimental-definition branch May 11, 2021 10:46
ddbeck added a commit to ddbeck/browser-compat-data that referenced this pull request May 20, 2021
ddbeck added a commit that referenced this pull request May 20, 2021
* Bump version to v3.3.4
* Add release note for #10353
* Add release note for #10347
* Add release note for #10336
* Add release note for #10415
* Add release note for #10417
* Add release note for #10449
* Add release note for #9933
* Add release note for #10354
* Add release notes for #6944
* Add release note for #10522
* Add release note for #10536
* Add release note for #10539
* Add release note for #10515
* Add release note for #10506
* Add release note for #10291
* Add release note for #10421
* Add release note for #10545
* Add stats
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docs Issues or pull requests regarding the documentation of this project. schema Isses or pull requests regarding the JSON schema files used in this project.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Improve documentation of experimental status
5 participants