Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve documentation of experimental status #6905

Closed
3 tasks
ddbeck opened this issue Oct 13, 2020 · 21 comments · Fixed by #9933
Closed
3 tasks

Improve documentation of experimental status #6905

ddbeck opened this issue Oct 13, 2020 · 21 comments · Fixed by #9933
Labels
docs Issues or pull requests regarding the documentation of this project. schema Isses or pull requests regarding the JSON schema files used in this project.

Comments

@ddbeck
Copy link
Collaborator

ddbeck commented Oct 13, 2020

Prompted by #6873.

#1528 proposes removing status.experimental. I'm not ready to head down that path yet, but, in the mean time, I am prepared to improve the way we set the experimental status.

The schema docs describe status.experimental as:

a boolean value that indicates this functionality is intended to be an addition to the Web platform. Some features are added to conduct tests. Set to false, it means the functionality is mature, and no significant incompatible changes are expected in the future.

This text doesn't quite capture the meaning of the data as we've come to update it. For consumers, we should revise the wording to more clearly articulate the meaning of the true and false states; for contributors, we should write contribution guidelines for choosing between the two values.

Once these docs and the guideline are written, then we should start follow-up issues to clean up any data accordingly.


Tasks:

  • Write new definition for experimental
  • Write data guideline for experimental
  • Open PR for new tooltip on MDN
@ddbeck ddbeck added schema Isses or pull requests regarding the JSON schema files used in this project. docs Issues or pull requests regarding the documentation of this project. labels Oct 13, 2020
@ddbeck
Copy link
Collaborator Author

ddbeck commented Oct 13, 2020

For the schema docs, I propose something like:

a boolean value that indicates whether the feature is a stable part of the Web platform. Set to false, Web developers should consider the feature subject to ongoing development and at high risk of compatibility-breaking changes; set to false, Web developers should expect the feature's behavior to remain the same over time.

For the data guidelines, I propose something like:

Choosing an experimental status (status.experimental)

If a feature is standard track and only one browser engine supports the feature without prefixes or flags, always choose true for status.experimental.

If a feature is standard track and more than one browser engine supports the feature without prefixes or flags, always choose false for status.experimental.

This feels rather definitive (in fact, we could probably automate this, as written). But it leaves out non-standard features. I can imagine scenarios were features are non-standard and experimental (e.g., some vendor decides to go it alone on something), or where features are non-standard non-experimental (e.g., a non-standard feature has been explicitly deprecated or in the process of removal). I'm not sure how exactly to phrase that and have run out of time to think about it this afternoon.

@ddbeck
Copy link
Collaborator Author

ddbeck commented Oct 13, 2020

Also, #5392 is important reading for this issue.

@foolip
Copy link
Contributor

foolip commented Oct 13, 2020

The most important effect of the experimental status in BCD is the flask icon with the title text "Experimental. Expect behavior to change in the future." on MDN. (example) Assuming that isn't changing on the MDN side, then I think the meaning of experimental in BCD should be close to "behavior is expected to change in the future".

Unfortunately, I don't think the number of implementations is a great proxy for that. What should matter for browser vendors' willingness to make breaking changes is how widely used something is. Once something is widely used that really locks the behavior in, if you change it you break the web.

I went looking and could pick out two single-engine where calling them experimental feels weird:

I guess the crux is that "experimental" sounds like "new" but if things remain single-engine for a long time they're really not experimental in that sense.

To make a concrete suggestion, I'd say that anything that's been shipping for 2+ years in any browser by definition can't be considered experimental. In some cases it might change in the future (being removed) but in others it's just there for the long term, unchanging.

@sideshowbarker
Copy link
Contributor

sideshowbarker commented Oct 14, 2020

The most important effect of the experimental status in BCD is the flask icon with the title text "Experimental. Expect behavior to change in the future." on MDN. (example)

Agreed that the icon is the most important effect. But IMHO the “Expect behavior to change in the future” text expansion of what that icon signifies is just one possible way to interpret what it’s marking. Another way to interpret is that it simply flags that feature as something that’s not actually a mature, agreed-upon part of the web platform.

I think that sense is what’s more important to developers, and I think we’d be way better off if instead of “experimental” we had some other term to indicate “not a mature part of the web platform” or “not an agreed-upon part” or some such.

Remembering what problem this flag should actually be trying to help solve

A well-known problem we have is that when developers see a feature documented in MDN, they tend to assume it’s mature and agreed-upon and “standard” and that if any browser doesn’t implement that documented-in-MDN feature, the vendor of that browser is bad for not following the standards and not implementing that feature.

So I think we have an obligation to clearly flag such “not-agreed-upon part of the web platform” features in MDN so that developers don’t make the mistake of assuming that there’s a vendor agreement about the feature.

The wider community of people involved in feature development for the web have had discussions about the broader issue around this in a number of different places; WICG/admin#102 for example.

In practice I guess it comes down to is what have commonly been labeled “single-vendor specs” or “single-vendor features’’. But I think the more accurate term would be “single-engine”.

I guess one obvious alternative to using the label/key/flag “experimental” is instead just using the word “single-engine”. (When considering that, there’s a natural question that comes up — I’ll post a separate comment about that.)

But if we don’t like “experimental” or “single-engine” as terms, then let’s please find some other word with the sense “not an agreed-upon part of the web platform” that we can use as a flag in BCD in place of “experimental”.

I don’t feel strongly about what particular word we end up using for that flag. But I do feel strongly that we need such a flag.

@sideshowbarker
Copy link
Contributor

In considering the idea of a “single-engine” flag (aka “single-vendor”), a thought naturally comes to mind: Well, we wouldn’t need any flag at all for that, because we can just compute “this feature is single-engine” based on the existing support data.

But I believe there’d be problems in practice with following that line of thinking; I believe we’d still want a static flag in BCD.

Here’s why:

What we can just compute “this feature is single-engine” would actually mean in practice is that “we” wouldn’t be computing it all, but that instead that work would need to be done bu others; in particular, it would need to be computed on the MDN side, in the code that consumes the BCD data and generates the Browser Compatibility tables — because within BCD itself, we have no processes that generate the data we’re storing is generated; it’s all static data.

And since we have other downstream consumers of BCD data, we can just compute “this feature is single-engine” would also mean that N different downstream consumers would each need to write their own custom code for computing it.

That might not sound like such a big deal — because after all, the data is there and it would seem relatively simple to compute “this feature is single-engine” from the data.

But speaking as a writer of downstream consuming code, having written code to do that computation, in several places and multiple languages, it’s not as simple and clear-cut as you might imagine.

For the curious, here are links to some of the code I’m using to do it —

I am not enthusiastic about imposing the burden on all other downstream consumers of (re)writing that same kind of code.

So to keep things as clear as possible and to avoid the need for all downstream consumers to write such code, it’s better for everyone if we have a static flag in BCD for this. And as I said in my other comment, I think if we can could find a good word that conveys “not an agreed-upon part of the web platform”, that would be best. But if we can’t find such a term, then I personally would be fine with just calling using “single-engine” as the name for the flag.

@sideshowbarker
Copy link
Contributor

I think the meaning of experimental in BCD should be close to "behavior is expected to change in the future".

Unfortunately, I don't think the number of implementations is a great proxy for that.

I agree. But the number of implementations is better as a proxy for measuring the level of browser vendor/engine agreement about ever making the feature an actual standard part of the web platform, so that developers can use it cross-browser.

Browser vendors vote on potential web-platform features by implementing them. So the lack an implementation for a certain feature in a particular browser engine is an indication that the vendor of that engine has not yet agreed in practice to make that feature part of the web platform.

I went looking and could pick out two single-engine where calling them experimental feels weird:

I agree that labeling those as "experimental" is a mismatch. But it seems like labeling them as “single-engine” would not be.

I guess the crux is that "experimental" sounds like "new" but if things remain single-engine for a long time they're really not experimental in that sense.

Agreed. Those features have instead simply just pretty clearly become “single-engine”.

To make a concrete suggestion, I'd say that anything that's been shipping for 2+ years in any browser by definition can't be considered experimental. In some cases it might change in the future (being removed) but in others it's just there for the long term, unchanging.

Agreed. But if something’s been shipping in one browser engine for a long time (2+ years or whatever) and not in any other browser engines, then I think it’s even more important that we make it clear to developers that’s a “single-vendor” feature — or “not an agreed-upon part of the web platform” feature.

@sideshowbarker
Copy link
Contributor

sideshowbarker commented Oct 14, 2020

https://github.com/mdn/browser-compat-data/wiki/Features‐in‐less‐than‐two‐engines is a dump of output from a script I ran to identify all BCD features that are either only implemented in a single engine, or not even in a single engine — and that aren’t yet flagged experimental. It’s 508 features.

@foolip
Copy link
Contributor

foolip commented Oct 14, 2020

The most important effect of the experimental status in BCD is the flask icon with the title text "Experimental. Expect behavior to change in the future." on MDN. (example)

Agreed that the icon is the most important effect. But IMHO the “Expect behavior to change in the future” text expansion of what that icon signifies is just one possible way to interpret what it’s marking. Another way to interpret is that it simply flags that feature as something that’s not actually a mature, agreed-upon part of the web platform.

That seems too broad to me, as it would also cover things that are in the process of being removed from the platform, even including some things that are still in two engines, like document.xmlEncoding. But perhaps we can address that by saying it's impossible to go from non-experimental back to experimental, and appropriately calling those things deprecated.

What we can just compute “this feature is single-engine” would actually mean in practice is that “we” wouldn’t be computing it all, but that instead that work would need to be done bu others; in particular, it would need to be computed on the MDN side, in the code that consumes the BCD data and generates the Browser Compatibility tables — because within BCD itself, we have no processes that generate the data we’re storing is generated; it’s all static data.

If we decide that any status should be strictly derived from other parts of the data, that doesn't mean we have to push that burden on data consumers. A script to update the status plus a lint to ensure it doesn't produce any changes would do the trick.

A "single-engine" status would avoid a bunch of interpretation, but in light of #6738 we're not going to remove the experimental status, so we still have to decide when to set it to true and false in the data we have.

I think calling something that just shipped in only one engine experimental is always appropriate, at least I can't think of any exceptions. But when enough time has passed, it no longer seems compatible with the typical use of the word to call it "experimental", and "Expect behavior to change in the future" will be increasingly inaccurate.

Here's a straw suggestion:

  • Anything shipped for the first time in the past year is experimental, regardless of the number of current implementations.
  • Anything shipped for the first time more than two years ago can no longer be considered experimental, even if there's only one implementation.

@ddbeck
Copy link
Collaborator Author

ddbeck commented Oct 14, 2020

This has been a really helpful discussion already. Thank you, @sideshowbarker and @foolip. It really clarified a lot of things for me. I apologize, but I wrote a bunch more words in reaction to this conversation:

There are multiple plausible interpretations of experimental. Which do we prefer to communicate to consumers and Web developers?

If we ignore what heuristic (or mechanic) we use to set the value, what do we mean for the value itself to communicate to consumers and Web developers? The discussion here highlights that we're understanding it to mean a) unsettled implementation, b) not yet part of the consensus of implementers (even if it's nominally standard track), or c) both.

To put this another way, I'm hearing that true, today, means we suspect or know any one (or more) of these things to be true:

  • The feature is implemented in one engine only, probably recently
  • The feature's API or behavior is at-risk of material changes
  • The feature's API or behavior is at-risk of deprecation or removal
  • The feature lacks consensus between browser implementers (even if it's nominally standards track)
  • The feature may be (or has been) rejected by one or more browser implementers

Have I missed any? If not, I would summarize these points together like this:

If status.experimental is true, then Web developers should experiment with this feature and provide feedback to browser vendors and standards authors about it; Web developers should not rely on the feature's continued existence in its current (or potentially any) form in future browser releases.

Does this feel like a better wording for the schema? Any suggestions or reactions to this?

Two notes on this rewrite:

  1. I've tried to reframe it in terms of Web developers' role in the experiment (hat tip to @sideshowbarker's point about articulating "what problem this flag should actually be trying to help solve"). To me, this feels like it unlocked something. What do I want Web developers to do with this seems like a much more tractable problem than defining a feature's relationship to the Web platform.
  2. I've walked away from defining false at all, except to mean not true. Something that's not experimental doesn't have to mean it's well-accepted or has a long-life ahead of it, but it probably means that Web developers should look to other data points (such as deprecated status, standards status, and number of implementations) to make a decision about how to use it.

What signals can we use to set the value of experimental?

What we mean to communicate to Web developers aside, actually setting that value is kinda thorny! Suppose we used the definition above. What signals can we use to recognize experimental features?

Possible evidence of experimental status:

  • One and only one implementation of the feature
  • Unresolved issues on the feature's spec
  • Few or zero unflagged or unprefixed implementations of the feature
  • "Anything shipped for the first time in the past year is experimental, regardless of the number of current implementations" –@foolip

Possible evidence of non-experimental status:

  • The feature is not widely implemented yet explicitly identified as a legacy feature in a specification (e.g., navigator.getUserMedia())
  • The feature is explicitly deprecated or superseded
  • Implementers have indicated that they don't plan to change the feature
  • "Anything shipped for the first time more than two years ago" –@foolip (though I'd narrow this slightly to mean, "anything shipped in its current form more than two years ago")

I think some of these can be suggestive, while others might be definitive. All of this stuff feels like material that could be folded into a comprehensive guideline about when to set true and false. I'm guessing there's other indicators we can deliberately invite, too (e.g., under my definition above, something like "the spec authors asked us to label this as experimental to solicit feedback from Web developers" would be a great outcome).

We should probably compile a comprehensive collection of these possible signals, as a start. So if you've got more, I want to hear them.

What about single-engine features?

After reading everything here, I think whether a feature is single-engine is a really useful bit of information for BCD consumers, independent of experimental status. This seems like a really easy-to-explain point of caution for Web developers that's somewhat adjacent to but not the same as experimental, deprecated, and standards track. The other nice thing about it is that it can be computed directly from BCD's support statements—we don't have to change any raw data to produce such a flag.

In this, I strongly agree with @sideshowbarker, except for the very last clause:

I am not enthusiastic about imposing the burden on all other downstream consumers of (re)writing that same kind of code.

So to keep things as clear as possible and to avoid the need for all downstream consumers to write such code, it’s better for everyone if we have a static flag in BCD for this.

I don't think this needs to be static. It seems to me that we could start providing APIs for routine consumer tasks against the data. See related issues #4106 and #6138. BCD's schema is quite descriptive, but it's gnarly to actually work with—I like that this is a concrete, easy-to-conceptualize entry point into doing that sort of thing. I'm inclined to open an issue for this separately, unless I'm missing something that points to baking this into the schema.

@sideshowbarker
Copy link
Contributor

If status.experimental is true, then Web developers should experiment with this feature and provide feedback to browser vendors and standards authors about it; Web developers should not rely on the feature's continued existence in its current (or potentially any) form in future browser releases.

Does this feel like a better wording for the schema?

Yes, it does to me.

Any suggestions or reactions to this?

Thanks much for coming up with this. I have no suggestions — I think that articulation you’ve come up could solve a lot of the problems we’ve been discussing. I’ll be interested to see what @foolip thinks of it.

@sideshowbarker
Copy link
Contributor

If status.experimental is true, then Web developers should experiment with this feature and provide feedback to browser vendors and standards authors about it; Web developers should not rely on the feature's continued existence in its current (or potentially any) form in future browser releases.

… Any suggestions or reactions to this?

On further reflection, I do actually have a suggestion to make: I think rather than framing “experimental” in the context of “future browser releases”, I think it would be better to just frame it within the context of the overall web platform — like this:

Web developers should not rely on the feature's continued existence in its current (or potentially any) form as a standard feature of the web platform.

@sideshowbarker
Copy link
Contributor

But when enough time has passed, it no longer seems compatible with the typical use of the word to call it "experimental"

I don’t agree. I think something can be quite old and still be experimental — it’s just that it’s an experiment that never ended up producing what it was hoped or expected it would.

I think in some contexts such experiments would be called “failed experiments” or “dead-end experiments” or some such.

But our context, there’s not the connotation of something being a failure or dead-end; instead it’s just if we consider those experiments as having the goal of producing a feature the becomes a cross-browser-implemented part of the web platform, they’re experiments that haven’t ended up achieving that goal. Or for younger features, they just haven’t yet.

and "Expect behavior to change in the future" will be increasingly inaccurate.

I agree that the wording "Expect behavior to change in the future" would be inaccurate for the cases we’ve discussed. But I think the solution to fixing that inaccuracy is to drop that wording — rather than continuing to try to constrain “experimental” to what that over-limiting wording constrains it to.

And I think @ddbeck’s proposed new wording achieves the goal of providing us with a useful project-specific framing of “experimental” that avoids over-constraining it in any undesirable way — especially my refinement applied:

If status.experimental is true, then Web developers should experiment with this feature and provide feedback to browser vendors and standards authors about it; Web developers should not rely on the feature's continued existence in its current (or potentially any) form as a standard feature of the web platform.

Another nice property of that refined form of the statement is that it doesn’t imply that the feature is necessarily at risk of being removed from any browser which has already implemented it, nor that there’s necessarily any risk of the behavior of the feature in the browsers which have implemented will change.

Instead, we’re simply stating it’s experimental with regard to its status as a standard, cross-browser-supported feature of the web platform.

@sideshowbarker
Copy link
Contributor

I think calling something that just shipped in only one engine experimental…

Anything shipped for the first time more than two years ago…

The feature is implemented in one engine only, probably recently…

One and only one implementation of the feature…

Apologies in advance for pointing out something that risks further muddying the waters, but in regard to the above, I realize we also need to consider features which BCD lists as having zero implementations, and never having shipped anywhere.

https://github.com/mdn/browser-compat-data/wiki/Features‐in‐zero‐engines is a dump of output from a script I ran to identify all BCD features that are not implemented even in a single engine — and that aren’t yet flagged experimental.

It’s 159 features.

(And incidentally, I notice that 64 out of that 159 — so 40% — are SVG features. That would seem to indicate either that the SVG spec defines a whole lot of vaporware features, or else that the BCD data for those SVG features may not be accurate; those features may in fact be implemented in some engine.)

@sideshowbarker
Copy link
Contributor

Web developers should not rely on the feature's continued existence in its current (or potentially any) form as a standard feature of the web platform.

It occurs to me that if we frame it in that way, a question which is going to naturally arise is: What becomes the difference between the experimental flag and the standard_track flag?

I think the answer to that question is, we shouldn’t have a flag named standard_track to begin with. It’s misnamed.

So a solution to any perceived redundancy in the purposes of the experimental and standard_track flags would be to rename the standard_track flag to better reflect what we’re actually using it for — or should be using it for.

I assert that we ideally shouldn’t be trying to use standard_track for something subjective to begin with. Whether or not something’s “standard” is subjective and arguable; and even whether or not something is “standards track” is subjective and arguable — for proof of that, see the issue at WICG/admin#102, which is basically about exactly that.

But something that is measurable and that’s not subjective and arguable is: Is this feature currently specified, somewhere?

I think “Is this feature currently specified, somewhere?” is what we mostly use the standard_track flag for in practice.

Certainly I can say that when I become aware of a feature having been dropped from a specification, I raise a PR to change the BCD standard_track value for it to false. And I’ve seen other contributors doing the same.

That leads me to believe that we could probably just globally replace the standard_track flag throughout all BCD data with a flag named instead is_specified (or something like that), and make a corresponding rewording to the schema documentation (similar to the way @ddbeck’s proposed rewording for experimental here).

Those changes would resolve any ambiguity around the difference between the experimental and standard_track flags.

@sideshowbarker
Copy link
Contributor

sideshowbarker commented Oct 15, 2020

That leads me to believe that we could probably just globally replace the standard_track flag throughout all BCD data with a flag named instead is_specified (or something like that), and make a corresponding rewording to the schema documentation (similar to the way @ddbeck’s proposed rewording for experimental here).

It further occurs to me that after we were to go ahead with #6765 (adding spec_url values for all features that are currently specified somewhere), then maybe we wouldn’t need to even have an is_specified (or standard_track) flag at all any more — because the presence of a spec_url for the feature would convey the same meaning.

It seems like it’s become clear the problem that’s common to both the experimental and standard_track flags is that they’re not conveying anything which is objectively measurable, but are instead conveying something subjective and arguable. And that’s why we’ve ended up spending a lot of time arguingdiscussing them without (yet) reaching a clear agreement.

And so that’s why it seems like we’re going to be better off in the end if, in place of the subjective notions of “experimental” and “nonstandard” we’ve been surfacing to developers through MDN, we instead surface to them the related characteristics of the feature that are measurable — specifically: the measurable property “has less than two implementations” in place of “experimental”, and the measurable property “not currently part of any specification” in place of “nonstandard”.

@foolip
Copy link
Contributor

foolip commented Oct 15, 2020

Thanks @ddbeck for coming up with a concrete proposal here!

If status.experimental is true, then Web developers should experiment with this feature and provide feedback to browser vendors and standards authors about it; Web developers should not rely on the feature's continued existence in its current (or potentially any) form in future browser releases.

Does this feel like a better wording for the schema? Any suggestions or reactions to this?

Will this go together with an update of the tooltip wording on MDN? That is the wording that I think ultimately matters, and it won't be possible to put all the nuance in there.

What we mean to communicate to Web developers aside, actually setting that value is kinda thorny! Suppose we used the definition above. What signals can we use to recognize experimental features? [...] We should probably compile a comprehensive collection of these possible signals, as a start. So if you've got more, I want to hear them.

The signals you listed look good to me. I have thoughts on two of them though:

Unresolved issues on the feature's spec

Just going by the issue trackers, virtually all specs have unresolved issues, and opinions about how serious they are will diverge. It will be hard for us in BCD to judge how serious the disagreement is, or what the risk of future incompatible changes are.

Implementers have indicated that they don't plan to change the feature

I think this is good, and it's something could apply to navigator.getUserMedia() right after it was shipped. However, I think the Chrome team would also make a commitment of backwards compatibility for almost everything that has significant adoption, even if in a niche. Examples that come to mind are the widely used Network Information API and the Web MIDI API which has low usage but is essential to a niche type of web application.

To exaggerate the situation, would BCD maintainers be happy removing the experimental flag for every feature where the Chrome team made a public comment like "we don't have plans to remove this and are committed to backwards compatibility", at least for features above a certain age?

@ddbeck
Copy link
Collaborator Author

ddbeck commented Nov 6, 2020

OK, I think these are the questions that still feel a little under-addressed. The first two seem to have answers, the last one does not. (Also, I've started a task list in the issue description).

Should BCD even have experimental or standards flags? #6905 (comment)

Raised by @sideshowbarker in #6905 (comment). I agree in principle it'd be much better if we didn't have to make a judgement call about each feature individually. But we've got these flags now and it's going to take time to come up with alternatives and get consumers to switch to them. I think it's worth making "experimental" less bad now, even if we're going to make substantially less bad later.

The same goes for the standard track flag, though I actually think that one is in better shape and farther along to being dropped, with the discussions already happening on the spec_url work.

Can we update the "experimental" tooltip on MDN? #6905 (comment)

I believe so, yes. It may be easier to do after Yari ships. We'll probably want to propose a terser version of our definition (maybe: "Experimental. Use this feature to provide feedback to browser vendors and specification authors. Risk of behavior change or removal.") but we should be able to get the gist of things into the tooltip.

Can single-implementation, non-standard features age out of "experimental" status?

I hope this is a fair summation of @foolip's question. This feels like the thorniest question to me, once I started to outline a data guideline. What if a feature only ever ships with one implementation, but the browser vendor commits to keeping the feature, unchanged as-shipped? Should circumstances allow for the feature to be marked as not experimental or is it experimental for all time?

So @sideshowbarker suggests this is a "dead-end" experiment but an experiment nonetheless; @foolip's question suggests that experiments should one day be allowed to conclude. I see the appeal of both of these choices. That said, I do wonder if it would be difficult to write the guideline for expired experiments.

Right now, we have a really good idea of when to mark things as experimental: when they ship with only one implementation. If we have an escape hatch, it becomes a bit of a complex sequence. How do we know the old experiment won't change in behavior? How official does it need to be? Do we require a wontfix on an issue suggesting removing the experimental feature? Are comment from the browser vendor on the BCD repo? How much time must have elapsed since the feature shipped? I'm not sure I want to choose something that requires a flow chart to decide.

Is this something anyone feels very strongly about? If not, I'm inclined to follow the path of least resistance in terms of maintaining the data in the hopes that derived data (e.g., a generated single-implementation flag) will someday supplant this.

@foolip
Copy link
Contributor

foolip commented Nov 30, 2020

I would like to submit https://developer.mozilla.org/en-US/docs/Web/API/Selection as a case that definitely shouldn't be considered experimental, this is an ancient API.

@ddbeck
Copy link
Collaborator Author

ddbeck commented Nov 30, 2020

Yeah, we should get a move on for this. Were there any strong opinions for the question of, can single-implementation, non-standard features age out of "experimental" status? If not, I'll proceed with writing a PR without the answer being no.

@foolip
Copy link
Contributor

foolip commented Dec 1, 2020

I do feel somewhat strongly about it, that having a single implementation is not by itself enough to imply experimental status. Saying nothing about the relationship between single-implementation and experimental status would avoid the issue, but clarity on this is in my mind the most important part of this issue.

As a conservative rule which applies to fewer things than I would perhaps think is ideal, I'd suggest "anything shipped in its current form more than two years ago".

@ddbeck
Copy link
Collaborator Author

ddbeck commented Dec 3, 2020

@chrisdavidmills, @Elchi3, @foolip, and I talked talked about this yesterday. To recap, we agreed that:

  • A two-year time limit on single-engine experimental features would be a satisfactory exit criteria for experimental status. As Florian put it, the two-year time horizon has served us well for irrelevant features and flags, so why not use it for experimental?
  • We'd retain the option of figuring out some shortcut out of experimental status in the future (e.g., if some vendor commits to not changing something that hasn't been taken up by other vendors), but we're setting that aside for now.
  • I'd start work on a PR (probably next week), so we could have a concrete diff to discuss and finalize.

Also, Chris alerted us to the existence of a definition of "experimental" on MDN.

Speaking for myself here: by my reading, the MDN definition is compatible with the things we've talked about on this issue. It's worded in such a way that we can't adopt it directly (it's specification oriented, which doesn't help us for stuff that doesn't or won't have a specification and it doesn't have anything to tell us about dead-end experiments). I suspect we'll learn a lot writing our own definition and implementing it. We may want to offer our definition upstream, after we've gained some experience with it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docs Issues or pull requests regarding the documentation of this project. schema Isses or pull requests regarding the JSON schema files used in this project.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants