Should we treat Safari 6.2 as 6.1 or a later version?

[foolip]

In https://github.com/foolip/mdn-bcd-collector, we collect results from various browsers, and map those to browser releases in BCD, since BCD doesn't have every single release but just the more noteworthy ones.

Given the current logic I have, Safari 6.2 on OS X would map to 6.1. However, the UA string of the version on BrowserStack is this:

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_5) AppleWebKit/600.8.9 (KHTML, like Gecko) Version/6.2.8 Safari/537.85.17

That "537.85.17" is larger than the "537.71" claims is the WebKit version for Safari 7. I don't know what "600.8.9" means, though.

This risks leading to spurious updates changing "7" to "6.1" based on results from Safari 6.2.8.

cc @gsnedders

[ddbeck]

@apple-web-evangelist and @jensimmons: based on our call earlier this week, I think this would be an example of a question that y'all are uniquely situated to help us with 😃

To add a little more context: I think this is typical of a Safari-related versioning question. We're not quite sure on what versions count as significant, or how to assign features to versions when we have to round things off to known-significant releases. (And, if I understand correctly, this one has a fun subplot of the WebKit version numbers we know of suggesting that Safari versions are sometimes non-consecutive.)

[gsnedders]

Safari bundle version (537.85.17 above) is mostly irrelevant to web engine support, what matters is the WebKit bundle version (600.8.9 above). They're frequently the same, but they aren't necessarily.

Without digging too deeply into the history and looking at purely public sources, it appears what happened is Safari 6.1 (macOS 10.7–8) shipped alongside Safari 7 (macOS 10.9 only) with the same (or roughly the same) WebKit version. Safari 6.2 (macOS 10.8) shipped alongside Safari 7.1 (macOS 10.9 only) with the same (or roughly the same) WebKit version. I can look deeper internally, but I doubt there's much more to say than that?

This presumably means we have features that shipped in Safari 6.2 and Safari 7.1, but that aren't in Safari 7.

[foolip]

Thanks @gsnedders that really helps explain what's going on here.

I wonder how we should deal with this in BCD then. In a situation like this, in order to pretend there was a strictly linear progression, perhaps it's best to not have the Safari versions which were only for older version of OS X at the time?

[foolip]

Thinking out loud a bit: while we could simply exclude Safari 6.2 from BCD, that would not be totally satisfactory, if Safari 6.1≈7. Deleting "6.1" and converting all of those to "7" would kind of make sense, except BrowserStack has no 7.0.x release, so for mdn-bcd-collector it would still not be possible to determine between 6 and 7. But one could perhaps take Safari 6.1 and pretend that's Safari 7. At any rate, special knowledge about 6.1 and 6.2 in the version mapping seems necessary. It might be a good idea to extract the "convert UA string to BCD browser+release" logic into a shared BCD util with plentiful tests with real old UA strings.

[saschanaz]

Perhaps letting them be 7 and making MDN to add some note about the versions? 🤔

[ddbeck]

So if I understand this correctly, we've got a situation where two possible version strings are equivalent, but don't sort linearly. That is, a feature could progress through either path in this terrible ASCII art:

6.0  --> 7.0 --> 7.1 --> 8.0
    \                    /^
     --> 6.1 --> 6.2 -->

If that's correct, then I think @foolip has the right idea:

1. Exclude 6.1 and 6.2 from our version history ("best to not have the Safari versions which were only for older version of OS X at the time"). This would mean rewriting all existing 6.1 values to 7.0.
2. Provide a very explicit map between versions, where we've added difficulty for point 1. ("extract the 'convert UA string to BCD browser+release' logic into a shared BCD util with plentiful tests with real old UA strings").

A nice third thing to add might be:

3. If you try to add a 6.1 or 6.2 value, the linter gives you a useful warning (in addition to regular rejected version number).

[queengooborg]

So I've done some more digging into this, and I've found that the situation is even more confusing interesting. After comparing the user agent strings of 6.2/7.1, I noticed something interesting about the WebKit bundle version reported in the UA: "600.8.9". That's when I realized that the number matched what's reported by Safari 8.

In summary, Safari 6.2 and 7.1 are a backport of Safari 8, so the timeline looks more like this:

6.0  --> 6.1 --> 6.2 --> 9.0
    \                    /^
     --> 7.0 --> 7.1 -->
            \          /^
             --> 8.0 --

There are some differences between each of the versions, mostly due to their dependency on the OS (7.x introduced notifications, Media Source Extensions wasn't around before Safari 8.x, etc.). But after comparing the results (thanks to the mdn-bcd-collector project), there's otherwise no differences between Safari 6.2, 7.1, and 8.0.

While I can't find any articles or other web resources that explicitly confirm this, Apple has posted an article about security vulnerabilities in Safari 6.2.1, 7.1.1, and 8.0.1 all together, and Wikipedia lists the release dates for all eight patch releases of these three Safari versions to be the exact same. (Only Safari 8.0.0 mismatched, but I believe that's since it was bundled with the new macOS release.)

@foolip and I were talking about this on Matrix, and we thought to ask Apple once again for an opinion on the matter, as well as BCD consumers (like CanIUse) what they think.

[foolip]

@Fyrd have you dealt with this issue in caniuse.com, and if so how did you resolve it?

[queengooborg]

My thought on this matter is that we should ignore Safari 6.1, 6.2, and 7.1 in BCD. The only purpose of these releases is to backport 7.0 and 8.0 to earlier versions of macOS, and other than a few OS-specific APIs, there aren't any differences in the compatibility between the main and backported releases. Additionally, we would have a pretty odd set of data, where we would have to state features were implemented in 6.2, removed in 7.0, and re-added in 7.1. This also retains consistency with our decision to remove Safari 4.1.

One downside to this is that it may be more challenging for those using BrowserStack to understand the difference in versions. This is because BrowserStack has these three releases (6.1, 6.2, 7.1), but doesn't have 7.0. We should find some way to document these versions at least.

[foolip]

I would also like to remove the backported versions from BCD. Whether we have them or not, there will be some difficulty in knowing what to conclude from test results from Safari 6.1.

[ddbeck]

@vinyldarkscratch: in the most-recent BCD call yesterday, I brought up the proposal to drop the backported releases (6.1, 6.2, and 7.1). @jdatapple gave assent to the idea. I think that means dropping 6.2 and 7.1 is now wholly unblocked, if that's something you want to start work on.

On the testing documentation part, can we come up with a strategy that will approximate testing for 7.0 that we could treat as good enough for BCD? e.g., "if testing in browser stack, if supported in 6.1 + 6.2 + 7.1 → assuming 7.0 is OK?" If we have evidence for more specificity, then great, but we might tolerate a certain amount of error as long as we're all producing the same error.

[gsnedders]

The OS specific feature thing is something we already have a problem with, and is arguably worse now (c.f. foolip/mdn-bcd-collector#571 and having no way to distinguish Safari 14.1 on Catalina from Safari 14.1 on Big Sur).

Apologies for missing yesterday (technical problems, if you didn't guess from dropping in and out!), but I think there's two options:

• Drop 6.2, keep the rest (this gives 6.0 -> 6.1 -> 7.0 -> 7.1 -> 8.0 -> 9.0)
• Drop all back ports, keep the rest (this gives 6.0 -> 7.0 -> 8.0 -> 9.0)

The advantage of the former is we correctly give things that were introduced in the underlying OS, but does give us two releases that are near identical, but implies that things aren't supported in 6.2. From the POV of what BrowserStack has, we're likely to incorrectly attribute things to 7.1 instead of 7.0 if they're OS-dependent.

The advantage of the latter is we minimise the number of releases we have, avoiding trying to keep data for near identical releases. From the POV of what BrowserStack has, we're likely to incorrectly attribute things to 8.0 instead of 7.0 due to the lack of 7.0, or we do something to try and approximate it per above.

[ddbeck]

@gsnedders Did you have a preference between the options? I kinda know what I'd choose if were up only to me, I'd rather defer to you and @jdatapple, as the closest to and most informed about the browser in question.

[gsnedders]

@gsnedders Did you have a preference between the options? I kinda know what I'd choose if were up only to me, I'd rather defer to you and @jdatapple, as the closest to and most informed about the browser in question.

Jon can correct me if I'm wrong, but last I knew there was no particularly strong preference here, as they're old enough releases that they're mostly of historic interest.

Would be interested in what @vinyldarkscratch and @foolip think is easiest from the point of view of mdn-results-collector, especially given availability of the difference versions, as what works best for that is probably the right answer?

[foolip]

There isn't an option which makes this trivial, some logic will be needed in any scripts dealing with Safari versions to avoid making incorrect inferences based on support in Safari 6.1 or 6.2, for example.

And there isn't an option where we'll have exactly the right Safari versions available in BrowserStack or Sauce allowing us to avoid such inferences entirely.

All that being said, I think that excluding backported releases with lower version numbers will make the most sense here. In particular it avoids issues where sorting version numbers can result in a false-true-false-true situation as with 6.1 - 6.2 - 7 - 7.1.

[queengooborg]

It sounds like the decision has been deferred to @foolip and I, and we both think that the best course of action is to remove all the backported releases. Does that sound accurate and reasonable to everyone?

[ddbeck]

We're good to proceed on this. The next step is to document the mapping with #11909.

[jensimmons]

I see this as choosing between:

1. simplicity that erases the reality that things got backported and were available in older releases and OSes later on (shipped in 7, got backported to 6.5), aka shipped in 2013.
2. simplicity that erases the fact things did not originally ship in the earlier releases (shipped in 7, got backported to 6.5), aka implies wrongly it was shipped in 2012.
3. complexity, keeping things complicated

Since this is about what happened in 2013, I do not believe there is any practical need to do #3. And no one wants unneeded complexity.

As many of said, this is about recording history. And as a person who does use this kind of history often — it's not the version number that matters much, it's the date. Option #1 is what accurately records the right date. The fact it was also backported is a detail we can let go of.

And I think the same is true for which OS is supported. Those details are important in the current moment. For example, WebP is supported in Safari 15, but only on macOS Monterey. Developers need to know that now. But they won't need to know that in 8 years. In 8 years, all that will matter is that Safari implemented WebP in 2021.

[foolip]

This has been resolved by #11909 and #11156.