Add fast path skipping UTF8 length counting

[gaearon]

Commits

• 8f9becc
• 6488907
• d7cc7d?w=1

What

Similar to #2817, I'm trying to avoid calling into TextEncoder().encode(str).byteLength for every string. After this change, I basically don't hit it in the app at all — the fast path always lets me out early.

The fast pass itself is pretty general. The idea is that .length counts UTF-16 code units, and each UTF-16 code unit corresponds to at most 3 bytes in UTF-8 encoding. So we can safely use value.length * 3 as an upper bound on what utf8Len(value) could possibly be. If this upper bound is below the minLength, the same is true for utf8Len. If this upper bound is within maxLength, the same is true for utf8Len.

Why * 3?

• Codepoints that fit into a single UTF-16 code unit become 1 to 3 bytes in UTF-8. (Worst case is 3x.)
• Codepoints that need two UTF-16 code units become 4 bytes in UTF-8. (Worst case is 2x.)

So .length * 3 should always give us a valid upper bound. But this needs a look from an expert.

I've added some test cases.

[bnewbold]

this seems reasonable, though I should probably re-read more carefully and maybe cook up more corner-cases. I kind of suspect that it won't be as much of a win as the earlier grapheme cluster and utf8 caching patch though? I guess UTF-16 to UTF-8 does cost something through, and this probably does help with the happy path, and we do a lot of these, hrm.

[devinivy]

Good thinkin! Re: the factor of 3 in here, I am quite sure that checks out.

[gaearon]

Alright, had to rebase to account for stylistic changes in #2817, but let's land this. Re: perf impact, in React Native calling into encoder/decoder goes between JS and native so it's not guaranteed to be super cheap and it would just be nice to not worry about it for the common case.