Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Please double-check the end of the EUDC range #53

Closed
hsivonen opened this issue Jun 1, 2016 · 8 comments
Closed

Please double-check the end of the EUDC range #53

hsivonen opened this issue Jun 1, 2016 · 8 comments

Comments

@hsivonen
Copy link
Member

hsivonen commented Jun 1, 2016

In the Shift_JIS decoder, the inclusive end pointer 10528 looks suspicious, since it means only one possible trail byte (the lowest possible) is allowed for the lead byte F9. One would expect either the special case to run to the end of the pointers whose lead byte is F8 (making 10528 an exclusive bound) or run to the end of the pointers whose lead byte is F9.

Please double-check that the range is correct and, if it is, please add a note saying that the range is weird on purpose.

cc @vyv03354

@annevk
Copy link
Member

annevk commented Jun 1, 2016

This range originated from https://www.w3.org/Bugs/Public/show_bug.cgi?id=24130. @vyv03354 can probably clear this up.

@vyv03354
Copy link
Collaborator

The entire F9 range is valid as the removed commit indicates:
651f672
This is a bug of the current decoder algorithm.

@annevk
Copy link
Member

annevk commented Jul 24, 2016

Okay, so it should be 10715 ((0xF9 - 0xC1) x 188 + 0xFC - 0x41 in decimal)?

@annevk
Copy link
Member

annevk commented Jul 24, 2016

@r12a not sure if you test this range, but you might have to update a few things here.

@vyv03354
Copy link
Collaborator

Okay, so it should be 10715 ((0xF9 - 0xC1) x 188 + 0xFC - 0x41 in decimal)?

I think so.

@annevk
Copy link
Member

annevk commented Jul 24, 2016

So 236196e introduced the current range and it seems I just made an error there by not including the full range of 0xF9.

@annevk annevk closed this as completed in f0cfd4f Jul 24, 2016
@annevk
Copy link
Member

annevk commented Jul 24, 2016

Thanks @vyv03354 for the help and @hsivonen for spotting this!

@r12a
Copy link
Collaborator

r12a commented Sep 16, 2016

@annevk i updated the js file that does the decoding for the sjis tests. The results were the same.

r12a added a commit to r12a/r12a.github.io that referenced this issue Sep 16, 2016
hsivonen added a commit to hsivonen/encoding_rs that referenced this issue Nov 21, 2016
r12a added a commit to r12a/app-encodings that referenced this issue Mar 23, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

4 participants