Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

doc: StringDecoder supported encodings #31907

Closed
thany opened this issue Feb 21, 2020 · 2 comments
Closed

doc: StringDecoder supported encodings #31907

thany opened this issue Feb 21, 2020 · 2 comments

Comments

@thany
Copy link

thany commented Feb 21, 2020

See here:
https://nodejs.org/docs/latest-v12.x/api/string_decoder.html

It should specify which encodings are supported by StringDecoder. The ones I've found are utf8 which is documented, and ucs2 which is not mentioned, so I found it purely by chance.

We have no way of knowing what other encodings are supported.

@himself65
Copy link
Member

encoding parameter both pass through these functions

node/lib/internal/util.js

Lines 122 to 176 in 21bd667

// Return undefined if there is no match.
// Move the "slow cases" to a separate function to make sure this function gets
// inlined properly. That prioritizes the common case.
function normalizeEncoding(enc) {
if (enc == null || enc === 'utf8' || enc === 'utf-8') return 'utf8';
return slowCases(enc);
}
function slowCases(enc) {
switch (enc.length) {
case 4:
if (enc === 'UTF8') return 'utf8';
if (enc === 'ucs2' || enc === 'UCS2') return 'utf16le';
enc = `${enc}`.toLowerCase();
if (enc === 'utf8') return 'utf8';
if (enc === 'ucs2') return 'utf16le';
break;
case 3:
if (enc === 'hex' || enc === 'HEX' || `${enc}`.toLowerCase() === 'hex')
return 'hex';
break;
case 5:
if (enc === 'ascii') return 'ascii';
if (enc === 'ucs-2') return 'utf16le';
if (enc === 'UTF-8') return 'utf8';
if (enc === 'ASCII') return 'ascii';
if (enc === 'UCS-2') return 'utf16le';
enc = `${enc}`.toLowerCase();
if (enc === 'utf-8') return 'utf8';
if (enc === 'ascii') return 'ascii';
if (enc === 'ucs-2') return 'utf16le';
break;
case 6:
if (enc === 'base64') return 'base64';
if (enc === 'latin1' || enc === 'binary') return 'latin1';
if (enc === 'BASE64') return 'base64';
if (enc === 'LATIN1' || enc === 'BINARY') return 'latin1';
enc = `${enc}`.toLowerCase();
if (enc === 'base64') return 'base64';
if (enc === 'latin1' || enc === 'binary') return 'latin1';
break;
case 7:
if (enc === 'utf16le' || enc === 'UTF16LE' ||
`${enc}`.toLowerCase() === 'utf16le')
return 'utf16le';
break;
case 8:
if (enc === 'utf-16le' || enc === 'UTF-16LE' ||
`${enc}`.toLowerCase() === 'utf-16le')
return 'utf16le';
break;
default:
if (enc === '') return 'utf8';
}
}

and I have opened a PR that add a link on the doc

@DerekNonGeneric
Copy link
Contributor

I think this can be closed, right?

/cc @himself65

@jasnell jasnell closed this as completed Jun 10, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants