Unpopular encoding

[DisaPadla]

I don't think this is critical, I just wanted to notify about it. At the moment, slowreader might incorrectly display data received from websites that use less common encodings, such as Windows-1251 or ISO-8859-1. The percentage of such websites is very small, so feel free to close the issue if it's not relevant.

[ai]

Yes, good idea. We should look at HTTP headers for encoding. It is also important for Japanese users where UTF-8 is not very popular for historical reasons.

Maybe we can add some autodetect, but I am not sure that autodetect code is worst of it since most of Internet right now on UTF-8.

[DisaPadla]

Thx for quick response. I'm not sure what do you mean under "autodetect", but for now, i don't know better solution than use arrayBuffer instead of text method. I’ve just started looking into this project, and this is the first thing that came to mind.

[ai]

I'm not sure what do you mean under "autodetect"

Browsers has algorithm to detect charset by heuristics even without Content-Type header. But I think we do not need it.

BTW, are we sure, that response.text() doesn’t do charset encoding correction automatically?

[DisaPadla]

I see this result for some web resources

[ai]

Can you give a link for the testing?

[DisaPadla]

Example
https://www.pravda.com.ua/

[DisaPadla]

I prepare a fix and I'm trying to fix unit test now. And looks like i can't push to repo. Maybe i do something wrong. i will check Guidelines again

[ai]

You need to push to your fork and the crate pull request