Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Do not expect UTF-8 from stdin #6

Open
kud1ing opened this issue Jun 22, 2018 · 2 comments
Open

Do not expect UTF-8 from stdin #6

kud1ing opened this issue Jun 22, 2018 · 2 comments

Comments

@kud1ing
Copy link

kud1ing commented Jun 22, 2018

On a German Windows this

echo ö|win32yank –i

gives something like

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error { repr: Custom(Custom { kind: InvalidData, error: StringError("stream did not contain valid UTF-8") }) }', src\libcore\result.rs:859 note: Run with `RUST_BACKTRACE=1` for a backtrace. 

A colleague has this issue when win32yank is called by NeoVim.

@equalsraf
Copy link
Owner

Indeed, in nvim I think the process output is utf8 (i.e. the win32yank input) - in your case I would assume it is utf16.

Maybe we can use OsString for that read in https://github.com/equalsraf/win32yank/blob/master/src/main.rs#L127

@equalsraf
Copy link
Owner

It is a bit trickier than that. These are the cases I am aware of

First, the one you mentioned happens because ö uses whichever codepage the console is using. It is possible to decode this by checking the current encoding used by the console

echo ö|win32yank –i

However it this could also fail if the data actually comes from a file. There is nothing we can do in this case for detection - it all depends on the file encoding.

cat test2.txt|win32yank –i

Colleagues have this issue when win32yank is called by NeoVim.

I assume this is possible if the fileencoding is not utf8. Perhaps the easiest way is to force encode as utf8 on the nvim side (assuming this is correct).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants