Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

set explicit encoding when reading and writing hex config #2405

Merged
merged 1 commit into from
Oct 20, 2020

Conversation

blandinw
Copy link
Contributor

(fixes erlef/rebar3_hex#184)

The issue was that the bytes were written "as is" on disk, while Erlang source files are expected to be utf8 (and no longer latin1) by default since Erlang/OTP 17.0 (as per https://erlang.org/doc/reference_manual/character_set.html)
e.g. byte <<200>> (0xC8) is not valid utf8, but was written as is, causing file:consult to raise an encoding error.

It is preferable not to rely on default behavior, and specify an encoding both at write and read time.
This PR specifies utf8 (probably more future-proof than latin1) when calling file:write_file and adds the encoding at the top of the file for future reads via file:consult (as per https://erlang.org/doc/reference_manual/character_set.html#source-file-encoding).

The test writes to the actual config at the moment. How should I stub it out, knowing that it should still use file:file_write and file:consult and have the same semantics? Maybe an in-memory file, etc

cc @starbelly @ferd

@ferd ferd merged commit d258d91 into erlang:master Oct 20, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

rebar3 hex user auth leads to cannot translate from UTF-8
3 participants