PCRE.exec error during REPL startup error with non-Unicode locales #27239
Comments
|
Same error. Deleting %USERPROFILE%.julia will help, but after installing any package, you will get the same error. |
|
In v0.7 the history file is located at |
When I delete And there is non-ascii characters When I changed the file into UTF-16, different error occurs. UTF-16 LE: UTF-16 BE: In summary, the problem is that Julia does not support other encodings except UTF-8, so it cannot handle different locales which are default on Windows. |
|
We clearly need to save the history file as UTF-8. |
|
I just realized that this problem does not happen with older version. The reason is Julia simply ignores first two lines of each history elements. δ = 1When I run the code above in 0.6.2 and open When I open it as UTF-8: |
|
The history file should only ever be UTF-8. We only write it as such. Did you open it in an editor that might have changed the encoding? |
|
From the second example, it looks like the timezone code is getting saved as EUC-KR. Where are we getting that from? |
@StefanKarpinski I used Notepad and Notepad++, and they do not change encoding automatically. Using Notepad++ I can specify the encoding to interpret. I guess the problem is saving EUC-KR encoded bytes directly into the file opened as UTF-8.
Line 457 in ca35cd9 Edit: When I call I tried "
But in Windows
|
|
As @alkorang said, main reason of this is I have run following code snippet. Result:
You can also check what difference between I think It could be a Windows bug, or visual C++ bug. It is nonsense because just calling a function unrelated to String changes encoding(It supposed to return UTF-16) . English Windows users may not suffer with this issue. I wanted to test with different OS, but I'm using English macOS and it works well with the macOS. (result : "Test 테스트 KST") Let's see if other languages have this issue and how to deal with without any encoding conversion method, because as far as I know, Julia doesn't have conversion function between |
|
Environment variables controlling program behavior in unpredictable ways is an unfortunate but widespread "bug by design". It definitely shouldn't work that way but it does and we should handle it better in any case on both ends—on output and on input. |
|
I was wrong. Lines 173 to 179 in deaefef
can this be a solution? |
|
I tried call Edit: |
|
By the way, we need
Anyway, I'm also confused about Lines 126 to 134 in 7144b6b The problem is that I don't know how to convert |
@appleparan That's simple: set argument types I wrote function wcsftime(fmt::AbstractString, tm::Libc.TmStruct)
wcfmt = transcode(Cwchar_t, fmt)
wctimestr = zeros(Cwchar_t, 128)
n = ccall(:wcsftime, Csize_t, (Ptr{Cwchar_t}, Csize_t, Ptr{Cwchar_t}, Ref{Libc.TmStruct}),
wctimestr, length(wctimestr), wcfmt, tm)
n == 0 && return ""
return transcode(String, wctimestr)
end |
|
possible to relate with NUL-terminated character? and you don't need to wrap like |
function wcsftime(fmt::AbstractString, tm::Libc.TmStruct)
wcfmt = push!(transcode(Cwchar_t, fmt), Cwchar_t(0))
wctimestr = zeros(Cwchar_t, 128)
n = ccall(:wcsftime, Csize_t, (Ptr{Cwchar_t}, Csize_t, Ptr{Cwchar_t}, Ref{Libc.TmStruct}),
wctimestr, length(wctimestr), wcfmt, tm)
n == 0 && return ""
return transcode(String, wctimestr[1:n])
endI added |
|
We should use |
|
Lines 103 to 111 in b5c0cb0 @appleparan Yes, I found out the way how to use Cwstring. Just passing a value of AbstractString and ccall will handle it.
function wcsftime(fmt::AbstractString, tm::Libc.TmStruct)
wctimestr = Vector{Cwchar_t}(undef, 128)
n = ccall(:wcsftime, Csize_t, (Ptr{Cwchar_t}, Csize_t, Cwstring, Ref{Libc.TmStruct}),
wctimestr, length(wctimestr), fmt, tm)
n == 0 && return ""
return transcode(String, wctimestr[1:n])
end |
|
I tweaked your code to adjust We need to create pull request. I think we should change function name
We need a discussion for this, and then, please make pr because your code is uploaded first. @alkorang |
|
Oh you finished. I think your code is better, use them. I should have not used |
@appleparan
Okay, Again, thank you for your help! |
alkorang
changed the title
|
https://discourse.julialang.org/t/strftime-strptime-bug-27239-is-present-on-all-platforms-not-just-windows/11191/3 Example of the Mac from the link: @kshyatt Could you please remove |
|
I think I'm having the same issue on Windows 10 with Julia beta using a German locale. Any idea how to fix this, yet? |
|
The PR that fixes this issue should help: #27273 |
With 32bit version on Windows 10:
With 64bit version on Windows 10:
I deleted
%USERPROFILE%\.julia_historyyet same error occurs.The text was updated successfully, but these errors were encountered: