Julia 0.7.0-alpha.0 ERROR: PCRE.exec error: UTF-8 error: isolated byte with 0x80 bit set

suiato · June 5, 2018, 10:59pm

Getting the following error with Julia 0.7.0-alpha.0 on Windows 10 Pro when I launch the command prompt. Will appreciate any advice on how to handle it.

              _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: https://docs.julialang.org
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.7.0-alpha.0 (2018-05-31 00:07 UTC)
 _/ |\__'_|_|_|\__'_|  |  Official http://julialang.org/ release
|__/                   |  x86_64-w64-mingw32

ERROR: PCRE.exec error: UTF-8 error: isolated byte with 0x80 bit set
Stacktrace:
 [1] error at .\error.jl:33 [inlined]
 [2] exec at .\pcre.jl:137 [inlined]
 [3] match(::Regex, ::String, ::Int64, ::UInt32) at .\regex.jl:198
 [4] match at .\regex.jl:196 [inlined]
 [5] match at .\regex.jl:211 [inlined]
 [6] hist_from_file(::REPL.REPLHistoryProvider, ::IOStream, ::String) at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v0.7\REPL\src\REPL.jl:407
 [7] setup_interface(::REPL.LineEditREPL, ::Bool, ::Array{Dict{Any,Any},1}) at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v0.7\REPL\src\REPL.jl:845
 [8] #setup_interface#49(::Bool, ::Array{Dict{Any,Any},1}, ::Function, ::REPL.LineEditREPL) at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v0.7\REPL\src\REPL.jl:756
 [9] setup_interface(::REPL.LineEditREPL) at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v0.7\REPL\src\REPL.jl:756
 [10] (::getfield(Pkg, Symbol("##1#2")))(::REPL.LineEditREPL) at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v0.7\Pkg\src\Pkg.jl:45
 [11] __atreplinit(::REPL.LineEditREPL) at .\client.jl:312
 [12] #invokelatest#1 at .\essentials.jl:670 [inlined]
 [13] invokelatest at .\essentials.jl:669 [inlined]
 [14] _atreplinit at .\client.jl:319 [inlined]
 [15] (::getfield(Base, Symbol("##840#842")){Bool,Bool,Bool,Bool})(::Module) at .\client.jl:355
 [16] #invokelatest#1 at .\essentials.jl:670 [inlined]
 [17] invokelatest at .\essentials.jl:669 [inlined]
 [18] run_main_repl(::Bool, ::Bool, ::Bool, ::Bool, ::Bool) at .\client.jl:340
 [19] exec_options(::Base.JLOptions) at .\client.jl:278
 [20] _start() at .\client.jl:427

[ Info: Disabling history file for this session
julia>

julia> versioninfo()
Julia Version 0.7.0-alpha.0
Commit 22590d529d* (2018-05-31 00:07 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: Intel(R) Core(TM) i7-2620M CPU @ 2.70GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.0 (ORCJIT, sandybridge)

julia>

kristoffer.carlsson · June 5, 2018, 11:04pm

See https://github.com/JuliaLang/julia/issues/27239

suiato · June 6, 2018, 5:19am

Thanks for the pointer.
Renaming (or deleting) the file repl_history.jl (under …\.julia\logs) solved the problem

suiato · June 6, 2018, 7:17pm

Sorry, but the PCRE error occurred again…

kristoffer.carlsson · June 6, 2018, 10:00pm

Yes the problem is that the date that is being written is corrupted. So every time new stuff get put into the history file, it will have a corrupted date and mess things up.

ScottPJones · June 6, 2018, 11:47pm

Have you tried setting the LC_TIME locale field to <whatever_your_language_is>.UTF-8?

You could do that directly in your ~/.juliarc.jl file.

See the discussion here: Strftime & strptime bug #27239 is present on all platforms, not just Windows - #3 by ScottPJones

You could use this code to set the locale for Julia to whatever your language is, but using UTF-8:

julia> setlocale(lc) = unsafe_string(ccall((:setlocale,"libc"), Cstring,(Cint,Cstring),0,lc))
setlocale (generic function with 1 method)
julia> setlocale("ko_KR.UTF-8")
"ko_KR.UTF-8"

suiato · June 7, 2018, 9:35am

Thanks for the tips.

I’m on Windows 10, and couldn’t find .juliarc.jl.

When I tried the code, I received the following error:

julia> setlocale(lc) = unsafe_string(ccall((:setlocale,"libc"), Cstring,(Cint,Cstring),0,lc))
setlocale (generic function with 1 method)

julia> setlocale("ja_JP.UTF-8")
ERROR: error compiling setlocale: could not load library "libc"
The specified module could not be found.

ScottPJones · June 7, 2018, 12:34pm

Can you set the LC_TIME environment variable, before starting Julia?

suiato · June 7, 2018, 3:22pm

In Git Bash, locale gives the following:

user@user-PC MINGW64 ~
$ locale
LANG=ja_JP.UTF-8
LC_CTYPE="ja_JP.UTF-8"
LC_NUMERIC="ja_JP.UTF-8"
LC_TIME="ja_JP.UTF-8"
LC_COLLATE="ja_JP.UTF-8"
LC_MONETARY="ja_JP.UTF-8"
LC_MESSAGES="ja_JP.UTF-8"
LC_ALL=

I’ve tried to run julia from the bash terminal as follows, but the result was the same.

user@user-PC MINGW64 ~/AppData/Local/Julia-0.7.0-alpha/bin
$ ./julia
               _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: https://docs.julialang.org
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.7.0-alpha.0 (2018-05-31 00:07 UTC)
 _/ |\__'_|_|_|\__'_|  |  Official http://julialang.org/ release
|__/                   |  x86_64-w64-mingw32

ERROR: PCRE.exec error: UTF-8 error: isolated byte with 0x80 bit set
Stacktrace:
 [1] error at .\error.jl:33 [inlined]
 [2] exec at .\pcre.jl:137 [inlined]
 [3] match(::Regex, ::String, ::Int64, ::UInt32) at .\regex.jl:198
 [4] match at .\regex.jl:196 [inlined]
 [5] match at .\regex.jl:211 [inlined]
 [6] hist_from_file(::REPL.REPLHistoryProvider, ::IOStream, ::String) at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v0.7\REPL\src\REPL.jl:407
 [7] setup_interface(::REPL.LineEditREPL, ::Bool, ::Array{Dict{Any,Any},1}) at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v0.7\REPL\src\REPL.jl:845
 [8] #setup_interface#49(::Bool, ::Array{Dict{Any,Any},1}, ::Function, ::REPL.LineEditREPL) at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v0.7\REPL\src\REPL.jl:756
 [9] setup_interface(::REPL.LineEditREPL) at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v0.7\REPL\src\REPL.jl:756
 [10] (::getfield(Pkg, Symbol("##1#2")))(::REPL.LineEditREPL) at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v0.7\Pkg\src\Pkg.jl:45
 [11] __atreplinit(::REPL.LineEditREPL) at .\client.jl:312
 [12] #invokelatest#1 at .\essentials.jl:670 [inlined]
 [13] invokelatest at .\essentials.jl:669 [inlined]
 [14] _atreplinit at .\client.jl:319 [inlined]
 [15] (::getfield(Base, Symbol("##840#842")){Bool,Bool,Bool,Bool})(::Module) at .\client.jl:355
 [16] #invokelatest#1 at .\essentials.jl:670 [inlined]
 [17] invokelatest at .\essentials.jl:669 [inlined]
 [18] run_main_repl(::Bool, ::Bool, ::Bool, ::Bool, ::Bool) at .\client.jl:340
 [19] exec_options(::Base.JLOptions) at .\client.jl:278
 [20] _start() at .\client.jl:427

[ Info: Disabling history file for this session

julia> versioninfo()
Julia Version 0.7.0-alpha.0
Commit 22590d529d* (2018-05-31 00:07 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: Intel(R) Core(TM) i7-2620M CPU @ 2.70GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.0 (ORCJIT, sandybridge)

The region in the control panel of Windows is set to Japanese, but I couldn’t find a way to know if it is ja_JP.UTF-8 or something else…

ScottPJones · June 7, 2018, 4:22pm

Did you delete the old history file before starting Julia in the Git bash?
Otherwise, it will still complain about the older non-UTF8 entries in the file.

suiato · June 8, 2018, 3:54am

Yes, I deleted/renamed the file repl_history.jl before starting Julia.
When I launch Julia again, it starts without the error, and creates a new repl_history.jl, which seems to be working OK as shown below. It’s not corrupt… But, when I restart Julia, the same PCRE error message is issued.

# time: 2018-06-08 00:39:36 東京 (標準時)
# mode: julia
	using Pkg
# time: 2018-06-08 00:39:49 東京 (標準時)
# mode: julia
	Pkg.update()
# time: 2018-06-08 00:40:46 東京 (標準時)
# mode: julia
	exit()
# time: 2018-06-08 00:42:23 東京 (標準時)
# mode: julia
	exit()

東京 (標準時) says Tokyo (Standard Time).

ScottPJones · June 8, 2018, 4:05am

I thought I recognized the 東京, although I’ve spent more time in 大阪市, and in 鳥取
I love Japan (and Japanese food!)

I’ll play around with this tomorrow, after I get some sleep, see if I can figure out what’s going wrong still.

suiato · June 8, 2018, 4:15am

I found the error didn’t occur when I deleted the characters 東京 (標準時) in the file without changing encoding (I opened/ edited/saved it using notepad++).

Been to 大阪 and 鳥取？！ Yes, Japanese foods are good for your health, esp. 和食（washoku, traditional Japanese foods）

ScottPJones · June 8, 2018, 4:22am

When I was spending long periods of time in Osaka (working on adding support for Japanese to a database/language, to replace the S-JIS and EUC based localized implementations, I was the architect of the new Unicode based design), one of my nephews was working in a little village in Tottori-ken, teaching English there for 2 years, near where the camels are, so I went to visit him whenever I was in Osaka. Loved it there, and going to the 温泉. I’d love to go back, it was in '95-'97 when I was there a lot, I’ve been a few times since, but last time was 10 years ago

suiato · June 9, 2018, 3:30am

Camels Aha, Tottori Sand Dune! It’s famous, though I’ve never been in Tottori. Must be nice there. I’m interested in Izumo Taisha (Shrine) in Shimane, next to Tottori, and hope to visit there sometime in the future.

When you visit Japan next time, you can explore the eastside of Japan, too. I’m in Yamagata, near Sendai. Perhaps you’ve heard of Zao for skiing. Famous of Snow Monsters. There’re lots of hot springs, too
https://www.jreast.co.jp/e/yamagata/index.html?src=gnavi