Julia install issue (foreign name)

Sihyun_Kim · May 21, 2024, 5:31pm

I installed Julia, and am trying to execute it by typing julia in Windows PowerShell. Then, the terminal shows

PS C:\Users\김시현> julia
ERROR: could not load library "C:\Users\김?�현\.julia\juliaup\julia-1.10.3+0.x64.w64.mingw32\lib\julia\sys.dll"
The specified module could not be found.

I think the error results from

김?�현
I don’t understand where this weird character comes from. How should I fix this?

Ahmed_Salih · May 21, 2024, 5:35pm

Perhaps it is your user name on your computer?

ufechner7 · May 21, 2024, 5:44pm

I think that indeed Julia will not work if you install it in a home folder with non-ascii letters or with spaces…
What you can do if you use bash:

export JULIA_DEPOT_PATH="/.julia"

if the folder /.julia exists and can be written as normal user…

You might need a different command for powershell…

The key message: Create and environment variable with the name JULIA_DEPOT_PATH and set it to a path that exists, is writable and contains only ASCII characters…

Then everything should work fine…

nhz2 · May 21, 2024, 8:20pm

This looks like a bug in either julia or juliaup on Windows. Do the 64 bit “portable” or “installer” manual downloads from Download Julia work?

mkitti · May 21, 2024, 8:32pm

You may also want to change the environment variable JULIAUP_DEPOT_PATH:

giordano · May 21, 2024, 10:04pm

Indeed, we’ve seen many reports of users having troubles to load libraries from paths which include non-ASCII characters: Failed process 'gdk-pixbuf-query-loaders.exe' during load on Windows · Issue #461 · JuliaGraphics/Gtk.jl · GitHub, (julia.exe:7396): Gtk-WARNING **: 21:07:31.246: Could not load a pixbuf from icon theme. · Issue #497 · JuliaGraphics/Gtk.jl · GitHub, julia.exe starting error with Korean path-name. · Issue #33486 · JuliaLang/julia · GitHub. This may have been fixed by update libuv to v2-1.48.0 by vtjnash · Pull Request #49937 · JuliaLang/julia · GitHub, which is in Julia v1.11, although I’m still not 100% libuv is related to this specific issue, I guess some Windows users will have to confirm that.

woclass · May 22, 2024, 12:23am

An update to libuv (pr#49937) is included in v1.11-beta.
I tested it in a clean windows sandbox:

Seems works fine.

“你好” means “hello” in chinese.

giordano · May 22, 2024, 12:32am

Did you experience issues in the same configuration with Julia < 1.11 though?

StefanKarpinski · May 22, 2024, 1:18am

The issue was that Windows paths use invalid UTF-16 to encode Korean names. Libuv uses UTF-8. Until recently libuv insisted on strict UTF-8 which means that when Windows gives it invalid UTF-16 (unpaired surrogates), it simply cannot handle it. There is, however, a thing called WTF-8, which is just UTF-8 allowing surrogate code points (they’re fine from UTF-8’s perspective, just technically disallowed), which allows handling arbitrary invalid UTF-16. @jameson and I finally got libuv to use WTF-8, so this is expected to be fixed now.

woclass · May 22, 2024, 2:57am

Bad news: after more careful testing, this issue still exists on v1.11-beta1.

The problem in my last test: my main system language is set to Chinese and the code page is 936, in the sandbox the language is set to English and the code page is also 936.

cp936 seems to maintain compatibility with UTF8, so no invalid UTF16 is generated.

Set the global language of the sandbox to Korean.
The default code page changes to cp 949.

Create a new test folder “테스트” again
Both v1.10.3 and v1.11.0-beta1 cannot load sys.dll
When the path contains only ascii characters, everything works fine.

The problem seems to be in the call to MultiByteToWideChar, where it is assumed that the input string is encoded in UTF8, while the actual incoming encoding is affected by the system language and code page settings.

github.com

JuliaLang/julia/blob/263928f9ad450a28d601c3f185040987e0bc45d5/src/dlload.c#L157


      
                seg_end = RoundUpTo(seg_end, jl_page_size);
                cb((void *)seg_start, seg_end - seg_start);
              }
            }
          }
          #endif
          
          #if defined(_OS_WINDOWS_)
          JL_DLLEXPORT void *jl_dlopen(const char *filename, unsigned flags) JL_NOTSAFEPOINT
          {
              size_t len = MultiByteToWideChar(CP_UTF8, 0, filename, -1, NULL, 0);
              if (!len) return NULL;
              WCHAR *wfilename = (WCHAR*)alloca(len * sizeof(WCHAR));
              if (!MultiByteToWideChar(CP_UTF8, 0, filename, -1, wfilename, len)) return NULL;
              HANDLE lib;
              if (flags & JL_RTLD_NOLOAD) {
                  lib = GetModuleHandleW(wfilename);
              }
              else {
                  lib = LoadLibraryExW(wfilename, NULL, LOAD_WITH_ALTERED_SEARCH_PATH);
                  if (lib)

nhz2 · May 22, 2024, 3:00am

I tested this on Julia 1.11 and got the same error. I was able to fix the issue and run Julia on a test user after enabling “Beta: Use Unicode UTF-8 for worldwide language support” in Time & Language → Language → “Administrative Language Settings”, and rebooting.

giordano · May 22, 2024, 8:15am

That makes more sense to me, thanks. What’s failing is basically our implementation of dlopen, which explains the issues I linked above (they’re about loading libraries), and has nothing to do with libuv, which explains why the mentioned PR didn’t solve this specific issue (as I expected).

StefanKarpinski · May 22, 2024, 2:46pm

Are you able to open an issue describing this or make PR to address this? Would be great if this just worked now that libuv supports it. Of
Course, I also thin like that everyone should just use UTF-8 for stuff like this, so I don’t see the “workaround” as a workaround really—it’s the way things should be done.