Module Loading Case Sensitivity on Windows

Hi all,

I’ve had some frustration when trying to load custom modules on Windows. Could someone clarify exactly what the case sensitivity rules are when trying to load modules, and any differences between file systems which are and are not case sensitive?

On Windows 10 (x64), I create the following file

module MyModule

println("Module loaded!")

end # module

and call it mymodule.jl. I put it in C:/Users/MyUserName/Desktop/test.

Then in the Julia v1.0.2 (x64) REPL I do the below:

julia> push!(LOAD_PATH,"C:/Users/MyUserName/Desktop/test")
4-element Array{String,1}:
 "@"
 "@v#.#"
 "@stdlib"
 "C:/Users/MyUserName/Desktop/test"

julia> using MyModule
ERROR: ArgumentError: Package MyModule not found in current path:
- Run `import Pkg; Pkg.add("MyModule")` to install the MyModule package.

Stacktrace:
 [1] require(::Module, ::Symbol) at .\loading.jl:823

Now I rename the file to MyModule.jl and repeat:

julia> push!(LOAD_PATH,"C:/Users/MyUserName/Desktop/test")
4-element Array{String,1}:
 "@"
 "@v#.#"
 "@stdlib"
 "C:/Users/MyUserName/Desktop/test"

julia> push!(LOAD_PATH,"C:/Users/MyUserName/Desktop/test")
4-element Array{String,1}:
 "@"
 "@v#.#"
 "@stdlib"
 "C:/Users/MyUserName/Desktop/test"

julia> using MyModule
[ Info: Recompiling stale cache file C:\Users\MyUserName\.julia\compiled\v1.0\MyModule.ji for MyModule [top-level]
Module loaded!

Hooray! Things work.

So questions:

  • Is this requirement documented anywhere?
  • How does this work if I say, clone someone’s repo who’s been on a case-insensitive filesystem?

The latter seems to be causing my issues. In that case, I think when Git hasn’t been setup to track case changes in file names, custom module loading can sometimes not work for inexplicable reasons.

Thoughts?

Hi all, sorry to bump my own post. I’m still having trouble with this. Can anyone help?

Can’t you just make the file name match the case of the module?

See https://github.com/JuliaLang/julia/pull/13542 … like in Python (PEP 235), we decided to make module loading (using Foo) case-sensitive on case-insensitive case-preserving filesystems.

Is this requirement documented anywhere?

This is documented in the manual under https://docs.julialang.org/en/latest/base/base/#Base.require

How does this work if I say, clone someone’s repo who’s been on a case-insensitive filesystem?

Virtually all modern filesystems are either case-sensitive or case-insensitive but case-preserving. On the latter (Mac and Windows) systems, the case information is not lost, so you are fine.

Do you have a real need to support source code on case-insensitive case-destroying filesystems?

4 Likes

Thanks very much for the replies.

This is documented in the manual under https://docs.julialang.org/en/latest/base/base/#Base.require

I notice the documentation says

require is case-sensitive on all platforms, including those with
case-insensitive filesystems like macOS and Windows.

So to be clear, this is saying that the name of a module must, on such filesystems, match (with case sensitivity) the name of the .jl file in which it is created? I wonder if this could be made clearer and put straight into https://docs.julialang.org/en/latest/manual/modules/ ?

Can’t you just make the file name match the case of the module?

Whilst it’s been a while, I think, when I first created this topic, part of the issue I was having was that sometimes it seemed to matter if the file and module name’s cases matched, and sometimes it didn’t.

It’s not clear to me why, but here are two cases which illustrate the problem (at least on my machine):

On Windows 10 (x64), I create seqtest.jl in C:/Users/MyUserName/Desktop/test. This contains

module SeqTest

println("loaded SeqTest")

end # module SeqTest

in the same folder I create procmsgseqtest.jl which contains

module ProcMsgSeqTest

println("loaded ProcMsgSeqTest")

end # module ProcMsgSeqTest

Then in the Julia v1.0.2 (x64) REPL I do the following:

julia> push!(LOAD_PATH,"C:\\Users\\MyUserName\\Desktop\\test")
4-element Array{String,1}:
 "@"
 "@v#.#"
 "@stdlib"
 "C:\\Users\\MyUserName\\Desktop\\test"

julia> using SeqTest
ERROR: ArgumentError: Package SeqTest not found in current path:
- Run `import Pkg; Pkg.add("SeqTest")` to install the SeqTest package.

Stacktrace:
 [1] require(::Module, ::Symbol) at .\loading.jl:823

julia> using ProcMsgSeqTest
[ Info: Recompiling stale cache file C:\Users\MyUserName\.julia\compiled\v1.0\ProcMsgSeqTest.ji for ProcMsgSeqTest [top-level]
loaded ProcMsgSeqTest

Why does it work in one case but not the other? Perhaps someone else could verify on a windows machine?

We try to enforce that package source file case matches the module case even on case-insensitive file systems to avoid having people develop package on case-insensitive file systems and then have those packages break on case-sensitive file systems. It’s possible that these extra checks are being avoided in one of these cases, but it looks like you’re doing the exact same thing with both of them, so I’m not sure how that’s possible.

Bottom line: the case of the source file that defined a package should match the case of the package name. If there’s a particular place in the docs where you feel like that could be clarified, please do open an issue or better still a pull request proposing such a clarification.

2 Likes

As requested! https://github.com/JuliaLang/julia/pull/31911

Code loading calls isfile_casesensitive, which for Windows is defined as:

function isfile_casesensitive(path)
    isfile(path) || return false  # Fail fast
    basename(Filesystem.longpath(path)) == basename(path)
end

Filesystem.longpath calls the Win32 function GetLongPathName which supposedly returns the case-preserved filename on NTFS.

According to this write-up,

GetLongPathName is specifically meant for 8.3 type paths. So as long as you use 8 character filenames and paths, it will in fact do the right thing.

So a somewhat hacky way to get this to work is to convert the full path to a short path first, then call GetLongPathName with the adjusted path.

So it turns out that case-sensitive check works for SeqTest because it is short, whereas ProcMsgSeqTest is too long.

2 Likes

2 Likes

Yes, see also this discussion about fixing realpath: inconsistent realpath on Windows · Issue #30588 · JuliaLang/julia · GitHub

We could switch to using realpath (which is fixed in Julia 1.2), but I don’t know if that would cause a problem with symlinks on Windows?

Given this seems to be an issue, I’ve gone ahead and created an issue for this : https://github.com/JuliaLang/julia/issues/31913 . I hope that’s useful and doesn’t just split discussion up!