I wanted to report the very slow and somewhat frustrating experience of a beginner trying to install and use JuliaPro. I apologize if this is not the proper forum category.
Note: I come from a Matlab background, which may explain some errors I am making and some expectations I have. I am considering to use Julia for my own data analysis but also for teaching to non-programmers (who are assumed to have no patience). JuliaPro was interesting for me because it purportedly offers a nice “out of the box” experience. I am using a recent laptop with plenty of RAM and a 1 TB SSD, so I don’t think the hardware can be blamed.
Installing JuliaPro took 16 min. I had downloaded the full package before so I found that quite long. After that almost everything was very slow, to a point I would qualify as non-responsive. I tried to follow the JuliaPro quick start guide, and I am giving details on what I did and the time it took below.
I am posting this for 3 reasons:
getting some advice in case I am doing something wrong, or in case better options to use Julia exist
provide some feedback to the JuliaPro developers (couldn’t find a category for that)
getting specific feedback from other Matlab users and/or teachers: how quickly do these frustrations (if you felt them!) go away?
So here are some details about the unresponsiveness:
After installation was completed, I launched Juno and tried to follow the JuliaPro quick start guide.
Interestingly, once in the Juno window, I am in fact in a program called Atom, not Juno. I can already see the students emails…
The first bits of code detailed in the quick start guide (not in screenshots) are about making a plot with Gadfly. So I type using Gadfly in the console, and… nothing happens for 1 minute. then I start getting INFO messages (precompiling), and the command completes in 1 min 30.
I then type Gadfly.plot(sin,0,25). After 30 seconds, the plot appears.
I later realized that typing the same command again produces the plot much faster, but I didn’t know that. So I search the web, and see something somewhere that Gadfly is slow and Plots is faster.
But Plots is not installed by default. So I type Pkg.add("Plots"). Nothing happens for 40 seconds, then I start getting some INFO messages, and the install completes after 2 min 10s.
But the last INFO message says the package is out-of-date (didn’t I just install it?). As instructed, I type Pkg.update(). Another 2 minutes waiting.
I can finally type using Plots, which takes 25 seconds (it actually seems quite fast, I am getting used to my computer not responding… the JuliaPro effect?)
I get warnings that there may be conflicts. I guess it comes from Gadfly, a quick internet search tells me there is no way to “unuse” a package, but workflow() does roughly what “clear” would do in Matlab, so I try that. But I after that I get into a seemingly endless series of errors and warnings when trying to redo “using Plots”, so I end up restarting Juno (or more exactly Atom).
After restarting, using Plots, creating x=linspace(0,25) and y=sin.(x), I can finally plot them. plot(x,y) takes about 20 seconds. Subsequent plot commands are much faster, fortunately.
All in all, not a very successful/pleasing first attempt. So I am looking forward to your advice!
I think this sort of report is useful for other first timers. Some aspects of Julia use are definitely unexpected coming from a Matlab or Python background. I would say I don’t believe any of your experiences would surprise the developers.
What you keep running into is (pre) compilation time. Roughly, the first time you call any function, the function must be compiled. I don’t know if you’ve used C, but it’s akin to using a C program. First you compile, which is slow, but then the execution is fast.
Packages will be pre-compiled the first time you call using Package. This takes a long time, but the work is saved (cached), so the next time you use Julia, calling using Package should be almost instantaneous. The exception is if you update with Pkg.update. After any update, the package will need to be pre-compiled again.
This is definitely annoying to newcomers. The first time you use Julia, nothing will be pre-compiled, so most commands will be slow. However, you shouldn’t run into this the next time.
I found the Julia REPL to be much snappier also consider trying VSCode’s Julia extension, which is also snappier. Juno has some nice features but can sometimes feel slow. I was doing using SomePkg and that pakcage wasn’t installed and I timed it, it took 40 seconds for Juno to respond with error. In the Julia REPL it was almost instant.
At least the slowness of package operations should be fixed soon in Julia 0.7. The fact that loading plotting packages is very slow is unfortunately known, and will hopefully improve at some point, but not right now. See also 'using Plots' takes 20 seconds or more.
Restarting Juno/Atom is unnecessary in that case (in most cases, actually) – you can just kill Julia with Ctrl-JCtrl-K (or type Ctrl-Shift-P to open the command pane and type in “Kill Julia”) and then restart it by evaluating something in the console or in an editor.
That seems quite extreme – using Foo takes roughly 10s for me (in Juno) when Julia isn’t started yet. While that’s obviously much slower than the ~1s it takes in the REPL (including startup time), we are looking into speeding that up.
Thank you for these informative answers and comments. I may give a try to VSCode, but I suspect the slow plotting will be a significant problem for me. Plotting (and interactively zooming, panning) large datasets is an important part of data analysis for me. I realize they may be workarounds, super-user tricks and all, but I would have to figure this out and explain it to my students in addition to switching from one language to another one (and loosing my collection of m-files), so I don’t think I am ready for “the big switch” (and neither are my students). After all Julia is only 0.6, so I will check back later.
Slow is matter of perception and students or non-software developers won’t notice that plotting in Julia is actually much faster than Excel or Google Spreadsheet.
Because unlike those commercial programs, Julia doesn’t have pretty loading screen and tricks to hide true loading time of application.
From a business perspective, if we want to advertise Julia as fastest numerical computation software in the world, optimization and benchmark numbers won’t have much effect on public-students, corporate bosses- opinion
Maybe hiring web designer and developer to make JuliaPro prettier would help Julia to appeal to a public more
(This is just some random idea, I always feel grateful for all the hard works done by Julia developers. I wish I can express those feeling in more sincere manner)
I think it is important to recall that Julia is still in development (we are at v0.6.2); this is probably one of the reasons why it cannot compete (in some aspects such as documentation/stability of code,…) with established programming languages.
I am a Windows user too. In my view:
the package manager is slow (maybe Pkg3 will improve this). Pkg.update() can take AGES!
using takes some time
precompilation and compilation also takes time (‘even more’ (subjectively) if you are not used to it)
I want to add
a) It is quite easy to create a fast programme for non-programmers (I am a mathematician) in Julia! This is possibly the foremost reason for me to use Julia
b) I would not compare all the precompilation and start-up times, but rather judge Julia by the “subsequent plot calls”
I have no argument for b) (I simply like Julia; I am biased)
As a side note: Interestingly the plot command actually crashes Julia for me, see below
_
_ _ _(_)_ | A fresh approach to technical computing
(_) | (_) (_) | Documentation: https://docs.julialang.org
_ _ _| |_ __ _ | Type "?help" for help.
| | | | | | |/ _ | |
| | |_| | | | (_| | | Version 0.6.2 (2017-12-13 18:08 UTC)
_/ |\__'_|_|_|\__'_| | Official http://julialang.org/ release
|__/ | x86_64-w64-mingw32
julia>
julia> versioninfo()
Julia Version 0.6.2
Commit d386e40c17* (2017-12-13 18:08 UTC)
Platform Info:
OS: Windows (x86_64-w64-mingw32)
CPU: Intel(R) Core(TM) i7-6600U CPU @ 2.60GHz
WORD_SIZE: 64
BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Haswell)
LAPACK: libopenblas64_
LIBM: libopenlibm
LLVM: libLLVM-3.9.1 (ORCJIT, skylake)
julia> using Plots
julia> x=linspace(0,25);
julia> y=sin.(x);
julia> plot(x,y) #R6034 Runtime Error (MS Visual C++ Runtime Library)
This application failed to start because it could not find or load the Qt platform plugin "windows"
in "".
Available platform plugins are: minimal, offscreen, windows.
Reinstalling the application may fix this problem.
This might be because I messed up my C++ installation somehow. If anyone else experiences this, I am happy to file an issue.
This is because Pkg2 uses a lot of small files and folders, and Windows (other filesystems as well) don’t like this at all. It just slows them to a crawl. Pkg3 uses larger manifest files. Done.
This is part of a larger problem that not enough precompiles. Some others have been playing with tools like SnoopCompile.jl to fix this though, and I think a much better solution can come in a 1.x
I do worry that we will have a very hard time with inexperienced users coming from R and Python who will perceive Julia as “slow” because it compiles. Since my background is largely in C++ and I have an easy time accepting that code must compile, I find it really frustrating that people would prefer an interpreted language simply to avoid a 500ms compile time.
The slowness of plotting however is a legitimate problem that I don’t really understand and needs to be fixed (i.e. this is certainly not just a 500ms issue). On the bright side I can say that I haven’t encountered any compile or load times outside of plotting that I considered to be truly problematic.
It’s because the backends are setup conditionally and lazily, making the code it touches not precompilable, along with the fact that precompilation isn’t comprehensive. Starting up a backend in Plots is literally the Julia worst case scenario right now.
Does this have something to do with them having non-Julia dependencies? I would think even if you load Julia modules conditionally, as long as you have the compiled binaries it wouldn’t be too bad.
They could be Julia dependencies and it would still happen. The code literally uses @eval to using modules in the backend setup calls (gr()), and then defines backend-specific functions as well. Thus a large portion of the global state doesn’t even exist at precompile time to be precompiled. The idea is to change this to modules with a dispatch structure on a small set of interface functions so it’ll work in a way that’s compatible with precompilation.
I’d like to expand a bit on what @ChrisRackauckas said. The thing is that Plots.jl managed to become rich in terms of features and pleasant to use (syntax-wise) thanks to the effort of many contributors but still has two structural issues:
Slow first plot time, due to lazy loading of the back-end plotting library (I started using Julia at a time where everything had a crazy long starting time, things have improved massively since then, so now Plots.jl is one of the few libraries that still takes long)
Plots.jl interfaces with the backends in a high level and not fully documented way, which makes it difficult to maintain a backend
There have been plans to solve 1) for a while with the creation of linkers modules (such as PlotsGR). However, recently the possibility of solving 1) and 2) at the same time emerged (with Makie ). It is a riskier approach as it’s hard to tell whether/when Makie can reach feature parity with Plots, but if it worked it would bring many advantages to the table (it can plots many points very quickly, can -experimentally- be statically compiled thus reducing start up time drastically, is extremely interactive). On the other hand, it slowed down progress on Plots side as it makes little sense to start refactoring Plots when another more radical refactoring is taking place.
I’m neither an expert on packages nor a user of JuliaPro, but it seems that there is a relatively simple workaround to the “beginning user” problem that doesn’t require any revision to the Julia core or packages. When a user downloads and installs JuliaPro, a dialog box pops up that says:
Precompile all packages now? (Warning: this may take > 60 minutes but will speed up package loading during subsequent JuliaPro usage. Refer to documentation.)
Yes
No
Then the user has a choice and is at least warned that if he/she chooses “no”, there will be a delay later each time a using command is issued.
I’m an old timer, but still consider myself a newbie. It’s been a while since I did any serious development in Julia and just started up again.
I have a lot of sympathy for the concerns in the original post. I don’t remember Julia being so excruciating slow or I wouldn’t have been an early believer / evangelist.
My own private package takes 11 seconds to load. That is not acceptable when you are developing a package. I know you guys are pushing for 1.0 and I know the discussions are happening elsewhere (and sorry I am not very interested in reading them), but I would say you are nowhere near 1.0 if this is the experience newcomers will have. 1.0 means, to me anyway and most people I think, “This package is ready for production development”. Production with this kind of miserable experience? I don’t think so.
Revise.jl looks awesome and can probably help a lot with the pain, but without a good user experience, I would delay the 1.0 moniker as it will bring a lot of disappointment to a lot of people.
I say this out of love for Julia. Maybe I am doing something silly, but 11 second delays between every minor code change is painful.
That’s not a terribly helpful way to participate.