Failed to precompile ScikitLearn and Plots

New to Julia
, , ,
1

Am trying to add ScikitLearn and Plots package into my Pluto note book but am getting an error.
I tried updating, adding python path to Julia but did not work. Any suggestions on what I can do.

error:
Failed to precompile ScikitLearn [3646fa90-6ef7-5e7e-9f22-8aca16db6324] to C:\Users\ASUS.julia\compiled\v1.8\ScikitLearn\jl_FB9A.tmp.

error(::String)@error.jl:35
compilecache(::Base.PkgId, ::String, ::IO, ::IO, ::Bool)@loading.jl:1707
compilecache@loading.jl:1651[inlined]
_require(::Base.PkgId)@loading.jl:1337
_require_prelocked(::Base.PkgId)@loading.jl:1200
macro expansion@loading.jl:1180[inlined]
macro expansion@lock.jl:223[inlined]
require(::Module, ::Symbol)@loading.jl:1144
top-level scope@Local: 1

2

Hi,

Can you do something like (and show me your result):

(@v1.9) pkg> st ScikitLearn Plots PyCall PythonCall
Status `~/.julia/environments/v1.9/Project.toml`
  [91a5bcdd] Plots v1.38.10 `~/.julia/dev/Plots`
  [438e738f] PyCall v1.95.1
  [6099a3de] PythonCall v0.9.12
  [3646fa90] ScikitLearn v0.7.0

I don’t know exactly what’s the problem for your environment (any version should work in theory), but I know that at least these combinations of versions work for me.

They are the latest for ScikitLearn, which relies on PyCall, also latest, and work for me:

julia> @time using PyCall
  0.572052 seconds (396.61 k allocations: 24.870 MiB, 2.31% compilation time)

julia> @time using ScikitLearn
[ Info: Precompiling ScikitLearn [3646fa90-6ef7-5e7e-9f22-8aca16db6324]
 78.669469 seconds (3.46 M allocations: 229.452 MiB, 0.19% gc time, 3.66% compilation time: 92% of which was recompilation)

julia> versioninfo()
Julia Version 1.9.0-rc1

Very likely should work for you on 1.8 too. I’m on Linux and PyCall is a bit different there, on Windows I know it downloads a version for you. I’m not sure if that’s part of the reason or it failed. In general I would use PythonCall instead of PyCall, but that’s out of your hands as the latter is a dependency of ScikitLearn.

I googled your problem, and found a slightly different error, still a precompile error, so worth to try:
https://github.com/cstjean/ScikitLearn.jl/issues/105#issuecomment-1001791572

I am not able to reproduce but did have a different issue that was resolved with

import Conda
Conda.update()

And sometimes the AI can (make code, or if not gets an error message can) figure errors out, so I got curious what it would say (and I pasted the error message in full, and just that in Bing Chat):

Searching for: Failed to precompile ScikitLearn [3646fa90-6ef7-5e7e-9f22-8aca16db6324]

Generating answers for you…

This error message is usually caused by a package that failed to precompile. The error message suggests that the ScikitLearn package failed to precompile. This could be due to a number of reasons such as missing dependencies or an outdated version of Julia. You can try updating your Julia version and then reinstalling the ScikitLearn package. If that doesn’t work, you can try removing the package and then reinstalling it again. Here is an example of how you can remove and reinstall the ScikitLearn package:

using Pkg
Pkg.rm("ScikitLearn")
Pkg.add("ScikitLearn")

I hope this helps!

I wasn’t too sure about that, but since I’m not having a problem, I can’t know for sure if this helps you, so I asked further:

Could it have to do with PyCall?

Searching for: PyCall ScikitLearn

Generating answers for you…

Yes, it could be related to PyCall. PyCall is a Julia package that allows you to call Python functions from Julia. ScikitLearn.jl uses PyCall to interface with the Scikit-Learn library in Python. If you are having issues with PyCall, it could be due to a number of reasons such as missing dependencies or an outdated version of Julia. You can try updating your Julia version and then reinstalling the PyCall package. If that doesn’t work, you can try removing the package and then reinstalling it again. Here is an example of how you can remove and reinstall the PyCall package:

using Pkg
Pkg.rm(“PyCall”)
Pkg.add(“PyCall”)
I hope this helps!

3

thank you @Palli

this is what am am getting:

julia> Pkg.status(“ScikitLearn”)
Status C:\Users\ASUS\.julia\environments\v1.8\Project.toml
[3646fa90] ScikitLearn v0.7.0

julia> Pkg.status(“PyCall”)
No Matches in C:\Users\ASUS\.julia\environments\v1.8\Project.toml

julia> Pkg.status(“Plots”)
Status C:\Users\ASUS\.julia\environments\v1.8\Project.toml
[91a5bcdd] Plots v1.38.10

julia> Pkg.status(“PythonCall”)
No Matches in C:\Users\ASUS\.julia\environments\v1.8\Project.toml

4

The issue seems related to PyCall.
My python is in anaconda. I tried adding the path but did not work using ENV[“PYTHON”] = C://Users//ASUS/anaconda3//python.ex

5

This is not a worry, you do have it installed, indirectly, just not in your environment, and I want to know the version you have.

It would be nice if what you did still showed that, and I got curious if it’s an option:

help?> Pkg.status("PyCall")
[..]
  Print out the status of the project/manifest.
[..]
  pkg> Pkg.status(; outdated=true)
  Status `Manifest.toml`

I do see two problems there, it doesn’t print out the “status of the project/manifest”, i.e. yes, the project/environment, but not the manifest.

Also the pkg> prompt there is the wrong one, it will only work at the >julia prompt (I wanted to make a PR to fix the docs, but I can’t actually find it so I’m not sure where to fix, in case anyone can help). Also even if it seems to be looking in the manifest file, I’m not sure that’s happening for me.

You said you updated, but up alone or even with or without the version may not work:

(@v1.9) pkg> up PyCall@v1.95.1

Can you try:

(@v1.9) pkg> add PyCall@v1.95.1

That should work. From memory having the version with up is ignored, also up yes, tries to upgrade, but it may be blocked from going to the latest version, and then will go to older, and that’s not considered a failure (since it should also work). But I’m trying to force the last version to rule out the older as the problem.

6

Yes, I think it might relate to PyCall. Thought that is rather invisible from the error message. Are you sure you have anaconda, or did you mean conda? It’s ok to have it, you could most likely use it, but you can also have conda that PyCall provides for your (or both):

By default on Mac and Windows systems, Pkg.add(“PyCall”) or Pkg.build(“PyCall”) will use the Conda.jl package to install a minimal Python distribution (via Miniconda) that is private to Julia (not in your PATH). You can use the Conda Julia package to install more Python packages, and import Conda to print the Conda.PYTHONDIR directory where python was installed.

You should be able to (install it and) do as I did:

julia> @time using PyCall

I want you to have a good experience with Julia. @stevengj’s PyCall is excellent, when it works, i.e. after it starts working, on its own or with Python packages installed. I feel like the newer PythonCall is better (with dependency handling), and it should worth with Python’s skikitlearn, while not (right now) with the Julia wrapper for it.

7

Thank you @Palli .

My PyCall is showing now.
julia> Pkg.status(“PyCall”)
Status C:\Users\ASUS\.julia\environments\v1.8\Project.toml
[438e738f] PyCall v1.95.1

8

Am also concerned about plots:
image

9

Do you still get an error for the following?:

julia> @time using ScikitLearn

I do see an older thread, with a slightly different error message (why not getting same hint here?), but the solution might still work:

using ScikitLearn
┌ Info: Precompiling ScikitLearn [3646fa90-6ef7-5e7e-9f22-8aca16db6324]
└ @ Base loading.jl:1260
ERROR: LoadError: LoadError: PyCall not properly installed. Please run Pkg.build("PyCall")
10

I’m starting to think your .julia folder might be corrupted, since I don’t see a relation between Plots and PyCall (by default that is, i.e. GR is used, but there would be a relation if you opted into Python-based plotting backend).

I’m assuming since you’re new on Discourse that you might be a new Julia user. So where did you get the notebook file, and can you share it somehow?

Thee are two options I see.

I would never nuke the .julia folder (that’s bad advice, to delete it, but renaming it is ok, and I always rather do it as a first step). You could start from scratch somehow, with 1.8.

You could try Julia 1.9, that I’m using. I would usually hesitate to suggest a release candidate, at least to a new user. The problem should also be solvable in still current Julia 1.8 (or older 1.6 or 1.7 if you want two different Julias concurrent). But 1.9 should be out any day now, and rather stable with rc2 (and rc3), that I feel it should be ok.

I recommend juliaup to download Julias, and it lets you switch between versions. I think in all cases the .julia folder is shared though. Even if it might be corrected, then I I’m not sure it would be a problem, since subfolders of it are different for 1.9 and 1.8 etc. Your call if you rename the folder first, then you can rename it back when you’ve figured out the problem.

11

I got the same error, But when I tried to run Pkg.build(“Pycall”)
this is he error that I got:

PyCall → C:\Users\ASUS\.julia\scratchspaces\44cfe95a-1eb2-52ea-b672-e2afdf69b78f\62f417f6ad727987c755549e9cd88c46578da562\build.log
ERROR: Error building PyCall:
failed to create process.
┌ Info: Using the Python distribution in the Conda package by default.
└ To use a different Python version, set ENV[“PYTHON”]=“pythoncommand” and re-run Pkg.build(“PyCall”).
[ Info: Running conda install -y numpy in root environment