Does Python 3.11 improve the performance of compatible imported python packages in Julia?

Scikit-Learn recently released wheels suitable for Python3.11. Now I wonder if Python3.11 improves the Python packages’ performance in Julia. PyCall.jl uses a specific python version, and one can install any installable package using it and import it to the environment; since they claim that Python3.11 is somewhat improved I want to know if this improvement can effect performance of the imported python packages.

PyCall is just calling libpython directly to run Python code, so the performance should be essentially identical to what you get from python.

4 Likes

So this means utilizing Python3.11 in julia should be at least slightly better than utilizing any other version. How one can use Python3.11 in Julia?

Thanks. How can I utilize Python3.11 in Julia? For the sake of using Python packages in Julia.

See PyCall: Specifying the Python version.

I tried to replicate this via executable python in my Python3.11 Conda environment. But I faced an error. Should I install a standalone Python3.11 on my machine?

If you can install Python 3.11 in Conda then you should be able to use it from Julia. Hard to say more without knowing what you mean by β€œI faced an error”.

julia> ENV["PYTHON"] = "C:/Users/Shayan/Anaconda/\envs/3.11/python.exe"
"C:/Users/Shayan/Anaconda/\envs/3.11/python.exe"

julia> using Pkg

julia> Pkg.build("PyCall")
    Building Conda ─→ `C:\Users\Shayan\.julia\scratchspaces\44cfe95a-1eb2-52ea-b672-e2afdf69b78f\6e47d11ea2776bc5627421d59cdcc1296c058071\build.log`
    Building PyCall β†’ `C:\Users\Shayan\.julia\scratchspaces\44cfe95a-1eb2-52ea-b672-e2afdf69b78f\53b8b07b721b77144a0fbbbc2675222ebf40a02d\build.log`
ERROR: Error building `PyCall`, showing the last 100 of log:
    - pycosat==0.6.4=py310h8d17308_1
    - pycparser==2.21=pyhd8ed1ab_0
    - pyopenssl==22.1.0=pyhd8ed1ab_0
    - pysocks==1.7.1=pyh0701188_6
    - python==3.10.6=hcf16a7b_0_cpython
    - python_abi==3.10=2_cp310
    - requests==2.28.1=pyhd8ed1ab_1
    - ruamel_yaml==0.15.80=py310h8d17308_1008
    - setuptools==65.5.0=pyhd8ed1ab_0
    - tk==8.6.12=h8ffe710_0
    - toolz==0.12.0=pyhd8ed1ab_0
    - tqdm==4.64.1=pyhd8ed1ab_0
    - tzdata==2022e=h191b570_0
    - ucrt==10.0.22621.0=h57928b3_0
    - urllib3==1.26.11=pyhd8ed1ab_0
    - vc==14.3=h3d8a991_9
    - vs2015_runtime==14.32.31332=h1d6e394_9
    - wheel==0.37.1=pyhd8ed1ab_0
    - win_inet_pton==1.1.0=pyhd8ed1ab_6
    - xz==5.2.6=h8d14728_0
    - yaml==0.2.5=h8ffe710_2


The following NEW packages will be INSTALLED:

  brotlipy           conda-forge/win-64::brotlipy-0.7.0-py310h8d17308_1005
  bzip2              conda-forge/win-64::bzip2-1.0.8-h8ffe710_4
  ca-certificates    conda-forge/win-64::ca-certificates-2022.9.24-h5b45459_0
  certifi            conda-forge/noarch::certifi-2022.9.24-pyhd8ed1ab_0
  cffi               conda-forge/win-64::cffi-1.15.1-py310h628cb3f_2
  charset-normalizer conda-forge/noarch::charset-normalizer-2.1.1-pyhd8ed1ab_0
  colorama           conda-forge/noarch::colorama-0.4.6-pyhd8ed1ab_0
  conda              conda-forge/win-64::conda-22.9.0-py310h5588dad_1
  conda-package-han~ conda-forge/win-64::conda-package-handling-1.9.0-py310h635b8f1_1
  cryptography       conda-forge/win-64::cryptography-38.0.2-py310h6e82f81_2
  idna               conda-forge/noarch::idna-3.4-pyhd8ed1ab_0
  libffi             conda-forge/win-64::libffi-3.4.2-h8ffe710_5
  libsqlite          conda-forge/win-64::libsqlite-3.39.4-hcfcfb64_0
  libzlib            conda-forge/win-64::libzlib-1.2.13-hcfcfb64_4
  menuinst           conda-forge/win-64::menuinst-1.4.19-py310h5588dad_1
  miniforge_console~ conda-forge/win-64::miniforge_console_shortcut-1.0-h57928b3_0
  openssl            conda-forge/win-64::openssl-3.0.5-hcfcfb64_2
  pip                conda-forge/noarch::pip-22.3-pyhd8ed1ab_0
  pycosat            conda-forge/win-64::pycosat-0.6.4-py310h8d17308_1
  pycparser          conda-forge/noarch::pycparser-2.21-pyhd8ed1ab_0
  pyopenssl          conda-forge/noarch::pyopenssl-22.1.0-pyhd8ed1ab_0
  pysocks            conda-forge/noarch::pysocks-1.7.1-pyh0701188_6
  python             conda-forge/win-64::python-3.10.6-hcf16a7b_0_cpython
  python_abi         conda-forge/win-64::python_abi-3.10-2_cp310
  requests           conda-forge/noarch::requests-2.28.1-pyhd8ed1ab_1
  ruamel_yaml        conda-forge/win-64::ruamel_yaml-0.15.80-py310h8d17308_1008
  setuptools         conda-forge/noarch::setuptools-65.5.0-pyhd8ed1ab_0
  tk                 conda-forge/win-64::tk-8.6.12-h8ffe710_0
  toolz              conda-forge/noarch::toolz-0.12.0-pyhd8ed1ab_0
  tqdm               conda-forge/noarch::tqdm-4.64.1-pyhd8ed1ab_0
  tzdata             conda-forge/noarch::tzdata-2022e-h191b570_0
  ucrt               conda-forge/win-64::ucrt-10.0.22621.0-h57928b3_0
  urllib3            conda-forge/noarch::urllib3-1.26.11-pyhd8ed1ab_0
  vc                 conda-forge/win-64::vc-14.3-h3d8a991_9
  vs2015_runtime     conda-forge/win-64::vs2015_runtime-14.32.31332-h1d6e394_9
  wheel              conda-forge/noarch::wheel-0.37.1-pyhd8ed1ab_0
  win_inet_pton      conda-forge/noarch::win_inet_pton-1.1.0-pyhd8ed1ab_6
  xz                 conda-forge/win-64::xz-5.2.6-h8d14728_0
  yaml               conda-forge/win-64::yaml-0.2.5-h8ffe710_2


Preparing transaction: ...working... done
Executing transaction: ...working... done
β”Œ Info: No system-wide Python was found; got the following error:
β”‚ Base.IOError("could not spawn setenv(`C:/Users/Shayan/Anaconda/envs/3.11/python.exe -c \"import distutils.sysconfig; print(distutils.sysconfig.get_config_var('VERSION'))\"`,[\"PATH=C:\\\\Users\\\\Shayan\\\\Anaconda3\\\\condabin;C:\\\\Users\\\\Shayan\\\\AppData\\\\Local\\\\Programs\\\\Python\\\\Python311\\\\Scripts;C:\\\\Users\\\\Shayan\\\\AppData\\\\Local\\\\Programs\\\\Python\\\\Python311;C:\\\\Users\\\\Shayan\\\\AppData\\\\Local\\\\Programs\\\\Python\\\\Python310-32\\\\Scripts\\\\;C:\\\\Users\\\\Shayan\\\\AppData\\\\Local\\\\Programs\\\\Python\\\\Python310-32\\\\;C:\\\\Users\\\\Shayan\\\\AppData\\\\Roaming\\\\Python\\\\Python37\\\\Scripts;C:\\\\Users\\\\Shayan\\\\AppData\\\\Roaming\\\\Python\\\\Python37;C:\\\\WINDOWS\\\\system32;C:\\\\WINDOWS;C:\\\\WINDOWS\\\\System32\\\\Wbem;C:\\\\WINDOWS\\\\System32\\\\WindowsPowerShell\\\\v1.0\\\\;C:\\\\WINDOWS\\\\System32\\\\OpenSSH\\\\;C:\\\\Program Files\\\\dotnet\\\\;C:\\\\MinGW\\\\bin;C:\\\\Program Files\\\\Git\\\\cmd;C:\\\\Users\\\\Shayan\\\\AppData\\\\Local\\\\Programs\\\\Python\\\\Python311\\\\Scripts\\\\;C:\\\\Users\\\\Shayan\\\\AppData\\\\Local\\\\Programs\\\\Python\\\\Python311\\\\;C:\\\\Users\\\\Shayan\\\\AppData\\\\Local\\\\Programs\\\\Python\\\\Python310\\\\Scripts\\\\;C:\\\\Users\\\\Shayan\\\\AppData\\\\Local\\\\Programs\\\\Python\\\\Python310\\\\;C:\\\\Users\\\\Shayan\\\\AppData\\\\Local\\\\Programs\\\\Python\\\\Python37\\\\Scripts\\\\;C:\\\\Users\\\\Shayan\\\\AppData\\\\Local\\\\Programs\\\\Python\\\\Python37\\\\;C:\\\\Users\\\\Shayan\\\\AppData\\\\Local\\\\Programs\\\\Python\\\\Python37-32\\\\Scripts\\\\;C:\\\\Users\\\\Shayan\\\\AppData\\\\Local\\\\Programs\\\\Python\\\\Python37-32\\\\;C:\\\\Users\\\\Shayan\\\\AppData\\\\Local\\\\Microsoft\\\\WindowsApps;C:\\\\Users\\\\Shayan\\\\AppData\\\\Local\\\\Programs\\\\Microsoft VS Code\\\\bin;C:\\\\Users\\\\Shayan\\\\AppData\\\\Local\\\\GitHubDesktop\\\\bin;C:\\\\Users\\\\Shayan\\\\AppData\\\\Local\\\\Programs\\\\MiKTeX\\\\miktex\\\\bin\\\\x64\\\\;C:\\\\Users\\\\Shayan\\\\AppData\\\\Local\\\\Pandoc\\\\;C:\\\\Users\\\\Shayan\\\\Anaconda3;C:\\\\Users\\\\Shayan\\\\Anaconda3\\\\Scripts;C:\\\\Users\\\\Shayan\\\\Anaconda3\\\\Library\\\\bin\", \"USERDOMAIN_ROAMINGPROFILE=SHAYANSNOTEBOOK\", \"WSLENV=WT_SESSION::WT_PROFILE_ID\", \"HOMEPATH=\\\\Users\\\\Shayan\", \"PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC\", \"SYSTEMROOT=C:\\\\WINDOWS\", \"APPDATA=C:\\\\Users\\\\Shayan\\\\AppData\\\\Roaming\", \"PSMODULEPATH=%ProgramFiles%\\\\WindowsPowerShell\\\\Modules;C:\\\\WINDOWS\\\\system32\\\\WindowsPowerShell\\\\v1.0\\\\Modules\", \"COMMONPROGRAMW6432=C:\\\\Program Files\\\\Common Files\", \"PROGRAMDATA=C:\\\\ProgramData\", \"PUBLIC=C:\\\\Users\\\\Public\", \"USERDOMAIN=SHAYANSNOTEBOOK\", \"OS=Windows_NT\", \"PROCESSOR_REVISION=3c03\", \"TMP=C:\\\\Users\\\\Shayan\\\\AppData\\\\Local\\\\Temp\", \"COMSPEC=C:\\\\WINDOWS\\\\system32\\\\cmd.exe\", \"WT_PROFILE_ID={0caa0dad-35be-5f56-a8ff-afceeeaa6101}\", \"ALLUSERSPROFILE=C:\\\\ProgramData\", \"COMPUTERNAME=SHAYANSNOTEBOOK\", \"USERNAME=Shayan\", \"=E:=E:\\\\AUT\\\\Thesis\\\\MyWork\", \"ONEDRIVECONSUMER=C:\\\\Users\\\\Shayan\\\\OneDrive\", \"USERPROFILE=C:\\\\Users\\\\Shayan\", \"PROCESSOR_LEVEL=6\", \"=C:=C:\\\\Users\\\\Shayan\\\\.julia\\\\packages\\\\PyCall\\\\ygXW2\\\\deps\", \"CONDA_SHLVL=0\", \"PROGRAMW6432=C:\\\\Program Files\", \"CONDA_EXE=C:\\\\Users\\\\Shayan\\\\Anaconda3\\\\Scripts\\\\conda.exe\", \"TEMP=C:\\\\Users\\\\Shayan\\\\AppData\\\\Local\\\\Temp\", \"HOMEDRIVE=C:\", \"PYTHONIOENCODING=UTF-8\", \"PYTHON=C:/Users/Shayan/Anaconda/envs/3.11/python.exe\", \"WINDIR=C:\\\\WINDOWS\", \"LOCALAPPDATA=C:\\\\Users\\\\Shayan\\\\AppData\\\\Local\", \"PROCESSOR_IDENTIFIER=Intel64 Family 6 Model 60 Stepping 3, GenuineIntel\", \"NUMBER_OF_PROCESSORS=8\", \"PROMPT=\\\$P\\\$G\", \"COMMONPROGRAMFILES(X86)=C:\\\\Program Files (x86)\\\\Common Files\", \"PY=C:\\\\Users\\\\Shayan\\\\AppData\\\\Roaming\\\\Python\\\\Python37\\\\Scripts\", \"COMMONPROGRAMFILES=C:\\\\Program Files\\\\Common Files\", \"ONEDRIVE=C:\\\\Users\\\\Shayan\\\\OneDrive\", \"WT_SESSION=9fee51fe-e622-4f2e-a641-4f8dfd742037\", \"PROGRAMFILES(X86)=C:\\\\Program Files (x86)\", \"PROGRAMFILES=C:\\\\Program Files\", \"=EXITCODE=00000001\", \"OMP_NUM_THREADS=2\", \"LOGONSERVER=\\\\\\\\SHAYANSNOTEBOOK\", \"DRIVERDATA=C:\\\\Windows\\\\System32\\\\Drivers\\\\DriverData\", \"JULIA_LOAD_PATH=@;C:\\\\Users\\\\Shayan\\\\AppData\\\\Local\\\\Temp\\\\jl_FJel6n\", \"SYSTEMDRIVE=C:\", \"CONDA_BAT=C:\\\\Users\\\\Shayan\\\\Anaconda3\\\\condabin\\\\conda.bat\", \"PROCESSOR_ARCHITECTURE=AMD64\"]): no such file or directory (ENOENT)", -4058)
β”” using the Python distribution in the Conda package
[ Info: Downloading miniconda installer ...
[ Info: Installing miniconda ...
[ Info: Running `conda install -y numpy` in root environment
ERROR: LoadError: IOError: could not spawn setenv(`'C:/Users/Shayan/Anaconda3/envs/3.11\Scripts\conda.exe' install -y numpy`,["PATH=C:/Users/Shayan/Anaconda3/envs/3.11\\Library\\bin;C:\\Users\\Shayan\\Anaconda3\\condabin;C:\\Users\\Shayan\\AppData\\Local\\Programs\\Python\\Python311\\Scripts;C:\\Users\\Shayan\\AppData\\Local\\Programs\\Python\\Python311;C:\\Users\\Shayan\\AppData\\Local\\Programs\\Python\\Python310-32\\Scripts\\;C:\\Users\\Shayan\\AppData\\Local\\Programs\\Python\\Python310-32\\;C:\\Users\\Shayan\\AppData\\Roaming\\Python\\Python37\\Scripts;C:\\Users\\Shayan\\AppData\\Roaming\\Python\\Python37;C:\\WINDOWS\\system32;C:\\WINDOWS;C:\\WINDOWS\\System32\\Wbem;C:\\WINDOWS\\System32\\WindowsPowerShell\\v1.0\\;C:\\WINDOWS\\System32\\OpenSSH\\;C:\\Program Files\\dotnet\\;C:\\MinGW\\bin;C:\\Program Files\\Git\\cmd;C:\\Users\\Shayan\\AppData\\Local\\Programs\\Python\\Python311\\Scripts\\;C:\\Users\\Shayan\\AppData\\Local\\Programs\\Python\\Python311\\;C:\\Users\\Shayan\\AppData\\Local\\Programs\\Python\\Python310\\Scripts\\;C:\\Users\\Shayan\\AppData\\Local\\Programs\\Python\\Python310\\;C:\\Users\\Shayan\\AppData\\Local\\Programs\\Python\\Python37\\Scripts\\;C:\\Users\\Shayan\\AppData\\Local\\Programs\\Python\\Python37\\;C:\\Users\\Shayan\\AppData\\Local\\Programs\\Python\\Python37-32\\Scripts\\;C:\\Users\\Shayan\\AppData\\Local\\Programs\\Python\\Python37-32\\;C:\\Users\\Shayan\\AppData\\Local\\Microsoft\\WindowsApps;C:\\Users\\Shayan\\AppData\\Local\\Programs\\Microsoft VS Code\\bin;C:\\Users\\Shayan\\AppData\\Local\\GitHubDesktop\\bin;C:\\Users\\Shayan\\AppData\\Local\\Programs\\MiKTeX\\miktex\\bin\\x64\\;C:\\Users\\Shayan\\AppData\\Local\\Pandoc\\;C:\\Users\\Shayan\\Anaconda3;C:\\Users\\Shayan\\Anaconda3\\Scripts;C:\\Users\\Shayan\\Anaconda3\\Library\\bin", "USERDOMAIN_ROAMINGPROFILE=SHAYANSNOTEBOOK", "WSLENV=WT_SESSION::WT_PROFILE_ID", "HOMEPATH=\\Users\\Shayan", "PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC", "SYSTEMROOT=C:\\WINDOWS", "APPDATA=C:\\Users\\Shayan\\AppData\\Roaming", "PSMODULEPATH=%ProgramFiles%\\WindowsPowerShell\\Modules;C:\\WINDOWS\\system32\\WindowsPowerShell\\v1.0\\Modules", "COMMONPROGRAMW6432=C:\\Program Files\\Common Files", "PROGRAMDATA=C:\\ProgramData", "PUBLIC=C:\\Users\\Public", "USERDOMAIN=SHAYANSNOTEBOOK", "OS=Windows_NT", "PROCESSOR_REVISION=3c03", "TMP=C:\\Users\\Shayan\\AppData\\Local\\Temp", "COMSPEC=C:\\WINDOWS\\system32\\cmd.exe", "WT_PROFILE_ID={0caa0dad-35be-5f56-a8ff-afceeeaa6101}", "ALLUSERSPROFILE=C:\\ProgramData", "COMPUTERNAME=SHAYANSNOTEBOOK", "USERNAME=Shayan", "=E:=E:\\AUT\\Thesis\\MyWork", "CONDARC=C:/Users/Shayan/Anaconda3/envs/3.11\\condarc-julia.yml", "ONEDRIVECONSUMER=C:\\Users\\Shayan\\OneDrive", "USERPROFILE=C:\\Users\\Shayan", "PROCESSOR_LEVEL=6", "=C:=C:\\Users\\Shayan\\.julia\\packages\\PyCall\\ygXW2\\deps", "PROGRAMW6432=C:\\Program Files", "TEMP=C:\\Users\\Shayan\\AppData\\Local\\Temp", "HOMEDRIVE=C:", "PYTHONIOENCODING=UTF-8", "WINDIR=C:\\WINDOWS", "LOCALAPPDATA=C:\\Users\\Shayan\\AppData\\Local", "PROCESSOR_IDENTIFIER=Intel64 Family 6 Model 60 Stepping 3, GenuineIntel", "NUMBER_OF_PROCESSORS=8", "PROMPT=\$P\$G", "COMMONPROGRAMFILES(X86)=C:\\Program Files (x86)\\Common Files", "PY=C:\\Users\\Shayan\\AppData\\Roaming\\Python\\Python37\\Scripts", "COMMONPROGRAMFILES=C:\\Program Files\\Common Files", "ONEDRIVE=C:\\Users\\Shayan\\OneDrive", "WT_SESSION=9fee51fe-e622-4f2e-a641-4f8dfd742037", "PROGRAMFILES(X86)=C:\\Program Files (x86)", "PROGRAMFILES=C:\\Program Files", "=EXITCODE=00000001", "OMP_NUM_THREADS=2", "LOGONSERVER=\\\\SHAYANSNOTEBOOK", "DRIVERDATA=C:\\Windows\\System32\\Drivers\\DriverData", "CONDA_PREFIX=C:/Users/Shayan/Anaconda3/envs/3.11", "JULIA_LOAD_PATH=@;C:\\Users\\Shayan\\AppData\\Local\\Temp\\jl_FJel6n", "SYSTEMDRIVE=C:", "PROCESSOR_ARCHITECTURE=AMD64"]): no such file or directory (ENOENT)
Stacktrace:
  [1] _spawn_primitive(file::String, cmd::Cmd, stdio::Vector{Union{RawFD, Base.Libc.WindowsRawSocket, IO}})
    @ Base .\process.jl:128
  [2] #725
    @ .\process.jl:139 [inlined]
  [3] setup_stdios(f::Base.var"#725#726"{Cmd}, stdios::Vector{Union{RawFD, Base.Libc.WindowsRawSocket, IO}})
    @ Base .\process.jl:223
  [4] _spawn
    @ .\process.jl:138 [inlined]
  [5] run(::Cmd; wait::Bool)
    @ Base .\process.jl:479
  [6] run
    @ .\process.jl:477 [inlined]
  [7] runconda(args::Cmd, env::String)
    @ Conda C:\Users\Shayan\.julia\packages\Conda\x2UxR\src\Conda.jl:128
  [8] add(pkg::String, env::String; channel::String)
    @ Conda C:\Users\Shayan\.julia\packages\Conda\x2UxR\src\Conda.jl:222
  [9] add (repeats 2 times)
    @ C:\Users\Shayan\.julia\packages\Conda\x2UxR\src\Conda.jl:220 [inlined]
 [10] top-level scope
    @ C:\Users\Shayan\.julia\packages\PyCall\ygXW2\deps\build.jl:79
 [11] include(fname::String)
    @ Base.MainInclude .\client.jl:476
 [12] top-level scope
    @ none:5
in expression starting at C:\Users\Shayan\.julia\packages\PyCall\ygXW2\deps\build.jl:43

Full log at C:\Users\Shayan\.julia\scratchspaces\44cfe95a-1eb2-52ea-b672-e2afdf69b78f\53b8b07b721b77144a0fbbbc2675222ebf40a02d\build.log
Stacktrace:
  [1] pkgerror(msg::String)
    @ Pkg.Types C:\Users\Shayan\.julia\juliaup\julia-1.8.2+0.x64\share\julia\stdlib\v1.8\Pkg\src\Types.jl:67
  [2] (::Pkg.Operations.var"#66#73"{Bool, Pkg.Types.Context, String, Pkg.Types.PackageSpec, String})()
    @ Pkg.Operations C:\Users\Shayan\.julia\juliaup\julia-1.8.2+0.x64\share\julia\stdlib\v1.8\Pkg\src\Operations.jl:1060
  [3] withenv(::Pkg.Operations.var"#66#73"{Bool, Pkg.Types.Context, String, Pkg.Types.PackageSpec, String}, ::Pair{String, String}, ::Vararg{Pair{String}})
    @ Base .\env.jl:172
  [4] (::Pkg.Operations.var"#107#112"{String, Bool, Bool, Bool, Pkg.Operations.var"#66#73"{Bool, Pkg.Types.Context, String, Pkg.Types.PackageSpec, String}, Pkg.Types.PackageSpec})()
    @ Pkg.Operations C:\Users\Shayan\.julia\juliaup\julia-1.8.2+0.x64\share\julia\stdlib\v1.8\Pkg\src\Operations.jl:1619
  [5] with_temp_env(fn::Pkg.Operations.var"#107#112"{String, Bool, Bool, Bool, Pkg.Operations.var"#66#73"{Bool, Pkg.Types.Context, String, Pkg.Types.PackageSpec, String}, Pkg.Types.PackageSpec}, temp_env::String)
    @ Pkg.Operations C:\Users\Shayan\.julia\juliaup\julia-1.8.2+0.x64\share\julia\stdlib\v1.8\Pkg\src\Operations.jl:1493
  [6] (::Pkg.Operations.var"#105#110"{Dict{String, Any}, Bool, Bool, Bool, Pkg.Operations.var"#66#73"{Bool, Pkg.Types.Context, String, Pkg.Types.PackageSpec, String}, Pkg.Types.Context, Pkg.Types.PackageSpec, String, Pkg.Types.Project, String})(tmp::String)
    @ Pkg.Operations C:\Users\Shayan\.julia\juliaup\julia-1.8.2+0.x64\share\julia\stdlib\v1.8\Pkg\src\Operations.jl:1582
  [7] mktempdir(fn::Pkg.Operations.var"#105#110"{Dict{String, Any}, Bool, Bool, Bool, Pkg.Operations.var"#66#73"{Bool, Pkg.Types.Context, String, Pkg.Types.PackageSpec, String}, Pkg.Types.Context, Pkg.Types.PackageSpec, String, Pkg.Types.Project, String}, parent::String; prefix::String)
    @ Base.Filesystem .\file.jl:764
  [8] mktempdir(fn::Function, parent::String) (repeats 2 times)
    @ Base.Filesystem .\file.jl:760
  [9] sandbox(fn::Function, ctx::Pkg.Types.Context, target::Pkg.Types.PackageSpec, target_path::String, sandbox_path::String, sandbox_project_override::Pkg.Types.Project; preferences::Dict{String, Any}, force_latest_compatible_version::Bool, allow_earlier_backwards_compatible_versions::Bool, allow_reresolve::Bool)
    @ Pkg.Operations C:\Users\Shayan\.julia\juliaup\julia-1.8.2+0.x64\share\julia\stdlib\v1.8\Pkg\src\Operations.jl:1540
 [10] build_versions(ctx::Pkg.Types.Context, uuids::Set{Base.UUID}; verbose::Bool)
    @ Pkg.Operations C:\Users\Shayan\.julia\juliaup\julia-1.8.2+0.x64\share\julia\stdlib\v1.8\Pkg\src\Operations.jl:1041
 [11] build(ctx::Pkg.Types.Context, uuids::Set{Base.UUID}, verbose::Bool)
    @ Pkg.Operations C:\Users\Shayan\.julia\juliaup\julia-1.8.2+0.x64\share\julia\stdlib\v1.8\Pkg\src\Operations.jl:898
 [12] build(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec}; verbose::Bool, kwargs::Base.Pairs{Symbol, Base.TTY, Tuple{Symbol}, NamedTuple{(:io,), Tuple{Base.TTY}}})
    @ Pkg.API C:\Users\Shayan\.julia\juliaup\julia-1.8.2+0.x64\share\julia\stdlib\v1.8\Pkg\src\API.jl:1029
 [13] build(pkgs::Vector{Pkg.Types.PackageSpec}; io::Base.TTY, kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ Pkg.API C:\Users\Shayan\.julia\juliaup\julia-1.8.2+0.x64\share\julia\stdlib\v1.8\Pkg\src\API.jl:156
 [14] build(pkgs::Vector{Pkg.Types.PackageSpec})
    @ Pkg.API C:\Users\Shayan\.julia\juliaup\julia-1.8.2+0.x64\share\julia\stdlib\v1.8\Pkg\src\API.jl:145
 [15] #build#99
    @ C:\Users\Shayan\.julia\juliaup\julia-1.8.2+0.x64\share\julia\stdlib\v1.8\Pkg\src\API.jl:144 [inlined]
 [16] build
    @ C:\Users\Shayan\.julia\juliaup\julia-1.8.2+0.x64\share\julia\stdlib\v1.8\Pkg\src\API.jl:144 [inlined]
 [17] #build#98
    @ C:\Users\Shayan\.julia\juliaup\julia-1.8.2+0.x64\share\julia\stdlib\v1.8\Pkg\src\API.jl:143 [inlined]
 [18] build(pkg::String)
    @ Pkg.API C:\Users\Shayan\.julia\juliaup\julia-1.8.2+0.x64\share\julia\stdlib\v1.8\Pkg\src\API.jl:143
 [19] top-level scope
    @ REPL[10]:1

Then I tried it on a standalone Python installation:

julia> ENV["PYTHON"] = raw"C:\Users\Shayan\AppData\Local\Programs\Python\Python311\python.exe"
"C:\\Users\\Shayan\\AppData\\Local\\Programs\\Python\\Python311\\python.exe"

julia> Pkg.build("PyCall")
    Building Conda ─→ `C:\Users\Shayan\.julia\scratchspaces\44cfe95a-1eb2-52ea-b672-e2afdf69b78f\6e47d11ea2776bc5627421d59cdcc1296c058071\build.log`
    Building PyCall β†’ `C:\Users\Shayan\.julia\scratchspaces\44cfe95a-1eb2-52ea-b672-e2afdf69b78f\53b8b07b721b77144a0fbbbc2675222ebf40a02d\build.log`
Precompiling project...
  βœ“ PyCall
  3 dependencies successfully precompiled in 17 seconds. 476 already precompiled. 1 skipped during auto due to previous errors.
  1 dependency precompiled but a different version is currently loaded. Restart julia to access the new version

Yes, it should. I installed it but didn’t confirm/test performance.

julia> using PythonCall  # Note 3.11 or newer may not be the default read the docs, but it's easy to get with this package.

julia> sys = pyimport("sys")
Python module: <module 'sys' (built-in)>

julia> sys.version
Python str: '3.11.0 | packaged by conda-forge | (main, Oct 25 2022, 06:47:48) [GCC 10.4.0]'

julia> ENV["PYTHON"] = PythonCall.C.CTX.exe_path

julia> import Pkg; Pkg.build("PyCall")

You can use PyCall with PythonCall (or either will do), but you may need to do the above (i.e. on Linux) for both (also other ways possible).

I did get the package but:

(@v1.8) pkg> conda add scikit-learn

  + scikit-learn      1.1.1  py310hffb9edd_0      conda-forge/linux-64        9MB
[..]
  Downgrade:
───────────────────────────────────────────────────────────────────────────────────

  - python           3.11.0  ha86cf86_0_cpython   conda-forge              Cached
  + python           3.10.6  ha86cf86_0_cpython   conda-forge/linux-64       30MB

I’m not sure why, there’s a newer scikit-learn available, but maybe not in Conda, or it depends on the older Python version?

There are also other ways to install (python and its) packages, e.g. pip.

You would be able to use all of skikit-learn’s Python API with PythonCall, but if you rather need the Julia wrapper ScikitLearn.jl for the Julia API, then you need to use PyCall.

1 Like