Ayush1999's Keras.jl on Julia 1.3.1 precompiling failure

I see that the package is a bit stale and has not been upgraded to work on Julia 1, however I was able to install it seamlessly in Julia 1.3.1 by executing Pkg.add(Pkg.PackageSpec(url="https://github.com/ayush1999/Keras.jl", rev="master")). Executing using Keras returned errors I wasn’t able to fix. The full output is below. Any suggestion is greatly appreciated.

[ Info: Precompiling Keras [031d7243-eb6a-5793-a290-e3c72503329c]
WARNING: importing deprecated binding Compat.Libdl into Blosc.
WARNING: importing deprecated binding Compat.Libdl into Blosc.
WARNING: Method definition iterate(DataFlow.ObjectArraySet{T} where T, Any...) in module DataFlow at /home/adude/.julia/packages/Lazy/mAoZN/src/macros.jl:285 overwritten at /home/adude/.julia/packages/Lazy/mAoZN/src/macros.jl:285.
  ** incremental compilation may be fatally broken for this module **

ERROR: LoadError: LoadError: LoadError: syntax: "..." expression outside call
Stacktrace:
 [1] top-level scope at /home/adude/.julia/packages/Keras/IoYuA/src/ops.jl:13
 [2] include at ./boot.jl:328 [inlined]
 [3] include_relative(::Module, ::String) at ./loading.jl:1105
 [4] include at ./Base.jl:31 [inlined]
 [5] include(::String) at /home/adude/.julia/packages/Keras/IoYuA/src/Keras.jl:1
 [6] top-level scope at /home/adude/.julia/packages/Keras/IoYuA/src/graph.jl:2
 [7] include at ./boot.jl:328 [inlined]
 [8] include_relative(::Module, ::String) at ./loading.jl:1105
 [9] include at ./Base.jl:31 [inlined]
 [10] include(::String) at /home/adude/.julia/packages/Keras/IoYuA/src/Keras.jl:1
 [11] top-level scope at /home/adude/.julia/packages/Keras/IoYuA/src/Keras.jl:4
 [12] include at ./boot.jl:328 [inlined]
 [13] include_relative(::Module, ::String) at ./loading.jl:1105
 [14] include(::Module, ::String) at ./Base.jl:31
 [15] top-level scope at none:2
 [16] eval at ./boot.jl:330 [inlined]
 [17] eval(::Expr) at ./client.jl:425
 [18] top-level scope at ./none:3
in expression starting at /home/adude/.julia/packages/Keras/IoYuA/src/ops.jl:13
in expression starting at /home/adude/.julia/packages/Keras/IoYuA/src/graph.jl:2
in expression starting at /home/adude/.julia/packages/Keras/IoYuA/src/Keras.jl:4
ERROR: Failed to precompile Keras [031d7243-eb6a-5793-a290-e3c72503329c] to /home/adude/.julia/compiled/v1.3/Keras/RBgJ9_OVTEM.ji.
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1283
 [3] _require(::Base.PkgId) at ./loading.jl:1024
 [4] require(::Base.PkgId) at ./loading.jl:922
 [5] require(::Module, ::Symbol) at ./loading.jl:917

Have you tried https://github.com/invenia/Keras.jl instead?

Not yet. I saw it’s based on a older version of both TensorFlow and Keras. Have you been using it recently?

No but it says it was updated for Julia 1.0 and it looks to be more actively maintained.

I’ll give it a try and post the results. Thank you for suggesting it!

Those are two unrelated packages.
https://github.com/invenia/Keras.jl is a PyCall based wrapper of Keras
https://github.com/ayush1999/Keras.jl is a way of loading keras serialized json models as Flux models.

I would not use either.
I would just use Flux

2 Likes

I fear @oxinabox is terribly right. I just finished to try the Keras wrapper (suggested by @tbeason) and I got the following nonsense…

Error: Error building `Keras`, showing the last 100 of log:

β”‚ This can take several minutes. Press CTRL-C to abort.

β”‚ failed

β”‚

β”‚ UnsatisfiableError: The following specifications were found

β”‚ to be incompatible with the existing python installation in your environment:

β”‚

β”‚ Specifications:

β”‚

β”‚ - keras==2.0.2 -> python[version='2.7.*|3.5.*|3.6.*']

β”‚ - keras==2.0.2 -> python[version='<3']

β”‚ - keras==2.0.2 -> python[version='>=2.7,<2.8.0a0|>=3.5,<3.6.0a0|>=3.6,<3.7.0a0']

β”‚

β”‚ Your python: python=3.7

β”‚

β”‚ If python is on the left-most side of the chain, that's the version you've asked for.

β”‚ When python appears to the right, that indicates that the thing on the left is somehow

β”‚ not available for the python version you are constrained to. Note that conda will not

β”‚ change your python version to a different minor version unless you explicitly specify

β”‚ that.

β”‚

β”‚ The following specifications were found to be incompatible with each other:

β”‚

β”‚

β”‚

β”‚ Package certifi conflicts for:

β”‚ python=3.7 -> pip -> setuptools -> certifi[version='>=2016.09|>=2016.9.26']

β”‚ Package setuptools conflicts for:

β”‚ keras==2.0.2 -> setuptools

β”‚ python=3.7 -> pip -> setuptools

β”‚ Package pip conflicts for:

β”‚ keras==2.0.2 -> python=2.7 -> pip

β”‚ python=3.7 -> pip

β”‚ Package ca-certificates conflicts for:

β”‚ python=3.7 -> openssl[version='>=1.1.1a,<1.1.2a'] -> ca-certificates

β”‚ keras==2.0.2 -> setuptools -> ca-certificates

β”‚ Package wheel conflicts for:

β”‚ python=3.7 -> pip -> wheel

β”‚ keras==2.0.2 -> python=2.7 -> pip -> wheel

β”‚

β”‚ [ Info: No valid (>=2.0.2) install of keras found.

β”‚ [ Info: Installing Keras via Conda.jl...

β”‚ [ Info: Downloading miniconda installer ...

β”‚ [ Info: Installing miniconda ...

β”‚ [ Info: Running `conda config --add channels defaults --file /home/adude/.julia/conda/3/condarc-julia.yml --force` in root environment

β”‚ [ Info: Running `conda update -y conda` in root environment

β”‚ [ Info: Running `conda config --add channels conda-forge --file /home/adude/.julia/conda/3/condarc-julia.yml --force` in root environment

β”‚ [ Info: Running `conda install -y keras==2.0.2` in root environment

β”‚ ERROR: LoadError: failed process: Process(setenv(`/home/adude/.julia/conda/3/bin/conda install -y keras==2.0.2`,["_CE_M=", "PATH=/home/adude/anaconda3/condabin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/adude/julia/julia-1.3.1/bin:/home/adude/.local/bin:/home/adude/bin", "PWD=/home/adude", "HISTSIZE=1000", "LESSOPEN=||/usr/bin/lesspipe.sh %s", "SHELL=/bin/bash", "SSH_CONNECTION=10.182.71.193 65510 10.63.36.22 22", "OPENBLAS_NUM_THREADS=8", "SELINUX_USE_CURRENT_RANGE=", "CONDARC=/home/adude/.julia/conda/3/condarc-julia.yml" … "_CE_CONDA=", "USER=adude", "SSH_TTY=/dev/pts/0", "SELINUX_LEVEL_REQUESTED=", "TERM=xterm", "HOME=/home/adude", "LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=01;05;37;41:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=01;36:*.au=01;36:*.flac=01;36:*.mid=01;36:*.midi=01;36:*.mka=01;36:*.mp3=01;36:*.mpc=01;36:*.ogg=01;36:*.ra=01;36:*.wav=01;36:*.axa=01;36:*.oga=01;36:*.spx=01;36:*.xspf=01;36:", "HOSTNAME=IRIAZLNXMdR105d", "OPENBLAS_MAIN_FREE=1", "PYTHONIOENCODING=UTF-8"]), ProcessExited(1)) [1]

β”‚

β”‚ Stacktrace:

β”‚ [1] pipeline_error at ./process.jl:525 [inlined]

β”‚ [2] #run#565(::Bool, ::typeof(run), ::Cmd) at ./process.jl:440

β”‚ [3] run at ./process.jl:438 [inlined]

β”‚ [4] runconda(::Cmd, ::String) at /home/adude/.julia/packages/Conda/kLXeC/src/Conda.jl:113

β”‚ [5] #add#1(::String, ::typeof(Conda.add), ::String, ::String) at /home/adude/.julia/packages/Conda/kLXeC/src/Conda.jl:184

β”‚ [6] add at /home/adude/.julia/packages/Conda/kLXeC/src/Conda.jl:183 [inlined] (repeats 2 times)

β”‚ [7] top-level scope at /home/adude/.julia/packages/Keras/LfJdf/deps/build.jl:19

β”‚ [8] include at ./boot.jl:328 [inlined]

β”‚ [9] include_relative(::Module, ::String) at ./loading.jl:1105

β”‚ [10] include(::Module, ::String) at ./Base.jl:31

β”‚ [11] include(::String) at ./client.jl:424

β”‚ [12] top-level scope at none:5

β”‚ in expression starting at /home/adude/.julia/packages/Keras/LfJdf/deps/build.jl:7

β”‚ caused by [exception 1]

β”‚ PyError (PyImport_ImportModule

β”‚

β”‚ The Python package keras could not be found by pyimport. Usually this means

β”‚ that you did not install keras in the Python version being used by PyCall.

β”‚

β”‚ PyCall is currently configured to use the Python version at:

β”‚

β”‚ /home/adude/anaconda3/bin/python3

β”‚

β”‚ and you should use whatever mechanism you usually use (apt-get, pip, conda,

β”‚ etcetera) to install the Python package containing the keras module.

β”‚

β”‚ One alternative is to re-configure PyCall to use a different Python

β”‚ version on your system: set ENV["PYTHON"] to the path/name of the python

β”‚ executable you want to use, run Pkg.build("PyCall"), and re-launch Julia.

β”‚

β”‚ Another alternative is to configure PyCall to use a Julia-specific Python

β”‚ distribution via the Conda.jl package (which installs a private Anaconda

β”‚ Python distribution), which has the advantage that packages can be installed

β”‚ and kept up-to-date via Julia. As explained in the PyCall documentation,

β”‚ set ENV["PYTHON"]="", run Pkg.build("PyCall"), and re-launch Julia. Then,

β”‚ To install the keras module, you can use `pyimport_conda("keras", PKG)`,

β”‚ where PKG is the Anaconda package the contains the module keras,

β”‚ or alternatively you can use the Conda package directly (via

β”‚ `using Conda` followed by `Conda.add` etcetera).

β”‚

β”‚ ) <class 'ModuleNotFoundError'>

β”‚ ModuleNotFoundError("No module named 'keras'")

β”‚

β”‚ Stacktrace:

β”‚ [1] pyimport(::String) at /home/adude/.julia/packages/PyCall/ttONZ/src/PyCall.jl:544

β”‚ [2] top-level scope at /home/adude/.julia/packages/Keras/LfJdf/deps/build.jl:8

β”‚ [3] include at ./boot.jl:328 [inlined]

β”‚ [4] include_relative(::Module, ::String) at ./loading.jl:1105

β”‚ [5] include(::Module, ::String) at ./Base.jl:31

β”‚ [6] include(::String) at ./client.jl:424

β”‚ [7] top-level scope at none:5

β”‚

β”‚ Full log at /home/adude/.julia/packages/Keras/LfJdf/deps/build.log

β”” @ Pkg.Operations /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Pkg/src/backwards_compatible_isolation.jl:649

I would avoid using packages that you can’t install via:
] add PackageName

Since either it isn’t registered, and thus has never been considered complete enough to make a release of (like Invenia/Keras.jl).

Or its not compatible with your julia version, like ayush1999/Keras.jl

Invenia runs Flux in production; and does not use Invenia/Keras.jl for anything these days.

Thanks Lyndon for getting back to me.

Do you know of any working example of importing Python TensorFlow pre-trained bidirectional LSTM in Julia?

no, I have never wanted to import python stuff into julia.
Not a thing I have ever had cause to do, so I’ve never really looked into it.

I would just implement one in Flux directly then train it.

Thanks again for your answers. Can you guys please point me to some good resource on implementing a bidirectional LSTM in Flux?