Ayush1999's Keras.jl on Julia 1.3.1 precompiling failure

Specific Domains Machine Learning
1

I see that the package is a bit stale and has not been upgraded to work on Julia 1, however I was able to install it seamlessly in Julia 1.3.1 by executing Pkg.add(Pkg.PackageSpec(url="https://github.com/ayush1999/Keras.jl", rev="master")). Executing using Keras returned errors I wasn’t able to fix. The full output is below. Any suggestion is greatly appreciated.

[ Info: Precompiling Keras [031d7243-eb6a-5793-a290-e3c72503329c]
WARNING: importing deprecated binding Compat.Libdl into Blosc.
WARNING: importing deprecated binding Compat.Libdl into Blosc.
WARNING: Method definition iterate(DataFlow.ObjectArraySet{T} where T, Any...) in module DataFlow at /home/adude/.julia/packages/Lazy/mAoZN/src/macros.jl:285 overwritten at /home/adude/.julia/packages/Lazy/mAoZN/src/macros.jl:285.
  ** incremental compilation may be fatally broken for this module **

ERROR: LoadError: LoadError: LoadError: syntax: "..." expression outside call
Stacktrace:
 [1] top-level scope at /home/adude/.julia/packages/Keras/IoYuA/src/ops.jl:13
 [2] include at ./boot.jl:328 [inlined]
 [3] include_relative(::Module, ::String) at ./loading.jl:1105
 [4] include at ./Base.jl:31 [inlined]
 [5] include(::String) at /home/adude/.julia/packages/Keras/IoYuA/src/Keras.jl:1
 [6] top-level scope at /home/adude/.julia/packages/Keras/IoYuA/src/graph.jl:2
 [7] include at ./boot.jl:328 [inlined]
 [8] include_relative(::Module, ::String) at ./loading.jl:1105
 [9] include at ./Base.jl:31 [inlined]
 [10] include(::String) at /home/adude/.julia/packages/Keras/IoYuA/src/Keras.jl:1
 [11] top-level scope at /home/adude/.julia/packages/Keras/IoYuA/src/Keras.jl:4
 [12] include at ./boot.jl:328 [inlined]
 [13] include_relative(::Module, ::String) at ./loading.jl:1105
 [14] include(::Module, ::String) at ./Base.jl:31
 [15] top-level scope at none:2
 [16] eval at ./boot.jl:330 [inlined]
 [17] eval(::Expr) at ./client.jl:425
 [18] top-level scope at ./none:3
in expression starting at /home/adude/.julia/packages/Keras/IoYuA/src/ops.jl:13
in expression starting at /home/adude/.julia/packages/Keras/IoYuA/src/graph.jl:2
in expression starting at /home/adude/.julia/packages/Keras/IoYuA/src/Keras.jl:4
ERROR: Failed to precompile Keras [031d7243-eb6a-5793-a290-e3c72503329c] to /home/adude/.julia/compiled/v1.3/Keras/RBgJ9_OVTEM.ji.
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1283
 [3] _require(::Base.PkgId) at ./loading.jl:1024
 [4] require(::Base.PkgId) at ./loading.jl:922
 [5] require(::Module, ::Symbol) at ./loading.jl:917
2

Have you tried GitHub - invenia/Keras.jl: A julia wrapper for https://keras.io instead?

3

Not yet. I saw it’s based on a older version of both TensorFlow and Keras. Have you been using it recently?

4

No but it says it was updated for Julia 1.0 and it looks to be more actively maintained.

5

I’ll give it a try and post the results. Thank you for suggesting it!

6

Those are two unrelated packages.
GitHub - invenia/Keras.jl: A julia wrapper for https://keras.io is a PyCall based wrapper of Keras
GitHub - ayush1999/Keras.jl: Run keras models with a Flux backend is a way of loading keras serialized json models as Flux models.

I would not use either.
I would just use Flux

2 Likes
7

I fear @oxinabox is terribly right. I just finished to try the Keras wrapper (suggested by @tbeason) and I got the following nonsense…

Error: Error building `Keras`, showing the last 100 of log:

│ This can take several minutes. Press CTRL-C to abort.

│ failed

│

│ UnsatisfiableError: The following specifications were found

│ to be incompatible with the existing python installation in your environment:

│

│ Specifications:

│

│ - keras==2.0.2 -> python[version='2.7.*|3.5.*|3.6.*']

│ - keras==2.0.2 -> python[version='<3']

│ - keras==2.0.2 -> python[version='>=2.7,<2.8.0a0|>=3.5,<3.6.0a0|>=3.6,<3.7.0a0']

│

│ Your python: python=3.7

│

│ If python is on the left-most side of the chain, that's the version you've asked for.

│ When python appears to the right, that indicates that the thing on the left is somehow

│ not available for the python version you are constrained to. Note that conda will not

│ change your python version to a different minor version unless you explicitly specify

│ that.

│

│ The following specifications were found to be incompatible with each other:

│

│

│

│ Package certifi conflicts for:

│ python=3.7 -> pip -> setuptools -> certifi[version='>=2016.09|>=2016.9.26']

│ Package setuptools conflicts for:

│ keras==2.0.2 -> setuptools

│ python=3.7 -> pip -> setuptools

│ Package pip conflicts for:

│ keras==2.0.2 -> python=2.7 -> pip

│ python=3.7 -> pip

│ Package ca-certificates conflicts for:

│ python=3.7 -> openssl[version='>=1.1.1a,<1.1.2a'] -> ca-certificates

│ keras==2.0.2 -> setuptools -> ca-certificates

│ Package wheel conflicts for:

│ python=3.7 -> pip -> wheel

│ keras==2.0.2 -> python=2.7 -> pip -> wheel

│

│ [ Info: No valid (>=2.0.2) install of keras found.

│ [ Info: Installing Keras via Conda.jl...

│ [ Info: Downloading miniconda installer ...

│ [ Info: Installing miniconda ...

│ [ Info: Running `conda config --add channels defaults --file /home/adude/.julia/conda/3/condarc-julia.yml --force` in root environment

│ [ Info: Running `conda update -y conda` in root environment

│ [ Info: Running `conda config --add channels conda-forge --file /home/adude/.julia/conda/3/condarc-julia.yml --force` in root environment

│ [ Info: Running `conda install -y keras==2.0.2` in root environment

│ ERROR: LoadError: failed process: Process(setenv(`/home/adude/.julia/conda/3/bin/conda install -y keras==2.0.2`,["_CE_M=", "PATH=/home/adude/anaconda3/condabin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/adude/julia/julia-1.3.1/bin:/home/adude/.local/bin:/home/adude/bin", "PWD=/home/adude", "HISTSIZE=1000", "LESSOPEN=||/usr/bin/lesspipe.sh %s", "SHELL=/bin/bash", "SSH_CONNECTION=10.182.71.193 65510 10.63.36.22 22", "OPENBLAS_NUM_THREADS=8", "SELINUX_USE_CURRENT_RANGE=", "CONDARC=/home/adude/.julia/conda/3/condarc-julia.yml" … "_CE_CONDA=", "USER=adude", "SSH_TTY=/dev/pts/0", "SELINUX_LEVEL_REQUESTED=", "TERM=xterm", "HOME=/home/adude", "LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=01;05;37;41:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=01;36:*.au=01;36:*.flac=01;36:*.mid=01;36:*.midi=01;36:*.mka=01;36:*.mp3=01;36:*.mpc=01;36:*.ogg=01;36:*.ra=01;36:*.wav=01;36:*.axa=01;36:*.oga=01;36:*.spx=01;36:*.xspf=01;36:", "HOSTNAME=IRIAZLNXMdR105d", "OPENBLAS_MAIN_FREE=1", "PYTHONIOENCODING=UTF-8"]), ProcessExited(1)) [1]

│

│ Stacktrace:

│ [1] pipeline_error at ./process.jl:525 [inlined]

│ [2] #run#565(::Bool, ::typeof(run), ::Cmd) at ./process.jl:440

│ [3] run at ./process.jl:438 [inlined]

│ [4] runconda(::Cmd, ::String) at /home/adude/.julia/packages/Conda/kLXeC/src/Conda.jl:113

│ [5] #add#1(::String, ::typeof(Conda.add), ::String, ::String) at /home/adude/.julia/packages/Conda/kLXeC/src/Conda.jl:184

│ [6] add at /home/adude/.julia/packages/Conda/kLXeC/src/Conda.jl:183 [inlined] (repeats 2 times)

│ [7] top-level scope at /home/adude/.julia/packages/Keras/LfJdf/deps/build.jl:19

│ [8] include at ./boot.jl:328 [inlined]

│ [9] include_relative(::Module, ::String) at ./loading.jl:1105

│ [10] include(::Module, ::String) at ./Base.jl:31

│ [11] include(::String) at ./client.jl:424

│ [12] top-level scope at none:5

│ in expression starting at /home/adude/.julia/packages/Keras/LfJdf/deps/build.jl:7

│ caused by [exception 1]

│ PyError (PyImport_ImportModule

│

│ The Python package keras could not be found by pyimport. Usually this means

│ that you did not install keras in the Python version being used by PyCall.

│

│ PyCall is currently configured to use the Python version at:

│

│ /home/adude/anaconda3/bin/python3

│

│ and you should use whatever mechanism you usually use (apt-get, pip, conda,

│ etcetera) to install the Python package containing the keras module.

│

│ One alternative is to re-configure PyCall to use a different Python

│ version on your system: set ENV["PYTHON"] to the path/name of the python

│ executable you want to use, run Pkg.build("PyCall"), and re-launch Julia.

│

│ Another alternative is to configure PyCall to use a Julia-specific Python

│ distribution via the Conda.jl package (which installs a private Anaconda

│ Python distribution), which has the advantage that packages can be installed

│ and kept up-to-date via Julia. As explained in the PyCall documentation,

│ set ENV["PYTHON"]="", run Pkg.build("PyCall"), and re-launch Julia. Then,

│ To install the keras module, you can use `pyimport_conda("keras", PKG)`,

│ where PKG is the Anaconda package the contains the module keras,

│ or alternatively you can use the Conda package directly (via

│ `using Conda` followed by `Conda.add` etcetera).

│

│ ) <class 'ModuleNotFoundError'>

│ ModuleNotFoundError("No module named 'keras'")

│

│ Stacktrace:

│ [1] pyimport(::String) at /home/adude/.julia/packages/PyCall/ttONZ/src/PyCall.jl:544

│ [2] top-level scope at /home/adude/.julia/packages/Keras/LfJdf/deps/build.jl:8

│ [3] include at ./boot.jl:328 [inlined]

│ [4] include_relative(::Module, ::String) at ./loading.jl:1105

│ [5] include(::Module, ::String) at ./Base.jl:31

│ [6] include(::String) at ./client.jl:424

│ [7] top-level scope at none:5

│

│ Full log at /home/adude/.julia/packages/Keras/LfJdf/deps/build.log

└ @ Pkg.Operations /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Pkg/src/backwards_compatible_isolation.jl:649
8

I would avoid using packages that you can’t install via:
] add PackageName

Since either it isn’t registered, and thus has never been considered complete enough to make a release of (like Invenia/Keras.jl).

Or its not compatible with your julia version, like ayush1999/Keras.jl

Invenia runs Flux in production; and does not use Invenia/Keras.jl for anything these days.

9

Thanks Lyndon for getting back to me.

Do you know of any working example of importing Python TensorFlow pre-trained bidirectional LSTM in Julia?

10

no, I have never wanted to import python stuff into julia.
Not a thing I have ever had cause to do, so I’ve never really looked into it.

I would just implement one in Flux directly then train it.

11

Thanks again for your answers. Can you guys please point me to some good resource on implementing a bidirectional LSTM in Flux?