Singularity, ERROR: Unable to find compatible target in system image

I’m trying to compile a Singularity container for running on a HPC cluster.

My container works on my laptop (where I build it), but when I transfer it over to the cluster, I get a:

ERROR: Unable to find compatible target in system image.

Does someone know what the build instructions are to make this work. I’ve tried

I’ve tried make --cpu_target=generic, but that does not seem to be a recognized option?

thanks
(from the singularity def build file)

        git clone https://github.com/JuliaLang/julia.git
        cd /julia
        git checkout v1.5.4
        ## from ERROR: Unable to find compatible target in system image.
        make -j 8 --cpu-target=generic 
        mkdir /opt/julia 
        export JULIA_DEPOT_PATH=/opt/julia
        /julia/julia -e 'using Pkg ; Pkg.add("SpecialFunctions")'
        /julia/julia -e 'using Pkg ; Pkg.add("BSON")'
        /julia/julia -e 'using Pkg ; Pkg.add("LinearAlgebra")'
        /julia/julia -e 'using Pkg ; Pkg.add("Printf")'
        /julia/julia -e 'using SpecialFunctions; using BSON ; using LinearAlgebra ; using Printf'

Solved… this seems to fix the Singularity build, Julia runs inside Singularity on the HPC cluster.

export JULIA_CPU_TARGET='generic;sandybridge,-xsaveopt,clone_all;haswell,-rdrnd,base(1)'

from https://docs.julialang.org/en/v1/devdocs/sysimg/

Next question, Is there a better way to build the packages I need? (I.e some magic with PackageCompiler?).

        ## from Singularity def build file
        ## build Julia from github source
        git clone https://github.com/JuliaLang/julia.git
        cd /julia
        git checkout v1.5.4
        ## from ERROR: Unable to find compatible target in system image. 
        ## this seams to solve it
        export JULIA_CPU_TARGET='generic;sandybridge,-xsaveopt,clone_all;haswell,-rdrnd,base(1)'
        make -j 8
        mkdir /opt/julia
        export JULIA_DEPOT_PATH=/opt/julia
        /julia/julia -e 'using Pkg ; Pkg.add("SpecialFunctions")'
        /julia/julia -e 'using Pkg ; Pkg.add("BSON")'
        /julia/julia -e 'using Pkg ; Pkg.add("LinearAlgebra")'
        /julia/julia -e 'using Pkg ; Pkg.add("Printf")'
        /julia/julia -e 'using SpecialFunctions; using BSON ; using LinearAlgebra ; using Printf'

You may be interested in GitHub - bcbi/SimpleContainerGenerator.jl: Create container images for using Julia packages (especially useful in environments without Internet access)

It generates Docker images, but it is very easy to convert Docker images to Singularity images. I routinely take the Docker images generated by SimpleContainerGenerator.jl, convert them to Singularity images, and run them on HPC clusters.

That looks awesome! Thanks! (strange that didn’t come up in my google searches.) I’m glad I struggled to burn my own from scratch (good learning experience), but once is plenty…

I have access to an academic HPC cluster, but not for long. Do you have any experience running Julia/Singularity containers in the google Cloud, AWS, or other commercial cloud vendors? Any tips would be greatly appreciated. Now that I can actually get my code running, I’m curious the best way to get results back from the cloud (database, git repos, etc).

thanks again