WORKDIR /data/code/Olorin/
ENV JULIA_PROJECT="/data/code/Olorin/"
RUN cd /data/code/Olorin/ && \
julia --project="/data/code/Olorin" -e 'using Pkg; Pkg.instantiate()' > /tmp/pkg_instantiate.log 2>&1
RUN ["julia", "-e", "using Pkg; Pkg.instantiate()"]
it doesn’t install packages. the log file has the below lines. If I logon to the server manually and check, I can see that it has activated the project, but not downloaded/installed any packages
Installing known registries into `~/.julia`
No Changes to `/data/code/Olorin/Project.toml`
No Changes to `/data/code/Olorin/Manifest.toml`
if, I however, run the below command manually or as part of shell file:
cd /data/code/Olorin/
julia -e 'using Pkg; Pkg.instantiate()'
I can see it installs all packages. I am very confused! can someone help with this
It would help us find the error if you included the entire dockerfile, not just a snippet. It should work as you described, I have a bunch of similarly working dockerfiles, too.
thanks for the response @HanD. Here you go. I have copied both the docker compose and the DockerFile. you will notice that I have tried different ways to install Packages, none seem to work. however, when I attach shell to the container, I can run the same commands successfully. I am sure I am doing something silly.
Stupid reply from me. Back in the mists of time I debugged a problem with the Gridengine batch system - nothing related to Docker.
There are differences between interactive and non-interactive shells.
Clearly when you shell into a Docker container you get an interactive shell.
When you are running the commands is there a difference in the shell environment?
The only diagnostic I can think is running the command ‘env’ in the dockerfile and via the shell. Then compare the results minutely - vimdiff is good for this.
I see where the problem is. Your Dockerfile doesn’t contain the Julia project code. When you build your docker image, there is no project, no Project.toml, no Manifest.toml, so no pakcages are installed. However, when you run your container (via docker compose), the project root (/data/code/Olorin/) is mounted from a volume, so it will contain whatever you copied in that volume. That’s why when you log in to the service, your project files are already there, and you can install packages from Julia as expected.
If you want your image to contain preinstalled and precompiled dependencies, you need to copy at least the Project.toml file into the image at build time, in your Dockerfile, via COPY.
HTH.
Also, I recommend starting with a julia base image, rather than a generic ubuntu image. E.g., julia:1.9-alpine or julia:1.9-bookworm, if you perfer debian and apt. Ubuntu based images are most probably an overkill for this purpose.
Share all the details about the runtime environment.
Temporarily try to modify the Dockerfile so that it downloads the test code using git clone. This is to filter out any potential issues between the filesystem and Docker.
@HanD : thanks a lot for the response and suggestions. you were absolutely right, the issue is indeed as you described and the resolution worked like a charm.
for everyone else, the below works:
DockerFile:
Use a base image with a Linux distribution
FROM ubuntu:latest
SHELL [“/bin/bash”, “-c”]
COPY install/docker-entrypoint.sh /data/deploy/
RUN apt-get update &&
apt-get install -y wget &&
apt-get install unzip
RUN apt-get update && apt-get -y install libgconf-2-4
RUN apt --fix-broken install
RUN echo “****************************** INSTALLING MINICONDA ******************************”
WORKDIR /apps/third-party
ENV PATH=“/apps/third-party/miniconda3/bin:{PATH}"
ARG PATH="/apps/third-party/miniconda3/bin:{PATH}”
RUN wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-aarch64.sh
&& bash Miniconda3-latest-Linux-aarch64.sh -b -u -p miniconda3
&& rm Miniconda3-latest-Linux-aarch64.sh
&& miniconda3/bin/conda init bash
&& source /root/.bashrc
RUN conda --version
RUN mkdir -p /data/deploy
COPY install/requirements.txt /data/deploy/
RUN pip install -r /data/deploy/requirements.txt
RUN echo “****************************** DONE INSTALLING MINICONDA ******************************”
RUN mkdir -p /root/.julia/config
COPY install/startup.jl /root/.julia/config/
ENV PATH=$PATH:/apps/third-party/julia-1.9.3/bin/
Install Julia project dependencies and activate the environment
WORKDIR /data/code/Olorin/
ARG CURRENT_ENV
RUN echo “Activating environment: $CURRENT_ENV”
COPY deploy/$CURRENT_ENV/Olorin/Manifest.toml /data/code/Olorin/
COPY deploy/$CURRENT_ENV/Olorin/Project.toml /data/code/Olorin/
RUN julia -e ‘using Pkg; Pkg.instantiate()’
RUN julia -e ‘ENV[“PYTHON”] = “/apps/third-party/miniconda3/bin/python”; using Pkg; Pkg.build(“PyCall”)’
RUN julia -e “using Pkg; Pkg.status()”
RUN julia -e ‘ENV[“PYTHON”] = “/apps/third-party/miniconda3/bin/python”; using Pkg; Pkg.build(“PyCall”)’
RUN echo “****************************** DONE INSTALLING JULIA ******************************”