Run Julia from Julia

General Usage
1

It would be comfortable if users could use a few of my packages like this:

mkdir demo
cd demo
julia --project="."
using Pkg
pkg"add MyPackage"
using MyPackage
install()
compile()

Then restart Julia, and then the freshly compiled package image is used and the examples are available.

The install() function would create the folders data and examples and copy some example input data and some example scripts.

The compile() function would launch a bash script that creates a system image and installs the packages needed to run the examples. The script could be similar to this one:

update=false
if [[ $# -gt 0 ]]; then
    if [[ $1 != "--update" ]]; then
        echo "Invalid parameter! Use:"
        echo "./create_sys_image"
        echo "or"
        echo "./create_sys_image --update"
        exit 1
    else
        update=true
    fi
fi

if [[ $(basename $(pwd)) == "bin" ]]; then
    cd ..
fi

julia_version=$(julia --version | awk '{print($3)}')
julia_major=${julia_version:0:3}
if [[ $julia_major == "1.1" ]]; then
    julia_major=${julia_version:0:4} 
fi
if [ -d .git ] || git rev-parse --git-dir > /dev/null 2>&1 ; then
    branch=$(git rev-parse --abbrev-ref HEAD)
else
    branch=""
fi
if test -f "kps-image-${julia_major}-${branch}.so"; then
    mv bin/kps-image-${julia_major}-${branch}.so kps-image-${julia_major}-${branch}.so.bak
fi
julia -e "using Pkg; Pkg.add(\"TestEnv\")"

if [[ $update == true ]]; then
    echo "Updating packages..."
    if test -f "Manifest.toml"; then
        mv Manifest.toml Manifest.toml.bak
    fi
    julia  --pkgimages=no --project -e "include(\"./test/update_packages.jl\");"
else
    if [[ $julia_major == "1.9" ]]; then
        cp Manifest-1.9.toml.default Manifest.toml
        echo "Using Manifest-1.9.toml.default ..."        
    else
        cp Manifest-1.10.toml.default Manifest.toml
        echo "Using Manifest-1.10.toml.default ..."
    fi
fi
julia --pkgimages=no --project -e "include(\"./test/create_sys_image.jl\");"
if [[ $branch != "" ]]; then
    SOFILE="bin/kps-image-${julia_major}-${branch}.so"
else
    SOFILE="bin/kps-image-${julia_major}.so"
fi
if test -f $SOFILE; then
    mv $SOFILE $SOFILE.bak
fi
mv kps-image_tmp.so $SOFILE
julia  --project -e "using Pkg; Pkg.precompile(); Pkg.resolve()"
if [ -d src ]; then
    cd src
    touch *.jl # make sure all modules get recompiled in the next step
    cd ..
fi
echo "Precompiling package KiteModels..."
julia --project -J $SOFILE -e "using KiteModels, KitePodModels, KiteUtils"
echo "Precompiling package ControlPlots..."
if [[ $branch != "" ]]; then
    julia --project -J $SOFILE -e "using TestEnv; TestEnv.activate(); using ControlPlots"
else
    julia --project -J $SOFILE -e "using ControlPlots"
fi

Open questions:

  • can I launch a new Julia instance from a Julia function?
  • can I modify startup.jl such that it automatically uses a system image that is present in a project?
2

Sure, you can use run and Base.julia_cmd(). See e.g. ThreadPinning.jl/test/tests_mpi.jl at main · carstenbauer/ThreadPinning.jl · GitHub for inspiration, where I spawn MPI ranks running Julia (for testing purposes).

(Disclaimer: I didn’t read the entire post.)

3

If your goal is to replace the current Julia process by a new one without the user noticing, you can do this as well. But since this is pretty nuts, I won’t describe it here and just link to the relevant bit of one of my darkest projects.

1 Like
4

Why is it nuts? Would it be better to add this functionality to the julia starter program provided by juliaup, written in Rust?

I mean, automatically using a system image with a pre-defined name that is present in the current project folder seams like a pretty standard feature which could be useful in many scenarios.

5

I do something a bit like this in the docs building of a package of mine.

2 Likes