Using Julia in a containerized Nextflow pipeline

General Usage
,
1

I’m using Julia scripts inside of a container-based Nextflow pipeline. Basically, the pipeline script will pull a fresh Julia Singularity image, add the bin directory associated with the pipeline directory to PATH, and then execute the Julia scripts. I’m wondering what the best/most Julian way to handle package dependencies in such a situation is.

My current setup is to commit my Project.toml and Manifest.toml inside of the bin dir, like

.
├── bin
│   ├── myscript
│   ├── library.jl
│   ├── Project.toml
│   └── Manifest.toml
├── main.nf
└── nextflow.config

The first few lines of myscript are then

#!/bin/bash
#=
julia --project="$(realpath "$(dirname "${BASH_SOURCE[0]}")")" -e 'using Pkg; Pkg.instantiate()'
exec julia --project="$(realpath "$(dirname "${BASH_SOURCE[0]}")")" "${BASH_SOURCE[0]}" "$@"
=#

which boils down to

  1. Set the project directory and run ] instantiate
  2. Set the project directory and execute the file using Julia

in a manner similar to that described in the manual.

This setup works well enough, but it feels kind of hacky and I’m afraid it might break at larger scales. Is there a better way?

2 Likes