# Advice on Command Line Usage on HPC with Julia

**URL:** <https://discourse.julialang.org/t/advice-on-command-line-usage-on-hpc-with-julia/108176>\
**Category:** Specific Domains\
**Tags:** hpc, slurm, command-line-options\
**Created:** [December 30, 2023, 5:28am UTC](https://discourse.julialang.org/t/advice-on-command-line-usage-on-hpc-with-julia/108176 "2023-12-30T05:28:15Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![TI36XPro](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ti36xpro/32/33658_2.png) [@TI36XPro](https://discourse.julialang.org/u/TI36XPro)\
**Post date:** [December 30, 2023, 5:28am UTC](https://discourse.julialang.org/t/advice-on-command-line-usage-on-hpc-with-julia/108176/1 "2023-12-30T05:28:15Z")

</div>

Hello,

I have been working and re-working the method I use to generate and submit runs on my HPC and I’d appreciate any advice or comments on the process (I’m sure there is plenty of room for improvement). Apologies if any of this seems unnecessarily complicated. This is just me fumbling my way through it.

Right now I use ArgParse.jl. I have a parser that basically looks like the following. I use keyword arguments because it makes it easier to enter the commands (since there are so many, otherwise I’d get confused)

```julia
function parse_my_commandline()

    s = ArgParseSettings()

    @add_arg_table s begin
        "--parm1"
        arg_type = String
        "--parm2"
        arg_type = String
        "--parm3"
        arg_type = Float64
        "--parm4"
        arg_type = Float64
        "--parm5"
        arg_type = Int
        # ... And so on to ~30 total 
        end

        parsed_args = parse_args(s)
        # Do some stuff based on default values of args, some error checking, etc.
        return parsed_args 

end

```

This works but the length of the command line is getting a little silly. Is there a better way to do this? Some type of config file I type up and then feed to the parser? Just so I don’t have to type up such a long command at the terminal? Or is this typical?

---

<div class="post-metadata">

**Author:** ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)\
**Post date:** [December 30, 2023, 7:32am UTC](https://discourse.julialang.org/t/advice-on-command-line-usage-on-hpc-with-julia/108176/2 "2023-12-30T07:32:25Z")

</div>

Note that you can add a short form for each argument. `-p` etc

---

<div class="post-metadata">

**Author:** ![fatteneder](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/fatteneder/32/33991_2.png) [@fatteneder](https://discourse.julialang.org/u/fatteneder)\
**Post date:** [December 30, 2023, 9:17am UTC](https://discourse.julialang.org/t/advice-on-command-line-usage-on-hpc-with-julia/108176/3 "2023-12-30T09:17:55Z")

</div>

I recommned using the TOML.jl stdlib to read in a .toml file as a dictionary and then use `get(dict, :my_parameter_name, default_value)` to query the parameters.

---

<div class="post-metadata">

**Author:** ![ehsani63](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ehsani63/32/38199_2.png) [@ehsani63](https://discourse.julialang.org/u/ehsani63)\
**Post date:** [December 30, 2023, 9:45am UTC](https://discourse.julialang.org/t/advice-on-command-line-usage-on-hpc-with-julia/108176/4 "2023-12-30T09:45:23Z")

</div>

Then for submitting several jobs at once, there should be several TOML files created, right? The good thing about command line arguments is that you can submit different jobs by changing some args in a bash loop, for example.

---

<div class="post-metadata">

**Author:** ![fatteneder](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/fatteneder/32/33991_2.png) [@fatteneder](https://discourse.julialang.org/u/fatteneder)\
**Post date:** [December 30, 2023, 10:10am UTC](https://discourse.julialang.org/t/advice-on-command-line-usage-on-hpc-with-julia/108176/5 "2023-12-30T10:10:08Z")

</div>

> [@ehsani63](#):
>
> Then for submitting several jobs at once, there should be several TOML files created, right?

Personally I prefer this over explicit command line args, because if I keep the parameter file around then it automatically documents what parameter values were used for which simulation.

> [@ehsani63](#):
>
> The good thing about command line arguments is that you can submit different jobs by changing some args in a bash loop, for example.

I also think it is quite easy to write a julia script with a loop to generate the needed configurations and dump them to a file, again using TOML.jl.  
[Although in practice I never do this and instead generate the files manually as most of the times this is faster.]

But all of this is personal preference, so YMMV.

---

<div class="post-metadata">

**Author:** ![pitsianis](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pitsianis/32/26588_2.png) [@pitsianis](https://discourse.julialang.org/u/pitsianis)\
**Post date:** [December 30, 2023, 9:55pm UTC](https://discourse.julialang.org/t/advice-on-command-line-usage-on-hpc-with-julia/108176/6 "2023-12-30T21:55:06Z")

</div>

Consider [DrWatson.jl](https://github.com/JuliaDynamics/DrWatson.jl) to prepare and submit jobs without a command line.

---

<div class="post-metadata">

**Author:** ![TI36XPro](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ti36xpro/32/33658_2.png) [@TI36XPro](https://discourse.julialang.org/u/TI36XPro)\
**Post date:** [December 31, 2023, 3:50am UTC](https://discourse.julialang.org/t/advice-on-command-line-usage-on-hpc-with-julia/108176/7 "2023-12-31T03:50:50Z")

</div>

Thanks for the suggestion. I actually am using DrWatson for this project. Did you have a specific functionality of DrWatson in mind to accomplish what I described above? I might’ve missed something in the docs.