How to run julia code that depends on other packages in Jenkins?

Hi guys, I try to run some codes with the help of Jenkins. OS is win10.

I tried this command but failed:

cd E:\\sim_1\scripts
set JULIA_NUM_THREADS=8
julia E:\\sim_1\scripts\main.jl

error says that julia is not a command, but I checked the system path and also run the above code in CMD, it works well.

Are there any tutorials on this issue? I searched around and couldn’t find it.

This is more a Jenkins than a Julia related question. Better ask this on “stack overflow”. See for example delphi - Why is Jenkins ignoring the %PATH% variable when using MSBuild? - Stack Overflow

1 Like

Thanks!

Yes, I also asked this question on stack overflow. I’m a beginner in Jenkins, so I wonder if someone from the Julia community has some experience with this. The explanations from this community would be better for me to understand.

Is julia installed in the jenkins worker and available in PATH?

1 Like

Its available in PATH but not installed in the Jenkins. I think I can run simple code in Jenkins now, like using CMD command julia E:\\sim_1\scripts\main.jl. But It still failed when it require some packages.

I have no idea about Jenkins, but this sounds like a project issue.
If your Julia project lives in E:/sim_1/ (i.e. Project.toml and Manifest.toml are located there), then use the --project flag when calling the script:

julia --project=E:/sim_1 E:/sim_1/scripts/main.jl

or change directory beforehand

cd E:/sim_1
julia --project=@. scripts/main.jl
1 Like

Thank you!
I have tried to set environment, and it also works in CMD.
While in Jenkins, error still comes like this :

ERROR: LoadError: ArgumentError: Package DrWatson [634d3b9d-ee7a-5ddf-bec9-22491ea816e1] is required

and I am pretty sure that DrWatson is included in the Project.toml.

Is that the complete error message? I would like to suggest that you need to instantiate the environment in Jenkins, but then the error message should be longer and more helpful.

2 Likes

Yes, I only present the main message of the error. Obviously, this is because Jenkins agent has no access to the packages or the Project.toml. If I wrote using XXX in main.jl, the error message would be the same type.

I config Jenkins agent in this way:

cd /d E:\XXX\Jenkins\workspace\test\JenkinsSim
set JULIA_PROJECT=@.
set JULIA_NUM_THREADS=8
julia E:\XXX\Jenkins\workspace\test\JenkinsSim\scripts\main.jl

The error message would be:

Running as SYSTEM
Building in workspace E:\XXX\Jenkins\workspace\test
[test] $ cmd /c call C:\Windows\TEMP\jenkins78245134749799026.bat

E:\XXX\Jenkins\workspace\test>cd /d E:\XXX\Jenkins\workspace\test\JenkinsSim 

E:\XXX\Jenkins\workspace\test\JenkinsSim>set JULIA_PROJECT=@. 

E:\XXX\Jenkins\workspace\test\JenkinsSim>set JULIA_NUM_THREADS=8 

E:\XXX\Jenkins\workspace\test\JenkinsSim>julia E:\XXX\Jenkins\workspace\test\JenkinsSim\scripts\main.jl 
ERROR: LoadError: ArgumentError: Package DrWatson [634d3b9d-ee7a-5ddf-bec9-22491ea816e1] is required but does not seem to be installed:
 - Run `Pkg.instantiate()` to install all recorded dependencies.

Stacktrace:
 [1] _require(pkg::Base.PkgId)
   @ Base .\loading.jl:1306
 [2] _require_prelocked(uuidkey::Base.PkgId)
   @ Base .\loading.jl:1200
 [3] macro expansion
   @ .\loading.jl:1180 [inlined]
 [4] macro expansion
   @ .\lock.jl:223 [inlined]
 [5] require(into::Module, mod::Symbol)
   @ Base .\loading.jl:1144
in expression starting at E:\XXX\Jenkins\workspace\test\JenkinsSim\scripts\main.jl:1

E:\XXX\Jenkins\workspace\test\JenkinsSim>exit 1 
Build step 'Execute Windows batch command' marked build as failure
Finished: FAILURE

and my Project.toml is located in E:\XXX\Jenkins\workspace\test\JenkinsSim

name = "JenkinsSim"
authors = ["Yibo"]

[deps]
BifurcationKit = "0f109fa4-8a5d-4b75-95aa-f515264e7665"
CairoMakie = "13f3f980-e62b-5c42-98c6-ff1f3baf88f0"
DelimitedFiles = "8bb1440f-4735-579b-a4ab-409b98df4dab"
DifferentialEquations = "0c46a032-eb83-5123-abaf-570d42b7fbaa"
DrWatson = "634d3b9d-ee7a-5ddf-bec9-22491ea816e1"
DynamicalSystems = "61744808-ddfa-5f27-97ff-6e42cc95d634"
ForwardDiff = "f6369f11-7733-5829-9624-2563aa707210"
LinearAlgebra = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e"
Parameters = "d96e819e-fc66-5662-9728-84c9c7592b0a"
Plots = "91a5bcdd-55d7-5caf-9e0b-520d859cae80"
ProgressMeter = "92933f4c-e287-5a05-a399-4b506db050ca"
Revise = "295af30f-e4ad-537b-8983-00126c2a3abe"
Setfield = "efcf1570-3423-57d1-acb7-fd33fddbac46"

[compat]
julia = "1.8.4"

Should I install packages in the Jenkins agent? and how can I do that?

Yes, the error message already tells you that:

So either in your main.jl you need to make sure that the project is instantiated, or modify your jenkins BAT script to instantiate it before you run the script:

cd /d E:\XXX\Jenkins\workspace\test\JenkinsSim
set JULIA_PROJECT=@.
set JULIA_NUM_THREADS=8
julia -E 'import Pkg; Pkg.instantiate()'
julia E:\XXX\Jenkins\workspace\test\JenkinsSim\scripts\main.jl
1 Like

Thanks! really helpful!

julia -E ‘import Pkg; Pkg.instantiate()’

It doesn’t work but I add import Pkg; Pkg.instantiate() to the head of main.jl and it works!

A word of advice is not to abbreviate too aggressively. Had you included the full line and the next we could immediately have told you what was the problem and how to solve it.

2 Likes

That should work though :thinking: It’s doing the exact same thing. Do you get an error? What exactly happens?

E:\XXX\Jenkins\workspace\test\JenkinsSim>julia -E import ‘Pkg; Pkg.instantiate()’
ERROR: syntax: incomplete: premature end of input
Stacktrace:
[1] top-level scope
@ none:1

E:\XXX\Jenkins\workspace\test\JenkinsSim>julia E:\XXX\Jenkins\workspace\test\JenkinsSim\scripts\main.jl
ERROR: LoadError: ArgumentError: Package DrWatson [634d3b9d-ee7a-5ddf-bec9-22491ea816e1] is required but does not seem to be installed:

  • Run Pkg.instantiate() to install all recorded dependencies.

Looks like you copied wrong - those apostrophes are at the wrong place. Note that in the code I posted they enclose the import too.

Really thank you, for correction!