Changed the ModiaMath travis.yml file to use Julia 0.7. Now, travis cl fails with the message:
$ julia --check-bounds=yes --color=yes -e "if VERSION < v\"0.7.0-DEV.5183\"; Pkg.test(\"${JL_PKG}\", coverage=true); else using Pkg; Pkg.test(coverage=true); end"
Testing ModiaMath
Resolving package versions...
ERROR: LoadError: ArgumentError: Package Test not found in current path:
- Run `Pkg.add("Test")` to install the Test package.
What needs to be fixed?
Maybe this will help
https://docs.julialang.org/en/latest/stdlib/Pkg/#Test-specific-dependencies-1
I have a similar question.
I tried the following
[targets]
test = ["Test"]
build = ["Libdl"]
but, although I added the “Libdl” in the targets part. I still get error when build. Does this mean that currently only test target is supported?
Thanks, this fixed the issue.
Correct, unfortunately the build target is not yet hooked up the way the test target is so for now your package will have to have build dependencies among the main dependencies just as they were in the old system. In the future, however, we will hook this up so that you can avoid having the main package depend on things you only need for building the package.
There is another complication with testing. The structure of the package is:
ModiaMath
src
test
runtests.jl
test_something.jl
# file runtests.jl
module Runtests
using Test
@testset "Test ModiaMath" begin
include("test_something.jl")
end
end
# file test_something.jl
modul Test_something
import ModiaMath
using StaticArrays # <==== gives error
end
Package StaticArrays is also imported in ModiaMath and is defined in the ModiaMath project.toml file.
If the tests are directly executed by including the file runtests.jl:
julia> import ModiaMath
julia> include("$(ModiaMath.path)/test/runtests.jl")
Test ModiaMath: Error During Test at ModiaMath\test\runtests.jl:9
Got exception LoadError [...] ArgumentError("Package StaticArrays not found in
current path:\n- Run `Pkg.add(\"StaticArrays\")` to
install the StaticArrays package.\n"))) outside of a @test
O.k. I could add StaticArrays in environment v0.7, but then users that use the ModiaMath package have to do this as well, but they get an error beforehand. It would be appropriate, that ModiaMath is configured so that no such error occurs for the user.
The question is therefore: How to configure ModiaMath, so that this error does not occur?
I tried to add the following to the project.toml file of ModiaMath:
[extras]
Test = "8dfed614-e22c-5e08-85e1-65c5234f0b40"
StaticArrays = "90137ffa-7385-5640-81b9-e52037218182"
[targets]
test = ["Test", "StaticArrays"]
But this did not help. “Somehow” one needs to add a project.toml file to file “runtests.jl”, but probably this is not possible.
I just recognized that it is even worse: In ModiaMath there is an examples directory where users can directly execute many examples:
ModiaMath
examples
Simulate_Pendulum.jl
Simulate_FreeBodyRotation.jl
...
src
...
test
...
In julia 0.6, users just used include("$(ModiaMath.path)/examples/Simulate_Pendulum.jl") to run one of the examples (the example itself is a module). In julia 0.7 they get errors, because in some of the examples other packages are imported (such as “StaticArrays”) and julia claims that it does not know these packages.
The general question is therefore: How to configure ModiaMath, so that modules outside of the src directory can still be executed and the packages imported in these modules are known to the julia package manager?
If those examples are self-contained examples of how to use the package, why not make them their own distinct modules with Project.toml and Manifest.toml? After all, that’s what “regular” packages using your package should look like on some level (unless they’re one off scripts, in which case they shouldn’t probably be modules in the first place).
As for the problems with your tests, I’m not quite sure what the correct way to structure them is, sorry.
EDIT: However, according to the docs runtests.jl doesn’t seem to be a module of its own, but a script with the necessary using statements. I think if your users clone your package, they should activate and instantiate in order to run the examples you provided (also see here).
The fact that Modia depends on StaticArrays does not mean that we can load StaticArrays in the REPL. For that, StaticArrays has to be explicitly added by the user.
I finally found a good solution (could be a template for other packages as well):
The source of the problem is that (a) there are external Julia “scripts” under ModiaMath/examples and ModiaMath/test that use package ModiaMath to demonstrate/test its features and (b) these “scripts” import other packages (besides ModiaMath) and the versions of these other packages are not defined. It is of course most natural to exactly use the package versions that are also used inside ModiaMath.
A reasonable solution with Julia 1.0.0 is therefore to reference extra packages used in examples
and test via the ModiaMath package:
File: ModiaMath/src/ModiaMath.jl
module ModiaMath
[....]
# Add import clauses used in examples and test (if not yet available here)
import StaticArrays
import Unitful
import DataFrames
@static if VERSION >= v"0.7.0-DEV.2005"
import LinearAlgebra
import Test
end
end
File: ModiaMath/examples/Simulate_FreeBodyRotation.jl
module Simulate_FreeBodyRotation
using ModiaMath
using ModiaMath.StaticArrays
@static if VERSION >= v"0.7.0-DEV.2005"
using ModiaMath.LinearAlgebra
end
[....]
end
File: ModiaMath/test/runtests.jl
module Runtests
@static if VERSION >= v"0.7.0-DEV.2005"
import ModiaMath
using ModiaMath.Test
else
using Base.Test
end
[....]
end
In the project.toml file there is only the [deps] section, but no longer [extras] and [targets] sections.
The only (minor) drawback is that package Test is loaded always with ModiaMath, even if it is not used (but I guess this overhead is small). The benefit is that individual Julia scripts under ModiaMath/test can be run directly with include which is the standard way how the tests are developed in ModiaMath and in Modia.
It might be useful to support the pattern above directly in the Julia language. Example:
module MyExample
using ModiaMath
using StaticArrays as_in ModiaMath # use same version of StaticArrays as in ModiaMath
end
Tom Short wrote here:
The code from 479cec8 seems like a kludgy solution. I’m not sure what’s the best approach. Maybe it points to the need for better Base tooling.
How about including an
examples/Project.tomlwith those dependencies? The user would have to activate that to run the code.
i hope that @StefanKarpinski could also have a look and give his opinion.
Let me add some more explanations: Currently, there are about 20 examples in ModiaMath and there will be much more, especially in Modia and in Modia3D, at some point in the future (say hundreds or thousands of such examples). All these examples are also executed from the test directory. So we have:
ModiaMath
examples # 100 ... 1000 independent examples (not handled by package manager)
src # source code handled by package manager
test # tests + calling all the examples (partially handled by package manager)
With the current package manager, via [extras] and [test] test runs can be handled, if the test is started in a special environment (]activate ModiaMath; test). However, if the user would like to run single examples from the examples directory or single tests from the test directory directly from his/her standard environment, then this might not work and the user has to add packages to his standard environment that are used in the examples directory (but the examples directory was potentially tested with another version).
My current solution in ModiaMath (as proposed above) is nice in the sense, that it always works and the user does not have to think about (just start the single example or single test in any environment in which ModiaMath is present, and it will work; this also includes ContinuousIntegration environments).
I agree that this solution looks not nice from a conceptual point of view and it would be better to have some nicer solution in this respect.
It is not clear how to add project.toml/manifest.toml files to the examples or test directory. In particular if single examples and single tests would like to be executed, then potentially for every single example or test separte project.toml/manifest.toml files are needed. If one has 100 files, it would be hard to manage and/or maintain this. In particular, all the examples and tests would be typically developed with the same base packages, so to define in every example that version xxx from package yyy is used would be a nightmare.
Why can’t you just Pkg.add what you need in the example scripts? Or simply put it in the documentation:
To run the following example you need to install Modia, LinearAlgebra and StaticArrays.
One problem with including calls to Pkg.add() is that it changes the current environment. The user may not want that.
Another option is to Pkg.activate("path/to/examples") and include a Project.toml there like you did for Literate.jl/docs.
I just don’t understand why you would try to hide the dependencies like this. Presumably the examples are there for users to study them, and to be able to modify/extend/write their own programs, and if the examples generally uses e.g. StaticArrays then why not be open with that such that users understand that in general you need more than just Modia?
I don’t think they’ll be hidden in that the using Whatever will still be there. I’m also good with the idea of adding documentation. The Pkg.add() part seemed klunky.
Right, but the commit linked above, where e.g. using StaticArrays was replaced with using Modia.StaticArrays is a bit weird, it looks more like an internal module than the actual StaticArrays package.
Agreed. That’s the reason for reaching out for other ideas.
I think that documenting what packages you need to run the examples, and also provide a examples/Project.toml with the required deps that users can instantiate is a good way.
I am sorry, but I do not fully understand the proposed alternative. Current structure:
ModiaMath
examples
Simulate_FreeBodyRotation.jl
Simulate_Pendulum.jl
If Project.toml, Manifest.toml files should be added, they need to be added for every example. This means the structure would have to be changed to something like:
ModiaMath
examples
Simulate_FreeBodyRotation # new directory
run.jl # previous Simulate_FreeBodyRoation.jl file
Manifest.toml
Project.toml
Simulate_Pendulum
run.jl
Manifest.toml
Project.toml
But how is the example then executed so that Julia uses its Project.toml and Manifest.toml files? I guess
julia> import ModiaMath
julia> include("$(ModiaMath.path)/examples/Simulate_Pendulum/run.jl")
would use the packages defined in the standard environment and the Project.toml and Manifest.toml file from the Simulate_Pendulum.jl directory would be ignored?
You can probably have the same project for all examples, no? Something like
ModiaMath
examples
Project.toml
Simulate_FreeBodyRotation.jl
Simulate_Pendulum.jl
and users would activate, instantiate and run with
using Pkg
Pkg.activate("path/to/examples")
Pkg.instantiate()
include("path/to/examples/Simulate_FreeBodyRotation.jl")