Changes of A.jl are not understood from B.jl that includes A.jl

New to Julia
, ,
1

Dear all,
I am new, so many things could be different than Matlab or any other C,C++,Fortran language here. But I experienced a weird behaviour. My script B.jl starts by including a function found in another script A.jl which implements 2 functions one calling the other. I usually run it from REPL by:
Julia > include("B.jl")
and then A.jl together with other scripts are called from B.jl returning results at the end of the day in a nice table. I could go more into what packages I am “using” but I think that the problem is more Julia-related and not package-related. I do hope I am wrong by the way and the problem is package related.

What happens unfortunately is that B.jl is using the very first version of A.jl, and does not understand the edits I did since the first run. Was it because at some point I ran it from the command line and not the REPL? In order to get correct results, compatible with my edits to A.jl, I had to rename A.jl to Anew.jl rename the function inside Anew.jl accordingly, and to be on the safe side, I also renamed B.jl to Bnew.jl. Then it worked and a @printf command was finally printing a message proving me that it was really compiled.
By the way from REPL I could call

Julia> include("A.jl")
Julia> x=5.0; myAfun(x)

and get correct answer. No problem there. I was hoping that the same is true when my A.jl is called from another script by the include(“A.jl”) directive. It seems this is not the case. What am I doing wrong? Every other language works like this. What is wrong with Julia or is it the packages I am “using” doing something weird for running benchmarks?

2

Could you give a complete minimal example to reproduce the problem?

I tried the following:

File A.jl containing:

f(x) = 2x
g(x) = f(x) + 1

File B.jl containing:

include("A.jl")
g(1)

First run in REPL:

julia> include("B.jl")
3

Then I change A.jl to replace g(x) = f(x) + 1 with g(x) = f(x) + 10 and redo the same in the REPL:

julia> include("B.jl")
12

How is your case different from this example? Are you sure the include("A.jl") line is indeed executed the second time? Is your code part of a package?

By the way, it’s good to try and understand what happens here, but in the end you might want to use Revise.jl.

3 Likes
3

Did you use Pkg.add to add A as a dependency to B? That would fully explain your issue since add records the current commit that repository and keeps on using that unless you call Pkg.update to change to newest version. If you want to use Pkg to just load whatever happens to be at some location on your machine you need to use Pkg.develop instead of Pkg.add.

The documentation might help you: Pkg.add and Pkg.develop

4

No, no, these are pure scripts and not packages. I mentioned that in my initial post.

5

I for one understood that B.jl and A.jl aren’t packages that are importing each other, but your post is legitimately confusing because it keeps mentioning packages and how you’re using them. You don’t describe the change to A.jl or what automatic changes you’d expect in B.jl, and it’s not apparent where compilation and benchmarks factor into this. Some phrasings are unintelligible: we include a .jl file, not a function; we can’t call .jl files; methods are compiled and packages are precompiled, .jl files are not.

If you don’t describe your problem as clearly as sijo’s step-by-step example does, then nobody can attempt to explain it. Here are some guidelines to help your readers follow along:

3 Likes
6

OK, I see where your confusion comes from. Basically I am doing what @sijo described in his sample code. I will try to reproduce with a small example later. I had to make sure however, before discussing about other possible problems due to packages imported such as the package:

using SolverBenchmark

that what @sijo described in his example does not force Julia to precompile A.jl the first time B.jl is called and then keep running B with the precompiled version every time the directive

include("B.jl")

is given to the REPL. Because from what I experienced this is what I was really afraid of. Namely, some Julia chief designers decided that precompiling A.jl, the 1st time B.jl is called is the right way to do things to save compilation time. I’m greatly relieved if this is not the case.

Now, what really happens is that B.jl is using A.jl as input to the SolverBenchmark package. This is done as described in JSOSolvers webpage to test different solvers and print the results using a DataFrame. So B.jl contains essentially the following code, where A.jl is an algorithm that minimizes an unconstrained optimization problem such as the ones that are described as ADNLPModels. A.jl always produces the correct results every time I run it from REPL using the include directive:

Julia > include("A.jl")

But when I include B.jl

Julia > include("B.jl")

then the results printed by the

pretty_stats(combined_stats)

command are always the same and do not change after the first time B.jl is executed no matter how many times I edit A.jl and/or run it using the include directive in REPL as above include("A.jl). I had to rename A.jl to Anew.jl as I described in my 1st post to see new results printed and the @printf to print something. So after the explanation of @sijo, I am wondering if the package SolverBenchmark is making/forcing precompilations of every solver in the dictionary solvers. Does this help you understand what the problem is?

B.jl contains

using Printf, LinearAlgebra, JSOSolvers, ADNLPModels, NLPModels, NLPModelsJuMP, OptimizationProblems, Optimization
Problems.PureJuMP, SolverBenchmark, SolverCore, DataFrames

include("A.jl")

 problems = [
   ADNLPModel(x -> x[1]^2 + 4 * x[2]^2, ones(2)),
   ADNLPModel(x -> (1 - x[1])^2 + 100 * (x[2] - x[1]^2)^2, [-1.2; 1.0]),
 ];

solvers = Dict(
  :a => model  ->  A(model)
)

stats = bmark_solvers(
  solvers, problems,
  skipif=prob -> (!unconstrained(prob) || get_nvar(prob) > 1000 || get_nvar(prob) < 2),
)

cost(df) = (df.status .!= :first_order) * Inf + df.elapsed_time
performance_profile(stats, cost)

# Create a new DataFrame with the desired columns
combined_stats = DataFrame(
    name          = stats[:a].name,
    nvars         = stats[:a].nvar,
    A_iters       = stats[:a].iter,
    A_obj         = stats[:a].objective,
)

# Display the combined stats table
pretty_stats(combined_stats)
7

Presumably, A.jl defines the name A appearing in B.jl and runs the @printf you keep mentioning. The obvious questions remain: what’s in A.jl, and what are the exact sequence of actions you are taking around its edits? These principles are covered in the linked guidelines.

For example, this seems to imply you’re only running include("A.jl") in the REPL after edits to A.jl, in which case the code previously evaluated from B.jl does not change, so it’s very plausible that nothing changes what pretty_stats(combined_stats) does. However, is that actually what happened? Running a copy include("Anew.jl") in the REPL should not do anything different, but you claim to see changes. Personally, I wouldn’t have ran include("A.jl") in the REPL at all, I would re-run include("B.jl") after changes because the include("A.jl") line in B.jl should serve that purpose, in addition rerunning B.jl’s code to incorporate those changes. Bear in mind that within the same REPL session, repeated imports do not reload the packages, so you only need to consider your files.

The ambiguity of a summary of what only you have observed is why it’s important you give a sequence of actions. A smaller example producing the observed effect like you suggested would also help.

That is definitely not happening, if only because precompilation is something done for packages, not anonymous functions in a script. Precompilation is distinct from compilation. If you are rerunning A.jl after changes but not B.jl, then solvers stays the same, using A from the obsolete A.jl. If you rerun B.jl after changes to A.jl, then an entirely new dictionary using A from the newest A.jl is made and assigned to solvers. The model -> A(model) will also be a new anonymous function that needs to be compiled for the first time; note that this does not imply A itself is assigned to a different instance, it would depend on what it is and what changes you made.

Unfortunately, I am not familiar with most of the imported packages, so any problem beyond Julia basics will elude me. Hopefully the problem can be identified and someone can spot the solution.

8

I don’t understand exactly what is going on. Because I don;t understand what steps are being followed.

But I know it is unreleated to precompilation,
and neither the files A.jl not B.jl will be precompiled.
because precompilation only happens for modules. (in particular it happens when they are loaded via using or import).
And those scripts are not modules and are being loaded via include.

idk if the confusion is between precompilation vs compilation? (they are different things and we use different words for them)
Or relation to cache of module loads: the module loads with the usings at the top of B.jl are cached. (This is the same as python IIRC) (they are also precompiled).

While I don’t understand what is happening,
when people say things like thing about things not being loaded after changing them the right answer is to use Revise.jl and includet instead of include.
Which will result in the files being tracked and any function defintions that are changed will automatically result in their defintions being reloaded
It will also mean that anything loaded by using or import is tracked and will be realoaded after changes (which will not happen).
But I am not sure that this applies in this case.

3 Likes
9

@DoctorDro any chance you can put a full self-contained example on GitHub or somewhere so that we can reproduce the problem ourselves?

10

I think you should be using Revise.jl in your workflow for running in the REPL, which handles what I interpret to be your issue.

julia> using Revise
julia> includet("B.jl")
julia> includet("A.jl")

Note use of includet instead of include. The prior tracks the file, including changes and updates your functions in the REPL so the most recent ones are used. Revise is great, but you may have issues when you change any structs you define and will have to restart the REPL.

EDIT: @oxinabox already mentioned this! Sorry for adding to the noise.

1 Like
11

I am working on it but it is not going to be simple. It seems the issue is related to the SolversBenchmark package. Probably it caches/precompiles the solvers it needs to benchmark so that the timings it will produce are optimal (release version who knows) and somehow it caches the results of the first compilation, so subsequently no compilations take place until the solver names change.

12

I tried that but it did not help at all. And I tried that in 2 different laptops.

13

A.jl implements a function, that takes an ADNLPModel as argument and minimizes the function defined in the model and outputs the number of iterations needed for the optimization to converge to user defined tolerance, objective values, gradient values etc.

Now in the A.jl script, there are 2 additional functions Alpha1 and Alpha2 that are called by the main function defined 1st in A.jl Amain. These 2 additional functions compute some constants that are needed in the main function Amain of A.jl. It is in one of them more precisely in Alpha1 that I wrote the @printf statement and the statement was not executed at all unless I renamed A.jl to Anew.jl and also change the code in B.jl to reflect that the solver to be benchmarked now is Anew and not A and also I changed the include statement at the top of B.jl, to include(“Anew.jl”). Just to make sure. This time everything worked and I got the results I waited for. I will try to make a short version so you guys can reproduce.

Thank you all for the support, your time and your interest. I really appreciate!

14

That is why I was so puzzled. I tried a minimalistic example with A.jl including 3 functions and a B.jl but I was not able to reproduce the problem. So now I have to try using the SolversBenchmark package because it seems this is causing the problem. I assume it somehow caches or precompiles the solvers it uses or something similar I do not know. But trust me when I am saying that I lost 2 days trying to figure out why I was getting the same results although I was editing the function in A.jl that was being benchmarked.

More precisely the DataFrame printed by the SolversBenchmark was reporting 5000 iterations and no convergence while when I manually from the REPL was running the same problem using
include("A.jl")
my method was converging in 26 iterations with the new edits. I was rerunning
include("B.jl")
and the DataFrame was reporting 5000 iterations and no convergence at all. It was really frustrating. Until I tried to put a printf statement inside the second function defined in A.jl and realized that it was not printing anything at all.

15

Is it possible that this mapping is compiled once with the old version of A.jl and later on since the code in B.jl is not changing, the mapping since it has been compiled once is not becoming aware of changes in A.jl, due to the fact that the code in B.jl is not changing anymore ?

16

No, because dictionaries are not compiled. If you rerun the solvers = Dict(... line after changing nothing, you get a new dictionary containing a new anonymous function model -> A(model), and that needs to be compiled anew, despite doing the same thing as the previous version. The documentation of SolversBenchmark does not mention anything about caching solvers, and even if it did, rerunning B.jl would make something new that has never been in a cache. The misconceptions about compilation and loose descriptions of the problem are not productive, so I suggest redoubling your efforts on a step-by-step example that reproduces the problem and avoid incorporating any assumptions you have about the cause. Is there a reason you’re avoiding posting A.jl along with B.jl? If you don’t reduce either, all you need to reproduce the problem is additionally list the line edits and commands in order.

17

I found the problem. It is both Julia’s and mine. But Julia does not spit a WARNING: as other compilers do. It seems that A.jl and C.jl which were both included by B.jl, implemented additional functions with the same signature apart from the main one. For example,
Contents of A.jl:

function MainA(blablabla)
end

function Helper1(blablabla)
end

Contents of C.jl:

function MainC(blablabla)
end

function Helper1(blablabla)
end

I was changing the Helper1 of A.jl but I had forgotten that the same function exists with an identical name in C.jl. It seems that the include directives in B.jl were in such order that one function survived and was always executed. But why the compiler would not WARN me about functions with identical signature? This is TOO wrong …

18

Which language are you coming from? I think you typically get a warning (or error) in this case in statically compiled languages, but I can’t think of any dynamic language where it’s the case. For example in Python, R, Ruby and Octave you get no warning. Julia is rather dynamic so it’s no surprise that you have no warning.

Think about the workflow you are yourself describing in the top post of this thread: you run include("B.jl") several times. Do you really want a warning for all the functions that are redefined by this call?

I think such warnings would be quite annoying in the REPL, but maybe it could be added in spefic contexts like precompilation…

1 Like
19

Of course it should spit a warning. Because it is very wrong from the developer side to have in different files functions with the same signature. Most of the time it is something he missed and/or overlooked. He should get a warning in order to fix it. This is why I mentioned a warning but after a second thought this should be an error because the function that would be executed at the end of the day is unpredictable or even if it is predictable it is not the one the developer would like to be executed.

20

I think in your top post you describe the following workflow:

  1. run include("B.jl")
  2. edit A.jl
  3. run include("B.jl") again (and you have created this thread because you don’t get the expected result here)

Maybe I misunderstood your workflow, but in any case many users are using Julia like that. And how would that work if you get an error in case of method redefinition?

For example your B.jl file contains this line:

cost(df) = (df.status .!= :first_order) * Inf + df.elapsed_time

so when you reexecute include("B.jl") you want to get an error because it redefines the cost function?