Changes of A.jl are not understood from B.jl that includes A.jl

New to Julia
, ,
81

There is one thing that I to make clear to future readers who have come this far (besides saying congratulations on making it this far).

Function redefinition is not just a Julia thing, it is a dynamic thing.

File A.py

def f():
    print('Definition A')

File B.py

from A import *
def f():
    print('Definition B')
f()

$ python B.py
Definition B

This doesn’t mean that Python is flawed, it just means that Python is dynamic - you can inspect and edit your program while it is running.

Julia is often seen as an easy target for being “not very good for real programming/creating software”, but I think that you would be reluctant to make the same case in a Python forum. Often, the logic for putting Julia in a different category from Python is “scientists are bad at programming, so Julia must be too.” Although, I may be slightly paraphrasing with that last quote, I have read good arguments from Jakob Nissen.

As pointed out in this thread, there are workflows that can ensure code integrity at each step (to do with modules and using/import). My personal favorite is Pluto notebooks. Working with Pluto notebooks really feels like you are editing a running program. They throw errors for the exact type of redefinition that you are talking about and tell you how to fix the issue.

3 Likes
82

Dear @JackDevine,
thank you for your inputs. I never, denied that my programming workflow is crap. But it was not intentional. I copied a file, I changed the first function (implementing some different method) and forgot to change the last 2 functions at the bottom of the file.

Other than that the whole discussion was about people trying to convince me that Julia should not spit out a warning message that functions are redefined because Julia is a dynamic language. Somebody mentioned modules to avoid such problems in the future. I will do that in the future. I am thankful to the person that pointed to good programming practices.

The point I wanted to make in this thread is that I do not believe that such a warning even from a dynamic language is such a difficult thing to implement. It is already there as an option activated with some command line parameters as somebody kindly pointed out. Why not making it the default to save people from agony and pain trying to figure out why the results are not changing while their code does? I lost 3 days trying to figure out why my code produces the same results even after significant edits. I could not have imagined that functions were being redefined without a warning. Dynamic or not it does not hurt to let you know.

All I asked here is a warning to let me know. For example, g++ had an option (I do not know if it is still there) to spit out warnings related to the book of Scott Meyers Effective C++ and More Effective C++.

If it warns you it promotes safer code at no extra cost, still maintaining its dynamic character. If it does not warn you it does not promote safer code and lets you screw up without knowing and your program has unexpected behaviour. Whether the programming practice is good or bad, this is the purpose of warning messages to WARN about potential issues. People were trying to convince me here that it should not WARN me because it is Dynamic or because my coding practice is wrong.

Yes, my coding practice was indeed crap, but it was unintentional and the language should let me know because it is indeed dynamic and allows redefinition and because redefinition might cause issues and unexpected code behaviour if it is unintended, that is why it should WARN ME. How can I explain it more simply? At the end of the day dynamic or not, the language should help the developer write correct code and avoid mistakes. If Julia is not helping in this direction it clearly goes in the wrong direction but people do not see it yet because they are more interested in making Julia dynamic than safe.

Do they realize that if for something so essential we have to discuss it for so long, and they are still not convinced (although the warning option is already there but hidden and activated with some command line options), how can I trust that Julia handles properly more complicated things concerning code safety, etc. I am really afraid to go on, because I do not know what I will discovered next.

Julia is not far away from becoming the main substitute of Fortran for scientific calculations. There is no other language at the moment as far as I know that allows partially LaTeX variable names. Who is this person who suggested and implemented this feature? He deserves a statue. Generations after generations will thank him :slight_smile:

But I am afraid at the last moment Julia took a wrong turn and it is going somewhere else. I am afraid that they do not know/decided where they are going. It is open and sounds like a kitchen of computer science fancy ideas or properties. It is not that bad, but at least it would be wise to maintain a fork of Julia which is stable and oriented to something that scientists could use without uncertainty. Some replacement of Fortran for example. It would be just a fork oriented to this scope and the scope is crystal clear. The main branch could be what it is right now. Some thoughts to consider for the future.

83

the point others have tried to make, at length, is that such a warning is not appropriate because the behavior you’re observing is intended and desirable

to be honest, I think it should be about time for this thread to wind to a close

4 Likes
84

I concur. I do not think anything more productive can come of this thread.

Here’s a patch if you really want it.

diff --git a/src/jloptions.c b/src/jloptions.c
index 570d021351..36bdbd1115 100644
--- a/src/jloptions.c
+++ b/src/jloptions.c
@@ -73,7 +73,7 @@ JL_DLLEXPORT void jl_init_options(void)
 #endif
                         JL_OPTIONS_CHECK_BOUNDS_DEFAULT, // check_bounds
                         JL_OPTIONS_DEPWARN_OFF,    // deprecation warning
-                        0,    // method overwrite warning
+                        1,    // method overwrite warning
                         1,    // can_inline
                         JL_OPTIONS_POLLY_ON, // polly
                         NULL, // trace_compile

I license the above patch under the MIT License.

It’s probably easier to use a shell alias or a one line shell script to enable this by default if you really want it.

1 Like
85

Thank you for your comment. You understand however, that this makes more unsafe Julia that what it could have been at no extra developing cost. Do you? Because if yes, then it is equivalent of saying that we would like to make Julia as unsafe as possible and as untrustworthy as possible? Are these features desirable for a modern programming language?

Wouldn’t it be better if it maintains its dynamic character but also at the same time promoting safety to the extent possible? How difficult is it to spit a warning? As far as I know Julia is made for scientists doing scientific calculations and not for GUIs or databases or HTML5 or Javascript. Scientists desire stability and reproducibility in their results, desire safety of results, desire even bit-by-bit identical results when their code runs in parallel. Got tell them that.

86

@mkitti thanks once again for the understanding, but it is not what I want only, it is what the community of engineers and scientists want. Check HSL MA97 claiming bit-by-bit identical results when the code runs in parallel for instance to get an idea how serious are engineers when it comes to results of calculations. Try to understand the community who will use Julia and the problems they are solving first before concluding that nothing good can come out of this thread.

87

yes, I’m aware of the consequences of using a dynamic language

3 Likes
88

My objection is: since the functionality to protect/warn users so that they will not waste their time in case of unintended includes, or forgotten function definitions, or even wrong workflow, is already there as @mkitti showed us above, why not compile it in so it is default behaviour and at the same time make the current dynamic language even more safe with minimal effort. Safety is a secondary objective which is also desirable in any language. I cannot think of a reason why one would hate to increase the language safety if the effort is minimal.

89

good thing the participants of this thread have told you the reasons. that way you do not have to try to think of them yourself

1 Like
90

It is sad that the focus is not improving the language. But use Julia as a platform for testing computer science fancy/modern ideas. The result will be a Russian salad of CS concepts since there is not interest in the predictability of the results of the compilation. Yesterday I got the same problem with Julia working on a dev fork of a package. I was changing the code under dev/package/src and Julia was not always recompiling. I had to exit Julia and start it again to compile the code and get correct results. Is this a bug I should report? Since the language is dynamic I do not know if it is a bug or a feature. I was constantly changing an @info message in SolverBenchmark.jl and I was rerunning my benchmark. Sometimes Julia was showing that it recompiles. After a few more edits of the source code (I was always saving the file) Julia was not recompiling and it was running the previous version of the code show what I had changed in @info was not showing. Is this a bug? I really do not know now. I am confused. I will discuss it with the package developer.

91

There is nothing unpredictable about julia. Over at github there is a ton of tests which must be satisfied whenever a new minor version is released, to make sure that nothing breaks or performs worse than before. However, you do not seem to be very interested in figuring out how the language and its tooling works, but pretend it should behave like some static compiled language like fortran or C. It does not.

Functions are compiled when they are called with a type signature they haven’t been called with before. They are not compiled when defined, be it via an include or by some other means. Although there are some exceptions when it comes to precompilation, invalidation and other stuff.

As an example, consider the function definitions:

f(x) = x+1
f(x::Real) = x+2
f(x::AbstractFloat) = x+3
f(x::Int) = x+4
f(x::Float64) = x+5

After the third method definition f(2.0) will return 5.0, whereas f(2) will return 4. After the fourth method definition f(2) will return 6, even though nothing has been redefined. After the fifth method definition, f(2.0) will return 7.0, still nothing has been redefined, but a program using the function f will likely change behaviour. Do you want a warning when something like this is being done?

1 Like
92

I forked a package, using ] develop, and I was editing it constantly until I manage the sought result. REPL was constantly “open” and after each edit inside the body of the function I was rerunning my include("myscript.jl") which was running a benchmark calling that package I forked in ~/.julia/dev/package_name/. Sometimes, after edits in the function body of the dev/package_name REPL after running myscript as above, was showing that compilation was taking place and the edits after a logging message @info blablaba where printed for each problem being solved. Changing @info blablaba to @info bloumbloumbloum was not understood somehow by Julia. So running my include("myscript.jl") I was surprised to see that I was getting ... blablaba instead of ... bloumbloumbloum. Exiting the REPL with CTRL+D and starting Julia again, after rerunning include("myscript.jl") I could see that this time Julia was compiling something and after that compilation I could see ... bloumbloumbloum. Keep editing and rerunning the first time Julia was correctly recompiling, and the output was correct. The second time I was editing the dev/package_name/src/file.jl Julia was not recompiling. Note that always I kept editing the same file.jl inside ~/.julia/dev/package_name/src/file.jl at the same line printing a message with the @info command. Basically I was changing the strings following @info. Now is this a bug, or is this a feature that belongs to the special cases you mentioned?

93

Of course. I always need a warning to WARN me whenever the result is either not predicted or something else so that I can check and be aware of the expected code execution sequence after compilation takes place. This is what warning stands for isnt it? It is not an error, it just warns you that something may be not as expected.

94

I have no idea, but the “special cases” I mentioned are not special cases, it is the core of julia, such things are being done all the time, all over the place. If you e.g. at some point have changed the formal argument list of some function, the definition with the old argument list still exists in your julia session, and will be used if the argument list matches. This can explain the behaviour you have seen. And this is not a bug or some fancy CS idiocy, but the mechanism which makes most everything in julia work.

95

No, no, I was not changing the function definition. This happened only once in the beginning. All subsequent edits took place inside the body of the function and the function signature was the same. I was just trying to beautify the logger message and until I get it right it took me sometime. So just editing the message, Julia from within the REPL sometimes it was compiling after my edits to the message and sometimes it was not. When it was not compiling I had to exit REPL and start REPL again, to find that this time it was compiling. Every time it was not compiling after my edits exiting the REPL and restarting it it was compiling as expected and after running I could see my edits to the logger message. This is weird cause the package I was editing was in .julia/dev/ which I forked with ] develop command. I followed the directions of the developer.

96

If this is reproducible I would report a bug, with a concrete example and instructions for how to reproduce the behaviour. That said, I have occasionally used the same workflow, and have never seen such things happen unless I have inadvertently messed up something. The danger with that workflow is that the workspace can gradually become polluted, either by intended changes or typos. Like in the f-function above, where everything is fine, f(2) == 6 until you happen to call f with a 32 bit 2 (which looks exactly the same), and it suddenly returns 4, like it did five hours ago.

2 Likes
97

If you have Pkg.develop-ed some package PackageName.jl, and you are editing that package’s source code locally (e.g. editing the files at ~/.julia/dev/PackageName.jl), then the edits you made will not take effect until you restart your Julia session. This is the expected behavior.

If you want your edits to show up without needing to restart your Julia session, then I’d recommend using Revise.jl.

3 Likes