I just found that in the latest version v.1.9.×, reloading a function of a package alters the original behavior of the same function of the package when being called independently. For instance, define a module “test” to reload the “Distributions: rand”,
module test
import Distributions: rand
function rand()
println(“reload!\n”)
end
end
then, edit a script to call them independently like this:
The result is that the last calling of “Distributions.rand()” is affected by the reloading operation of the module test, i.e. to display the string of “reload!” rather than a random number! Such a behavior change is very unexpected. Any suggestion?
This is the normal behavior, although it is pretty unexpected at first. What you’re doing is called “type piracy” and it is frowned upon for that very reason
As a general rule, you can implement
existing functions on your own types
your own functions on existing types
but never both at the same type, which is what you did by redefining Base.rand for no argument
I’m not really sure why this is unexpected for you, but I’m guessing you expected Distributions.rand loaded in your test to belong to the module test. It’s actually the opposite, imported functions definitely do not belong to the module they’re imported in. They may not even belong to the module you imported them from, because that module may have imported it from yet another module. You can check with parentmodule(rand), which tells you Base. A function’s multiple methods can be defined in multiple modules, like test or Distributions, but they’re just extending the function in its parent module. test.rand === Distributions.rand === Base.rand.
Instead of overriding an existing function, you could make your own rand function that belongs to test. However, if you want to use Base.rand, you will have to write out the name since the name rand will refer to your function in the scope of module test. When you import from test to other modules, you’ll also have to make the decision of which rand is more important. Imports can also rename now, so name collisions are manageable.
julia> module Test2
function rand() println("reload!\n") end
rand()
println(Base.rand())
end
reload!
0.4336162318813531
Main.Test2
Many thanks, I just read about the type piracy and understand my code problem. But occasionally I need to extend a package by only modifying the intermediate function. For instance, provided that we already have a package:
Now, we want to import it into another new package, and want c() to be reloaded by only modifying "needRelaod() " without changing “b()”. In such a case, the most hassle-free manner, however, could be the type piracy
import pkg1: needRelaod
function needRelaod()
do something special
end
Note that, we can’t reload needRelaod() by other types, since we still want Pkg1.c() can call it. So, the type piracy can meet the requirement. Now, in order to avoid the type piracy, I have to copy the binary file of the pkg1 into my new module folder and take pkg1 as a local file for reloading. I feel such a manner is still not convenient. Could you please share better suggestions?
if the author of Pkg1 intends to support “user customization”, then they should reflect this in their library’s design, one possible way is to change how c() is defined:
c(func::F = needReload) where F = func() + b() + 3
or more sophisticated design that involves abstract type and downstream user can specialize to their own types and customize intermediate behavior.
another possibility is if you’re the author of both pkg1 and pkg2, and they form an “ecosystem”, it might be convenient and more understandable to commit “piracy” since you essentially control both pkgs
And another option is to commit the piracy in a glue package called something like Pkg1Pkg2Utils.jl. Sure, piracy redefines the behavior for every downstream user, but since when they import that glue package explicitly, we can assume they agree to that. Of course things are different when the glue package is an implicit dependency of something else.
All right, the idea of using glue package Pkg1Pkg2Utils.jl is great, but it still cannot meet the requirement of calling both of Pkg1Pkg2Utils.needRelaod() and pkg1.needRelaod() without interfering with each other.
Great share! But, I think that package developers are not always aware of all users’ future demands, and confining the ecosystem of a reloaded package to its parent package might be very helpful.
They wouldn’t have to be, jling’s code lets the user input their custom functions. That function’s limitation is it must support addition with b() and 3. That is an expectation that should be documented. You wouldn’t need to make end-users able to use this, you could do this internally to customize behavior in different packages.
You are right in the case that a package developer intends to let a customer to extend a function as will, but unfortunately, not all cases. Such an awareness of extending functions might require far-sighted code planning. I still believe in the value of limiting piracy type in the scope of a reloading module. Packages, if developed as such, can be shared in more flexible manner.
But it could be easily solved if you provide your own type (some struct MyBehaviourType end and then implement rand(::MyBehaviourType) with your expected version of rand?
Otherwise you might “disturb” other peoples code, when the expect rand() to behave ad Base defines it (I hope no one else does, otherwise that is already type piracy)
You could define MyPackage.rand(), and not import Base.rand (or any other rand() coming from there), then you even can do it without a new type. But then please do not export it, otherwise loading your package issues a warning like and both Base.rand and yours have to always be prefixed with the package name to resolve the ambiguity (in which rand Is meant).
Without exporting it, yours has to always be prefixed, bt at least its not type piracy and not breaking other peoples code.
I believe in strongly discouraging any type piracy, since that breaks other people’s code.
Then I probably do not yet get your full use case here, because even after carefully reading this thread I do not yet see any good case where type piracy should ever be used.
Briefly, for instance, one just wants to reload b() without altering anything else. Then Pkg1.c() can be called to employ the reloaded b(), in the meanwhile, Pkg1.b() is desired to behave the original strategy.
But that is impossible. Either you do type piracy, then you overwrite even the behaviour of b in this case (that is you import Pkg1.b and redefine its zero argument definition) or you define a (second) b() outside of Pkg1 then it does not have an effect _within Pkg1. So either way, you do not achieve both things you want at the same time.
I would consider this still type piracy, just that no one can see this any longer, since you hide it in binary files.
But also if you now change anything in Pkg1 you have to manually copy over the binary?
I am also not sure how “take pkg1 as a local file for reladong” would work in practice, but anything that defines code but also requires manually copying files – should even less be in anything productive/published/open source, even less than type piracy.
The source of error (by someone forgetting the manual copy) is just too large.
