# Possibility of \`local import\` statements in future?

**URL:** <https://discourse.julialang.org/t/possibility-of-local-import-statements-in-future/10611>\
**Category:** Internals & Design\
**Tags:** scope\
**Created:** [April 29, 2018, 8:41pm UTC](https://discourse.julialang.org/t/possibility-of-local-import-statements-in-future/10611 "2018-04-29T20:41:39Z")\
**Posts on this page:** 20\
**Page:** 3

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [April 30, 2018, 8:42pm UTC](https://discourse.julialang.org/t/possibility-of-local-import-statements-in-future/10611/41 "2018-04-30T20:42:42Z")

</div>

How do you deal with generic functions calling interface method extended in another module.?

---

<div class="post-metadata">

**Author:** ![TsurHerman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tsurherman/32/1234_2.png) [@TsurHerman](https://discourse.julialang.org/u/TsurHerman)\
**Post date:** [April 30, 2018, 9:26pm UTC](https://discourse.julialang.org/t/possibility-of-local-import-statements-in-future/10611/42 "2018-04-30T21:26:25Z")

</div>

> [@yuyichao](#):
>
> How do you deal with generic functions calling interface method extended in another module.?

You mean like `FileIO`?

the interface function which get extended for dispatch would need to be a macro(which is appropriate because  
it is a kind of magic … which is what macros do … from a programmer perspective) which calls:

`function load(s::Stream{format"PNG"}; keywords...)`

based on the extension.  
Thus forcing the programmer to state each format she wants to register for use in this scope. but also frees her to choose an implementation for a specific file format.

EDIT:  
not sure if it is possible with a macro … you are the expert, what do you say is it possible?

---

<div class="post-metadata">

**Author:** ![chakravala](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chakravala/32/6832_2.png) [@chakravala](https://discourse.julialang.org/u/chakravala)\
**Post date:** [April 30, 2018, 10:05pm UTC](https://discourse.julialang.org/t/possibility-of-local-import-statements-in-future/10611/43 "2018-04-30T22:05:45Z")

</div>

One of the other ways I am using macros to handle method dispatch is using `Reduce` switches,

```nohighlight
julia> using Reduce

julia> @rounded @factor x^3-2x+1
:((x + 1.61803398875) * (x - 1) * (x - 0.61803398875))

```

In this example, any sequence of REDUCE switch macros can be used, and they will all be automatically rewritten into a single command to the software, instead of communicating multiple times.

```nohighlight
@force using Algebra

```

However, some of the extensions currently available in the package will now be moved to the sub-module `Algebra`, but will still be optional to have the full extensions of all methods in any module with `@force` macro.

---

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [April 30, 2018, 10:40pm UTC](https://discourse.julialang.org/t/possibility-of-local-import-statements-in-future/10611/44 "2018-04-30T22:40:15Z")

</div>

I don’t see what’s the difference between a macro and a method here. It’s not just `FileIO`, it’s **everything** or at least any abstract types. Custom number types, custom array types, etc. You just can’t require the caller of the function to know which module the method is going to come from. I do believe adding namespace lookup based on type in certain cases is good, however, it shouldn’t change how method/function/dispatch works.

---

<div class="post-metadata">

**Author:** ![TsurHerman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tsurherman/32/1234_2.png) [@TsurHerman](https://discourse.julialang.org/u/TsurHerman)\
**Post date:** [May 1, 2018, 7:55am UTC](https://discourse.julialang.org/t/possibility-of-local-import-statements-in-future/10611/45 "2018-05-01T07:55:04Z")

</div>

What about the following inference rule, if M is a the calling module and B is the module in which the function being called is defined then any function call inside the called function is inferred first based on the local  
scope (module B) as defined in my original post, and then based on the calling module scope(given that it is exported in the calling module).

The most concrete match is chosen, if there are 2 matches with have the same concreteness then the one from  
module B(the function location) is chosen.

Module Main will be special in the sense that any function in Main is considered exported.

I believe this change in architecture is not far fetched or a drastic changed compared to what is going on with inference today.  
I just believe it is better and clearer from the programmer perspective. giving some encapsulation to multiple dispatch

@jeff.bezanson @StefanKarpinski , If you have a few minutes spare I would highly appriciate your opinion on the matter.  
I appologize if this idea is not perfectly articulated … and I would understand if you choose not to comment.

---

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [May 1, 2018, 12:44pm UTC](https://discourse.julialang.org/t/possibility-of-local-import-statements-in-future/10611/46 "2018-05-01T12:44:07Z")

</div>

1. It’s not inference, but dispatch.
2. There’s nothing special about the caller’s module. Your call could be a few levels of function calls deep and easily have many modules in between.
3. Making functions behavior sensitive to the callers’ scope will both make it impossible to pre-compile and very hard to understand.

---

<div class="post-metadata">

**Author:** ![chakravala](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chakravala/32/6832_2.png) [@chakravala](https://discourse.julialang.org/u/chakravala)\
**Post date:** [May 1, 2018, 1:15pm UTC](https://discourse.julialang.org/t/possibility-of-local-import-statements-in-future/10611/47 "2018-05-01T13:15:25Z")

</div>

To update on what’s happening with the implementation, I made a comparison of load times using the original form of extending the `Base` functions and the new way of containing them locally. There is performance gain.

On the original [master](https://github.com/chakravala/Reduce.jl) branch, the compile cache is `2.5 MiB` and the load time is

```nohighlight
julia> @time using Reduce
Reduce (Free PSL version, revision 4588), 28-Apr-2018 ...
 11.211845 seconds (7.25 M allocations: 402.106 MiB, 3.00% gc time)

```

While on the [force-user](https://github.com/chakravala/Reduce.jl/tree/force-user) brach, the compile cache is `1.7 MiB` and the load time is

```nohighlight
julia> @time using Reduce
Reduce (Free PSL version, revision 4588), 28-Apr-2018 ...
  6.455135 seconds (3.57 M allocations: 191.129 MiB, 4.70% gc time)

```

This difference can be attributable to the fact that the package contains definitions for over 50 conflicting methods from `Base`, such as arithmetic, matrix, trigonometric, and other elementary / special functions.

With the new definitions, the original dispatch from `Base` is essentially reduced down to a single one that encapsulate all of the `Base` methods, plus the additional new ones defined by the package.

Now, the multiple-dispatch is handled completely differently, which is why the tests are failing. The multiple-dispatch had to be very finely tuned to be in harmony with `Base`, and there were no issues or conflicts due to the precise fine tuning of the dispatch. However, since a simpler dispatch is now used using the

```nohighlight
module Algebra
    export +
    +(r...) = Base.:+(r...)
end

```

using that form extension causes havoc on the multiple dispatch, since it was so finely tuned to work with the hundreds of existing methods from `Base`. This means that they fine tuning of dispatch has to be started over, since none of the dispatch structure from `Base` could be copied over to the new module.

---

<div class="post-metadata">

**Author:** ![TsurHerman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tsurherman/32/1234_2.png) [@TsurHerman](https://discourse.julialang.org/u/TsurHerman)\
**Post date:** [May 1, 2018, 6:36pm UTC](https://discourse.julialang.org/t/possibility-of-local-import-statements-in-future/10611/48 "2018-05-01T18:36:00Z")

</div>

I disagree, the more I run it my head it is clearer to me that this is the right approach ,  
Modules will be more encapsulated , importing packages will not alter the inner workings of other packages . It is easier to understand what gets export d and extended .

The conceptual difference is that dispatch is rooted at the current module and not in a common ancestor.

Actually this makes modules easier to compile (to binary not just into IR) because only exported functions will be “dynamic” whereas today as with the example I gave before I can import a function that will crush the whole system in an instant.

About the few layers deep of function calls … I fail to see how this is different or more complicated than what’s currently happening.

Maybe this idea is to advanced for you at the moment…

---

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [May 1, 2018, 7:19pm UTC](https://discourse.julialang.org/t/possibility-of-local-import-statements-in-future/10611/49 "2018-05-01T19:19:26Z")

</div>

> [@TsurHerman](#):
>
> importing packages will not alter the inner workings of other packages

This is not very well defined and it’s the biggest problem. See below.

> [@TsurHerman](#):
>
> Actually this makes modules easier to compile

Right. But the point is that this makes everything useless. You cannot create any abstract interface anymore.

> [@TsurHerman](#):
>
> Actually this makes modules easier to compile

It won’t unless you remove your special treatement of module. It’ll make compiling strictly worse, by a lot.

> [@TsurHerman](#):
>
> I can import a function that will crush the whole system in an instant.

There are very few general purpose systems that stops you from doing that in all possible ways, as long as you still want your program to be useful. As long as you don’t do things that are strongly recommended against (type piracy in this case), there won’t be a problem.

> [@TsurHerman](#):
>
> About the few layers deep of function calls

You were proposing making the caller module of a function special. However, the actual user of the function you want the callee to see may not be the caller of the function since there may be many layers of module in between. From what I can tell, you are saying that `f(a, b) = a + b` will behave differently if it’s implemented as `f(a, b) = f2(a, b); f2(a, b) = a + b` since the `+` in the second function won’t know which module `f` is being called anymore.

> [@TsurHerman](#):
>
> Maybe this idea is to advanced for you at the moment…

You are welcome to try a more advanced language or forum. If anything I talked about above is too advanced or at least too abstract for you, feel free to ask about the precise part that’s unclear.

---

<div class="post-metadata">

**Author:** ![TsurHerman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tsurherman/32/1234_2.png) [@TsurHerman](https://discourse.julialang.org/u/TsurHerman)\
**Post date:** [May 2, 2018, 6:02am UTC](https://discourse.julialang.org/t/possibility-of-local-import-statements-in-future/10611/50 "2018-05-02T06:02:45Z")

</div>

you made a lot of assumptions regarding what I meant … instead of joining into the mental exercise of listing the steps that make this possible.

And no I do suggest all functions will depend on the caller module , just exported function or better yet “interface” functions who get their dispatching inference rules from the calling module (like macros which get evaluated at dispatch time scope and not where their code was defined). All other functions have their dispatch rules according to the module they were defined.

- 

Do you have a self destruct example for Python? Ruby? Go? Rust? Something as simple as the one liner example I gave?

My wish is that instead of putting lots of effort of showing me were I am wrong … you would use your superior experience to join in and see what would actually make this a reality … not nessecearly a reality for Julia.

I personally think Julia is a brilliant concept , but it does not mean that a better and more concise way of doing things doesn’t not exist.

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [May 2, 2018, 6:14am UTC](https://discourse.julialang.org/t/possibility-of-local-import-statements-in-future/10611/51 "2018-05-02T06:14:24Z")

</div>

> [@TsurHerman](#):
>
> My wish is that instead of putting lots of effort of showing me were I am wrong … you would use your superior experience to join in and see what would actually make this a reality … not nessecearly a reality for Julia.

With respect, I don’t think this is how it works in the open source world. Specifically, inviting other people to use their time, effort, and expertise to work out the implementation for something that _you_ are interested in but they are explicitly skeptical about is unlikely to lead to any results. Especially after suggesting

> [@TsurHerman](#):
>
> Maybe this idea is to advanced for you at the moment…

---

<div class="post-metadata">

**Author:** ![TsurHerman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tsurherman/32/1234_2.png) [@TsurHerman](https://discourse.julialang.org/u/TsurHerman)\
**Post date:** [May 2, 2018, 6:58am UTC](https://discourse.julialang.org/t/possibility-of-local-import-statements-in-future/10611/52 "2018-05-02T06:58:39Z")

</div>

This post was temporarily hidden by the community for possibly being off-topic, unfocused, inappropriate, or spammy.

---

<div class="post-metadata">

**Author:** ![TsurHerman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tsurherman/32/1234_2.png) [@TsurHerman](https://discourse.julialang.org/u/TsurHerman)\
**Post date:** [May 2, 2018, 7:18am UTC](https://discourse.julialang.org/t/possibility-of-local-import-statements-in-future/10611/53 "2018-05-02T07:18:59Z")

</div>

More over , it seems to me that sometime in the past there was an actual flash of light coming out of Julia’s behind , some are still blinded by it mistaking it for the sun…

> [@TsurHerman](#):
>
> module SelfDestruct  
> import Base.+  
> (+)(s::Int64,t::Int64) = “muhahahaha”  
> end

This is the most alarming thing about the current implementation…

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [May 2, 2018, 8:38am UTC](https://discourse.julialang.org/t/possibility-of-local-import-statements-in-future/10611/54 "2018-05-02T08:38:15Z")

</div>

Why is this alarming? That it is possible at all? You are running arbitrary code, so that it is possible to crash the program is not very surprising.

The fact that this is possible at all is a powerful feature of Julia that allows many cool things to be done. I would be sad to see that go.

---

<div class="post-metadata">

**Author:** ![TsurHerman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tsurherman/32/1234_2.png) [@TsurHerman](https://discourse.julialang.org/u/TsurHerman)\
**Post date:** [May 2, 2018, 10:43am UTC](https://discourse.julialang.org/t/possibility-of-local-import-statements-in-future/10611/55 "2018-05-02T10:43:12Z")

</div>

I agree that the language does cool things It would be even cooler to be able to do all of those in a way that is not so breakable.

I think multiple dispatch can be achieved in a way that keeps the language more or less as it is while offering safety and guarantees that if a function f in module M was compiled for some concrete types then… as long as you don’t change M or its imported packages … that binary holds.

What a I suggested is something along the lines of:  
Drop the concept of extending and importing functions

A) imported functions having the same name gets fused into one function with several methods.

B)  
resolve the right binary method to call in the current module … and in the calling frame … and in the calling frame of the calling frame … etc… all the way to the top and chose the most specialized and “near” implementation.

It will take me some time to formalize this …or go over the implications : I was hoping to go into discussion of … in order to do that you’ll need A B C etc  
which in the end may turn out to be too complicated or … better than the current way dispatch works.

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [May 2, 2018, 11:09am UTC](https://discourse.julialang.org/t/possibility-of-local-import-statements-in-future/10611/56 "2018-05-02T11:09:10Z")

</div>

> [@TsurHerman](#):
>
> in a way that is not so breakable.

Can you please clarify which feature of the language you have a problem with? The ability to write methods

1. for **functions** defined in another module,
2. for **functions and types** defined in other modules (not necessarily the same),
3. for **functions and types, all defined in the same module** ,
4. that **overwrite existing methods** , defined in another module?

Guidelines about [type piracy](https://docs.julialang.org/en/latest/manual/style-guide/#Avoid-type-piracy-1) are not formally enforced by the language, but catch (2) and (3), and also (4) if the other definition that would be overwritten avoids type piracy. OTOH, restricting (1) would kill a very useful feature.

---

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [May 2, 2018, 11:56am UTC](https://discourse.julialang.org/t/possibility-of-local-import-statements-in-future/10611/57 "2018-05-02T11:56:14Z")

</div>

> [@TsurHerman](#):
>
> you made a lot of assumptions regarding what I meant … instead of joining into the mental exercise of listing the steps that make this possible.

AFAICT, what I’ve stated does not contradict with what you’ve said so all the assumptions that I (and everyone  
reading what you’ve said must) made are everything that you failed to mention. In another word, it’s impossible to understand your full proposal without those assumptions.  
As an example, I don’t believe you’ve mentioned the relation between exported function and the sensitivity to calling modules until this reply so there’s no way I could have guess that. (Not that this assumption actually matter though, see below.)

> [@TsurHerman](#):
>
> My wish is that instead of putting lots of effort of showing me were I am wrong … you would use your superior experience to join in and see what would actually make this a reality … not nessecearly a reality for Julia.
> 
> I personally think Julia is a brilliant concept , but it does not mean that a better and more concise way of doing things doesn’t not exist.

By all mean, the **ONLY** way to push such an idea forward is to identify why it won’t work and then how it can be fixed. Just asking others to admire what you’ve proposed will not lead to anywhere. For a big change like what you’ve proposed, it’s very important to figure out what impact does it have. In particular, how does it affect [usecases that we must support](https://discourse.julialang.org/t/possibility-of-local-import-statements-in-future/10611/56). Identifying how you want to support these usecases and avoiding seemingly obvious issues is also one of the best ways to understand all the non-obvious detail of your proposal.

> [@TsurHerman](#):
>
> And no I do suggest all functions will depend on the caller module , just exported function or better yet “interface” functions who get their dispatching inference rules from the calling module (like macros which get evaluated at dispatch time scope and not where their code was defined). All other functions have their dispatch rules according to the module they were defined.

As far as what I can tell, this does not make a difference and all what I’ve said about my `f` and `f2` still apply. In this case, `f` is the exported function and `f2` is “all other functions”. This basically forbid one from using these “all other functions”.

> [@TsurHerman](#):
>
> Do you have a self destruct example for Python? Ruby? Go? Rust? Something as simple as the one liner example I gave?

What you have actually does **NOT** crash julia. What it crashes is the REPL. All core components are actually well protected by that.

---

<div class="post-metadata">

**Author:** ![TsurHerman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tsurherman/32/1234_2.png) [@TsurHerman](https://discourse.julialang.org/u/TsurHerman)\
**Post date:** [May 2, 2018, 12:14pm UTC](https://discourse.julialang.org/t/possibility-of-local-import-statements-in-future/10611/58 "2018-05-02T12:14:35Z")

</div>

> [@Tamas\_Papp](#):
>
> restricting (1) would kill a very useful feature.

This is exactly my point, if we handle dispatch differently you could be restricting (1) without loosing  
what is (positively)enabled by it.

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [May 2, 2018, 12:22pm UTC](https://discourse.julialang.org/t/possibility-of-local-import-statements-in-future/10611/59 "2018-05-02T12:22:04Z")

</div>

I am afraid I don’t see the details of how your proposed mechanism would work. Perhaps you could open a [Julep](https://github.com/JuliaLang/Juleps) which describes it in detail, considering its impact on all current use cases.

---

<div class="post-metadata">

**Author:** ![TsurHerman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tsurherman/32/1234_2.png) [@TsurHerman](https://discourse.julialang.org/u/TsurHerman)\
**Post date:** [May 2, 2018, 2:01pm UTC](https://discourse.julialang.org/t/possibility-of-local-import-statements-in-future/10611/60 "2018-05-02T14:01:49Z")

</div>

> [@yuyichao](#):
>
> ust asking others to admire what you’ve proposed will not lead to anywhere

I am not in it for admiration, but for open mindedness ,learning ,the joy of solving abstract stuff together.  
And you disappoint me because you are obviously capable and probably better at this stuff (with respect to Julia internals) than me, and yet you too easily regard any idea that is different than the status quo as being wrong.

Anyway I did come up to a very very preliminary synopsis of the idea , eventually, in this discussion.

it goes like this:

> [@TsurHerman](#):
>
> Functions will be named meta containers bound to a module, it will contain all methods (functions with the same name
> 
> but different signature in the arguments) defined inside the module and a list of all modules that exported the same function name and were imported into the current module

dispatch can be defined recursively as follows: for a function F who is being called from function G  
resolve the most appropriate method to call in the `Module` containing F, and also the most appropriate method  
in the `Module` containing G if and only if F is exported (not sure this restriction is needed…) ,choose the most specialised method.  
if there is no G then selection is trivial.

```julia

module IterInterface
    start(iter) = throw("unimplemented")
    next(iter, state) = throw("unimplemented")
    done(iter,state) = throw("unimplemented")

    export start,next,done #actual definition of the interface is redundant
end

module ForLoop
    using IterInterface #for formality .. not really needed

    forloop(f::Function,iter) = begin
        state = start(iter)
        while !done(iter,state)
            elem,state = next(iter,state)
            f(elem)
        end
    end
    export forloop
end

module Foos
    struct Foo
        v
    end
    start(iter::Foo) = false
    next(iter::Foo, state) = (iter.v,true)
    done(iter::Foo, state) = state

    export Foo,start,next,done
end

module Main
    using Foos
    using ForLoop

    forloop(println,Foo(10))
end

```

in this example the start(iter) method called from forloop which is called from Module Main will resolve correctly.

[Previous page](https://discourse.julialang.org/t/possibility-of-local-import-statements-in-future/10611.md?page=2)

[Next page](https://discourse.julialang.org/t/possibility-of-local-import-statements-in-future/10611.md?page=4)