Possibility of `local import` statements in future?

[yuyichao]

@TsurHerman

if forloop2 is something that needs to be extended (like IndexingStyle etc) then yes it needs to be exported.

Base is pretty flat structure , I think you’ll find a lot of “part of the implementation” functions exported anyway

I assume by “needs to be extended” you don’t me extending the function in julia sense but seeing user definitions. In that case “part of the implementation” functions are basically every functions and they are clearly not all exported. As an example, just look for all methods starts with _collect here (there are two functions and many methods). They all need to see the user defined iterator protocol.
Or really, just look for any function in base, they (edit: almost) all need to see user defined methods.

Also, according to what I understand your proposal to be, exporting the function doesn’t really affect what the function see when called from the same module. Even if forloop2 is exported, as long as it is called from forloop and not by the user directly, it still can’t see the scope of the caller of forloop.
That is, unless by caller you don’t mean caller but the toplevel scope (current_module() on 0.6) or the first caller from a different module or something in between. Each of them has their version of the same problem and if you indeed me one of those, you should clarify so that I can be more specific. Listing them all here will be too long… (edit: though I just realized that I inevitably listed most of them below…)

@TsurHerman

You can have one module defining graph types and two modules defining the implementation of the iterator interface differently and you could have:

OK, so as a more concrete example of what I mean by the problem when you have multiple modules involved in a way similar to what you’ve listed here.

# All modules are using all other modules that the module is aware of so no new using/import can be added.
module IfaceDefAndUse
# This module cannot be aware of any other modules
export g, f
function g end
f(a, b) = g(a, b)
end

module WrapperOfIfaceUse
using IfaceDefAndUse
export k
k(a, b) = IfaceDefAndUse.f(a, b)
end

module IfaceImpl
using IfaceDefAndUse
export T, f
struct T end
IfaceDefAndUse.f(::T, ::T) = 1
end

module Library
using IfaceDefAndUse
using WrapperOfIfaceUse
using IfaceImpl

export l()
l() = WrapperOfIfaceUse.k(IfaceImpl.T(), IfaceImpl.T())
end

module User
using IfaceDefAndUse
using WrapperOfIfaceUse
using IfaceImpl
using Library
Library.l()
end

Now, how can IfaceDefAndUse.f(a, b) know which g to call? The direct caller of IfaceDefAndUse.f is WrapperOfIfaceUse.k, which isn’t directly aware of IfaceImpl. It’s caller, Library is, but there isn’t any reason to pick this module to use unless you want to include all modules in the call chain. User as it is now, is also aware of IfaceImpl and it is the toplevel caller so there’s some reason to treat it in a special way, but there’s no reason User need to using IfaceImpl (in practice, there’s a lot of reason you don’t want to using a dependency used by a module you use) and if you don’t, you can’t pick it up from here either.

So for this usecase, I can see this working only if you make IfaceDefAndUse aware of all modules in the call chain, or making it aware of the toplevel module and require the toplevel module to import all modules. Either way though, it’ll require the function to be aware of a lot of other modules/context in this particular call chain and this is exactly what I mean by making the compiler problem worse.

@TsurHerman

s that this method guarantees that for a method f in module M and arguments args the binary code will not change as long as you don’t change (f ,M , args) , caching is clear … no type piracy.

Except that M is a long list of modules making the compilation result not reusable at all. Type piracy is also not really a compiler problem, it’s a code organization problem. Compiler is totally fine with it. Only the user will be surprised.

@TsurHerman

there can be true incremental precompilation in the sense that if module A and module B both use StaticArrays then the binary/IR code of a functions in A and B involving StaticArrays … belong to the cache of A and B and not StaticArrays.

This is really solving a non-problem. Being able to incrementally precompile isn’t even a problem. The usefulness of the result is. You can always do compilation and nothing can stop you from doing that. The only question is how useful the compilation result is for future use (i.e. the reusability of it). Doing compilation without maximumly reusing the result will cause strictly worse performance.
What you are suggesting here is basically that by making things more context dependent there are a lot more to compile and by construct those compilation are more independent (there’ll be much less sharing). That’s exactly how you make it less compiler friendly and kill the performance.