Trouble understanding how scopes work when using @testset

Hello!

This is a continuation of the discussion here: https://github.com/JuliaLang/julia/issues/32453

You see this really bothers me. The guy who answered me stated I probably have a deep misunderstanding of Julia and that is probably true.

English is not my native tongue so please bear with me

I have attached his first response below:

What do you mean by scope? Are you talking about variable scope? If yes, then it’s impossible since that concept does not exist when the macro is called yet.
It seems that you assume every scope is a module (or something module-like) and you can access variables in that scope via fields. Such things does not exist in julia by design. Your macro is also doing evaluation at the macro expansion time rather than runtime which is wrong if you want to access local variables in the calling scope. Please read the macro hygene section in the document and ask any further questions on

What I do not understand here is his notation of variables and local scope and how my issue would relate to hygienic macros?

For instance, if I did not have @testset my toy code linked in the issue. It would run. Big thanks to Fredrik post @testset scoping I manage to make my client macro work pseudojulia below:

module fooScope
include("./minEx.jl")
import .myMod
function bar()
end
using Test
@testset begin #This works fine without @testset
include("someotherfile.jl")
end
end
#someotherfile.jl
module somemodule
#assume we import the package here as well 
function foo()
  0
end
 println(myMod.@client)
end

Using code similar to the code above my macro would indeed be able to find foo(). However, the Julia developer who responded to me claimed that I can never find foo during macro expansion time. Which after some modification this code clearly does. I have only worked on Julia for about a month but I am very curious about the different use cases in this scenario. Since it confused me a lot when I encountered it

Sorry for a long post

All the best

John

Hi John, perhaps you might explain a bit about what you’re trying to do in a larger sense. Typically, tests are not used inside a module. That is, not between module Foo and end. Packages typically have a tests/ folder that contains a file or files with eg

using Foo
using Tests

@testset "test foo" begin
#...
end

Perhaps explaining why you’re trying to use @testset inside a module, or what you’re trying to accomplish more generally will help people understand where your confusion is coming from.

2 Likes

Your macro looks up the global foo in the calling module at macro expansion time. If you put the function definition in a test set then it is local to that test set, so there is no global foo to get, which I’m guessing is the error you got. (You didn’t report the error, so I don’t know what it was—this is generally helpful to have.)

That’s the problem you’re having, but there’s also the issue that you shouldn’t be looking at global state at all during macro expansion. Macros should be a pure syntax transformation: they transform input syntax into output syntax, that’s all.

Hopefully that helps explain what’s going on and what @yuyichao was talking about. It’s generally better to ask about confusing things like this on Discourse before concluding that it must be a bug. There certainly are sometimes bugs in macro expansion and scope but chances are that’s not what it is.

2 Likes

Thank you so much Stefan!

Yup, that is the idea. However, should I even be able to do that? According to @yuyichao what I did should not work at all, maybe I missunderstood? So I thought that might be some kind of a bug

From the response I got it sounded like whatever I was doing was insane :sweat_smile:

I know that. However, I am writing a really ugly set of hackish macros out of necessity to make functions inherit each other thus I need my macro to be able to call methods and code_lowered at expansion time. Compile time reflection. Currently, it does work but the scoping of @testset confused me since my tests were broken but my “real” code did work as expected. I resolved it by having modules in testset which does work with the approach above which is for me a bit confusing. I tried to state that on my github issue. What confused me more was that when I lowered the IR it did look like some lookup occurs under the hood.

In a sense yes. However, I still do not get what macro hygiene has to do with anything here?

Thanks again Stefan!

@kevbonham

Thank you for your time Kevin, see the response I sent to Stefan. I am at a trainstation at the present time so my connection is a bit messed up

We’re not big on disallowing things. How would one even do that? You’re evaluating normal Julia code—there aren’t separate macro expansion semantics—you have a module object, so you can look things up in it. Disallowing that would be a complex feature. Worse still, we’d need an even more complex feature on top of that to let people opt into doing it anyway if that’s what they really want.

However, I am writing a really ugly set of hackish macros out of necessity to make functions inherit each other thus I need my macro to be able to call methods and code_lowered at expansion time. Compile time reflection.

I don’t understand this well enough to give advice other than to say it sounds brittle. Perhaps you could, as Kevin suggested, explain what you’re trying to accomplish at a higher level?

However, I still do not get what macro hygiene has to do with anything here?

Depends on what you consider macro hygiene. Evaluating globals during macro expansion isn’t strictly hygiene in the traditional sense, but it’s definitely a way to write macros that work in some places and break in others.

3 Likes

Now my tests seem to run https://github.com/JKRT/MetaModelica.jl/blob/master/src/functionInheritance.jl comments might be a tad bit outdated since this is still more or less WIP. However, it showcases the use case of what I am currently working on, it was a bit to much code to add as an issue so I tried to illustrate it with a minimal example. Look at the difference between the last commit and the previous commit in my test directory. Then you can see my issue exactly. It works for the use case I have. However, it is not pretty. Still, from your answer, I guess the response on my issue was incorrect?

The code I have linked does things that goes against the advice I was given on my issue

There’s absolutely no way you can get accessed to foo during macro expansion time. It’s not even created when your macro is called.

Logically, almost all the confusion you got comes from over generalizing everything. Things “work” under specific constraints and tweaks on those constraints that seem minor for you can be (and are) fundamantal.

All what I’m saying is that the enclosing local scope is not something you could access at macro expansion time by design. It is related to micro hygene since that’s exactly about how to access variables in the macro generated code in the correct scope, i.e. it is the section that talks about how to “access the enclosing local scope” at macro run time (i.e. in the code generated by the macro). In this case, you must not generalize something you could do for global scope to something you could do to local scope. And that’s not just an API limitation, it’s fundamantal in so many ways.

Again, don’t overgeneralize what I said and also please don’t make major change to the code between the two posts that invalidate my post. The code I was replying to, the only foo from the short github thread, was,

using Test
@testset begin #This works fine without @testset
function foo()
  0
end
 println(myMod.@client)
end

If you haven’t realized that yet, moving something from inside an include to directly at the calling site completely changed the scope it runs in (there are many threads on this and I’ll perfer not talking about it in this thread). This might be a fundamantal change in the constraints that you didn’t realize. And as for why I said you can’t access foo in this case (i.e. the original case on github), that is still absolutely true. But that is only about when you define it in a local scope (@testset scope) for the reason that “It’s not even created when your macro is called.” as I said in the github post. More on that below.

So now about two things you definitely need to learn that I didn’t want to post on github.

  • First thing is the one you constantly misinterpreting what I said about foo and looking for all signs/execuses/chances to call me wrong on it.

    Again, all what I said is the foo on github, a function defined in a local scope, is impossible to access during macro expansion time. There’s absolutely no comment from other people you’ve seen in this thread that conflicts that and there won’t be any from anyone that knows what they are saying…

    And now there’s the why.

    First, macros are transformations on the code. Only after macro expansion of the code it could be run. This is done for each global/toplevel expression. This is the smallest unit one can run the code/interleave macro expansion and execution. The macro expansion result could alter how a global statement could be run so it cannot be done before the macro expansion finishes. What this means is that if you have

    begin
        foo() = ...
        @your_macro
    end
    

    @your_macro is called before any part of this block of code is evaluated. I hope it’s easy to understand that if no part of this code has run, the foo() = ... would not have been evaluated, and there’s no foo for you to use anywhere at that time.

    Next, local scope. You cannot access a local scope externally. This is fundamental to the design that worth a separate thread if you have question about it. There are certainly some internal data structure you can use to do something close in certain conditions but they are implementation details and messing with them comes at your own risk both for crashing and for breaking your code. Local scope also means that things in it does not exist until the scope is created, which is hopefully easy to see. Of course the object might exist if they don’t need to be created but by all mean there’s no way to access them with the name in that scope.

  • The second one is about what you thin you saw in the following code you posted on github:

    julia> code_lowered(bar)
    1-element Array{Core.CodeInfo,1}:
     CodeInfo(
    1 ─      barbar = %new(Main.:(#barbar#10))
    │   %2 = (barbar)()
    └──      return %2
    )
    

    I assume you saw the Main.:(#barbar#10) and thought

    So it does seems that at least nested functions have the declaration as some kind of field.

    Well, Main.:(#barbar#10) is the type of the function. It is not the function itself. barbar, i.e. %new(Main.:(#barbar#10)) is the function. In general you cannot go from the type directly to the value. Even though it actually is in this case since the function is not a closure (it doesn’t enclose/capture any local variable) that type is not yet defined when your macro is called and just the name is not even enough to get the type you need.

And now for what you should/could do for your “real code”. If you just claim it works by accessing the module at macro expansion time and it’s only the test issue then you have the following constraints.

  1. Your macro can only access global functions. There’s no way you can access local functions.
  2. The function defined before your macro is called. Since that’s when you are doing the work, there’s absolutely no way around it.

And your fix would be simply to NOT define any local function to use in the test. In another word, move your foo out of the @testset and all enclosing top-level code blocks and you’ll be fine.

The only way to get around these liminations is to not do the work at macro expansion time. You need to figure out a way to do it at runtime. Depending on what you want to do with the function, this may or may not be possible.

Finally, just because a package that does something like this exists does not mean it should be done or is the right thing to do or is the right way to do it. Please don’t use that as an argument for anything. There are so much code out there that has caused trouble that I really wish would not be there (to be fair, this one seems to be pretty mild on the overall scale). In this case, just because it works in some cases does not mean it should work in other cases. Thare are some liminations put in by design and you should not try to do certain things (reflection for example) at the wrong time.

2 Likes

I have not done any editing on github between what I posted the link is provided in the original post?
I am not out to get you in any sort of way.

I was confused because I had done exactly what you wrote I could not do. Accessing variables during expansion, those that are known statically atleast and from my example, in my first post I wrote about that the variable was in a new scope of @testset and not the model scope, I just wanted to find away around that

Yup. That was what I thought and as you state it is what happens! However, it does do the lookup for the declaration which you admit it to do

Please do not make this a discussion about right or wrong you are clearly one of the top Julia devs and a very experienced develope much more so then I. I do not question your competence on Julia. I interpreted what you wrote regarding foo literally, and from your tone I assumed I was doing something very wrong. That is why I became confused and brought the discussion here in the first place. I am very interested in Julia as a language but this behaviour did confuse me.

I do not quite see what I have done to be flamed/patronised. As I stated I do not much about Julia

I said don’t make major change to the code. I said you cannot access foo in the specific code you post on github (as in there will be no changes you can hope for to access it). But here you posted completely different code and then question why you could access foo in that code. I made no comment on the new code you refer to since it wasn’t even posted when I made the original comment and of course what I said about the original code doesn’t apply to the new one.

I have no doubt that you did not intentionally come out to get me. However, as I put in the very first sentence, you did not pay any attention to the condition of, well, many things (code, statements, etc) which is why you are confusing yourself and it’s hard for other’s to clear it up.

Errr, it does a lookup. It does not do the lookup that you are thinking about. In particular, it does not use a lookup to find the function or in another word, you cannot use a lookup to find the function. Just because there is a related lookup in the code does not mean it’s the lookup you think you are looking for. The lookup you are looking for does not exist and I did not admit its existance.

The point is that you must interpret what I said more literally, especially regarding the context.

I assume you want to access the foo in the @testset begin function foo() end; .... end case, the original code on github. That’s what you asked about in the github issue and what you are doing is indeed very wrong. If my tone gets stronger and stronger that’s because you have to understand this before the discussion can even proceed to the why part.

Your confusing is fine. But please read people’s reply more carefuly and try to understand them first. If there’s anything that bother’s me from the very beginning about this all the way to your last reply is that it seems that you still don’t see that there’s very fundamental difference between the following two cases.

  1. what you posted on github:

    @testset begin #This works fine without @testset
    function foo()
      0
    end
     println(myMod.@client)
    end
    
  2. And the code you posted above.

    module somemodule
    #assume we import the package here as well 
    function foo()
      0
    end
     println(myMod.@client)
    end
    

Please first accept that there’s a fundamental difference between the two cases regarding the relation betwen the function foo() and the myMod.@client. That should clear up/explain why what you observe appears to be conflicting with what I said.

After that, you can read my explaination about the difference tthat I posted above and I’m happy to explain more if you have more questions about it.

2 Likes