Optional macro invocation

Yes, I was leaning towards a macro myself.

If this is the only way to go, I guess it should be something at one level of abstraction higher, to pass the macro, the flag, and the expression (so that it will work with @async but also with @time and others).

And again, I guess one of the issues will be that macros can’t be passed around (referenced) as arguments (or at least I don’t know how).

Yeah. I thought the same thing about a more general macro. This is probably already implemented in a few utility files scattered around github…

macro maybemacro(asyncflag, themacro, expr)
    mccall = Expr(:macrocall, Symbol("@", themacro), LineNumberNode(@__LINE__), expr)
    quote
        if $(esc(asyncflag))
            $mccall
        else
            $(esc(expr))
        end
    end
end
julia> @maybemacro true async cos(1)
Task (done) @0x00007f115df46980

julia> @maybemacro false async cos(1)
0.5403023058681398

julia> @maybemacro true time cos(1)
  0.000002 seconds (5 allocations: 176 bytes)
0.5403023058681398

julia> @maybemacro false time cos(1)
0.5403023058681398
2 Likes

This is where it’d be nice to have first-class macros like first-class functions.

While others have given nice solutions with macros, frankly I would just use a closure, possibly with some syntactic help like

doit(f, ::Val{true}) = @async f()
doit(f, ::Val{false}) = f()

doit(async_flag) do 
    # stuff goes here
end
2 Likes

Thanks so much, @jlapeyre and @Tamas_Papp - I’ll give the two approaches a try today, to see how they work.

They both look great although in the general case I believe that the macro approach could be more efficient as it won’t evaluate the input on every function call. Similar to https://docs.julialang.org/en/v1/manual/metaprogramming/index.html#Hold-up:-why-macros?-1

@Mason first-class macros would be awesome. It looks like lisp-style macros could be “upgraded” to first-class macros. I’d love to see that in Julia.

I may be missing something, but if you branch (either via methods or an if) then it should be called only once.

Sorry, I don’t think I properly articulated what was going through my head.

I have these global (environment) variables which are set upon application initialisation and don’t change throughout the lifetime of the script. Things like DEBUG = true | false, ENV = dev | test | prod, etc. These affect whether or not subsequent macros like @time, @async, etc are used (ie if ENV == dev then many statements are prefixed by the @time macro; but in prod they are not).

I was thinking that with macros I can remove the IF evaluation altogether and just use the corresponding branch throughout the lifetime of the script. So the IF/ELSE is evaluated at parse time and the resulting expression is used from that point on. Similar to the parse time vs run time example here: https://docs.julialang.org/en/v1/manual/metaprogramming/index.html#Hold-up:-why-macros?-1

If you use a global const, the compiler should just eliminate the dead branches for you costlessly. Especially if you dispatch on type.

2 Likes

Constant propagation seems to be already enough for an inlineable function, as long as the global is a const:

doit(f, bool) = bool ? f() : f() + 22
const async_flag = true
function test()
  doit(()-> 22, async_flag)
end
@code_warntype test()
Body::Int64
163 1 ─     return 22 
1 Like

Interesting - wondering how does that work, given that Julia allows changing the value of the const? I presume the function will be recompiled when that happens?

Oh, yes, that’s a good point. They’re not actually constants but fields in a config object. I could se tup some key constants, like const DEBUG = config.debug. Or maybe just refactor the config type to be immutable - that should have a similar effect as the constants, I expect.

No, that’s exactly why you get a warning when redefining a const - because you need to manually recompile a function to see any new const!

2 Likes

I am not sure (though it may not be “undefined” and just happens to work below).

julia> VERSION
v"1.2.0-DEV.28"

julia> const A = 1
1

julia> f() = A
f (generic function with 1 method)

julia> const A = 2
WARNING: redefining constant A
2

julia> f()
2

No:

julia> const A = 1
1

julia> f() = A
f (generic function with 1 method)

julia> f()
1

julia> const A = 2
WARNING: redefining constant A
2

julia> f()
1

as f only gets compiled when called.

2 Likes

That’s very interesting. Looks like a closure and quacks like a closure, doesn’t it?

1 Like

here’s a more efficient, in terms of code generation, macro to optionally apply another macro:

macro maybethread(flag, loop)
    if flag
        return quote Threads.@threads $loop; end
    else
        return quote $loop; end
    end
  end

thanks so much @jlapeyre for your example above.

@Tamas_Papp i did not see a way to optionally thread a loop with closures. would be curious about such an alternative solution if you have one.

EDIT: i realize now a huge limitation here is that flag has to be literally false or true and can NOT be a boolean variable :frowning:

2 Likes

I was actually recently trying to figure out this exact case of making @threads optional, based on a flag passed to a function. Would the macro you’re proposing be applicable for something like the following?

function func(args...; use_threads=false)
    @maybethread use_threads for i = 1:10
         do_something_with(i, args...)
    end
end

If not, is there a macro that would work for this kind of use case?

i don’t know how to pass in a variable without duplicating the loop code-gen. so instead, i’m now testing an alternative where a global use_threads is defined.

in my case, the loop is actually bit slower if i haven’t started julia with the -t flag, so i also check for that and skip the @threads macro if there is just one thread.

macro maybethread(loop)
  if use_threads && Threads.nthreads()>1
    return quote Threads.@threads $loop; end
  else
    return quote $loop; end
  end
end

I suppose one fundamental distinction is whether you want to make the threads optional at runtime vs at compile time. I think you (and the OP) were primarily looking for a compile-time switch, whereas I’m looking for a runtime switch.

Based on @jlapeyre’s answer, it occurred to me that maybe the easiest way to do this for @threads is to just modify the definition of the macro from the standard library. Indeed, the following seems like it works perfectly for me:

module ConditionalThreads

using Base.Threads
using Base.Threads: threadid, threading_run

macro threadsif(cond, loop)
    if !(isa(loop, Expr) && loop.head === :for)
        throw(ArgumentError("@threadsif requires a `for` loop expression"))
    end
    if !(loop.args[1] isa Expr && loop.args[1].head === :(=))
        throw(ArgumentError("nested outer loops are not currently supported by @threadsif"))
    end
    quote
        if $(esc(cond))
            $(Threads._threadsfor(loop.args[1], loop.args[2], :static))
        else
            $(esc(loop))
        end
    end
end

end

I’ve left out the schedule from the original implementation for conciseness since that’s currently limited to :static anyway. It wouldn’t be hard to add that back in.

Edit (usage example):

I currently have the above macro definition “in production” at QuantumControlBase.jl, and it is used e.g. here:

1 Like

version 2 of my maybethread macro above, which optionally threads at compile time, is now hygienic:

macro maybethread(loop)
  if use_threads
    quote Threads.@threads $(Expr(loop.head,
                             Expr(loop.args[1].head, esc.(loop.args[1].args)...),
                             esc(loop.args[2]))); end
  else
    # @warn "running single threaded"
    quote $(esc(loop)); end
  end
end

i typically define use_threads = Threads.nthreads()>1.

thanks @chakravala for your post which showed me how to do this.

figuring this out took a long time. maybe i’m stupid. maybe macros are hard. but i wholeheartedly agree with @Chris_Foster that we should rethink them for julia 2.0.