Repeated compilation for same function with extreme type instability

General Usage
1

Here is a minimal reproducible example: Repeated compilation for same function with extreme type instability - #5 by guoyongzhi

My function consistently experiences prolonged compilation times with each execution. However, upon @snoopr inspection, I consistently receive empty invalidations. What steps should I take to address this issue?

using WordCloud
using SnoopCompileCore
using SnoopCompile
wordcloud("aa bb cc dd")
invalidations = @snoopr wordcloud("aa bb cc dd")
@show length(uinvalidated(invalidations))
@time wordcloud("aa bb cc dd")

length(uinvalidated(invalidations)) = 0
0.669331 seconds (438.85 k allocations: 30.645 MiB, 75.80% compilation time: 4% of which was recompilation)

2

I think this issue might be related to unstable types, as I’m certain the problem lies within function WordCloud.getstylescheme, where there are some types determined by random numbers.

using WordCloud
WordCloud.getstylescheme(["a"], [1]);
@time WordCloud.getstylescheme(["a"], [1]);

0.181821 seconds (204.89 k allocations: 13.782 MiB, 98.69% compilation time)

Are there any inspection tools in Julia for this kind of issues?
Is there a way in Julia to mark a dynamic function to be executed by interpretation instead of time consuming re-compilation?

3

Maybe you can try @nospecialize:

https://docs.julialang.org/en/v1/base/base/#Base.@nospecialize

4

The random number is generated within the function body, and since the function can accept zero arguments, the @nospecialize decorator can not help me. Thank you regardless.

5

Thankfully, I have created a minimal reproducible example:

f1() = rand() > 0.5 ? 1 : rand()
f2() = rand() > 0.5 ? 1 : rand()
f3() = rand() > 0.5 ? 1 : rand()
f4() = rand() > 0.5 ? 1 : rand()
f5() = rand() > 0.5 ? 1 : rand()
f6() = rand() > 0.5 ? 1 : rand()

ff() = f1() + f2() + f3() + f4() + f5() + f6()

@time ff()

0.007261 seconds (7.20 k allocations: 485.763 KiB, 98.70% compilation time)

@time (f1(), f2(), f3(), f4(), f5(), f6())

0.000019 seconds (6 allocations: 240 bytes)

Is there a way in Julia to mark ff() to be executed by interpretation instead of time consuming re-compilation?

2 Likes
6

I found that: GitHub - JuliaDebug/JuliaInterpreter.jl: Interpreter for Julia code

7

I am unsure whether I understand your problem. Let me try to summarize what I got:
You have some function WordCloud.wordcloud and this is slow, because everytime you call it something gets compiled. You think this is due to some type-instability which is caused by randomness and provide an example showcasing that.

The compilations in your example stem indeed from type-instability. In this case, I think it is because a+b+c+d is transformed into +(a,b,c,d) and so a new version of this multi-argument + needs to be compiled for every new combination. This hypothesis can be checked by putting in parenthesis:

ff() = (f1() + f2()) + ((f3() + f4()) + (f5() + f6()))

Here all + are 2-argument and it does not show any compilations.

I also ran wordcloud a couple of times and noticed that the style or something is randomized. So I think the compilations stem from the fact that each run generates a combination of types passed to another function that was not encountered before. So a new method needs to be compiled for that type-combination. Note that after a couple of runs, new combinations become rarer and often times no compilation happens.
To reduce the compilations, you’ll need to reduce the number of type combinations. I don’t know how much influence you have on that, since I don’t know at what level these happen in the code and which level you can influence. To reduce the number of type combinations there are a couple of ways to achieve this:

  1. break up functions to reduce the combinatorics, e.g. split (if possible) f(randtypeA, randtypeB) into f1(randtypeA) and f2(randtypeB)
  2. Reduce types e.g. by removing parameters on parametric types - this introduces type-instabilities but reduces the compilation times
  3. Instead of the above, use SumTypes.jl (or DynamicsSumTypes.jl) to “pack” multiple “subtypes” into a single type.
1 Like
8

Thanks for your nice reply. You’ve understood my problem precisely. Your advice to reduce the number of type combinations is very inspiring. Let me elaborate further on the context.

My type-unstable function acts as a random scheme generator (corresponding to ff()). It’s not performance-critical and is naturally written (and likely to be run) in a Pythonic way. For instance, the color generator (corresponding to f1()) might return either "black", 0.7, (255,255,255) or a Color object. Similarly, the picture size generator (corresponding to f2()) might return 512, (512, 768), 0.5, or :original.

I believe such scenarios are not uncommon, so I’m wondering if there’s a way to instruct Julia to operate in an interpreted mode for a specific function. We already have @nospecialize after all. I came across JuliaInterpreter.@interpret. It works, but it seems not tailored for this purpose and is quite slow.

1 Like
9

While this is not possible to my knowledge, maybe it would be an idea to reduce the optimization level. This can either be done globally by starting Julia with -O0 or -O1 or even --compilation=min (this effectively turns Julia into an interpreter).
If your program does not do much more than the plotting, then one of these might be the easiest option.

Alternatively there is Base.Experimental.@optlevel where you can set the optimization level for a module (but as the name suggests this is a experimental setting). As this is on a per-module basis, you’d need to separate out the parts that should get less optimization into their own submodule.

2 Likes
10

There is no all-powerful programming language.

11

…yet :laughing:

12

It’s still just an experimental interface, but there’s Base.Experimental.@compiler_options. Try something like:

module InterpretedModule

Base.Experimental.@compiler_options optimize=0 compile=min infer=no max_methods=1

...

end
4 Likes