Memory leaks from compiled functions: Solved

I have been told about one known issue with Julia, compiled function doesn’t get garbage collected. So since I wanted to dynamically create functions for objects in my game engine, it was a big problem and I had to solve it since objects in a game are created and destroyed at an erratic rate.
So I thought about a way to solve that and it’s just to reuse functions.
Yeah cool, but I do we reuse it ?
Simple, we override methods of unused functions.
When we want to reuse a function, we just redefine it so that the old version is overrided by the new one and I made a little module FunctionPooling to automatically solve that.
The result substain my idea. I tried dynamically create 1000 functions.
here is the snippet:

include(joinpath("..","src","FunctionPooling.jl"))

using .FunctionPooling

pool = FunctionPool()

## Warming to avoid compilation overhead.
@pooledfunction pool function update(x::Int)
	x + 5
end

N = 1000 # Number of functions that will be created
total_memory = Sys.total_memory()

## Part 1: Standard Julia version
used_memory = (total_memory - Sys.free_memory())/(1024^2)

# We will generate `N` standard function and make sure they compile by calling them
for i in 1:N
	symb = gensym() # The name doesn't matter
	eval(quote function $symb(x::Int)
		x + 5
	end 
    $symb(1)
end)
end

# We call the GC so we free the unused memory.
GC.gc()

used_memory = (total_memory - Sys.free_memory())/(1024^2) - used_memory
println("Memory in use: $used_memory Mo")

## Part 2: Version with reused functions
GC.gc()
used_memory = (total_memory - Sys.free_memory())/(1024^2)

for i in 1:N
	symb = gensym()
	eval(quote @pooledfunction pool function $symb(x::Int)
		x + 5
	end
	$symb(1)
	free($pool,$symb)
end)
end

# We free the unused memory
GC.gc()
used_memory = (total_memory - Sys.free_memory())/(1024^2) - used_memory

println("Memory in use: $used_memory Mo")

And here is the results (It vary but the main point still stand anyway):

# Standard way
Memory in use: 55.5 Mo

# With recycled functions
Memory in use: 9.9 Mo

I think overriding method instance solve that problem.
I’m not sure yet about why it works (just that it seems logical for me that it would work)
Maybe it’s because overriding the function increment the world age and since the old compiled code is in a deprecated world it’s garbage collected, it seems like the most logical explanation to me.

I’m not thinking about making a package for that (FunctionPooling is neat but not my priority so it’s probably not optimal), so if anyone would push that idea further it would be really really cool.

This won’t work. Replaced methods aren’t garbage collected either. We keep them around in case they need to be called in an old worldage.

julia> f() = "An old method";

julia> world = Base.get_world_counter();

julia> f() = "A fresh method!";

julia> f()
"A fresh method!"

julia> GC.gc()

julia> Base.invoke_in_world(world, f)
"An old method"
1 Like

I understand. But if it’s the case, then I really don’t know why the reused version take less memory. Maybe it’s because a new method table isn’t create.
Any way, I would be glad if anyone can explain the drastic reduction of the memory in use.
Anyways, It now made sense, the version using recycled functions still leaks some memory from the old methods, just less than the regular version so you the old methods are effectively kept.

Nobody knows how you implemented FunctionPooling or what you mean by “reusing” functions.

#######################################################################################################################
################################################### FUNCTION POOLING ##################################################
#######################################################################################################################

"""
    module FunctionPooling

Julia has alway suffered from a well known issue: functions doesn't get garbage collected.
Even if a function is no more used, it stay in memory because Julia has no clear way to know when a function is unused.
So this package propose to pool function in order to reuse them instead of creating new one.
This package works with methods. When you reuse a function, to override the old method to create a new function.
"""
module FunctionPooling

export FunctionObject, FunctionPool
export @pooledfunction
export getfunc, getfield, free

"""
    struct FunctionObject <: Function
		id::Int
		name::Symbol
		func::Function
		pool::WeakRef

This struct is a reusable function instance.
- `id`: It's the position of the function object in the pool.
- `name`: It's the name of the reusable function. It's used to override his methods.
- `func`: It's the actual function that will be reused.
- `pool`: A weak reference to the central pool object.
"""
mutable struct FunctionObject <: Function
	const id::Int
	const name::Symbol
	const func::Function

	## Constructor

	function FunctionObject(id, name, func)
		obj = new(id, name, func)
		return obj
	end
end

"""
    mutable struct FunctionPool
		pool::Vector{FunctionObject}
		free::Vector{Int}

This object manage pooling.
- `pool`: The list of all the function object existing in the pool
- `free`: THe list of the functions ready to be reused.

## Constructor

    FunctionPool()

Will construct a new empty pool for functions.
"""
mutable struct FunctionPool
	pool::Vector{FunctionObject}
	free::Vector{Int}

	## Constructors

	FunctionPool() = new(FunctionObject[], Int[])
end

"""
    @pooledfunction pool function ... end

This create or reuse a function.
IF there is a function ready to be reused, this will override his method.
If there is no available function, then a new one is created.
This is also compatible with `@generated` functions.
"""
macro pooledfunction(p, func)
	func = QuoteNode(func)
	pool = esc(p)
	return quote
		id = 1
		nm = :()
		func = $func
                
        # The name of the function will be used to create a variable
		old = func.args[1].args[1]

        # If there is no function to reuse
		if isempty($pool.free)
            # We create a new symbol and id for it
			nm = gensym()
			id = length($pool.pool)+1

             # We then change the name of the old function
			func.args[1].args[1] = nm 

			fn = eval(func)
			obj = FunctionObject(id, nm, fn)
			push!($pool.pool,obj)

            # We then assign the new object to `old`
			$__module__.eval(Expr(:(=),old,obj))

			return obj
		else
			id = pop!($pool.free)
			nm = $pool.pool[id].name
		end

		func.args[1].args[1] = nm

		fn = eval(func)
		obj = $pool.pool[id]
		eval(Expr(:(=),old,obj))
	end
end

#################################################### Functions ########################################################

(f::FunctionObject)(args...) = f.func(args...)
getfunc(f::FunctionObject) = getfield(f, :func)
getname(f::FunctionObject) = getfield(f, :name)

"""
    free(f::FunctionObject)

This mark a pooled function as reusable
"""
free(pool,f::FunctionObject) = begin
    push!(pool.free, f.id)
end

end # module
1 Like

I think reusing functions drastically reduce memory leaks not because old method are deleted but because it reduce the number of method table created.

Thanks, I think I get some of it now: you’re reusing the method table of a free-d function by renaming @pooledfunction’s input function expression to the cached gensym-ed name and assigning the method-edited function to the input name as a non-const global variable. You’re right that doesn’t make as many new function types and associated method tables, but I’m still not sure how much of the observed numbers are saving memory on method tables or the OS caching allocations in disk. free_memory can’t tell.

As mentioned, replaced methods aren’t GCed in order for world age interactivity, so that’s still going to accumulate in a program that indefinitely evals methods, which might defeat the purpose. Games are mostly AOT compiled and the interpreted parts don’t usually generate code, so the purpose might not be needed anyway. But if you do want to avoid making new methods while changing behavior, a couple things come to mind:

  1. Callable objects. You don’t change the method body at all in order to simplify the example, but if your method bodies would practically only change internal values that you don’t want through your arguments, you could simply instantiate new functions with those internal values. The method would take those functions and the internal values as input, so the compiler can’t optimize it as much as baked-in values, but that also means the same compiled method works for any of the functions.
julia> struct Logb{B<:Real}<:Function  b::B  end

julia> (lb::Logb)(x) = log(lb.b, x)

julia> log5 = Logb(5) # one of many possible functions
Logb{Int64}(5)

julia> log5(25)
2.0
  1. DynamicExpressions.jl lets you build a callable expression without an associated method, though it does make inputs a bit strange. You just have to provide the component operators and compile those, which should stabilize for a fixed set of numerical types no matter how many expressions you build. The timings in the README have user issues, but it is indeed pretty fast if most of the work is done by the operators. I’ve explained how this addresses the whole eval deal in a fuller answer to a related question, so I’ll also link that. DynamicExpressions also has a lot of other features, but some of it is stored in extensions so you wouldn’t have to load it unless you also involve the other trigger packages.
1 Like

Yeah, it’s right, thx.
In a game, dynamically creating function isn’t that hard to manage since once the game is compiled, everything is clear and set.
For what I proposed, it just minimize the problem but doesn’t solve it. I made in case I need to dynamically recreate functions.
I think using DynamicExpression.jl is a good compromise, since the whole function body need to be re-created.

I don’t understand this bit. Dynamic code generation is generally extremely difficult to manage, and AOT-compilation adds the extra difficulty of throwing away a lot of the necessary higher level information.

I should have also mentioned type inference issues come with evading the compiler and its drawbacks. FunctionPooling also actually runs into this despite reusing bona fide functions because it exposes a non-const global variable, so caller methods won’t statically dispatch calls. DynamicExpressions calls are also vulnerable to non-const global variables, but you might be able to reuse a type for several callable expressions inside local scopes. The callable expressions are parameterized with input types, so also watch out for an explosion of types.

Another option is JuliaDebug/JuliaInterpreter.jl: Interpreter for Julia code. I’m sure one could create a callable struct InterpretedFunction that contains code as an Expr, and just passes it to JuliaInterpreter. Then it won’t leak memory at all IIUC. There’s a global in JuliaInterpreter that specifies which functions/modules to run-compiled, so as long as the code is small, there should be no issue.

That said, unless your game is very unusual,

So since I wanted to dynamically create functions for objects in my game engine, it was a big problem and I had to solve it since objects in a game are created and destroyed at an erratic rate.

makes me think that there might be a misunderstanding somewhere. Could you explain what kind of functions you are creating? Typically, one can get away with using functors, and those are just plain Julia objects that will not leak memory.

3 Likes

I don’t think I ever found out explicitly, but I’m pretty sure that Expr and everything in it except for Symbols get GCed. I would be concerned if arbitrarily many symbols and variables are being generated (and interned), that could also be a problem for DynamicExpressions.

1 Like

hat I mean is, each object in a game can have an update function, it will be called every frame. When a new type of object is create, a new update function is created and when an object is deleted, we no more need his function so normally it should be GCed.
That is why I tried pooling function, so when an object is deleted, his update function is reused to for another type of object.

That still sounds like something to be solved by functors! Could you give an example? Like this kind of code is legitimate:

struct Player
    update::Function
end

but then update will be chosen from a set of 10 or 30 functions (or maybe functors) and there will be no memory leak, because while the function themselves will never be GC’ed, what is stored in player.update is not a copy of the function, but just a pointer to it.

Or am I misunderstanding?

Reusing methods of functors is what I’d expect for objects of the same type (or maybe categorized outside the type system), especially for something like a game that would want to maintain the same code size for hours, maybe days. But this was also said:

which is the exact opposite of functors.

Maybe the intent is to dynamically generate types with diverging behaviors in separate functions, but to truly discard them after not needing them. That’s just not done because it compromises performance (if you want something to be GCed, you need to reassign their references to something else, and non-const references are unknown to the compiler), safety (if you don’t have a GC, you can end up with dangling references), and the aforementioned world age interactivity. Typical Julia isn’t capable of dynamic loading and unloading any more than an AOT compiler could insert and delete things from active compilation units, and we have yet to see if juliac could introduce something like that.

Can you think of a game type where

the whole function body need to be re-created

? I cannot, but maybe I’m too old and lack imagination. Even for something wildly experimental, I would start with a tree of functors, then profile to see if DynamicExpressions.jl is warranted

What I mean is that functions for objects are not static. In fact they will change. for example I may not want the player to update a certain way on certain conditions so I change his function. But if each time I make a new behavior with a new function, there are memory leaks (expecially if I have many special objects in game, those with unique behaviors) it will be a problem.
Anyways, what are functors ?

A functor is a callable object. Consider:

mutable struct Player
    movement::Function
    x::Int
end
move(p::Player) = p.movement(p)

then a functor called Run might be implemented like this:

struct Run <: Function
    x_speed::Float64
end
function (r::Run)(p::Player)
    p.x += r.x_speed
end

Or, using QuickTypes.jl it can be written more succinctly as

using QuickTypes

@qfunctor function Run(x_speed::Float64)(p::Player)
    p.x += x_speed
end

So then you’d have

fast_player = Player(Run(10), 0)
slow_player = Player(Run(1), 0)
# call `move` to make them move

By creating other functors (MoveRandomly, Teleport, etc), you can mix-and-match behaviour for the entities in your game.

mutable struct Player
    movement::Function
    dying_behaviour::Function
    angry_behaviour::Function
end

Run(1) is, in some sense, “creating a function”, but it’s way cheaper than the alternatives discussed above, because it’s just a struct with a single Float64.

I’ve never written a game so take everything here with a grain of salt, but I think you’re talking about game logic. eval of expressions (or strings) is overkill even in languages that can clear obsolete code, and bear in mind that conditional behavior is present in games that can’t generate code at all for their limited systems. You’ll provide the basic behavioral components (maybe functions) in advance, and the idea is to run those in a variety of combinations and orders depending on the game state. From what I can tell in documentaries and streams, a lot of malleable behavior can be done just with conditional branching (if-else, switch). If you need a variable or collection of functions but want more type stability like function pointers in other languages, FunctionWrappers.jl kinda gets you there, though note its documented limitations as well as this weird failure to adapt to method changes if previously compiled to a constant (not sure what the exact conditions for the bug is, sometimes needs 0 arguments, sometimes not). That’s all I can really contribute, it’s better to study game logic and game scripting languages and see what you can bring to Julia.

Yeah, just that behavior are not that fix. Using if/else is cool when you don’t need to pass behaviors between objects.
Whetever, having a limited set of functions seems better. If everything is AOT, then no more need to dynamically create behaviors, at least it’s some unique kind of game, so the compromise may be to provide the tools (functors, FunctionWrappers.jl, etc) warn the user about the risks at help him as we can to manage it if he need creating dynamic functions so badly.

1 Like