I have been told about one known issue with Julia, compiled function doesn’t get garbage collected. So since I wanted to dynamically create functions for objects in my game engine, it was a big problem and I had to solve it since objects in a game are created and destroyed at an erratic rate.
So I thought about a way to solve that and it’s just to reuse functions.
Yeah cool, but I do we reuse it ?
Simple, we override methods of unused functions.
When we want to reuse a function, we just redefine it so that the old version is overrided by the new one and I made a little module FunctionPooling to automatically solve that.
The result substain my idea. I tried dynamically create 1000 functions.
here is the snippet:
include(joinpath("..","src","FunctionPooling.jl"))
using .FunctionPooling
pool = FunctionPool()
## Warming to avoid compilation overhead.
@pooledfunction pool function update(x::Int)
x + 5
end
N = 1000 # Number of functions that will be created
total_memory = Sys.total_memory()
## Part 1: Standard Julia version
used_memory = (total_memory - Sys.free_memory())/(1024^2)
# We will generate `N` standard function and make sure they compile by calling them
for i in 1:N
symb = gensym() # The name doesn't matter
eval(quote function $symb(x::Int)
x + 5
end
$symb(1)
end)
end
# We call the GC so we free the unused memory.
GC.gc()
used_memory = (total_memory - Sys.free_memory())/(1024^2) - used_memory
println("Memory in use: $used_memory Mo")
## Part 2: Version with reused functions
GC.gc()
used_memory = (total_memory - Sys.free_memory())/(1024^2)
for i in 1:N
symb = gensym()
eval(quote @pooledfunction pool function $symb(x::Int)
x + 5
end
$symb(1)
free($pool,$symb)
end)
end
# We free the unused memory
GC.gc()
used_memory = (total_memory - Sys.free_memory())/(1024^2) - used_memory
println("Memory in use: $used_memory Mo")
And here is the results (It vary but the main point still stand anyway):
# Standard way
Memory in use: 55.5 Mo
# With recycled functions
Memory in use: 9.9 Mo
I think overriding method instance solve that problem.
I’m not sure yet about why it works (just that it seems logical for me that it would work)
Maybe it’s because overriding the function increment the world age and since the old compiled code is in a deprecated world it’s garbage collected, it seems like the most logical explanation to me.
I’m not thinking about making a package for that (FunctionPooling is neat but not my priority so it’s probably not optimal), so if anyone would push that idea further it would be really really cool.
I understand. But if it’s the case, then I really don’t know why the reused version take less memory. Maybe it’s because a new method table isn’t create.
Any way, I would be glad if anyone can explain the drastic reduction of the memory in use.
Anyways, It now made sense, the version using recycled functions still leaks some memory from the old methods, just less than the regular version so you the old methods are effectively kept.
#######################################################################################################################
################################################### FUNCTION POOLING ##################################################
#######################################################################################################################
"""
module FunctionPooling
Julia has alway suffered from a well known issue: functions doesn't get garbage collected.
Even if a function is no more used, it stay in memory because Julia has no clear way to know when a function is unused.
So this package propose to pool function in order to reuse them instead of creating new one.
This package works with methods. When you reuse a function, to override the old method to create a new function.
"""
module FunctionPooling
export FunctionObject, FunctionPool
export @pooledfunction
export getfunc, getfield, free
"""
struct FunctionObject <: Function
id::Int
name::Symbol
func::Function
pool::WeakRef
This struct is a reusable function instance.
- `id`: It's the position of the function object in the pool.
- `name`: It's the name of the reusable function. It's used to override his methods.
- `func`: It's the actual function that will be reused.
- `pool`: A weak reference to the central pool object.
"""
mutable struct FunctionObject <: Function
const id::Int
const name::Symbol
const func::Function
## Constructor
function FunctionObject(id, name, func)
obj = new(id, name, func)
return obj
end
end
"""
mutable struct FunctionPool
pool::Vector{FunctionObject}
free::Vector{Int}
This object manage pooling.
- `pool`: The list of all the function object existing in the pool
- `free`: THe list of the functions ready to be reused.
## Constructor
FunctionPool()
Will construct a new empty pool for functions.
"""
mutable struct FunctionPool
pool::Vector{FunctionObject}
free::Vector{Int}
## Constructors
FunctionPool() = new(FunctionObject[], Int[])
end
"""
@pooledfunction pool function ... end
This create or reuse a function.
IF there is a function ready to be reused, this will override his method.
If there is no available function, then a new one is created.
This is also compatible with `@generated` functions.
"""
macro pooledfunction(p, func)
func = QuoteNode(func)
pool = esc(p)
return quote
id = 1
nm = :()
func = $func
# The name of the function will be used to create a variable
old = func.args[1].args[1]
# If there is no function to reuse
if isempty($pool.free)
# We create a new symbol and id for it
nm = gensym()
id = length($pool.pool)+1
# We then change the name of the old function
func.args[1].args[1] = nm
fn = eval(func)
obj = FunctionObject(id, nm, fn)
push!($pool.pool,obj)
# We then assign the new object to `old`
$__module__.eval(Expr(:(=),old,obj))
return obj
else
id = pop!($pool.free)
nm = $pool.pool[id].name
end
func.args[1].args[1] = nm
fn = eval(func)
obj = $pool.pool[id]
eval(Expr(:(=),old,obj))
end
end
#################################################### Functions ########################################################
(f::FunctionObject)(args...) = f.func(args...)
getfunc(f::FunctionObject) = getfield(f, :func)
getname(f::FunctionObject) = getfield(f, :name)
"""
free(f::FunctionObject)
This mark a pooled function as reusable
"""
free(pool,f::FunctionObject) = begin
push!(pool.free, f.id)
end
end # module
Thanks, I think I get some of it now: you’re reusing the method table of a free-d function by renaming @pooledfunction’s input function expression to the cached gensym-ed name and assigning the method-edited function to the input name as a non-const global variable. You’re right that doesn’t make as many new function types and associated method tables, but I’m still not sure how much of the observed numbers are saving memory on method tables or the OS caching allocations in disk. free_memory can’t tell.
As mentioned, replaced methods aren’t GCed in order for world age interactivity, so that’s still going to accumulate in a program that indefinitely evals methods, which might defeat the purpose. Games are mostly AOT compiled and the interpreted parts don’t usually generate code, so the purpose might not be needed anyway. But if you do want to avoid making new methods while changing behavior, a couple things come to mind:
Callable objects. You don’t change the method body at all in order to simplify the example, but if your method bodies would practically only change internal values that you don’t want through your arguments, you could simply instantiate new functions with those internal values. The method would take those functions and the internal values as input, so the compiler can’t optimize it as much as baked-in values, but that also means the same compiled method works for any of the functions.
julia> struct Logb{B<:Real}<:Function b::B end
julia> (lb::Logb)(x) = log(lb.b, x)
julia> log5 = Logb(5) # one of many possible functions
Logb{Int64}(5)
julia> log5(25)
2.0
DynamicExpressions.jl lets you build a callable expression without an associated method, though it does make inputs a bit strange. You just have to provide the component operators and compile those, which should stabilize for a fixed set of numerical types no matter how many expressions you build. The timings in the README have user issues, but it is indeed pretty fast if most of the work is done by the operators. I’ve explained how this addresses the whole eval deal in a fuller answer to a related question, so I’ll also link that. DynamicExpressions also has a lot of other features, but some of it is stored in extensions so you wouldn’t have to load it unless you also involve the other trigger packages.
Yeah, it’s right, thx.
In a game, dynamically creating function isn’t that hard to manage since once the game is compiled, everything is clear and set.
For what I proposed, it just minimize the problem but doesn’t solve it. I made in case I need to dynamically recreate functions.
I think using DynamicExpression.jl is a good compromise, since the whole function body need to be re-created.
I don’t understand this bit. Dynamic code generation is generally extremely difficult to manage, and AOT-compilation adds the extra difficulty of throwing away a lot of the necessary higher level information.
I should have also mentioned type inference issues come with evading the compiler and its drawbacks. FunctionPooling also actually runs into this despite reusing bona fide functions because it exposes a non-const global variable, so caller methods won’t statically dispatch calls. DynamicExpressions calls are also vulnerable to non-const global variables, but you might be able to reuse a type for several callable expressions inside local scopes. The callable expressions are parameterized with input types, so also watch out for an explosion of types.
Another option is JuliaDebug/JuliaInterpreter.jl: Interpreter for Julia code. I’m sure one could create a callable struct InterpretedFunction that contains code as an Expr, and just passes it to JuliaInterpreter. Then it won’t leak memory at all IIUC. There’s a global in JuliaInterpreter that specifies which functions/modules to run-compiled, so as long as the code is small, there should be no issue.
That said, unless your game is very unusual,
So since I wanted to dynamically create functions for objects in my game engine, it was a big problem and I had to solve it since objects in a game are created and destroyed at an erratic rate.
makes me think that there might be a misunderstanding somewhere. Could you explain what kind of functions you are creating? Typically, one can get away with using functors, and those are just plain Julia objects that will not leak memory.
I don’t think I ever found out explicitly, but I’m pretty sure that Expr and everything in it except for Symbols get GCed. I would be concerned if arbitrarily many symbols and variables are being generated (and interned), that could also be a problem for DynamicExpressions.
hat I mean is, each object in a game can have an update function, it will be called every frame. When a new type of object is create, a new update function is created and when an object is deleted, we no more need his function so normally it should be GCed.
That is why I tried pooling function, so when an object is deleted, his update function is reused to for another type of object.
That still sounds like something to be solved by functors! Could you give an example? Like this kind of code is legitimate:
struct Player
update::Function
end
but then update will be chosen from a set of 10 or 30 functions (or maybe functors) and there will be no memory leak, because while the function themselves will never be GC’ed, what is stored in player.update is not a copy of the function, but just a pointer to it.
Reusing methods of functors is what I’d expect for objects of the same type (or maybe categorized outside the type system), especially for something like a game that would want to maintain the same code size for hours, maybe days. But this was also said:
which is the exact opposite of functors.
Maybe the intent is to dynamically generate types with diverging behaviors in separate functions, but to truly discard them after not needing them. That’s just not done because it compromises performance (if you want something to be GCed, you need to reassign their references to something else, and non-const references are unknown to the compiler), safety (if you don’t have a GC, you can end up with dangling references), and the aforementioned world age interactivity. Typical Julia isn’t capable of dynamic loading and unloading any more than an AOT compiler could insert and delete things from active compilation units, and we have yet to see if juliac could introduce something like that.
? I cannot, but maybe I’m too old and lack imagination. Even for something wildly experimental, I would start with a tree of functors, then profile to see if DynamicExpressions.jl is warranted
What I mean is that functions for objects are not static. In fact they will change. for example I may not want the player to update a certain way on certain conditions so I change his function. But if each time I make a new behavior with a new function, there are memory leaks (expecially if I have many special objects in game, those with unique behaviors) it will be a problem.
Anyways, what are functors ?
mutable struct Player
movement::Function
x::Int
end
move(p::Player) = p.movement(p)
then a functor called Run might be implemented like this:
struct Run <: Function
x_speed::Float64
end
function (r::Run)(p::Player)
p.x += r.x_speed
end
Or, using QuickTypes.jl it can be written more succinctly as
using QuickTypes
@qfunctor function Run(x_speed::Float64)(p::Player)
p.x += x_speed
end
So then you’d have
fast_player = Player(Run(10), 0)
slow_player = Player(Run(1), 0)
# call `move` to make them move
By creating other functors (MoveRandomly, Teleport, etc), you can mix-and-match behaviour for the entities in your game.
mutable struct Player
movement::Function
dying_behaviour::Function
angry_behaviour::Function
end
Run(1) is, in some sense, “creating a function”, but it’s way cheaper than the alternatives discussed above, because it’s just a struct with a single Float64.
I’ve never written a game so take everything here with a grain of salt, but I think you’re talking about game logic. eval of expressions (or strings) is overkill even in languages that can clear obsolete code, and bear in mind that conditional behavior is present in games that can’t generate code at all for their limited systems. You’ll provide the basic behavioral components (maybe functions) in advance, and the idea is to run those in a variety of combinations and orders depending on the game state. From what I can tell in documentaries and streams, a lot of malleable behavior can be done just with conditional branching (if-else, switch). If you need a variable or collection of functions but want more type stability like function pointers in other languages, FunctionWrappers.jl kinda gets you there, though note its documented limitations as well as this weird failure to adapt to method changes if previously compiled to a constant (not sure what the exact conditions for the bug is, sometimes needs 0 arguments, sometimes not). That’s all I can really contribute, it’s better to study game logic and game scripting languages and see what you can bring to Julia.
Yeah, just that behavior are not that fix. Using if/else is cool when you don’t need to pass behaviors between objects.
Whetever, having a limited set of functions seems better. If everything is AOT, then no more need to dynamically create behaviors, at least it’s some unique kind of game, so the compromise may be to provide the tools (functors, FunctionWrappers.jl, etc) warn the user about the risks at help him as we can to manage it if he need creating dynamic functions so badly.