I’m trying to do some program synthesis in Julia. This means generating a lot of programs (Julia Exprs) evaluating them, and scoring them. These Expressions can create new types, structs, functions, etc. The most reasonable way to do this seems to be to dynamically create a new module. As far as I can tell, creating a module is permanent. Hence it seems like this will inevitably produce memory leaks. Is this true? If so, is there a way around this? Is there a better alternative? Below is a toy example to demonstrate what I mean, which does indeed accumulate increasing amounts of memory:
function genprog(n = 10)
for i = 1:n
y = rand()
mdname = gensym()
expr = :(module $mdname
f(x) = 2*x + $y
end
)
@show expr
m = @show eval(expr)
@show Base.invokelatest(m.f, 3))
end
end
To try to avoid the XY Problem , could you elaborate a bit on what you are generating all this code for? Is it some sort of genetic algorithm? Without knowing any of this, it seems like you might get better results from building and evaluating data structures rather than generating code.
I am doing Bayesian program synthesis, which means to generate programs that are likely to have to have generate data. This is not quite the same as genetic programming but there are many similarities. I have an internal DSL that I compile to Julia. I could of course write an interpreter for my DSL, but I am hoping to make use of Julia’s compiler instead.