The ideas are interesting. One of my Ph.D. students does some of it in Python for a small model… one of the extensions of Pohjanpalo’s idea. It may be necessary to combine symbolic treatment of models with AD and high precision types.
Very feasible in 2019. It’s just a Groebner basis algorithm, and those have been the basis of computational topology for awhile now so they have gotten very optimized. We just need a version of a solver in Julia.
I wouldn’t use AD here. I’d use ModelingToolkit because you want to interact with the polynomials. Luckily, with Julia we can already transform numerical codes into symbolic codes:
http://juliadiffeq.org/DiffEqTutorials.jl/html/ode_extras/01-ModelingToolkit.html
Anyways, that’s where we’re going.
1 Like
Nice! I’ve been thinking that the Grobner basis algorithm could be handy here, but haven’t found a package for that in Julia… [Btw. Buchberger was the perfect PhD student… naming his algorithm after his supervisor… :-o ]
A simple approach might be to use Pohjanpalo’s idea of a series expansion in the output to collect (multivariable) monomials in parameters, and then use a Gröbner basis algorithm to find clusters of identifiable parameters. The “problem” with most of these ideas is that in theory, there might be more information if the expansion goes to infinity; if one takes a finite expansion, there might be more information if one takes one more term.
Perhaps it is possible to say something about how many terms are useful before information drowns in noise? That could at least give some idea of when to stop in the expansion.
You could give SurrogateModelOptim.jl a try. It is probably far to slow to be useful in general for your application but it will scale the input dimension and return these results. They are scaled based on the leave-one-out loss so that, given enough samples, a skateboard ramp shaped function basically becomes a line in the scaled space. If you run the smooth variant of your function with the defaults in the package and increase the iterations you should get an idea of the parameter importance.
1 Like
OK – I guess that this package builds up a surrogate model of the cost function, and then optimizes the surrogate model?
Thanks for suggestion! I’ll keep it in mind!
1 Like
Precisely, the surrogate creation is an optimisation problem itself, one that takes a lot of time. Increasing the speed is on the wish list but for now it will give you an estimate of the importance of each parameter overnight (as well as the best found design). The surrogate model is also returned so that it can be used to create sensitivity plots, although in this case I think your function is more than fast enough for you to not need this functionality!