The best way to pass intermediate value to auxiliary functions

General Usage Performance
1

I’m writing a Monte Carlo program and I need a lot of auxiliary function dependent on one or two fields of the parameter model of the main function sweep!(model) in the following way:

function sweep!(model, n_sweep)
    α = some_expression_of_parameters_in(model)
    β = some_other_expression_of_parameters_in(model)
    # ... 
    # α, β, ... don't change in the following code
    #...
    for _ in 1 : n_sweep
        my_auxiliary_function() # The definition of the function is dependent on α, β, ...
    end
end

The auxiliary functions are called in each loop of sweeping, so I want them to be as fast as possible. Currently I define them in sweep!, so that α, β, … aren’t passed through the parameters of my_auxiliary_function. This is not very readable (causing sweep! to be a 300-line function), so I want to define my_auxiliary_function outside.

But here comes the problem of how to pass α, β, … to my_auxiliary_function. I will frequently change the number and definition of these quantities, and my_auxiliary_function appears in multiple places, so if I just define my_auxiliary_function as my_auxiliary_function(α, β, ..., other_parameters), whenever I change the number and definition of α, β, …, I need to change all my_auxiliary_function. Besides, since α, β, … never change after they are calculated, this kind of calling convention is rather distracting.

I can also build a struct to contain α, β, …, and pack these values into one struct, and pass the struct to my_auxiliary_function. But unpacking a struct takes time.

What should I do, then, to move my_auxiliary_function out of sweep!?

2

Create a struct, you comment

But unpacking a struct takes time.

what is your empirical base for it? If the struct is immutable, then unpacking is basically aliasing a address in the stack and incurs in no runtime penalty (maybe slightly more compilation time). If you struct is mutable each unpacked field is just one pointer de-reference to a value probably already in L1 cache; is your auxiliary function so lightweight that this de-reference is a considerable overhead?

2 Likes
3

Why not just define a closure (or a similar callable object) and pass around that? For example, define:

function sweep!(model, n_sweep)
    for _ in 1:n_sweep
        model()
    end
end

and then call it as e.g.

sweep!(() -> my_auxiliary_function(α, β, ..., other_parameters), n_sweep)

or even (using Julia’s nice do syntax) as:

sweep!(n_sweep) do
    my_auxiliary_function(α, β, ..., other_parameters)
end

This way, you let the anonymous function (closure) “capture” all of the parameters that you need and pass them through to sweep! or whatever other functions might need to call your model, and you can change the model as much as you want while leaving sweep! unchanged.

There are lots of examples of this sort of higher-order function in Julia (and other languages). e.g. a close relative of Monte-Carlo methods is numerical integration (“quadrature”), in which you have a function of the form integrate(f, a, b) that computes \int_a^b f(x)dx. If you want to pass additional parameters to your integrand you just use a closure integrate(x -> f(x, params...), a,b).

3 Likes