Sanity check with optimization function

New to Julia
1

Hey I have a quick question about my workflow with a problem.
I am writing a little logistic regression function for Andrew Ng’s ML course on coursera. I have X and y, where y is 0 or 1. I want to make sure that the scopes I am using are correct

function logit(X, y)
    function costFunction(theta)
        cost = f(theta, X, y) # X and y are only available to this function, so they are type stable
        return cost
    end

theta_0 = [0, 0, 0] # X has width of 3, for example
theta = optimize(costFunction, theta_0) # not the complete code or correct syntax
return theta

I am writing this because the course is done in Octave / Matlab. There, the syntax is

[theta, cost] = ...
    fminunc(@(t)(costFunction(t, X, y)), initial theta, options);

In Julia, it seems that Optim.jl and JuMP don’t allow your function to take in anything other than your parameters of interest, so having X and y in the cost function wouldn’t work. This has to do with the way Julia optimizes for the types of variables, so my procedure above is the Julian way of working with this problem.
Is my understanding correct?

PS perhaps this should be in gitter? I just want to ensure I have the right understanding.

2

If you look at your matlab code you’ll see that it does so as well. The line @(t)(...) is an anonymous function of just the parameters to optimize. In julia you could write it as t->constFunction(t, X, y)

1 Like
3

This looks good. This “building another function that encloses data” (here, costFunction which encloses X and y) is usually called a closure. You can also much more simply use an anonymous funciton: theta->f(theta,X,y). Julia compiles the closures so you don’t need to worry about speed with this setup, it’s basically just an interface change.

4

Note that in Julia you can define functions by assignment and returns are implied, so a somewhat more compact version of your code might be

function logit(X, y)
    costfunction(θ) = f(θ, X, y)
    θ₀ = zeros(Float64, 3)
    optimize(costfunction, θ₀)
end

in case you were interested in what common practice in the language is like.

2 Likes
5

Thanks all for the help! Julia is definitely more transparent than Matlab on this stuff.

1 Like