Struct Encapsulation Question for Optimization Test Set

New to Julia
1

I am building a test set of unconstrained optimization problems and each program is held in the struct:

mutable struct UncProgram
    name::String
    f::Function
    g!::Function
    fg!::Function
    n::Int
    x0::AbstractVector{<:Float64}
end

The structure is declared at the top of the main package module UncNLPTestSet, where I proceed in the file to include each test problem as:

TestSet = Dict{AbstractString, UncProgram}()
for p in readdir(joinpath(@__DIR__, "problems"))
    include(joinpath("problems", p))
end

Where each file in problems/* has the form:

function f(x) .... end
function g!(x, g) .... end
function fg!(x, g) .... end

TestSet["Name"] = UncProgram(f, g!, fg!,)

This leads to the following overwritten warning when I compile my package

julia> using UncNLPTestSet
WARNING: Method definition f(Any) in module UncNLPTestSet at /Users/daniel/.julia/dev/UncNLPTestSet/src/problems/LANGER.jl:21 overwritten at /Users/daniel/.julia/dev/UncNLPTestSet/src/problems/LIARWHD.jl:18.
  ** incremental compilation may be fatally broken for this module **
.
.
.

Possible Solutions:

  1. Create a bunch of different modules for each problem … silly idea
  2. Rename f, g!, fg! for each problem to have a unique (prefix the problem name in front of the function).
  3. Encapsulate each problem script in a let ... do block (where the let condition is empty) to create a local scope.

I am leaning towards number three, since it may be quicker and encapsulate direct calls to <An UncProgram>.f(x) .. from the end user. However, very new to Julia and I could be missing something. I want to ensure declaring the functions in the local-scope block won’t make things inefficient or break when I throw them into ForwarDiff.jl.Thanks in advance for any assistance!

2

A few things:

  • Looping over files to run include is a somewhat unusual approach. I would rethink if it’s truly necessary to take this approach. It’s not evaling files as strings, but it feels a bit like it.
  • Creating named functions here seems to be purely a source of problems for – it seems like you just want to capture three anonymous functions per file and probably should do precisely that.
3

Thanks for the prompt response!

In response to looping over the files, thank you for the warning. I honestly didn’t put much thought into it because it worked. My approach to loop over the files was to avoid having writing 44 include statements. Hopefully, more problems will be added in time to make the matters worse. How would you do it?

As for the anonymous function suggestion, that is a great idea apart from one issue! Ssome specifications are really long and it would be very illegible. So I need to be able to define it by some reference then put it into my struct, which implies the function is no longer anonymous.

4

I don’t know enough about your problem to give a strong suggestion. Often I’d tend to explicitly write out an array of functions, but that assumes each file’s content is fairly short. If not, using include might be the right approach.

I’m not sure I follow. In Julia, there is a distinction between a named function and an anonymous function bound to a name.

julia> f = x -> begin
     y = x + 1
     z = y * 2
     return z^2
end
#1 (generic function with 1 method)

julia> function g(x)
    y = x + 1
    z = y * 2
    return z^2
end
g (generic function with 1 method)

julia> f(1)
16

julia> g(1)
16

Does that address your concern?

5

No, it isn’t silly. You can create, all at once, a bunch of different modules by for loop.

Code:

module O

struct Program f::Function end
const TestSet = Dict{Symbol, Program}()

filenames = ("dir/A.jl", "dir/B.jl", "dir/C.jl")
rmext(x) = replace(x, r"\.[^.]*$"=>"")

for fn in filenames
    s = rmext(basename(fn))
    m = Symbol(s)
    @eval module $m
        using ..O
        f() = "f defined in the module O." * $s # or include(fn)
        O.TestSet[Symbol($s)] = O.Program(f)
    end
end

end

O.TestSet

Output:

Dict{Symbol, Main.O.Program} with 3 entries:
  :A => Program(f)
  :B => Program(f)
  :C => Program(f)

Code:

O.TestSet[:A].f()

Output:

"f defined in the module O.A"

The same thing can be done by

O.A.f()

Output:

"f defined in the module O.A"

The advantage of splitting into modules is that the function f defined in the file A.jl will be simply accessible with O.A.f.

6

Well, I stand corrected! That is a clever idea and potentially very useful. Your approach is giving me ideas to easily query a test set of problems! Thank you!

7

johnmyleswhite
Does that address your concern?

Yes, that does indeed. I wasn’t aware of the begin statement. I have only ever written one lined anonymous function, but that makes a lot of sense. So the g is the named and the anonymous is f, which still has a reference.