Generalizing functions to accept inputs with 1 or 2 (or with 2 or 3) dimensions

New to Julia
1

I wish to create a function which can accept one input and perform calculations, or can accept multiple inputs, choose the best one, and then perform calculations. I’m not sure how to approach it. I originally thought that I would keep the “options” in the last dimension of an AbstractArray. A toy example is below, but I cannot get sort to work. It gives a method error when I pass a Vector or a Range. I next thought about writing one method for AbstractVectors and another for AbstractArrays. However, I do not think that approach would extend to higher dimensions, because I do not know how I would call different methods on a 2D vs 3D array. Finally, I thought about creating a vector of arrays where the outer vector construct would contain the “options”, but that may make the code more complex than it needs to be. Is there a preferred way to handle this situation? (Coding critiques are also welcome.)

function f(x)
    il=size(x,1)
    jl=size(x,2)

    #sort!(x,dims=1) # this works for f(b), but breaks for f(a) and f(c).

    y=Array{typeof(x[1])}(undef,il)
    for i in 1:il
        y[i]=sum(x[i,:])
    end

    (minsum,jminsum)=findmin(sum(x,dims=1))
    z=x[:,jminsum]

    return y,z
end

a=[4,2,3,1]
b=[[5,8,6,7] [11,10,9,12]]
c=10:-2:0

f(a)
f(b)
f(c)
2

I’m not sure of understanding the problem entirely, but with respect to this:

sort (and sort!) methods for 2D, 3D, and N-D arrays are just the same, so that should not be a problem, I think:

julia> x = cat(rand(3,2), rand(3,2), dims=3)
3×2×2 Array{Float64,3}:
[:, :, 1] =
 0.460061  0.727659
 0.155417  0.241911
 0.148384  0.285468

[:, :, 2] =
 0.782055  0.269738
 0.126793  0.786585
 0.851543  0.56357

julia> sort!(x, dims=1)
3×2×2 Array{Float64,3}:
[:, :, 1] =
 0.148384  0.241911
 0.155417  0.285468
 0.460061  0.727659

[:, :, 2] =
 0.126793  0.269738
 0.782055  0.56357
 0.851543  0.786585
3

That seems like a pretty unusual way of doing things in Julia, and I think that’s part of why you’re having trouble expressing this cleanly in code.

I think it’s worth spending a little bit of time thinking about what the nouns in your problem are and then figuring out the right set of verbs to operate on them.

For example, if a single input to your problem contains a vector of data and a boolean option, then I would start by creating a struct to hold precisely that:

struct Input
  data::Vector{Float64}
  option::Bool
end

Suddenly the two cases become much clearer: you either have an Input or a Vector{Input}. That’s it. Now your function could look like:

function perform_calculations(input::Input)
  do_stuff(...)
end

function select_best(inputs::AbstractVector{Input})
  # iterate over `inputs`, find the best one, and return it
end

function my_function(inputs::AbstractVector{Input})
  perform_calculations(select_best(inputs))
end

function my_function(input::Input)
  perform_calculations(input)
end

We’ve accomplished a bunch of things all at once: First, the core perform_calculations function no longer has to worry about whether it’s in the multi-input or single-input case. Second, the selection of the best input is a nice standalone function that we can test and iterate on. And third, the two cases (multiple inputs or a single input) are easily separated by multiple dispatch on my_function.

6 Likes
4

Yes, the issue is that sort(x,dims=1) will not work for the 1D case.

5

What Nathan means is that:

julia> a = [3, 2, 1];
julia> sort(a)
3-element Array{Int64,1}:
 1
 2
 3

julia> sort(a; dims = 1)
ERROR: MethodError: no method matching sort!(::Array{Int64,1}; dims=1)
[...]

julia> b = [3 2 1];
julia> sort(b)
ERROR: UndefKeywordError: keyword argument dims not assigned
[...]

julia> sort(b; dims = 1)
1×3 Array{Int64,2}:
 3  2  1
1 Like
6

Then you can make and use a function that generalizes for both cases:

sortvectororarray!(x::AbstractVector; kwargs...) = sort!(x)
sortvectororarray!(x::AbstractArray; kwargs...) = sort!(x; kwargs...)

Depending on how you use it, it might be safely used without producing type instabilities.

1 Like
7

That makes sense. My Matlab habits are dying hard. I am used to just adding dimensions to my array when I need to store more data and adding more lines to my code when I need to add more operations. I’ve never used structs, but it is probably time to start. Although for my current problem the input is just a vector.

I did oversimplify a bit since I need to perform most of the calculations on all of the inputs before I know which one is best. Currently, I am keeping track of the proper indices to spit back out after I find the best one. However, based on your outline, I think it may be cleaner to just call the function again once I know which input to use.

2 Likes
8

Not the best solution. The first method (AbstractVector) should remove the dims of the kwargs if it is present and pass the rest to sort!, otherwise it will be impossible to access any other options of sort!.

9

True, but I don’t mean sortvectororarray! to be of general use, but an ad-hoc function, specific for that problem. So, if no other options of sort! have to be used, there is no need to make them available.

10

I am not sure if this is what you want, but this example is, literally, what conforms to that description (here the “best” input is the smallest of the numbers, and the “best” is multiplied by 10 at the end):

julia> function f(args...)
         best = args[1]
         for i in 2:length(args)
           if args[i] < best
             best = args[i]
           end
         end
         return 10*best
       end
f (generic function with 1 method)

julia> f(1)
10

julia> f(1,2)
10

julia> f(0.5,1,2)
5.0
2 Likes
11

Yes! Do it! :slight_smile:

The ability to structure your data with little or no cost to performance is one of Julia’s superpowers, and it’s something you simply cannot do in MATLAB or Python.

2 Likes
12

I should also point out that there are other inputs to my function which will be constant for the set of input “options”. That is, it may look like f(x::Int, y::Vector, z::Array) where x and y will apply to every column of z.

13

You mean something like that (supposing you want to change the parameter z)?

julia> function f(x::Int, y::Vector, z::Array)
           xy = x .* y
           (col -> col .+= xy).(eachcol(z))
           z
       end
julia> x = 1;
julia> y = [0, 1, 2];
julia> z = [3 3 3; 2 2 2; 1 1 1]
3×3 Array{Int64,2}:
 3  3  3
 2  2  2
 1  1  1

julia> f(x, y, z)
3×3 Array{Int64,2}:
 3  3  3
 3  3  3
 3  3  3

julia> z
3×3 Array{Int64,2}:
 3  3  3
 3  3  3
 3  3  3
14

With that syntax you could use the first parameters as the “non-variable” ones.

julia> function f(x :: Int, y :: Vector, args...)
         z = Vector{Float64}[]
         for i in 1:length(args)
           @. args[i] = args[i] + x
           push!(z,args[i]*y)
         end
         z
       end
f (generic function with 1 method)

julia> f(1, rand(3), rand(3,3), rand(3,3))
2-element Array{Array{Float64,1},1}:
 [2.1299518632077437, 2.438409415790726, 3.321686548573605]
 [3.1192576982367894, 2.5495899125063506, 2.4974216223219736]
15

I just meant that I have other parameters that need to be kept alongside the main input options.

For @lmiq’s code, I think f(x,y,args...) works fine as long as I pass a Vector of Vectors as args. (Edit: This method expects one or multiple Vector arguments which then are then slurped into a Tuple.)

For @rdeits’s code, I wasn’t sure whether to pass the other parameters to each function along with input and inputs, or whether they should be part of the Input struct (even though they will not change for a given set).

16

If you have that already it does not make much sense to use the ... notation in your function, it makes more sense to just accept that vector of vectors as the third variable. I think it is more reasonable in general to define one method that operates on that vector of vectors taking care of the dimensions properly than to generalize the function definition such that one method specializes to each number of arguments.

1 Like
17

Oh, I understand now. The ... in your code is slurping the arguments up into a Tuple. I was thinking it was splatting when I wrote my last post. Are you saying it would be better to require a Vector of Vectors rather than slurping the remaining arguments then?

18

I think it is a more controlled alternative. Probably the code is easier to maintain and bugs are easier to find if your function can only accept one type of argument, and the same number of arguments always.

What I said before (that specialized methods are generated for each number of arguments) appears to be wrong, only one method remains which operates on a tuple always. That is ok if you define the type of every tuple element, with, for example:

julia> g(x :: Float64 ...) = x
g (generic function with 1 method)

julia> g(1.)
(1.0,)

julia> g(1.,2.)
(1.0, 2.0)

julia> g(1.,3)
ERROR: MethodError: no method matching g(::Float64, ::Int64)

otherwise you might get type instabilities probably. If using this, passing the vector of vectors, or many arguments, is probably equivalent.

Actually the methods specialize depending on what you do with the arguments in the function, which is pretty smart:

julia> f(x...) = 2*x[1]
f (generic function with 1 method)

julia> @code_typed f(1,2)
CodeInfo(
1 ─ %1 = Base.getfield(x, 1, true)::Int64
│   %2 = Base.mul_int(2, %1)::Int64
└──      return %2
) => Int64

julia> @code_typed f(1,"a")
CodeInfo(
1 ─ %1 = Base.getfield(x, 1, true)::Int64
│   %2 = Base.mul_int(2, %1)::Int64
└──      return %2
) => Int64

julia> @code_typed f(1,2,3)
CodeInfo(
1 ─ %1 = Base.getfield(x, 1, true)::Int64
│   %2 = Base.mul_int(2, %1)::Int64
└──      return %2
) => Int64

Note that the fact that the second argument is a string does not affect the code generated, and the fact that the third call has three arguments neither.

2 Likes
19

If you think convenience is the priority, then the modified version of my function below can take any number of matrices or vectors (because eachcol works on Vectors too)

2 Likes
20

To add another twist, the x::Int argument is currently optional i.e. x::Int=3. I don’t imagine there is a way to use both optional arguments and argument slurping simultaneously?