Looking for a script that runs the same code with different set of parameters and saves the results in different folders

Specific Domains Data
1

Hi guys, I am running a set of experiments and each of experiment differs from the rest by the set of the parameters which are picked by the same pool of parameters. I would like to run the experiments in a loop automatically till all the possible sets of parameters run out. I also would like to save the results in different folders named after the set of the parameters used for that same experiment.

As I hate to reinvent the wheel, my question is if there exist any piece of code that does this and if not is it possible to give me some hints how you would have solved this problem.

I thank you in advance for your time and help.

Cheers.

Ergnoor

2

https://github.com/JuliaDynamics/DrWatson.jl

2 Likes
3

You have a extremely vague description of what you are looking for. Each sentence in the first paragraph is basicallly literally one line of code, saving the complex part that you didn’t mention. (In another word, you basicallly said you want to write a loop and save data to files).

You won’t be reinventing any wheels if that’s exactly what you need. If you have any other requirements then that’ll be time to see what others do.

1 Like
4

hi guys and thank you very much for your time and attention payed to my problem. Following I am giving a NOT working piece of code extracted from my not looping code that I intend to convert into looping one.

abstract type Metric end

struct Euclidean <: Metric end

struct PowerEuclidean <: Metric
    α::Int64
end

function (covMat::Euclidean)(covA::AbstractVecOrMat{<: Number}, covB::AbstractVecOrMat{<: Number})
    distance = sum(log.(F.values).^2)
end

function (covMat::PowerEuclidean)(covA::AbstractVecOrMat{<: Number}, covB::AbstractVecOrMat{<: Number})
    α = covMat.α
    PEcovA = covA^α
    PEcovB = covB^α
    distance = (1/α) * norm(PEcovA - PEcovB, 2)
end

function calFeatFunction(img::AbstractVecOrMat, featParam::whateverWillBeThisObject)
     imHeight, imWidth = size(img[:, :, iN])
     x = repeat(collect(1:imHeight), outer = (1,imWidth))
     y = repeat(transpose(collect(1:imHeight)), outer = (imHeight,1))
     Gy, Gx = imgradients(Gray.(img[:, :, iN]), KernelFactors.ando3)
     angles = atan.(Gy, Gx) * (180/π)
     magnitudes = sqrt.(Gy.^2 + Gx.^2)

     return angles, magnitude # suppose I am returning these two features
end


distSetParams = (Euclidean, PowerEuclidean) # what type of object is suitable to contain this set of metrics? 
featSetParams = (x, y, Gy, Gx, angles, magnitudes) # what type of object is suitable to contain this set of features?

for distParam in distSetParams # supose 'distParam' is "PowerEuclidean", here it is alright to select only one distance
    for featParam in featSetParams # I need to be able to select more than one feat here, supose we selected "angles" and "magnitude"
        calFeats = calFeatFunction(img, featParam)
	result = myFunction(img, calFeats, distParam()) # here the problem is that 'distParam' could be even 'distParam(α)'

	# save result in a folder with name in this case "PowerEuclidean_angles_magnitudes"
	# how to do this part?
    end
end

Notice that I have asked the questions next to the relevant parts of the code itself.

I again thank you guys for your time and support.

Cheers.

Ergnoor

5

I’m not sure exactly in which part you are having trouble, but let me give you a few hints.

For iterating over a bunch of parameter lists you could either write a nested for loop for p1 in p1_list, p2 in p2_list, p3 in p3_list (which is basically what you already did) or you could use Iterators.product(p1_list,p2_list,p3_list).

For generating the folder name you can use string interpolation folder_name="my_awesome_experiment_p1_$(p1)_p2_$(p2)_p3_$(p3)", then you can create the folder with mkdir. You might want to use $(float(p1)) instead to make sure that p1=1 and p1=1.0 end up in the same folder. Alternatively take a look at the DrWinston.jl package as suggested above whose purpose is to help with organizing experiments, data and projects.

6

@orialb Hi orialb and thank you for your help. Now that I have explored a little bit the area related to my problems I know that I can do the following with tuples of arguments of a function:

julia> myTupleOne = (7)
7

julia> myTupleTwo = (4, 6, 3)
(4, 6, 3)

julia> F(a, b, c, d, e, h, j, k) = +(a, b, c, d, e, h, j, k)
F (generic function with 1 method)

julia> F(5, myTupleOne..., 4, 8, myTupleTwo..., 9)
46

I also know that there is this Combinatorics.jl that is able to generate the Powerset of a vector. My question here is if there exists any package similar to Combinatorics.jl that generates the Powerset of a tuple, for example:

julia> myTuple = (1, 2, 3)
(1, 2, 3)
julia> tuplePowerset(myTuple)
(1, 2, 3, (1, 2), (1, 3), (2, 3))

The other problem for me is the following

result = myFunction(img, calFeats, distParam()) # here the problem is that 'distParam' could be even 'distParam(α)'

how would you have managed this situation?

Thank you very much for your time and support.

Cheers.

Ergnoor

7

I have never used Combinatorics.jl, but I am guessing that if the function you are talking about works for an array it should also work for a tuple. Did you try just passing the tuple?

I don’t understand what is the other problem you are referring to.

8

@ararslan

Hi Alex, I hope this message will not bother you.

I have seen that you dealt with the powerset function in Combinatorics.jl.

My question is if it possible to build a powerset function for some tuple?

You see I have a tuple of features and I would like to get the powerset of subtuples of that tuple in order to run experiments with all possible combinations of those features.

Thank you very much in advance for your time and support.

Cheers.

Ergnoor