Alternative to Any[ ] ? or efficient usage idea?

New to Julia
, ,
1

Hello everyone, Im trying to avoid use of any[] in my code but at the moment situation is such that I need to save results for unknown number of interactions (stochastic in nature). So, I’m using any[] and pushing results in it for each interaction.

Are there any tips on using any[] that i should know to make it memory efficient ?

Thank you

2

Is your intention to just create an initial array for pushing into? If all your results are of the same type, say T, you can just do T[] and then push into it, e.g. something like

results = T[] # same as Vector{T}()
while condition() # replace with whatever you're using to determine when to stop iteration
    result = simulate()
    push!(results, result)
end

If you have a rough estimate for the number of interations, say n, you should also do sizehint!(results, n) before the loop

1 Like
3

At the moment i’m saving different types stored in a tuple. So something like this

push!(x, (x1, x2, x3, x4, x5)). # which has float, int also symbol
4

You could use e.g. T = Tuple{Float64, Int, Symbol, (whatever else)}. Something more common would be to define a struct for your results and then push into a vector of that, e.g.

struct SimulationResult
    x1::Int 
    x2::Symbol 
    x3::Float64 
    < and so on >
end 
results = SimulationResult[]
< the rest >
2 Likes
5

i will try this and come back :slight_smile:

1 Like
6

As a general aside: don’t worry too much about optimising until you’ve got something that works and you can profile it. If your code is spending most of it’s time doing simulations then this may well not be critical.

4 Likes
7

My code is working fine. Only issue I have right now is too much time consumption and memory. So, just for 5 simulation it consumes almost 5 GiB which seems insane. :joy:

8

That would probably indicate that this Any might be 0.00001% of your troubles :sweat_smile:.

My solution does answer what you asked originally, but you may need to look at Performance Tips · The Julia Language (bit long, so browse the section titles first to see what might be most relevant to you - memory usage like that sounds something like a type instability somewhere / global variables etc.).Make sure to use functions everywhere rather than staying in global scope!

3 Likes
9

It’s hard to comment on possible causes without seeing some code or a better understanding of what it’s doing.

Possibly-relevant tips:

  • Creating an empty vector and repeatedly push!ing into it isn’t super efficient. Sometimes this can be addressed by generating the vector using map, e.g. when you expect every function call to result in an entry. You could also look into whether a sizehint! might help this, but that might be a long shot.
  • If you’re running any operations where vectors/matrices are updated without using in-place operations, you can create a situation where new memory is allocated but old memory isn’t immediately freed, which can sometimes lead to memory usage multiple times greater than you expected.
1 Like
10

Quick tip:

Whenever you have a for loop like

ys = []
for x in xs
  push!(ys, f(x))
end

replace it with the map-do syntax:

ys = map(xs) do x
  f(x)
end

It will infer the resulting type even when f is a very complicated algorithm.

4 Likes