Any tips on finding about memory leaks please?


#1

My code running in REPL causes memory usage (as shown in ‘top’) of julia to go up gradually. Calling gc() specifically doesn’y help much. Oftentimes, I have to restart julia.

The code basically reads in a DataFrames via readtable, then it allocates some memory and does some processing, then it may plot some graphs with Plots package, then repeats with a different data file, and so on. Some of the above procedures may be manual, with re-including the code file and re-launching the program function. Even with re-launching the program function, the memory usage still goes up through time. I don’t have global memory allocation, and all allocations are within functions. So I am quite suspicious that it is not my code leaking memory (am I correct in this?).

But anyhow, what can I do to find out about memory leaks?


#2

Are you working in pure Julia, or calling external libraries (eg in C) that allocate memory?

If you are only using Julia, memory usage per se should not be a concern, the garbage collection will take care of it as needed.


#3

Thanks for replying.

I am using pure Julia (0.6.4), and its packages.

The problem for me is that memory usage can go very high so the system has to use swap space (on Ubuntu 18.04) at times. And there is no sign that gc kicks in to help. The only solution is to restart julia.


#4

Hard to say more without the actual code, but I would first investigate memory usage of objects with Base.summarysize. Also see this question.


#5

Thanks for the tip.

The summarysize of my data structures stay quite constant through the iterations, but the size of julia grows. I even tried to set some large data to “nothing” before quitting the function and then call gc(). All does not help much. Very strange.

Anyone know if this following type of statement can give gc trouble? I have yet done a specific test on it.

push!(getfield(myData, fields[i]), 0.)