There was a discusison on this already here: How to release memory from jupyter notebook? - #5 by nilshg
You can try the empty!(Out)
command, but that would only do anything if you have large outputs in your cells.
I’ve been experiencing similar issues and have never fully been able to deal with them, nor produce anything reliably reproducible and diagnoseable to file any issues (given that this is just something that crops up after a few hours of analysis on large data sets). One thing that has helped me is switching large string columns in my data to ShortString
s and PooledArray
s, which generally speeds up all sorts of operations on DataFrames, but also seems to mitigate the slow memory creep you describe.