I found a workaround: if I add a GC.gc() inside the loop the memory is released. However, it has no effect (i.e. the memory is not released) if I add it outside and after the loop. Also, note that GC.gc() can be added anywhere in the loop and memory is released (perhaps something to do with the local scope).
using PyPlot
ioff()
for i = 1:2000
println(i)
figure()
plot(randn(30000))
savefig("figure_$i.png")
close()
GC.gc()
end