# OOM Killer on Linux

**URL:** <https://discourse.julialang.org/t/oom-killer-on-linux/22722>\
**Category:** Julia at Scale\
**Created:** [April 3, 2019, 9:25pm UTC](https://discourse.julialang.org/t/oom-killer-on-linux/22722 "2019-04-03T21:25:15Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![versipellis](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/versipellis/32/7878_2.png) [@versipellis](https://discourse.julialang.org/u/versipellis)\
**Post date:** [April 3, 2019, 9:25pm UTC](https://discourse.julialang.org/t/oom-killer-on-linux/22722/1 "2019-04-03T21:25:15Z")

</div>

I’m curious if anyone else has run into this.

I’m on Ubuntu and trying to run an analysis on some really large datasets (40gigs binary, 1.5billion rows, 1600 chunks) using JuliaDB. I’ve frequently run into the issue where I can’t utilize all 8 threads on my CPU because I run out of memory before then - Linux kills off the processes as that happens, and then the whole execution fails. The only solution I have is to limit myself to only 2 or 3 threads, maybe 4 if I feel lucky, or to chunk out even smaller.

Is there a solution to throttling the processes as I run out of memory rather than killing off a process 70% of the way through a 30-minute-long script, so that it finishes?

---

<div class="post-metadata">

**Author:** ![zgornel](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/zgornel/32/217487_2.png) [@zgornel](https://discourse.julialang.org/u/zgornel)\
**Post date:** [April 4, 2019, 7:23am UTC](https://discourse.julialang.org/t/oom-killer-on-linux/22722/2 "2019-04-04T07:23:08Z")

</div>

I think not. You may try increasing the amount if swap available.

---

<div class="post-metadata">

**Author:** ![mauro3](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mauro3/32/292_2.png) [@mauro3](https://discourse.julialang.org/u/mauro3)\
**Post date:** [April 4, 2019, 7:42am UTC](https://discourse.julialang.org/t/oom-killer-on-linux/22722/3 "2019-04-04T07:42:54Z")

</div>

`pmap` is fault tolerant, at least when working with `addproc` processes (not sure about when using threads). [Fault tolerant `pmap` when worker goes down - #2 by mauro3](https://discourse.julialang.org/t/fault-tolerant-pmap-when-worker-goes-down/21648/2)

---

<div class="post-metadata">

**Author:** ![johnh](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/johnh/32/3615_2.png) [@johnh](https://discourse.julialang.org/u/johnh)\
**Post date:** [May 9, 2019, 12:38pm UTC](https://discourse.julialang.org/t/oom-killer-on-linux/22722/4 "2019-05-09T12:38:34Z")

</div>

Ooooh… as @zgornel points out I dont think there is a throttling mechanism.

I would start to witter on about containers, which depend on cgroups.  
But I don’t think this will help [https://www.kernel.org/doc/Documentation/cgroup-v1/memory.txt](https://www.kernel.org/doc/Documentation/cgroup-v1/memory.txt)

Linux specific warning. However at one stage when working with large simulations I implemented ZRAM. This works in a kind of counter intuitive fashion - you allocate some RAM as swap space BUT you compress this. So if your data is compressible you effectively get more RAM.

> **[zram](https://en.wikipedia.org/wiki/Zram)**
>
> zram, formerly called compcache, is a Linux kernel module for creating a compressed block device in RAM, i.e. a RAM disk with on-the-fly disk compression. The block device created with zram can then be used for swap or as general-purpose RAM disk. The two most common uses for zram are for the storage of temporary files (/tmp) and as a swap device. Initially, zram had only the latter function, hence the original name "compcache" ("compressed cache").
> After four years in the Linux kernel's drive...

Give me a message and I can try to help you set it up.

---

<div class="post-metadata">

**Author:** ![johnh](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/johnh/32/3615_2.png) [@johnh](https://discourse.julialang.org/u/johnh)\
**Post date:** [May 9, 2019, 12:41pm UTC](https://discourse.julialang.org/t/oom-killer-on-linux/22722/5 "2019-05-09T12:41:38Z")

</div>

there is the freezer with cgroups

> **[Linux Kernel Documentation :: cgroups : freezer-subsystem.txt](https://mjmwired.net/kernel/Documentation/cgroups/freezer-subsystem.txt)**
>
> Linux Kernel Documentation

But that is not a gentle throttle. don’t see much point in putting a high memory using process in the freezer. When you thaw it out it will only continue to grow.

---

<div class="post-metadata">

**Author:** ![jpsamaroo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jpsamaroo/32/46804_2.png) [@jpsamaroo](https://discourse.julialang.org/u/jpsamaroo)\
**Post date:** [May 9, 2019, 3:57pm UTC](https://discourse.julialang.org/t/oom-killer-on-linux/22722/6 "2019-05-09T15:57:57Z")

</div>

The real issue here is JuliaDB’s large memory footprint, IMO. It should be possible to process arbitrarily large datasets with JuliaDB with a fixed RAM budget (at the cost of lost performance, but there’s no avoiding that).

Trying to throttle a process in the hopes that it will cause it (and its friends) to allocate less seems like a poor solution to the problem; ZRAM or swap is probably the better option in the short term.