Stray Julia processes in the background on VSCode

General Usage
1

Hi there, I am facing some issues when using Julia in VSCode, using Jupyter notebooks, where stray processes are left running in the background consuming a lot of CPU and RAM.

I’ve tried to work out a MWE Jupyter notebook, but I have failed to do so.
I have tried chipping away the Jupyter notebook to identify the portion of the code which is causing the issue, but while doing do, I am no longer able to reproduce it…
Attached, I am sending that Jupyter notebook, and the auxiliary

However, in it’s full form, I can consistently reproduce this issue with the following steps:

  1. Open up VSCode and the troublesome Jupyter notebook;
  2. Run all cells;
  3. Once it finishes running, CPU usage goes back to normal;
  4. Restart the kernel and run all cells again;
  5. Once it finishes, there will be a single stray Julia process in the background with 100% usage on a single core;
  6. Closing VSCode will not stop the process, only a pkill command will close it and release the CPU/RAM.

After a full morning of work, htop reported that my CPU had a load average of approximately 6, in a machine with only 4 cores. Killing the stray Julia processes reduced this to idle CPU usage and released about 3.5 GB of RAM (out of 16 GB available) and 5 GB of Swap (of 16 GB available).
The stray processes reported by htop are always of the form /usr/bin/julia -i --color=yes /home/undercover/.julia/packages/IJulia/Vo51o/src/kernel.jl /home/undercover/.local/share/jupyter/runtime/kernel-v2-295311QNiiKkX9ib6d.json

The troublesome Jupyter notebook, which will not run, given that it depends on files available locally in my machine is available here.
At first I thought that this was an issue with the animation of Plots.jl, so I issue a garbage collect after each animation is successful, but as it turns out, it did not help.

Could anybody advise on how to further debug this issue?

2

I also found out that there are several leftover directories in /tmp/jl_* which include several png files that aren’t being cleaned up when I restart the kernel or close VS Code.
Now whenever there are stray processes or I close VS Code, I run pkill julia && rm -r /tmp/jl_*, and everything seems to be left clean.

3

I have found a way to reproduce this bug! The steps are:

  1. Create a new Jupyter notebook file and open it in a new VS Code session;
  2. Write the following piece of code in a single cell;
using Plots
anim = @animate for i=1:10
    plot()
    for n=0:2
        plot!(rand(3), rand(3), label="n = $n")
    end
end
mp4(anim)
  1. Run the cell, wait for it to finish, and run it again (running it twice is important! I haven’t been able of replicating the bug if I only run it once!);
  2. Save and close the notebook when it finishes running;
  3. There should be now a stray Julia process in running in the background at 100% CPU usage on a single core, and two temporary directory managed by Julia, in /tmp/jl_*, that weren’t cleaned up.

Shutting down VS Code does not kill this process nor cleans any directories.
What was probably happening on my end for having a lot of these stray processes, was that I was replicating this pattern over and over again until it would fill up my memory and my CPU.
Since I never shutdown my laptop, only hibernate it, they would pile up and cause performance degradation.

Anybody has any hints on how to further debug this?
Could this be some issue with the IO that is stuck in an endless loop trying to clean up the directories maybe?

3 Likes
4

I’ ve also have several closed projects left processes that has many threads with vscode remote. Which cause other new processes fail to fork because hitting the ulimit -u max. They don’ t consume CPU like yours, though. And I don’ t use the Jupyter extension, just cell evaluation.

5

So in standard .jl files?
Can you provide me with the steps to replicate the issue you have on your machine? Maybe it also takes place in mine and we’ll be able to narrow it down even further.

6

Yes. I don’ t know how to reproduce yet because those are projects windows that are closed months ago. I will comment if I can reproduce it. Your issue could be different from mine, you can open an issue at Issues · julia-vscode/julia-vscode · GitHub if you have the reproduce steps.

7

I don’t think the Julia VS Code repository is the best place to file this bug report, because I am currently using the Julia kernel available in my machine.
Additionally, I can only reproduce this when creating animations using Plots.jl, so I think that would be the best place to present my case.

On a positive note, I think I have found a fix to this problem.
After an entire day of work, I haven’t seen any stray processes nor leftover temporary directory after running the following block of code right after creating my animation:

# store the video in a temporary directory and embed it in this notebook
tempdir = mktempdir()
video = mp4(anim, "$tempdir/output.mp4", fps=32)
display(video)

# remove the video and the animation object because the video is now embedded in this notebook
rm(anim.dir, recursive=true)
rm(tempdir, recursive=true)
anim = nothing
GC.gc()

I will now test this in a clean notebook to see if it does in fact solve the problem.

8

Given that I am able of replicating this bug consistently, I have created an issue at [BUG] Stray Julia process with high CPU and RAM usage · Issue #4834 · JuliaPlots/Plots.jl · GitHub.
If anybody has any ideas then it might be better to post them there.

9

Is this with the Julia-extension-native notebooks or do you also have the Jupyter extension installed?

10

I am running my local kernel with IJulia installed.

1 Like