# Various BSODs when precompiling many packages in WSL on ARM64

**URL:** <https://discourse.julialang.org/t/various-bsods-when-precompiling-many-packages-in-wsl-on-arm64/119611>\
**Category:** General Usage\
**Created:** [September 19, 2024, 4:36pm UTC](https://discourse.julialang.org/t/various-bsods-when-precompiling-many-packages-in-wsl-on-arm64/119611 "2024-09-19T16:36:38Z")\
**Posts on this page:** 16\
**Page:** 1

<div class="post-metadata">

**Author:** ![BioTurboNick](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bioturbonick/32/6380_2.png) [@BioTurboNick](https://discourse.julialang.org/u/BioTurboNick)\
**Post date:** [September 19, 2024, 4:36pm UTC](https://discourse.julialang.org/t/various-bsods-when-precompiling-many-packages-in-wsl-on-arm64/119611/1 "2024-09-19T16:36:38Z")

</div>

I figured I should just post this in case others encounter this, because it seems like something that may be quite complicated to actually pin down.

I have a Windows on ARM system (Surface Pro 11), and so need to run Julia in WSL to use the version compiled for ARM. So I have VSCode set up to run connected to WSL, and to use Julia inside WSL.

When I tried to precompile a package, which includes dependencies on Enzyme and Flux among others, from scratch inside VSCode’s REPL, the entire system hangs (display and all) somewhere variable while Enzyme is being precompiled. Eventually the system throws up a DPC\_WATCHDOG\_VIOLATION blue screen.

However, if I repeatedly precompile, it eventually gets through everything.

I also don’t seem to see the issue if I precompile from scratch just within a WSL Terminal window. CORRECTION: Not true

---

<div class="post-metadata">

**Author:** ![BioTurboNick](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bioturbonick/32/6380_2.png) [@BioTurboNick](https://discourse.julialang.org/u/BioTurboNick)\
**Post date:** [October 25, 2024, 1:21am UTC](https://discourse.julialang.org/t/various-bsods-when-precompiling-many-packages-in-wsl-on-arm64/119611/2 "2024-10-25T01:21:57Z")

</div>

I’ve now seen a whole family of BSODs. It’s very reliable.

IPI\_WATCHDOG\_TIMEOUT  
CLOCK\_WATCHDOG\_TIMEOUT  
DPC\_WATCHDOG\_VIOLATION

Turns out that the WSL team wants BSODs to be reported to Microsoft’s Security team, so I’ve sent them four minidumps from the crashes.

And this is what I sent them as a reproducer:

```julia
# 1. install Julia
curl -fsSL https://install.julialang.org | sh
# 2. Execute a long-running process in Julia
julia -e "import Pkg; Pkg.add([\"Flux\", \"Enzyme\", \"Symbolics\", \"Plots\"])"

```

---

<div class="post-metadata">

**Author:** ![BioTurboNick](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bioturbonick/32/6380_2.png) [@BioTurboNick](https://discourse.julialang.org/u/BioTurboNick)\
**Post date:** [November 12, 2024, 2:08am UTC](https://discourse.julialang.org/t/various-bsods-when-precompiling-many-packages-in-wsl-on-arm64/119611/3 "2024-11-12T02:08:02Z")

</div>

Microsoft determined that the problem is resource exhaustion. They recommended that `.wslconfig` ([Advanced settings configuration in WSL | Microsoft Learn](https://learn.microsoft.com/en-us/windows/wsl/wsl-config)) be used to set the number of processors available to the VM (default is all), and to reserve 1 or 2 for Windows. On my 12-core machine, I set it to 10 cores and Julia worked perfectly. (Didn’t try 11.)

```julia
# Settings apply across all Linux distros running on WSL 2
[wsl2]

# Sets the VM to use ten virtual processors
processors=10

```

---

<div class="post-metadata">

**Author:** ![BioTurboNick](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bioturbonick/32/6380_2.png) [@BioTurboNick](https://discourse.julialang.org/u/BioTurboNick)\
**Post date:** [November 21, 2024, 7:51pm UTC](https://discourse.julialang.org/t/various-bsods-when-precompiling-many-packages-in-wsl-on-arm64/119611/4 "2024-11-21T19:51:35Z")

</div>

I take that back, it suddenly started happening again. I even reduced CPUs to 6.

Next attempt: setting `autoMemoryReclaim` to `dropcache`

---

<div class="post-metadata">

**Author:** ![abraemer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/abraemer/32/51403_2.png) [@abraemer](https://discourse.julialang.org/u/abraemer)\
**Post date:** [November 22, 2024, 7:56am UTC](https://discourse.julialang.org/t/various-bsods-when-precompiling-many-packages-in-wsl-on-arm64/119611/5 "2024-11-22T07:56:42Z")

</div>

I think, this just sounds like a hardware problem to me - not specific to Julia.  
Did you try running some CPU stresstest (e.g. [Prime95](https://www.guru3d.com/download/prime95-download/)) to test this hypothesis?

---

<div class="post-metadata">

**Author:** ![sumiya11](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sumiya11/32/207147_2.png) [@sumiya11](https://discourse.julialang.org/u/sumiya11)\
**Post date:** [November 22, 2024, 8:22am UTC](https://discourse.julialang.org/t/various-bsods-when-precompiling-many-packages-in-wsl-on-arm64/119611/6 "2024-11-22T08:22:45Z")

</div>

You can try to give more RAM to WSL. I think the default is 4 GB and no swap.

This helped me before when a huge package hanged in precompilation in WSL:

```julia
[wsl2]
memory=6GB
swap=4GB

```

---

<div class="post-metadata">

**Author:** ![BioTurboNick](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bioturbonick/32/6380_2.png) [@BioTurboNick](https://discourse.julialang.org/u/BioTurboNick)\
**Post date:** [November 22, 2024, 4:52pm UTC](https://discourse.julialang.org/t/various-bsods-when-precompiling-many-packages-in-wsl-on-arm64/119611/7 "2024-11-22T16:52:50Z")

</div>

Should WSL hanging also crash the host OS, if that was the only problem? Though I could try that. My system RAM usage is already near max though, with 16 GB, so not a lot of room.

---

<div class="post-metadata">

**Author:** ![BioTurboNick](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bioturbonick/32/6380_2.png) [@BioTurboNick](https://discourse.julialang.org/u/BioTurboNick)\
**Post date:** [November 22, 2024, 4:56pm UTC](https://discourse.julialang.org/t/various-bsods-when-precompiling-many-packages-in-wsl-on-arm64/119611/8 "2024-11-22T16:56:06Z")

</div>

I’ll try that, though Microsoft’s security people told me it looked like resource exhaustion. Julia does spin up a lot of processes to do the precompilation steps. Maybe there’s another resource being exhausted… like file system operations or handles?

I’ve run a long-running C compiler in WSL and nothing hanged there.

---

<div class="post-metadata">

**Author:** ![sumiya11](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sumiya11/32/207147_2.png) [@sumiya11](https://discourse.julialang.org/u/sumiya11)\
**Post date:** [November 22, 2024, 6:08pm UTC](https://discourse.julialang.org/t/various-bsods-when-precompiling-many-packages-in-wsl-on-arm64/119611/9 "2024-11-22T18:08:44Z")

</div>

> Should WSL hanging also crash the host OS, if that was the only problem?

On my laptop (windows 10, x86\_64) the host does not crash when WSL is OOM. WSL hangs and does not restart successfully until host reboot.

---

<div class="post-metadata">

**Author:** ![BioTurboNick](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bioturbonick/32/6380_2.png) [@BioTurboNick](https://discourse.julialang.org/u/BioTurboNick)\
**Post date:** [November 23, 2024, 7:49pm UTC](https://discourse.julialang.org/t/various-bsods-when-precompiling-many-packages-in-wsl-on-arm64/119611/10 "2024-11-23T19:49:19Z")

</div>

None of the other settings seems to work. Setting `ENV["JULIA_NUM_PRECOMPILE_TASKS"] = 4`, or restricting WSL to 4 virtual cores or less seems to work more reliably… at least I haven’t seen a BSOD with it yet.

Maybe I’ll try a stress test.

EDIT: 20 min of Prime95 on Windows directly - no issue. Also tried stress-ng on WSL - no issue.

---

<div class="post-metadata">

**Author:** ![BioTurboNick](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bioturbonick/32/6380_2.png) [@BioTurboNick](https://discourse.julialang.org/u/BioTurboNick)\
**Post date:** [November 23, 2024, 9:06pm UTC](https://discourse.julialang.org/t/various-bsods-when-precompiling-many-packages-in-wsl-on-arm64/119611/11 "2024-11-23T21:06:44Z")

</div>

I’m trying to see if I can get it to happen with anything other than Enzyme. So far no luck.

It seems like there are a handful of vulnerable points during the process that I’ve seen so far, during Enzyme precompilation and during precompilation of an extension of Enzyme.

Since precompilation actually executes code, I wonder if it’s a problem in Enzyme proper… EDIT: nope, Images (either ImageCore or TiffImages?) seems to have triggered it too.

---

<div class="post-metadata">

**Author:** ![Palli](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/palli/32/3380_2.png) [@Palli](https://discourse.julialang.org/u/Palli)\
**Post date:** [November 24, 2024, 4:44pm UTC](https://discourse.julialang.org/t/various-bsods-when-precompiling-many-packages-in-wsl-on-arm64/119611/12 "2024-11-24T16:44:21Z")

</div>

> [@BioTurboNick](#):
>
> None of the other settings seems to work. Setting `ENV["JULIA_NUM_PRECOMPILE_TASKS"] = 4`, or restricting WSL to 4 virtual cores or less seems to work more reliably…

I suggest a PR to limit to 4, or even 3, or 1? 1 is safest, but we could start with 4 and always lower this. It doesn’t matter to much what is chosen, people could always override with `JULIA_NUM_PRECOMPILE_TASKS`.

A small modification to:

> <https://github.com/JuliaLang/Pkg.jl/pull/2552/files>
>
> Fixes https://github.com/JuliaLang/Pkg.jl/issues/2404
> 
> Note that users can sti…ll override this via setting \`ENV\["JULIA\_NUM\_PRECOMPILE\_TASKS"\]\`

To implement that PR, I needed isWSL() and I started with that:

> <https://github.com/JuliaLang/julia/pull/56671>

It could strictly be merged separately (though I need to fix some minor issue with it, feel free to suggest a change for that, or even for the two in one PRs), and then a PR building on it. Or I was thinking, would I get away with doing those in one PR?

---

<div class="post-metadata">

**Author:** ![BioTurboNick](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bioturbonick/32/6380_2.png) [@BioTurboNick](https://discourse.julialang.org/u/BioTurboNick)\
**Post date:** [November 24, 2024, 4:58pm UTC](https://discourse.julialang.org/t/various-bsods-when-precompiling-many-packages-in-wsl-on-arm64/119611/13 "2024-11-24T16:58:45Z")

</div>

I appreciate the suggestions for a PR. Would be nice to know if others on Windows ARM systems experience the same thing?

I’d like to understand better what the real cause is here if possible…

---

<div class="post-metadata">

**Author:** ![BioTurboNick](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bioturbonick/32/6380_2.png) [@BioTurboNick](https://discourse.julialang.org/u/BioTurboNick)\
**Post date:** [November 24, 2024, 5:08pm UTC](https://discourse.julialang.org/t/various-bsods-when-precompiling-many-packages-in-wsl-on-arm64/119611/14 "2024-11-24T17:08:39Z")

</div>

Might be related to this: [windows 10 - WSL process causing DPC\_WATCHDOG\_VIOLATION - Super User](https://superuser.com/questions/1362149/wsl-process-causing-dpc-watchdog-violation)

Maybe WSL generally has trouble with lots of threads doing filesystem operations?

---

<div class="post-metadata">

**Author:** ![jdad](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jdad/32/4739_2.png) [@jdad](https://discourse.julialang.org/u/jdad)\
**Post date:** [November 24, 2024, 5:30pm UTC](https://discourse.julialang.org/t/various-bsods-when-precompiling-many-packages-in-wsl-on-arm64/119611/15 "2024-11-24T17:30:45Z")

</div>

In your PR I see a second definition for Sys.iswindows rather than a definition for isWSL ?

Edit. It seems now corrected. Thanks!  
May be two further comments: 1) possibly add a bullet in the 1.12 Readme? 2) is there an interest to back port it to 1.11? ( 1.10??)

---

<div class="post-metadata">

**Author:** ![BioTurboNick](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bioturbonick/32/6380_2.png) [@BioTurboNick](https://discourse.julialang.org/u/BioTurboNick)\
**Post date:** [November 24, 2024, 11:36pm UTC](https://discourse.julialang.org/t/various-bsods-when-precompiling-many-packages-in-wsl-on-arm64/119611/16 "2024-11-24T23:36:38Z")

</div>

Another thought I had is that it could be related to ARM64 systems not having any concept of hyperthreading