Warning: macos 10.12.* bug with SharedArray: shm_open() failed

Specific Domains Julia at Scale
1

The following program triggers a bug in macos 10.12.*, but not in 10.13.* :

(nprocs()>1) || addprocs(1)

function sharedarray_parallel_sum()
    sa = SharedArray{Float64}(5)
    @sync @parallel for i=1:5; sa[i] = timingfun(); end#for
    sum(sa)
end#function

@everywhere function timingfun();  return sqrt(2.0); end;

println("Number of Processors: $(nprocs()).")

## works with 100, but dies with 1000
for i=1:1_000; sharedarray_parallel_sum(); end

the precise bug is

Number of Processors: 2.
ERROR: LoadError: On worker 2:
SystemError: shm_open() failed for /jl047505QBT91bdhwS4cz86EgGYm: Too many open files
_link_pipe at /Applications/Julia-0.6.app/Contents/Resources/julia/lib/julia/sys.dylib:?
...

I have run it up to 100_000 on 10.13.*, and it does not trigger this bug.

2

Any updates on this issue, a work around o something?. I hit the same problem. In my case sometimes works for 100, but for 200 definitely fails. And I am using MacOs 10.14.2

3

Never mind. I just called * finalize * once the SharedArray is not longer needed, and it works just fine.

No. Sometimes works other don’t. The minimal example that I’m using is the following.

55%20PM
55%20PM555×609 49.9 KB

4

Hi,
Late reply, but might be relevant.

I recently had a similar error in Linux. What I found that there is a limit on the number of files you can create at a time in an operating system. And a shared array is like “creating a new file” in the memory.

Therefore, we can have a certain number of shared arrays at a time. I suggest rather than creating many small-sized shared arrays, create a few big-sized shared arrays.

Thanks.