POSIX Shared Memory and different behaviour on Linux/Mac

General Usage
1

Hi.

First. I don’t know if this is the right place to ask, but I hope posting here might help others looking at shared memory for Julia in the future.

I have attempted to share memory between C++ and Julia in order to move large amounts of data efficiently to Julia for processing.

I have made a test repository where I have set up POSIX shared memory from both Julia and C++, and connected to the shared memory from both Julia and C++.

I am using the Inter Process Communication package in Julia, and the standard C library in C++.

I have tested the combinations of generating and connecting to the shared memory on both Ubuntu and my M1 Mac. Sharing memory works neatly on Linux, but on my Mac I can only get the from C++ to C++ route to work.

On Mac, when I generate the shared memory from Julia, I get the error open: Permission denied when trying to access from C++. When I try to access from Julia, even with the same Julia REPL, I get the following:

ERROR: SystemError: shm_open: Permission denied
Stacktrace:
 [1] throw_system_error(mesg::String, errno::Int32) (repeats 2 times)
   @ InterProcessCommunication ~/.julia/packages/InterProcessCommunication/rpDdn/src/utils.jl:733
 [2] SharedMemory(name::String, flags::Int32, mode::UInt16, len::Int64, volatile::Bool)
   @ InterProcessCommunication ~/.julia/packages/InterProcessCommunication/rpDdn/src/shm.jl:138
 [3] #SharedMemory#24
   @ ~/.julia/packages/InterProcessCommunication/rpDdn/src/shm.jl:120 [inlined]
 [4] top-level scope
   @ REPL[13]:1

It seems that there might be some access issues on the shared memory when creating it from Julia on Mac.

On Mac, when I create the shared memory from C++ and try to access it from Julia, I get the following:

ERROR: LoadError: stat returned zero type for a valid path
Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:35
 [2] stat(fd::RawFD)
   @ Base.Filesystem ./stat.jl:156
 [3] stat
   @ ./stat.jl:168 [inlined]
 [4] filesize
   @ ./stat.jl:456 [inlined]
 [5] SharedMemory(name::String, flags::Int32, mode::UInt16, len::Int64, volatile::Bool)
   @ InterProcessCommunication ~/.julia/packages/InterProcessCommunication/rpDdn/src/shm.jl:153
 [6] #SharedMemory#24
   @ ~/.julia/packages/InterProcessCommunication/rpDdn/src/shm.jl:120 [inlined]
 [7] top-level scope
   @ ~/100-132-SharedMemory/julia_consumer/src/main.jl:7
 [8] include(fname::String)
   @ Base.MainInclude ./client.jl:476
 [9] top-level scope
   @ REPL[17]:1
in expression starting at /Users/erikbuer/100-132-SharedMemory/julia_consumer/src/main.jl:7

Does anyone know what might be the issue with doing this on Mac?
Perhaps something with the Julia installation causing access problems?

On Ubuntu this seems to work fine. The ubuntu installation was made with Snap.
On Mac the installation was made with Homebrew brew install --cask julia.

I am somewhat new to Julia, so if I am making any obvious mistakes or bad practices, I’d love to hear about it!

Thanks, Erik.

1 Like