It is a Julia issue. The embedding mechanism is not designed for multiple embeddings. Running the atexit hooks is meant to do things such as flushing buffers or unlocking files. It is not necessarily tearing down and explicitly freeing every structure.
I’m not sure why it should be possible to repeatedly shutdown and initialize Julia within the same process. Java is another example where this restarting the runtime is not supported.
I suppose if you really wanted to do this, you could start a MATLAB worker and call this just before you terminate the worker. Another option would be to do the embedding via a mex file and call this during mexAtExit:
There might still have a similar issue if you try to reload the mex file.
I will refer you to @Sukera’s post in another thread.