For memory bound operations, memory performance dominates.
I kept this in mind for my build and went with DDR4-3600 RAM as well as an M.2-2280 NVME SSD 
. I also got inspired to start a thread for showing off Julia performance on PCs that people build/have so check it out! Thanks so much for your response!!