I’ve compiled it on an 8Gb RAM Pi4 in maybe an hour or two. You do need at least 4Gb RAM now. What version are you using that it takes almost a day?
I’ve compiled Julia on many SMBC through the years without issues or particularly long compile times.