No, no and no.
Julia uses a generational garbage collector (which is better, an optimization), that’s the more correct term. It doesn’t use mark-and-sweep, or at least that’s a simplification. Yes, I believe that’s the most correct term (plus generational), as apposed to mark-compact, which at least isn’t used (because of allowing calling C).
What is “released in time”? What’s the hurry? Yes, there are pros and cons (delayed deallocation isn’t a huge problem, or memory use, nor rather delaying closing files which which a GC doesn’t address, but e.g. Rust and destructors in C++ do), but lots of dead objects are released very quickly because of generational, which is its point, but not all. It’s not so problematic for a GPU, because Julia already targets them (CUDA on of the official tiers, yes, strictly through a package/s), and then no GC is used, yes, with GPUs with GC would be problematic.
A lot of Julia code avoids the GC, and thus the issues, including some hard-realtime code, and for GPU use. But yes, you need to do that, limit yourself then.
That’s a good thing, since users then have a mental load (and may screw things up). Julia does escape-analysis, and can and does stack allocate.
I need to run, may answer more completely later, couldn’t read further. At least welcome to the Julia community. If you think Julia has a problem, you’re likely wrong, or at least there’s a non-default workaround, e.g. static executables possible.