Is it possible to write Julia completely in Julia using the PackageCompiler? Why is C/C++ needed at the moment? Just curious. 
1 Like
We don’t have simple to use low level language features. We have nothing that guarantee free of runtime calls.
I was wondering whether developing more low level features is on the roadmap, to make Julia also viable as a “systems” language. If not, is it because it’s not why Julia was created, or more because of devtime constraints.
It was never on any roadmap I’m aware of and it’s not even clear what is a system language. It’s not necessarily incompatible with “why julia was created” but also not because no one has time working on it.
In general, I don’t think making something a “system” language is ever a useful goal, unless you have a specific usecase in mind that requires it. Adding functions that is part of a “system” language could be useful and is probably being done from time to time depending on what you mean by system language just not for the purpose of making a “system” language.
This is similar for writing julia in julia, which is similar but certainly not the same. I also don’t see much value of getting rid of the C/C++ part completely. However, there are certainly cases that do and they are done from time to time. Just like adding “system” language features, they are done mostly as a mean to fix other problem for specific cases (performance, reduce duplicate, easy of maintainance) and not to achieve writing julia in julia.
6 Likes
Yea I agree with what you say. To me what I would like is a more easy way of interacting with/controlling the GC. Anyway thanks for the clarification!
Well before Julia existed it had to be written in a language that already existed…Granted I believe Go wrote their “reference implementation” in C/C++, then once they had that, wrote the compiler again all in Go…but then they had google’s money behind them.
I’m sure if you wanted to fund a pure Julia compiler someone would be happy to build it 
. Baring that, it’s probably better for the language if people focus on new features rather than re-writing existing working code. Well at least until they get threads fully implemented…I want that first!
3 Likes
That has nothing to do with how the GC is written. If the GC is ever written in julia itself, it’ll be hidden in a way that’s almost impossible for you to touch, only exposing necessary API just like what you have now.
If you give enough money for just this project and nothing else, sure. But still, the question is why you want to do it. If you don’t have some particular limitation of the current approach in mind (and these should be real/practical limitation rather than some abstract/vague ones that people likes to throw around) then it makes no sense to do it. If you do have some limitation in mind, then that’s an issue that could be specifically solved, which may or may not involve rewriting part of the runtime in julia.
6 Likes
Yea I know, it was more a comment on low level language features. Sorry for going off topic
Not really OT. What I mean is that whether the GC is written in julia, or whether julia get’s more low level features, or both, it’ll not make interacting with the GC easier. The GC, and much of the runtime, is meant to be hidden, they are implementation detail and must maintain certain invariance and are not meant to be touched by the user freely.
Now if you mean you want a easier/guaranteed way to not interact with the GC, that’s indeed something that has come up before and AFAICT the status of that is that we have enough support (unsafe_* llvmcall, ccall) that for anything that doesn’t need a guarantee it’s mostly good enough. There hasn’t been much (any?) usecase that needs such a guarantee either.
4 Likes
Interesting topic, could someone explain to me what makes a language, a system language and why Julia can not be considered one ?
I’m on Windows 10 and i can call 100% of the Win32 API from Julia and do all sort of stuff, like windows services, some networking stuff, create windows, close them, list installed fonts, get the hostname, etc…
I wouldn’t say i can create kernels in Julia because i don’t know how to create one.
But if somebody could define “System programming language” for me and why Julia is not one, i would appreciate.
That’s exactly why I said,
I believe it’s more or less talking about language you can write a OS kernel with or something on that level. It’s definately not about interacting with OS API. You can almost always make them work in any language.
It’s more about a set of features that you need to have to give you enough low level control. I don’t think it’s a sharp line either… That’s why I don’t like talking about “system” language… It’s more useful to say what you want to use julia for and which feature(s) you are missing. There are enough different “systems” that I don’t think it’s useful to talk about them altogether especially without a universal definition of “system”…
8 Likes
I understand the reasoning to keep the GC unexposed, but wouldn’t it be pretty nice if one could annotate code to help the GC be faster. One scenario that springs to mind would be heap-allocated work buffers inside a function that 100% certainly will not be used anywhere outside the function. If I could say at the end of the function @gc_destroy_allowed buffer1, buffer2 (or whatever variation), wouldn’t that speed up the GC’s work a bit? I’m not really familiar with how the internals of the GC work so I might be wrong.
This is indeed a little ot. But anyway, I realized that I haven’t said his for long enough that it’s probably worth putting it up again.
One thing to realize is that freeing is not free. You must do work to free memory, work needed so that you can use the memory again, or else why are you freeing the memory in the first place…
So there are basically two aspects of freeing memory afa the GC is concerned, to mark it as not needed and to actually free it. In a tracing GC, the first is free (dead memory cause no time in marking) and does the second in bulk. Both of these are very important for the performance of the GC. In particular, the bulk freeing is actually one of the main reason, I believe, that a GC can achieve higher throughput than manual memory management. That this means is that if you don’t do freeing in bulk, you can easily loose performance, rather than gaining it and just marking something free is useless since it doesn’t take the GC any time to ffigure that out anyway.
Now this is not to say there’s nothing you can do to help the GC, but it won’t be trivial at all. It would almost have to be a very integrated solution, which might involve changing part of the GC to have different kinds of memory, maybe changing the compiler to take advantage of this (another way of having a different type of memory), and possibly other ways. And even with these, I think the most useful thing you can do is to give the compiler more escape information and then it isn’t necessarily a “help GC” anymore and we’ve certainly done a lot about it.
There are also other ways you could “help” the GC, but I think in general you can’t trivially help the GC in any meaningful way. There are certainly still areas you can improve the GC, but there are very few that doesn’t involve a trade off.
11 Likes
Actually, one of the easiest way you can help GC is to declare an allocation perminant so that the GC will never need to look at it. We already do that automatically for some object. I imagine it goes against what most people imaging though… The GC really don’t want to free things when it doesn’t have to…
5 Likes
Nice, thanks for this explanation, I wasn’t aware that finding what memory to free was … free. Then indeed what I was thinking of won’t do anything at all. The bulk freeing argument makes complete sense to me too!
As go was mentioned, it seems a garbage collected language can be used to write a fast interpreter. I think that maybe there is I problem in generating a binary without including the original interpreter. Writing an interpreter in a compiled language seems like an easier task.
I believe the DLang compiler is written in DLang…and to be fast they disabled the garbage collector…which really makes me worry about their garbage collector implementation. They do have a “lowmem” flag to enable it…
Compilers and interpreters can be written in GC language just fine and we basically do. GC is really hard to be written to manage itself though.
And in the end there’s still the why question. Its not impossible to add all the bits to make it possible. We are pretty far from being benefit from any hypothetical advantage writing everything in julia provide anyway…
2 Likes
You may find these previous discussions helpful:
https://github.com/JuliaLang/julia/issues/32282
Note in particular the article Jeff posted:
https://tratt.net/laurie/blog/entries/the_bootstrapped_compiler_and_the_damage_done.html
7 Likes
I used to believe that’s what people meant by “systems language” but then at some point people shifted the meaning and started to use the term to refer to languages for writing servers and such, which Julia definitely can do. So if you want to write a kernel, you could do it in Julia but it would be a weird choice. If you want to write servers, Julia is a totally fine choice—the I/O model makes writing concurrent servers pretty simple. Being self hosted has nothing to do with either of these.
5 Likes