Because of limited manpower and the rather high barriers to entry (learning the packaging workflow for a Linux distribution is quite a time investment), and in some cases, release cycles, a lot of software lags behind by 6+ months on many distros.
That said, Ubuntu and Debian (testing) both have a 1.* version now, and for Windows and OS X, install binaries are just one click away. I don’t think it is reasonable to expect more for Julia — it is unlikely that it would be installed by default on systems.
I’m both a big fan of Python’s subprocess module and a vocal critic. It’s a bear to learn, but once you know it, you can do pretty much anything and, as you mention, it abstracts all the difficult parts of IO away.
I have no particular objection to a Popen.communicate-like interface being added to process.jl. This is more of a sidetrack as someone who has both written process handling interfaces and called a lot of processes in scripts:
What’s the actual usecase for writing to stdin and reading from the stdout of the same command or pipeline inside of a Julia script? It’s not like you’re actually going to be using grep or sed as a data filter inside of Julia. I suppose this might be useful for wrapping a tool like pandoc or other more complex text transformation tool, but I honestly don’t see much of a case for capturing the output of something like that either. (The output is eventually going to go to stdout or into a file anyway, which programs like that can handle directly, unless you’re going to do additional transformations on the output, which is already a very specialized usecase.)
As a process library implementer, this is fun problem to try and solve, but as a scripter, I run processes for two main reasons:
As a side-effect: The process just runs and does something on the system. I might pipe something to the process, but I’ll rarely do anything with the output.
To get information that cannot be easily obtained in another way (Julia’s ccall interface makes most of this unnecessary, but sometimes it’s still easier to parse command output than to bother learning the C API)
Using a process as a filter inside of a Julia/Python/Ruby/Perl script is fairly rare and sort of inelegant, since these languages are perfectly capable of doing their own text filtering.
And, as a side note to this side note, the error Popen.communicate makes is that it deals with strings rather streams. stdout and stderr should be returned as IO objects in Julia, against the event that their content may be too large to fit into RAM. The user can then decide if they want to call read or eachline or whatever.
For what it’s worth I think you’re entirely right about the reasons that one can’t just send a Julia script to someone and expect that it will just work for them. However, I’m not sure that it makes sense for Julia to aspire to be where Perl 5 or Python are in terms of that. Even if that might happen eventually it will take too long. We need other ways to make it possible to send Julia code to people. Being able to compile a small stand-alone program like you can in Go would be great. Being able to send a Julia script with a manifest file including info about the required Julia version and have it “just work” would be great too.
Can’t agree more! I can compile the same code flawlessly to Windows or Linux to one binary. Please don’t be like Rust where it’s difficult to even find out how to compile a standalone binary!
I agree this for grep and sed but there are non-trivial APIs provided as CLIs like I just mentioned. Piping is a good “RPC” for simple things. It’s nice to be good at it.
The communication between the VS Code julia extension and the LanguageServer.jl, and then the communication between the LanguageServer.jl and the SymbolServer.jl is all based on that model. The architecture there is quite different from something like say the VegaLite.jl case that @tkf mentioned already: in the latter case we start a subprocess, it processes something, ends, and we process the return case. The VS Code example, on the other hand, is more a long running RPC type scenario, where multiple processes stay around for a long time and exchange messages via STDIN and STDOUT.
So I just freed up around 3-4 GB in my .julia directory by deleting some .cache and .trash directories which were hidden and now it is only about 1.1 GB (most of that is for Nemo).
If you were a user of previous Julia editions, such as 0.4, 0.5, or 0.6, then you might have some hidden folders laying around with some gigabytes of cached data.
How about a command line argument that starts Julia in a project environment with the equivalent of a REQUIRE file, and a shebang syntax that does the same thing?
For example, if the first line of my script is:
#!julia --require "julia 1.1; Plots 0.23"
then Julia would make sure that I’m running v1.1 or later, and it would ask me to Pkg.add("Plots") if I hadn’t done so.
(Including an entire manifest is technically better, but would be much more cumbersome. Even for simple scripts it would not comfortably fit on a command line.)
Coming back to your original post (and github pointer(s)). I’m very interested in trying out Julia to hunt down the various many bash/csh/zsh/{yourinitials}sh scripts lying around (rather lengthy processes, so don’t mind the warmup overhead). Many of my colleagues have recently made a similar switch to python, so it’s a nice opportunity for (soft) benchmarking (including UX).
Any chance to see some (albeit minimal) progress on the tutorial you started? This would be very highly appreciated
I’m working on the last chapter now, which is about string manipulation. I suppose I can start working on the scripts to build out the file and post it in its own thread here for feedback.
Honestly, there isn’t that much to add to the official docs. I just put a few examples made some links to a few things in the manual. It’s not like Python where all the utilities you need for admin/workflow scripting are spread out in a bunch of crazy modules. Basically everything you need is already in Base.
