I wanted to share two packages which provide useful functionality but are currently not available in Julia. The following two would make Julia even more attractable for users in / with #hpc.
Maybe this is a good idea for a GitHub list. Like a “Most Wanted” type of thing?
As always these kind of lists provide very little if any help. Instead of collecting lists and discussing about the lists the effort should be in implementing these missing packages. Alternatively, if you are missing some package or feature and you cannot implement it yourself then the most useful thing is to collect money for someone who can do it.
I have developed and maintain several registered packages as well as proprietary ones. The first step in deciding as an author to invest time and effort into developing something is knowing that there is a real need / benefit of making it available to the community. Identifying gaps is really helpful to me as a developer and I imagine to others as well. Assuming I have the knowledge, time, and interest to develop and maintain those, even then, there might be others equally or more capable to do that. For that reason, the discussion in the public forums can aid in coordinating efforts. I strongly disagree with the premise.
You are right it’s better to find all who are interested in the topic before you start. And it’s nice to hear that you might have some time to contribute new packages as well. Currently, I don’t have direct need for either packages, but I think I could need both of them in the future.
I imagine the SSH tunneling might be best serve by someone from the JuliaWeb community with some threading experimentation. A pretty common use case for SSH tunnels is also when connecting to databases. @quinnj am I correct in that assessment?
As for the TeX package, I would imagine it would be some of those talented JuliaPackaging folks that understand Yggdrasil magic. @staticfloat would BinaryBuilder be a sound approach for that?
A Tex binary distribution could certainly be built within Yggdrasil; I think there are competing, more lightweight alternatives today rather than the full texlive distribution that many of us are familiar with. I seem to remember a rust reimplementation being talked of positively a while back, but I don’t know what the status of that is.
The TinyTeX is a customized distribution that plays really well with R markdowns. From the TinyTeX documentation on its motivation
You have to either install a basic version that is relatively small (several hundred MB) but basically doesn’t work, because it is very likely that certain frequently used LaTeX packages are missing; or you install the full version that is several GB, but in your whole life, you probably will only use 1% of the packages.
I would imagine a direct porting or light customizing using that as the base distribution might work well for Julia literate programming tools (e.g., IJulia, Weave, Literate). I usually just use Overleaf for LaTeX but don’t want to install the full TeXLive distribution for a simple PDF from a Julia markdown and a few tables…
The full TeX distribution of all packages is large, and is easier to maintain at an operating system level. What I dont like about binary artifacts is that they take up a ton of unnecessary duplicate space with OS packaging. It would be nice to have TeX binaries available in CI jobs, but I wish these binaries would be used from the operating system packaging if possible, especially because the full set of them is large data.
PS, I have some programming tools which build and generate TeX documents, VerTeX.jl but it uses local operating system binaries.
There also seems to be a new package that does so Tectonic.jl. It might just check the second box. @mike does it play well with Weave?
Not tried it with Weave yet, but those are the kind of place that this should be suitable for, i.e. where you need PDF generation, but don’t really want/need to install the full TeX Live distribution.
There is an ongoing discussion on Yggdrasil about packaging a small TeX distribution
There are other threads like this one.
I think there are two important missing packages:
MICE is crucial for ML & currently super slow in R. Our ecosystem& the ML community would greatly benefit if there was a Julia implementation
I’m trying right now to get a Swirl equivalent for Julia.
I’m calling it DrJulia… I’ll make an RFC soon probably let me know if you wanna contribute.
For the first one, there is also an OpenSSH jlll, so you could do something like:
using OpenSSH_jll
ssh() do ssh
@async run(`$ssh -L 8000:localhost:1234 user@myserver.com`)
end