Sometimes I think of projects that I may or may not ever get around to doing.
Other times people come on to chat and say “What does julia need?”.
I’m not sure how well they line up, but anyway I thought we could maybe have a list.
MD5.jl
A pure julia MD5 implementation.
We have MD5 functions in MedTls.jl and Nettle.jl,
but they call into a backend,
and the packages are not ultra light weight (unlike SHA.jl).
DataDeps.jl would like this, because it is easier to find only MD5 hashs for datasets.
I suggest such a package should extends SHA.jl (using its conventions and namespace).
XML/JSON serialization
This is part of the C# /.Net standard library, I was surprised to find that it isn’t common in other languages. Simple Xml is a java library for it.
Apparently Pyxser is a python library for it, i’ve not used that though.
This is a bit less simple than MD5.jl.
One needs to implement decorators, probably with a macro,
so that some fields can be serialized to attributes and others to child elements.
And there are issued with infinite recursion etc, but clearly these have been handled acceptably in other languages; so that solution can be ported to here.
I think this is very useful, as it is much easier to work with in language types than with XML direct manipulation (as in LightXML.jl and EzXML.jl).
What random self contained projects have you been thinking about?
Not sure about small, but having a pure Julia implementation of the algorithms in NLopt is something I would love to see one day. When I get time and/or need them, I will do some of these algorithms myself but the list is big, and I think grabbing any one of them would be a nice optimization and Julia exercise. Each folder in the link above has a reference that can be used to understand and write the algorithm.
Not sure about small, but having a pure Julia implementation of the algorithms in NLopt2 is something I would love to see one day. When I get time and/or need them, I will do some of these algorithms myself but the list is big, and I think grabbing any one of them would be a nice optimization and Julia exercise. Each folder in the link above has a reference that can be used to understand and write the algorithm.
I wouldn’t call that a self-contained julia project (though it is a worthy one, perhaps).
That would be an extension to Optim.jl.
They could start as self-contained though, then they can get interfaced or merged later. By self-contained here, I meant none of them really needs to depend on the others, maybe our interpretations of the word are different
I’ve made some benchmarks of blackbox optimisation recently and it seems that Python’s CMA-ES is still the best for most problems. There’s some Julia implementations of CMA-ES around, but they are incomplete or not very good. One issue is that there’s many versions of the algorithm, so it’s not even super clear which one to implement (maybe this one).
Separable functions
In statistics you often have problems where some functions is separable (e.g. P(x,y) = P(x)P(y)), you can sometimes exploit this property for optimisation (e.g. turning two imbricated loops into two serial ones). It would be nice to have a generic way to deal with this. I made some prototype at some point, that was working alright, but I should get back to it. I found the problem to be quite interesting.
Generic GUI framework ?
This is more a pipe dream more than anything else but the idea would be to have some basic GUI framework where the drawer (e.g. Cairo, OpenGL, Web), the windowing system / event handler (SDL, GLFW) can be abstracted away. I would focus more on making writing widget as easy/elegant as possible rather than providing a lot of widgets. Maybe Visualize.jl is already going a bit in the direction though.
Just to mention this: “Generic GUI framework” contradicts with “small and self-contained”; because you will drown in testing effort on various platforms.
What might work is a very small, consistent subset as a julia API with backends in some places.
For those watching at home @jw3126 and have, in the best kind of open source merged our implementations together.
With @jw3126’s high performance, and my reuse of SHA.jl’s methods (which took a minor modification to SHA.jl, but was actually pretty simple because SHA.jl is pretty cleanly written)
Another one (and I won’t be doing this one myself :-P)
Download.jl
Build a (nearly) pure julia replacement for Base.download, on top of HTTP.jl, (or Requests.jl).
with more support for things like using the new logging progress bars,
and doing all that magic stuff to make downloads faster.
(See https://github.com/axel-download-accelerator/axel etc)
