Too much time on my hands

Community
1

TL;DR An old(ish) guy looking for a meaningful application of his modest coding skills.

whoami A semi-retired R&D engineer: “tangible things engineer”, not “software engineer”, with broad (not necessary deep) knowledge and hands-on experience in Physics / Material Science / Chemistry as well as electrical / chemical / mechanical Engineering. Maths was taught on a reasonaly serious level at the university, but that was >40 years ago.

The last few decades worked at the Fraunhofer Society. Now after retirement working part-time for a small company in the field of chemical apparatus engineering, and have some free capacity.

Over the decades I learned and forgot like a dozen of programming languages. I consider myseld fluent in LabVIEW (which is probably irrelevant), and more-or-less in Julia. Not completely forgotten are Python and JS/JQuery/CSS/HTML. Programming in Julia has been of some use in my work, but was also hobby to a large part. I’ve authored following registered packages: LabVIEW0.jl & LVServer.jl, Mendeleev.jl, YAArguParser.jl, GivEmXL.jl.

I’d like to start some project of general use for the Julia community of some part of it. Ideally I’d work mostly independently, but with some support from those who are interested in the results. Any suggestions?

One idea is a GUI for creation of Julia projects or packages. I’ve started it about a year ago, then put aside for varius reasons. The motivation has been, that especially for a fresh Julia user it is difficult to orient himself in the plethora of available options. Personally I have, after trying PkgTemplates.jl and PkgSkeleton.jl, settled with just using ] generate command, with copying all the rest from my other packages if necessary.

In this project (working title PackageInABlink ), Blink.jl is used for GUI, and PkgTemplates.jl as the backend. In the meanwhile I’ve however learned about BestieTemplate.jl, which might offer an advantage of being able to update packages. You can see mockup here , and some programming behind it is also implemented. I chose Blink.jl because of some experience with HTML and JS (though I dislike JS), but if there a better alternative, I’d consider it. @abelsiqueira , @gdalle - what’s your opinion’?

18 Likes
2

I would personally love something like the targets R package within Julia for reproducible analysis pipelines. Given your wide range of expertise and depth of knowledge, this might be a really fun project. Here’s a little ChatGPT summary of the package with some light editing from myself:

targets is a tool for managing reproducible and efficient data analysis pipelines. It organizes workflows into directed acyclic graphs (DAGs), where each node represents a step in an analysis and edges represent dependencies. By tracking these dependencies, targets ensures that only the necessary parts of a given pipeline are re-run when changes occur, optimizing computational resources. The package also supports parallel computing and various storage options, making it scalable for large projects.

targets integrates well with version control systems like Git, allowing for collaborative and trackable development. It provides flexibility by enabling the incorporation of custom R code and the use of other R packages, and it includes functions for visualizing pipelines to aid in understanding and debugging workflows. In short, targets streamlines the creation of reproducible and scalable data analysis projects in R.

1 Like
3

I’d recommend using Bonito.jl instead of Blink, since Blink isn’t really maintained as much anymore.
It should work quite well with electron, the browser or VSCode plotpane, and offers basic UI elements to create the app, but you’re free to use whatever HTML/CSS/JS you want :wink:
Of course I’m a bit biased here, being the author of Bonito.jl :wink:

6 Likes
4

I imagine “tooling” packages would be of very high impact, making everyone’s life better. E.g. helping with Infiltrator.jl, TestItems.jl, JuliaSyntax.jl, JuliaLowering.jl (to name some I care about). Some such packages are dark magic, but others are much more “reasonable engineering” focused.

In the monthly newsletter I now also post “calls for contributors” where maintainers of packages announce if they would like to have more folks helping with specific projects: Newsletter - Julia Programming Language

11 Likes
5

I started working on that with Waluigi.jl, but the project I was making it for got cancelled, so I haven’t had time to work on it.

I’d love to pass the reins if @Eben60 wanted it.

1 Like
6

:grinning:

Well, I had a look onto the targets R package - I don’t think it’s realistic for me to port it or write something similar, especially as that is completely outside of my field of knowledge.

7

That is outside of my field of knowledge just as well, but at least the code size is manageable, the original package is in Python which I can read, and substantial work already done. I’m not sure this is what I’m looking for, but might throw a closer look onto it.

8

isn’t really maintained “as much”, or isn’t really maintained?

Blink.jl has some 50 packages downstream - I hoped it’d stay maintained for years to come.

I’ve already written some code for Blink.jl . But on the other side, coming back a year later makes starting afresh not a big difference. I’d have a look.

9

The bones are there, and the concept is pretty straightforward! If you do decide to take a crack at it, I can take some time to improve the internals documentation and testing a bit. I’d love to have a nice language for scheduling jobs with Dask.

Feel free to Slack me if you want to chat!

10

@Eben60 your past work on ChemFormulae.jl and other chemistry efforts to convert chemical formulas into compositions was really useful. Maybe you could refactor the code and make a single package for chemistry work in Julia? Could this package be Mendelev.jl?

1 Like
11

I don’t see that as realistic.

ChemFormulae.jl was a pretty simple thing made within a couple of days, but that was just at the at the entrance of an endless rabbit hole :slight_smile: - and for a descent into a rabbit hole I’d prefer to know beforehand what are my objectives.

2 Likes
12

I remember coming across Waluigi.jl once upon a time! I thought it was such a great initiative but sorry to hear it got cancelled! That said, let me also CC @jpsamaroo and @pszufe as I think they’ve also been looking at something related to this discussion; maybe they might have more thoughts to add here in case @Eben60 is interested at all in this.

1 Like
13

Ah whoops – I was more getting at the fact that it sounds like you have exposure to a lot of various domains but I can see my contradiction in :joy: :man_facepalming:

14

While more work is needed, DataToolkit.jl should be able to do exactly this sort of stuff. See the graphs under “Recipe Hashing” on Cache · DataToolkitStore.jl.

Currently using it like targets/Waluigi is a bit awkward, but mostly because I haven’t had much cause to use it that way.

5 Likes
15

First of all thank you for your motivation!
I agree with @Krastanov that general Julia tooling is likely to have the most impact, so I wouldn’t necessarily encourage personal scientific projects to pile up here, especially if they’re outside of your expertise.

I think this could be a really nice idea, and lower the barrier of entry for beginners. I like your mockup, and it could be a more pleasant experience than the terminal interface for many people.

Indeed PkgTemplates.jl has been the standard for a long time, but after chatting with @oxinabox at JuliaCon we might recommend @abelsiqueira’s BestieTemplate.jl as the new default. One reason is the update mechanism, and the other is that neither Frames not I have much time for maintenance of PkgTemplates.

5 Likes
16
1 Like
17

My original intent was to keep Waluigi.jl’s backend generic so that people could use Dask.jl directly, have it construct DataToolkit.jl projects, or use some not-yet-existing backend that uses SQS queues and such.

When I read the docs of DataToolkit.jl, I get the sense that its audience is scientists who need reproducibility, so a static DAG is assumed. Where pipeline orchestration needs each node to be able to dynamically generate requirements and outputs.

But maybe I’m wrong about what Datatoolkit.jl can do!

18

The VSCode extension for Julia is another potential candidate project @Eben60.

19

I think kept alive is the right word :wink:
The initial authors are gone and most people who have put some time into it also haven’t committed in a long time…

1 Like
20

Would be fantastic if the GUI is accessible within VSCode, e.g. as an extension. Imagine just clicking the menu “Project” → “New Project”.

Once a project has been created already, a GUI package manager would be nice to have, so users could click “update”, “resolve”, “test” etc.

4 Likes