Best way to wrap a Julia package for use by those that can't code?

Goal

My goal is to gather input from the user with as little barrier to entry as possible. Then process it with Julia, and finally collect tables, graphs, and text into a report (preferably PDF). The user inputs I need are a few file locations and floating point numbers. The user should have a place to conveniently enter these inputs and then just click “run” to generate the report from their data. I don’t want them to have to interact with the package functions directly.

Background

I know Julia and some very basic LaTeX, but that is about it. I have no experience with GUIs or the Web Stack, but I am willing to learn any new technology that would help. I am looking for some direction on what to focus on learning to achieve my goal above.

Current State

Processing

I have the package itself in place already with functions for transforming, plotting, and printing the data.

Pre-Processing

I have included a directory called “run” in the repository containing the package with a “runthis.jl” and some auxiliary files. The “runthis.jl” file is the only file the user needs to change and run. It defines the input data I need with commented instructions next to each line. Then it includes all the auxiliary files in the “run” directory which in turn call the package functions with the correct syntax in the correct order.

Post-Processing

The post-processing functions are also called by the auxiliary files inside the supplemental “run” directory. I am using PrettyTables.jl and Plots.jl to print output to the REPL and save figures to disk. Is Weave.jl still the best choice to get these into a PDF instead? I haven’t had a chance to try it out yet. (I don’t intend to show any code in the report.)

Issues

The current set-up is working okay, but there must be a way to simplify it further.

  1. The user still has to dive into the repository and sift through code to enter their inputs.
  2. The user usually copies the “run” directory from the repository and stores it next to their data before changing the inputs. The “runthis.jl” file has using MyPackage at the top, so this works fine. Except then I don’t know how to maintain version control between the two. If I update the package after the user has copied the “run” directory, then “runthis.jl” will probably break, and the user won’t know why.

I’d appreciate any tips. Thanks.

I have no experience with GUIs or the Web Stack, but I am willing to learn any new technology that would help.

Indeed, these were invented to solve problems like yours. If you are willing to learn, then I suggest learning how to stand up a web service that runs your julia code. The service should serve an HTML page that includes GUI elements that allow the user to set the parameters. The page calls back to the service, and the service can respond with the .pdf if it is useful to the user or write it to file if it is only useful to you.

If you implement this as a web service with a GUI HTML page, then the user doesn’t even need to install julia.

Start with HTTP.jl.

4 Likes

Can your package be provided as a web server? Then you could build a very simple “upload” site for the data, run the package in the server in the background, and report the result. I have done that with a few packages.

Otherwise, I think a good alternative would be to provide a double-clickable julia script that launches a very simple GUI in which the user would provide the data and click “Run”. I would also be interested in advice on how to provide that. This kind of thing, maybe: GitHub - GenieFramework/Stipple.jl: The reactive UI library for interactive data applications with pure Julia.

Yeah, I think I will need to focus on something local for now since the data files can be large.

I think the biggest annoyance right now (besides #2 Version Control) is having to copy the file locations into the code as a String. Do any simple GUI file selectors already exist?

As an alternative to a web server, you could also provide a Pluto notebook. The user can interact with it using PlutoUI elements, including File Picker.

I thought about that. Is it easy to create an “installer” (for windows users, in particular) that creates a double-clickable file in the desktop that will launch that Pluto notebook?

1 Like

You could create a Windows link file with the Target
C:\Users\xxx\AppData\Local\Programs\Julia-1.6.1\bin\julia.exe -e "using Pluto; Pluto.run(notebook=\"test.jl\")"
However, for installing Pluto and creating this link there is no ready-to-use solution yet.

2 Likes

Am I able to keep all this traffic on a shared local server rather than through the web with HTTP.jl?

I would guess the easiest path is keep everything on local machine, via localhost, like jupyter. Sharing on LAN just require some open ports? And it definitely will need some network configuration to share it to the whole world. You can run a local server on http://localhost using HTTP.jl.

1 Like

Yes, as @cchderrick mentions, your service can open a port that is not exposed outside the local server, or outside the local network.

Nathan, I got a couple of Questions to ask

  1. What platform should the “solution” run on?
    Windows? Mac? Linux?

  2. How many users need to use the “solution”? 1? 10? 100? 1000?

  3. How big are the files that the users provide? 1kb? 1Mb? 1 Gb?

  4. Where should the users run the “solution”? On their desktop? Do you have to install it manually on their desktop?

  5. How often do you need to “update” your code? Daily? Weekly? Monthly? Yearly? Never?

2 Likes

Currently,

  1. Windows OS
  2. 6 users
  3. Between 10Mb and 1Gb total input size
  4. Running on personal desktops. All users have access to a shared network drive which is mapped to their desktops. The package is stored and accessed from there.
  5. I push updates to the shared version about monthly. I am still actively developing the features.

Improvements based on the current requirements is obviously the first priority, but cross-platform and easily sharable solutions would be good to know too.

With so few users that access the code through a shared drive, perhaps most expedient until you need a broader scope would be a simple gui in GTK.

1 Like

Perhaps longer term you could deploy the proposed web service in the cloud and the users can upload their data to cloud storage (like AWS S3) and point your service to their storage.

For a small group like that I would go the easy way: Install the Julia binaries from the site, double-click it (which will open the REPL), and just show them one step-by-step example with copy/paste input parameters. It can be simple as:

] add YourPackage

copy/paste this to run it:

using YourPackage

run_yourfunction(
    inputfile1="mdata.dat", 
    parameter1=10, parameter2=20,
    output="myoutput.dat"
)

With that approach telling them to up YourPackage whenever you want is also quite easy.

I’ll look into it.

That would avoid having to manage the “run” folder, which would be nice. I would just have to handle the different input options and analysis types with multiple dispatch. One concern would be that I now have to store documentation separately from the inputs, so it wouldn’t be as obvious if I change how you have to input something. Another would be more frequent typos, especially as the number of parameters change.

Maybe Pluto is the right move after all as long as I can keep the file management simple. It would provide GUI elements and Markdown documentation in place.

Or with keyword parameters, which might be easier, which can handle all default parameters, if these do not change every time (avoiding, then, some typos).

The current Pluto main contains the new automatic package management (see 🎁 Package management · fonsp/Pluto.jl Wiki · GitHub), which makes managing of package dependencies much easier. If you need non-published packages, you can set up a local Pkg registry.
For file transfer you can use the PluoUI File Picker for smaller files (up to a few MB). For larger files you can use remote storages or network folders shared between the users and the Pluto server.
One step further, you could set up a GitHub - JuliaPluto/PlutoSliderServer.jl: Web server to run just the @bind parts of a Pluto.jl notebook so that the users cannot manipulate the notebooks themselves anymore (and therefore not break them), but just interact with the UI elements.

2 Likes

Cool! I was just thinking if that was possible!

It should be, but I have not tried it myself yet.

1 Like