Stipple: Reactive data dashboards with Julia [WIP]

I wanted to show a quick teaser of an early prototype of reactive UIs with Julia and ask for some early feedback on the API/look-and-feel. This is a data table component that I’m currently working on (two-way reactive form inputs and text elements are already functional).

I took a declarative approach where one sets up a ReactiveModel - and Stipple automatically sets up the reactive UI.

The example shows the default rendering of a DataFrame loaded via CSV.

Quick demo:

22 Likes

It is very nice the component!

About the API I find it very nice. Only about the look-and-feel to change by default the color background depending on the row (odd or even), and remark more the names of the different columns.

Great work!

Thanks!
Yeah, in terms of formatting there will be more options, the objective is to expose all the various options presented here:

1 Like

Interesting! I’m actually going to be doing something similar to your example (displaying a table on a webpage from a CSV), but I was planning on using Franklin.jl and JQuery DataTables. I guess the difference with Stipple is that it is more of a web app whereas mine is going to be a static page (even if the table itself has interactivity)? Also your example is much more condensed and clean than what I imagine I’ll need to do.

But Quasar looks pretty dang cool too… I should look more into that. Are you wrapping it generally or just the Table API?

Very cool! How does this compare to Dashboards.jl? That exposes all of Plotly Dash’s components — could you lean on that work to easily import lots of reactive components?

Yes, Quasar is pretty powerful. That was actually a difficult decision, choosing one of the many available UI frameworks, with the final round between Vuetify and Quasar. In the end I went with the less popular Quasar due to having more powerful components and better docs.

For the prototype (alpha) the objective is to wrap all the Quasar components. But one of the main architectural objectives is for the framework to be easily extendable with any Vue component.

They will do the same thing, but the way they do it is different. Dashboards.jl / Dash.jl is about adding Julia as a Dash backend. Stipple is about building and publishing interactive data dashboards with Julia (just like Genie, building a truly native Julia experience). And this difference, I think, is fundamental:

  • I’m focusing (read obsessing) on architecting an efficient, terse, readable, pleasant Julia API. As the resulting frontend is abstracted away, the Julia API is the frontend for the developers. I believe this is easier when you can build from scratch and don’t have to integrate with an existing framework. This was a key factor.
  • a very difficult decision was the choice of frontend framework. Dash uses React.js - Stipple uses Vue.js. React is pretty good but it has a very steep learning curve. It’s just hard (per a recent article from Dash, they have some 50 devs, most of them working on React - Dash defines itself as a React company). Vue, on the other hand, is very easy to learn and I’m hoping it’s going to make it very easy to extend Stipple with extra components. Stipple uses “plain” Vue components (so no JS processing is needed) and all it takes is writing the Julia backend, by specialising a few methods to teach Stipple how to sync the data.

To conclude, they won’t be compatible. And in terms of look and feel, Stipple will most likely be closer to RShiny, possibly combined with RapidMiner (for instance, planning on providing some built in statistical widgets and would definitely like to provide a visual drag-drop editor at some point).

13 Likes

that sounds awesome

The data table API is almost ready! :partying_face: Next, plots!

An example with all the available options, rendering a data table and its associated stats, fully reactive and synchronized.

Output

Code

14 Likes

Hello.
The link doesn’t seem to work:
https://github.com/essenciary/Stipple.jl/

The repo is private. We have set up a small team (me doing Julia and JS and a friend doing design and JS) and we’re planning on having the project released in time for JuliaCon. At this point we’re not ready to release it as an open source project as all our time and effort goes into building and we can’t support it.

Plan is to release code, docs and any other info in approx 3 months time. We submitted a proposal for a 30 minutes presentation at JuliaCon and if that’s accepted, that will be the great reveal :slight_smile:

I’ll keep updating the progress here and we’ll release code as soon as possible.

14 Likes

Is this just for data stuff or can it be used to build front ends to general web apps?

TLDR; Yes, it will be appropriate for any kind of web apps.


In terms of architecture there are 3 layers and they go from lower level/more flexible to higher level/less flexible.

Lowest level is Genie which implements a full HTML API. It’s the least smart but most flexible. If we want to create a table, we have to manually write every part, something like

table(class="table") do 
  tr() do [
      td("Foo")
      td("Bar")
    ]
  end
end

Or we can use the HTML based templating language to write HTML with interpolated Julia which would generate the above code internally.

The big advantage of using Genie as the foundation is that we can use everything it has or will have, like session/cookie management, MVC structure, generators, caching, authentication, routing, etc. And this is key for writing generic web applications.

Mid layer are Stipple elements, which are higher-level, like for example there is a table(df::DataFrames.DataFrame; args...) which renders a DataFrame. So one doesn’t have to manually build a table, but is harder to customize or control every little thing in how the table is rendered. This outputs a static HTML table in a line of code.

Last layer are the Stipple components which are reactive but they require web sockets and they need Vue.js among other things. They provide a lot of features out of the box, but already one needs to opt-in to using Vue.js, Twitter Bootstrap, and other libraries which are bundled.

For everything except plots we’re using the Quasar UI components library (the Vue components: https://quasar.dev). The focus at this point is on the Stipple/Quasar/Vue components and for the first release we want to have reactive tables, plots, forms plus layouts (also optionally reactive as in show/hide parts of the page depending on the state) and typography (so H1-6, paragraphs, etc).

One can mix and match APIs from all these layers (Genie is a package, Stipple is a different package which has Genie as a dependency, StippleQuasar is yet another package which has Stipple as a dependency, and so on).

In phase 2 we will add wrappers for the rest of the Quasar components, as I’m sure any non-trivial app will need things like alerts, tabs, lists, images, popups, trees, avatars, etc. But I architected this to be easily extendable so hopefully other developers will contribute. There are thousands of Vue.js free components out there and ultimately we’d like to see this as a thriving ecosystem which would include both free and commercial high quality user contributed packages, themes, plugins, etc (from UI components to hosting, monitoring, scaling, pushing notifications, analytics, and what not).

We’re currently discussing with some startup incubators to help us realise this vision as obviously we’ll need a larger team to go beyond the first, maybe 2nd phase of the project. If anybody has experience, tips or recommendations in regards to funding such projects, that’d be very appreciated.

7 Likes

That was actually a difficult decision, choosing one of the many available UI frameworks, with the final round between Vuetify and Quasar.

I’ve developed several applications with Vue/Vuetify and I’m a big fan. I just read an article this morning about some of the features coming in Vue 3 and I think it’s going to be a fantastic framework.

Nonetheless, what you have here looks really great! Keep up the excellent work, I’m a huge Genie.jl fan.

Do you have plans to include plots? If so, what it would look like?

Makie, VegaLite, or anything else? What about the balance between performance and interactivity for data-intensive apps, choosing between WebGL / Canvas / SVG for different plots?

What about GUI and plots interactions, like clicks on plot elements (data points or bars etc.), popup menus, range selections, drag, zoom, crossing cursor, and so on?

Thank you very much @mthelm85 :slight_smile:

Plots, yes!

The architecture is: data on the backend, rendering on the frontend. Vue has a model and so does Stipple. Every field in the Stipple model (the model is an instance of ReactiveModel) is automatically exposed in the Vue model and is automatically set up to be watched for updates.

When a watched value changes on the front, Vue updates the corresponding field in the Vue model, and gets pushed over the sockets to Stipple, which updates the corresponding field in the ReactiveModel. If the field is a Reactive value (which is a wrapper around Observables at the moment) all the associated listeners are triggered. The reverse also holds: updating a field in the ReactiveModel from Julia updates the values in the Vue model and the changes are propagated through the UI.

Plots, as long as they are wrapped in Vue components will just adhere to this workflow: their data is part of the Vue model and the plot’s events trigger updates in the Vue model which are then propagated onto the backend, where the corresponding observables are triggered and the data is synced. Or the other way around (so we can for example “stream” data to a time series).

So whatever the JS plotting libraries provide, that’s what we’ll have. Very simply put, Stipple is a framework for running reactive Vue components on a Julia backend. For the first release the focus is on nailing the architecture, building a prototype and showing off what is possible and not on packing advanced features. The first plotting library (I’m working on integrating it now) will be Apexcharts https://apexcharts.com/

Like I said, we can later add support for any Vue plotting library through additional packages. Here are a few and there are lots more: https://madewithvuejs.com/blog/top-vue-js-chart-components

Data on the backend, rendering on the frontend also means no server side plotting out of the box - but of course, Julia plots can be added to the app “by hand” (generating a PNG/SVG or a JS via PlotlyJS, etc). The benefit here is that there is no time to first plot and everything fits nicely into the framework. The downside is that we’ll have to see how well the JS plotting will behave with large amounts of data. It goes without saying that this is all very experimental - we’re making some design choices and we’ll have to see how they hold against reality.

6 Likes

Thanks for reply!

So, if there are some grouping or aggregations, where thousands data points are transformed into a single bar, all that points will be transferred to JS side, but not a single bar primitive?

Correct!

For instance, this is what the data behind this chart looks like:

series: [
        {
          name: "Marine Sprite",
          data: [44, 55, 41, 37, 22, 43, 21]
        },
        {
          name: "Striking Calf",
          data: [53, 32, 33, 52, 13, 43, 32]
        },
        {
          name: "Tank Picture",
          data: [12, 17, 11, 9, 15, 11, 20]
        },
        {
          name: "Bucket Slope",
          data: [9, 7, 5, 8, 6, 9, 4]
        },
        {
          name: "Reborn Kid",
          data: [25, 12, 19, 32, 25, 24, 10]
        }
      ]

The resulting chart is animated, fully interactive, etc:

And:

1/ transferring JSON over Ajax/WebSockets is fast, the numeric payload is small. Binary JSON or MessagePack could also be used in future versions for extra compression (with extra CPU usage on both client and server - yup, there’s no free meal).

2/ JavaScript is (was?) inappropriate as a full fledged programming language but some things it does very well. For instance, it can use the GPU and it can render 1,000,000 points on a scatter plot quite fast! https://jsfiddle.net/gh/get/jquery/1.7.2/highslide-software/highcharts.com/tree/master/samples/highcharts/boost/scatter

5 Likes