Shiny for Julia [work in progress, early request for feedback]


#1

I have recently begun working on developing a shiny-like web UI stack for Julia. This is very early work (maybe 10h or so) but since I’m building it on top of Genie, the low and mid-level tiers are already done, so I can focus directly on the UI layer – thus development should be reasonably fast.

I’m very interested in early feedback – about everything really, but especially about:
1 - architecture
2 - API
(@ChrisRackauckas per our chat in London, I’d appreciate your thoughts).


The objective is to provide a high-level API which allows building web pages directly from Julia (no HTML, CSS or JS). This includes, in phase 1:

  • wrappers around all the standard HTML 5 elements (done)
  • wrappers around the Twitter Bootstrap CSS framework. The idea here is that Twitter Bootstrap provides a powerful, responsive (grid-based) layout library, a multitude of extra UI components and powerful theming capabilities (thousands of free and commercial themes available to style anything). The wrappers should make it simple to use, so that instead of <div class="container"> we’ll just say container(). This means there’s no need to learn the CSS classes in Bootstrap. (this phase is work in progress).

My first attempt (actually working code):

import Genie.TwitterBootstrap.Layout: container, row, col
import Genie.Flax: doc, html, head, body, link, h1
import Genie.Renderer: html!
import Genie.Router: route
import Genie.AppServer: startup

view = container(id = "main", class = "black", fluid = true) do; [
            row() do; [
                col() do; [
                    h1() do; [
                        "I'm the cool title"
                    ]end    
                ]end
            ]end
            row(data_json = "http://someurl", alignitems = "center") do; [
                col() do 
                   "I'm first!" 
                end
                col() do 
                   "I'm number 2!" 
                end
                col() do 
                   "Nooo, I'm last :((" 
                end
            ]end
        ]end

"<div id=\"main\" class=\"black container-fluid\"><div class=\"row\"><div class=\"col\"><h1>I'm the cool title</h1></div></div><div class=\"row align-items-center\" data-json=\"http://someurl\"><div class=\"col\">I'm first!</div><div class=\"col\">I'm number 2!</div><div class=\"col\">Nooo, I'm last :((</div></div></div>"

layout = html(lang = "en") do; [
            head() do; [
                link(rel = "stylesheet", 
                     href = "https://stackpath.bootstrapcdn.com/bootstrap/4.1.3/css/bootstrap.min.css")    
            ]end
            body() do; [
                view
            ]end
        ]end

"<html lang=\"en\"><head><link rel=\"stylesheet\" href=\"https://stackpath.bootstrapcdn.com/bootstrap/4.1.3/css/bootstrap.min.css\"></head><body><div id=\"main\" class=\"black container-fluid\"><div class=\"row\"><div class=\"col\"><h1>I'm the cool title</h1></div></div><div class=\"row align-items-center\" data-json=\"http://someurl\"><div class=\"col\">I'm first!</div><div class=\"col\">I'm number 2!</div><div class=\"col\">Nooo, I'm last :((</div></div></div></body></html>"

route("/") do 
    html!(layout)
end

startup(8001)

This builds a responsive web page using Twitter Bootstrap:

There are a lot more layout options which are already available (per https://getbootstrap.com/docs/4.1/layout/grid/)


1. Architecture

At the moment, invoking these functions returns HTML code as a string. I like this because it’s fast. The alternative is to get back HTMLElements objects (with properties and children and a string representation) – but I’m very concerned that holding a large structure in memory to represent a big HTML document will be very resource intensive. This is a complicated problem with all the front-end frameworks (especially when searching for nodes) leading to approaches like delayed and bulk updates, shadow-DOM, etc. So I’m quite happy with the strings, in this regard.

The issue with the strings though is that this is basically immutable data. Once you invoke a function to build an element you get back the string representation and you can’t change its properties (other than through clunky string manipulation). This can become a problem in the future when I’m planning on having more complex UI elements with two-way communication (the UI elements will update their Julia representation in the backend, to reflect user input). However, I think this can be addressed with a purely functional approach. Instead of building and manipulating objects, we can define a callback – a function which will be invoked on the backend when a certain event is triggered on the frontend.

A possible middle ground might be to provide a few helper methods which accept some HTML string, pass it into Gumbo (to turn it into a DOM structure), apply the changes and return the modified string.

Thoughts?

2. API

At the moment, the wrappers take a variable number of keyword arguments and a function which represents the children. When more than one child, the function should return an array, hence the [ ... ] square brackets.

I think this does a good job of representing a nested DOM structure but it’s too noisy due to the ]end garbage. Any suggestions on how to make this more beautiful and readable?

Update 1

I justed extended the API a bit so now we can do:

view = container(id = "main", class = "black", fluid = true, [
            row(
                col(
                    h1("I'm the cool title")
                )
            )
            row(data_json = "http://someurl", alignitems = "center", [
                col("I'm first!")
                col("I'm number 2!")
                col("Nooo, I'm last :((")
            ])
        ])

Lispyyyy! :smiley: Definetely cleaner so maybe we have a winner?


Also any other feedback is welcome. I’m not a heavy shiny user so I’m now learning a lot more about it (and Plotly’s Dash) – the idea is to come up with something inspired by these, but 100% Julian and obviously better :smiley:


#2

I think the main aspects that need to be addressed are:

  1. Scalability
  2. Interactivity
  3. Deployability

This seems like a good start, but I think that the API is still very low level. I am pretty sure (1) will be addressed since Julia code will be fast enough when properly engineered, but (2) is hard to do without a really nice API that does a lot of hand holding. What we need out of Julia-based webapps for scientific computing and data science is an easy way to directly take Julia code for visualizations and plop that on the web. This JuliaCon talk shows some great strides in that direction:

Notice that it’s features like integration with Plots.jl and the ease of creating sliders and button that really makes this a workable. Interact.jl and WebIO.jl have the API down IMO, and being able directly integrate that into Genie.jl is what I think would be helpful.

Then this gets to deployability. I think there’s two types of deployability that we are looking for:

  1. Upload some code to someone else’s server, pay for the hosting service and have my application run without worry.
  2. Setup my own server and add some pages with interactive Julia-based applications.

I think that integrating Interact.jl with Genie.jl would be a nice concrete way to get (2), but that’s for the hardcore users. (1) is very interesting to think about: could someone setup a SaaS service for hosting Julia-based webapps? I’ve been talking to some folks about setting up such a service based on Interact.jl applications.

So here’s where my thoughts have evolved to (over the last few days). Julia has this idea of Projects and Applications

https://docs.julialang.org/en/stable/stdlib/Pkg/index.html#Glossary-1

It would be nice if a webapp could just be a deployed Project that works locally. Interact.jl is already a usable framework for locally building and testing web applications, so a Project with a a main that builds a webapp via Interact.jl could at least be a good starting point. Then to make this a real web application, we would need tie-in from Genie.jl that let us just put such an app as a folder in our Genie.jl project and then the functionality to generate a page (or embed it in another page) would be deployment. Additionally, an online service where we could upload the Project (or point to a Github repo) and pay for hosting would be a nice avenue for those who don’t want to run a server themselves.

Of course, it doesn’t need to be based on Interact.jl but it would need interactivity with data (DataFrames/JuliaDB support), plotting libraries, and some dead simple Julia code to tie together widgets, and Interact.jl already has it.


#3

I’m really glad that work is being done on this! If cool web apps with a nice “powered by Julia”-logo start popping up then that could be a great new way of marketing Julia.

A thought: you probably don’t need to go overboard on the functionality of the layout & formatting API. Just implement the absolute essentials so that simple stuff can be written in pure Julia, but then make it easy to fall back on HTML + CSS for anything advanced or customized.

But I’m fuzzy on the interactivity part. We can’t run Julia in a browser, or at least not until we get full webassembly support. So would anything interactive be restricted to a number of premade widgets (implemented in Javascript), or does your approach allow more flexibility than that? Can you clarify how that would work?

For example, suppose I made a web app to explore the Mandelbrot set, and I wanted to let users zoom into the chart by drawing a square to zoom in on. Then the coordinates of that box would be sent to the server, and the chart would be updated. Could I implement this myself, or would I need to wait on draw-a-box-to-zoom support from your package?

I assume we can never reach the utopia of full interactivity with this approach. For example, here is a demo of monotonic cubic spline interpolation that I wrote in Javascript using PlotlyJS. Note that you can drag the breakpoints, and the interpolated curve gets redrawn while dragging. We won’t be able to write something like this in pure Julia and have it run in a browser, will we? (Again, I mean not until webassembly.)


#4

There was already some discussion on how to combine the two approaches (Interact - Genie) here.

I would really hope for Interact to become a general ecosystem for interactive recipes (we’ve finalized the recipe design with @shashi in London, I’ll be writing a blog post + announcement about it soon) and there are already a fair number of widgets in the Interact ecosystem (both “atomic widgets” like sliders, buttons, toggles, latex, notifications, etc as well as complex widgets like StatPlots GUI and the ones showcased in the video above). In terms of layout component, I’ve wrapped a bunch of stuff from Bulma CSS (a Bootstrap “competitor”) and plan to wrap more with a Julia interface (it’s trivial but requires a bit of thought when these components have some javascript values that you want to share with Julia).

To deploy Interact currently the best way is to have a server and deploy with Mux. It is reasonably straightforward (webio_serve(req -> slider(1:100)) gives you a web page with a slider) but requires the user to have a server. I’m working on a Interact demo website to complement the Interact docs, based on this approach. Alternatively, there was some discussion with @avik in London to add Interact to JuliaBox and use the new (I think unreleased) “deploy app” feature of JuliaBox.

As I mentioned in the issue linked above, if Genie could help simplify deployment, that’d be awesome!

I do not know how that’d work in Genie, but I can tell a few words on how that’s done in Interact. Basically what happens is that WebIO can share a Observable (a Ref that can do some callbacks when the value is changed) between Julia and javascript. Generally it is quite easy to sync the WebIO observable to the Julia value (for example here is how it updates the WebIO Observable). Normally the javascript library you are binding exposes some variable that you can link to your observable. For simpler cases like form data you can easily bind it with Knockout.jl (see here) but that’s generally not needed as most form input is already wrapped in Interact.

Check out the WebIO docs. You can definitely pass javascript variables to Julia as Observable and work with those so I don’t see any obstacle to this kind of visualization in principle.


#5

I am a total newbie with Julia but this sounds like great work in progress! Is Predix UI still part of the plan?

https://discourse.julialang.org/t/is-anyone-developing-a-spa-package-like-r-shiny-or-python-dash/5733/17


#6

@ChrisRackauckas Thanks very much, that’s some great feedback!

I’ve been thinking about these things myself, but I didn’t want to go too far in this early request for specs.

Definitely, this is low level - but it made sense to start with the basics (laying things out plus simple components) and grow from there. Next step is to wrap the Predix-UI components which provide rich UI elements for pretty much everything. Predix-UI is developed by GE to be used for building dashboards for their data. They have a wide array of UI components, from data tables, to plotting, to time series visualisation and everything in between: https://www.predix-ui.com/#/gallery

Hosting - I was thinking about the same. Just have a simple API to publish the app onto AWS, Azure, Heroku, etc. As a bonus, at some point it would be nice to have a UI editor, maybe based on https://grapesjs.com – this could also have a “Publish” button that would automatically set it up.

Interact is super cool, but it seems to me that the focus is on Jupyter Notebooks. I care more about dashboards (so data-focused web pages that run on their own). That being said, a generic wrapper around the Jupyter API (so that any Jupyter enabled components would work) would be amazing.

What do you think of the Predix-UI components? Would they serve the purpose?

Re other aspects, I need to look into these: watch the video – also, I’m still a bit in the dark about the distinction between app and project, but what you propose does make sense.


#7

For more advanced UI elements my idea is to wrap Predix-UI components (https://www.predix-ui.com/#/home).
There are a few very cool things about this:

1 - they use web components – which are encapsulated UI elements which extend the HTML standard, so for example a heat map has a dedicated tag: <px-vis-heatmap></px-vis-heatmap> (https://www.predix-ui.com/#/elements/vis/px-vis-heatmap). The beauty of this is that it nicely separates frontend from backend and everything is configured through data exchanges as JSON.

2 - these components know how to trigger relevant events. My idea is to have glue code (already provided by the framework) which automatically captures these events and sends them towards the Julia backend (via websockets). In the backend, the developers would be able to subscribe and be notified about the events they care about (by associating a handler, a function). And send back data in response, which would cause the UI element on the frontend to update accordingly.

Does this make sense?

Re webassembly, I’ve read that it’s already gotten pretty good support in most browsers. However, what people seem to forget is that the problem of rendering UIs still remains. Even if we can run Julia in the browser, the issue of rendering UI components and interacting with them still remains. Think about Julia desktop apps: Blink (so web UIs) is widely used because good UI libraries are scarce. I’m not sure how that would work - maybe by using the native platform UI elements, or using something like a game engine (like the Unreal engine for web)?


#8

That is excellent! I would love to learn more about using Interact outside Jupyter notebooks. Do the widgets work in any web page?

Re data exchange, I think architecture-wise we’re on the same page :slight_smile:


#9

Yes, exactly - please see my replies in this thread. Looking forward to getting some feedback on Predix from the Julia community. It definitely fits in terms of architecture (web components). But I’m curious if the UI components themselves would serve our purpose (if they’re good enough).


#10

At the moment we use Mux to serve them in a webpage (I think it doesn’t have to be Mux, it’s just handy because of the WebSocket integration, otherwise you need to write the WebSocket stuff on your own). I’m working on a website of InteractDemos (repo here). You can clone it (it’s on Julia 1.0), instantiate the manifest and run it locally or on a server. Unfortunately instantiating the manifest is not quite enough until this PR gets merged, for now you need to manually ] dev Mux and checkout that PR.

I think it should be feasible to serve them using Genie as the Mux integration is very little code. I definitely don’t know enough about WebSockets or Genie but it’d be really cool if you could give this a shot!


#11

I don’t think it has a focus on Jupyter notebooks. It just is an API for creating structured apps, and one of the places it can render to is Jupyter notebooks, but you can also use Blink.

That is a nice set of widgets, but it still needs to tie into Julia’s visualization libraries. Plots.jl has a link to Plotly and GR (for pngs), and using these will be much more powerful since the recipe system and 3D plots are essential.


#12

Re webassembly, I’ve read that it’s already gotten pretty good support in most browsers. However, what people seem to forget is that the problem of rendering UIs still remains. Even if we can run Julia in the browser, the issue of rendering UI components and interacting with them still remains. Think about Julia desktop apps: Blink (so web UIs) is widely used because good UI libraries are scarce. I’m not sure how that would work - maybe by using the native platform UI elements, or using something like a game engine (like the Unreal engine for web)?

Yes that’s how that works currently: they just paint gui widgets from a lib to canvas and don’t use the DOM.


#13

agreed on that. I’d rather see plotly used rather than introduce yet another plotting/GUI lib


#14

How about reactivity? This is an essential part of the Shiny architecture and what makes it high level and very fast to implement something.


#15

+100. For this usecase PlotlyJS (that is getting very good WebIO integration) or a future Makie WebGL backend really are ideal solutions.


#16

Yeah, I fully agree - any solution should leverage the existing Julia ecosystem. So support for Interactive would be a must. And maybe going the reverse route of adding Predix as Interactive custom widgets.


#17

Thanks for sharing the repo - looking into it :slight_smile:

No biggie, I can follow the steps to get it running, thanks for the info.


#18

@piever The Observables documentation seems to be broken.

Also, looking over the examples in the docs. There’s one with a button where one can be notified about clicks. Wondering if it’s possible to subscribe to other events, like say “mouseover”?


#19

D’oh! I’ll try to fix it, in the mean time it’s also available here.

It is and there are a couple of ways. You can add a javascript callback to any event (see here) and if you want to handle the event in Julia, the easiest is to update a Observable in the WebIO scope, for example here:

    w(
      dom"button"(
        "generate random",
        events=Dict("click"=>@js () -> $obs[] = Math.random()),
      ),
    )

you could then add a listener to obs as a Julia observer.

Otherwise, if you’re binding the javascript variables using Knockout, you could also pass the data-bind attribute. For example use attributes = Dict("data-bind" => "event: { mouseover: mycallback"}) where mycallback is you javascript method. This option I think is only worth it if you’re already working in a Knockout scope and you want the callback to interact with the Knockout observables.


#20

That looks really cool. @essenciary I just want to say that as an end user, I’d be very happy if Genie integrated with the webio/interact stack. It would make life easier for me as a user/casual developer to have less stuff to learn because it ameliorates the two language web problem and there is the ecosystem commonality.

And that’s beside the potential benefits for easing maintenance burden and boosting community interop/innovation/widget diversity.