Can the two language problem ever be solved for GUI programming

guis

#34

Yes, but hardly convincing when one looks at the “Contribution activity” at https://github.com/andlabs


#35

I actually do not think we want to solve the n-language problem for GUI programming.

When someone wants to improve GUI programming, in mainstream/application programming languages they tend to Introduce an additional language/DSL.
Like XAML for C#/.Net, or as the JS/HTML/CSS split (Ok, I think that was more up to chance, than design).

I will also say that the only GUI worth worrying about is basically based on HTML.
Either is is a website, or it is something running in Electron etc.
Web is the future of GUI, whether we like it or not.
Thus even for local stuff one might as well run a website on localhost.
(e.g. as in jupyter)

What we really need is minimum effort to go from julia code running as a server,
to something the looks half-decent on in the browser,
and that someone with actual design skill can customise easily

I suspect this involves Page.jl/Mux.jl/HTTP.jl being using to expose a function bunch of restful end-points,
and then something on the browser that runs the javascript to do AJAX style stuff,
to talk to those end-points.
Possibly this front end can be automatically generated to some extent.

Which basically gets you something a like R’s Shiny.
(Like Shiny is basically exactly what is useful)


#36

Taking HTML as inevitable for GUIs ignores mobiles (which are dominated by native apps) and also forgoes high performance.

I find GLVisualize.jl an attractive approach to GUIs, based on OpenGL, that is compatible with Julia’s high performance emphasis. This is in contrast with the “everything is web” model.


#37

Taking HTML as inevitable for GUIs ignores mobiles (which are dominated by native apps) and also forgoes high performance.

My experience of mobile dev, which was admittedly very limited,
was that people were just writing webapps and either making them mobile compatible,
or using a tool that converts them to web-app (React/React-Native I think was the big one in that space).

This does indeed forgo high performance, I feel like it catches most of the low hanging fruit.
And to me that is people who are using Mathlab/Mathematica/Shiny:
“A few buttons/sliders and a plot”

Though you are right, high performance Vis is a thing, plotting thousands of points via SVG sucks.
Displaying ultra-HD medical scans, also sucks.
to me that is almost a separate thing from GUI. (almost)


#38

Let’s start with a sustainable Shiny-like in Julia first. I thought Escher.jl would be it but it’s kinda not up to date now. There is also Electron.jl now.


#39

Maybe not for immediately but next to a html/JS framework that would be wrapped in julia I think that for the nearby future a julia webframework that compiles to webassembly would be great. People think that webassembly is only good for low level static languages like C/C++, not garbage collected languages (because webassembly itself currently doesn’t provide a GC although that’s planned) but Microsoft just proved that wrong by implementing the clr in webassembly which has the added benefit that it can run directly .NET IL without compiling all the application code to webassembly (something e.g. TeaVM or Kotlin/native do in the JVM world). See https://blogs.msdn.microsoft.com/webdev/2018/02/06/blazor-experimental-project.
See http://blog.stevensanderson.com/2018/02/06/blazor-intro/ for a nice technical overview.
The demo video looks really cool and impressive with good download & runtime performance. I really think this is the future of webprogramming.


Make a webapp framework/ecosystem as awesome as R's Shiny: a challenge to Julia's community
#40

That’s very interesting! I also believe that webassembly (or another implementation of the same idea) will be the future of web programming.
Thanks for the heads-up about the M$ project!


#41

You’re welcome :slight_smile:


#42

WRT UIs, I think people are possibly missing the elephant in the room, namely AR/VR. IMO (based on having/using a VR system), I don’t think I’m going out on a limb predicting that all UIs will be AR/VR-based in the not too distant future. In particular, data manipulation/visualization will be one of the first areas to move in that direction due to the significant benefits of working in that kind of environment. If you doubt this, I urge you to check out VR (not the, currently, lame cellphone type, though) - you’ll most likely be converted. Sure, today’s hardware isn’t quite there in terms of comfort/resolution, but we’re not very far off.

If you buy into the above, performance, compatibility with AR/VR rendering and AR/VR HCI considerations should be a key factors when thinking about UIs which won’t soon become irrelevant.


#43

Hopefully web stuff will quickly move toward WebAssembly, because the performance of Electron apps is quite bad right now… We can’t even have spinning cogs in Atom!


#44

You can get quite decent performance for 3D or plotting a lot of 2D points via webgl. Maybe not for very high density 3D medical scans or scientific visualizations but for that you would use something like VTK on a high graphics workstation.


#45

My impression of WebVR (which uses WebGL) is that there’s a very significant performance drop with that vs running VR out of the browser, so, while you might consider it adequate for some things, I’m not sure I’d want to use it given a choice. So far, when it comes to glitch-free (no pause animation, instant updating, etc) UIs, I can’t say that the browser platform has been very impressive. I’d say that it’s great to be able to use a current-gen browser while in AR/VR, but I would want as much of the AR/VR specific UI to be implemented outside of that.


#46

These “web GUI / Visualization has good enough speed” arguments sound exactly like the arguments in favour of using Python for scientific computing…


#47

Agreed. Until Julia came along, it was “common knowledge” that you couldn’t solve the two language problem for technical computing. While it seems like the problems for graphics are somewhat different from those for numerical work (although there’s a fair amount of overlap), and at this point less work on Julia has been towards things like minimizing the time to first plot, in general the problems in graphics seem surmountable and overall easier than problems those that have already addressed. In short, it doesn’t seem like there’s any fundamental reason why Julia couldn’t solve the two language problem in graphics with some more time and effort.


#48

No. Firstly: visualization & computation are 2 different things. Secondly: if you read my post about webassembly and understood it you would see that I envision a future where it’s all julia (with possibly some html as a GUI DSL): julia on the server doing the backend number crunching and julia (compiled to webassembly or at least a julia runtime in webassembly) running in the browser for the GUI & interactivity & visualizations. So one language (and one compilation target= webassembly) to rule them all. No 2 language problem. Webassembly is native code (kind of) so nothing like Python (and BTW all the heavy lifting in Python is done in native code (C/C++/Fortran) and gives good performance (OK maybe not C or fortran but close enough for GUI & viz).

My comment about good enough speed was about WebGL (which is also kind of native and much more performant compared to the other web vector graphics tech: SVG). WebGL is certainly good enough for showing a 3D model and rotating it interactively without a hitch (see for example 3D plots on the plotly site). Maybe not for a hi res VR simulation at 50 FPS but that’s a very specific niche and I don’ t see that coming anytime so to browsers or even mainstream. I’ve seen the VR hype more than 20 years ago. Didn’t go anywhere and it seems the same VR hype we had a couple years ago is dead already again. So what 's the problem with standard scientific plotting in the browser? Nothing at all. Totally doable and in fact it has been done: See D3.js, plotly.js and that’s (mostly) in javascript. Probably could be faster when webassembly analogs appear.


#49

Uh, VR of 20 years ago is not today’s VR. Like I said, try it out. Minimum spec for the better HMDs is about 90 Hz with displays soon to be 4K resolution per eye - so two different buffers at 90 Hz. That’s a lot of graphics needing to be shuffled around.

Take a fluid simulation “experience” like Chroma Lab, http://store.steampowered.com/app/587470/Chroma_Lab. With my i7 6700K, 16 GB RAM, and 1080 GTX I can simulate about 800,000-1,000,000 particles with good response. Going higher, things start to slow down.

Another demo of a force-directed graph, https://medium.com/@zachkinstner/devup-force-directed-graph-in-vr-9e6bf3e1a351, shows off really nicely what you could expect with a VR UI. I’d like to see it being able to handle millions of nodes.

As far as VR being a hype tech like 3D on TVs, I think you’re getting bad info. Once AR/VR glasses/HMDs become a bit more comfortable (and wireless) soon, you’ll see adoption soar, especially for commercial use. You will not be able to compete at your job with with people that choose to use AR/VR instead of limiting themselves to a 2D world and traditional HCI. Remember, in AR/VR you can have the 2D and a lot more.


#50

@asmpal I agree. Look at the current Microsoft standard for mixed reality headsets. Several manufacturers make them - hP, Dell, Asus. They do not need a room set up with sensors, just the headset plus two controllers. Fro Windows users, there is an app which tests if your system meets the minumum specifications. My HP Spectre laptop meets them - just. That’s why I bought it. OR you coudl have a higher end ‘gaming’ laptop.
Just think - you can carry around a VR system in a laptop bag. (HP actually have a backpack mounted VR setup, which basically means carrying a PC on your back… hmmm)

I agree there will eb a big takeup of VR for visualisation and design work using these portable setups.


#51

That’s me, whenever I leave the house - always have my laptop with me on my back!
:nerd_face:


#52

This is the HP backpack
http://www8.hp.com/us/en/campaigns/vrbackpack/overview.html
I see it having applications, but only in training or simulation facilities which have big budgets.

You could build a functional VR rig using a laptop with a Thunderbolt output plus an e-GPU enclosure (a thunderbolt cabled enclosure with a more powerful GPU)


#53

That’s pretty damn cool!