You’re welcome 
WRT UIs, I think people are possibly missing the elephant in the room, namely AR/VR. IMO (based on having/using a VR system), I don’t think I’m going out on a limb predicting that all UIs will be AR/VR-based in the not too distant future. In particular, data manipulation/visualization will be one of the first areas to move in that direction due to the significant benefits of working in that kind of environment. If you doubt this, I urge you to check out VR (not the, currently, lame cellphone type, though) - you’ll most likely be converted. Sure, today’s hardware isn’t quite there in terms of comfort/resolution, but we’re not very far off.
If you buy into the above, performance, compatibility with AR/VR rendering and AR/VR HCI considerations should be a key factors when thinking about UIs which won’t soon become irrelevant.
2 Likes
Hopefully web stuff will quickly move toward WebAssembly, because the performance of Electron apps is quite bad right now… We can’t even have spinning cogs in Atom!
You can get quite decent performance for 3D or plotting a lot of 2D points via webgl. Maybe not for very high density 3D medical scans or scientific visualizations but for that you would use something like VTK on a high graphics workstation.
My impression of WebVR (which uses WebGL) is that there’s a very significant performance drop with that vs running VR out of the browser, so, while you might consider it adequate for some things, I’m not sure I’d want to use it given a choice. So far, when it comes to glitch-free (no pause animation, instant updating, etc) UIs, I can’t say that the browser platform has been very impressive. I’d say that it’s great to be able to use a current-gen browser while in AR/VR, but I would want as much of the AR/VR specific UI to be implemented outside of that.
These “web GUI / Visualization has good enough speed” arguments sound exactly like the arguments in favour of using Python for scientific computing…
5 Likes
Agreed. Until Julia came along, it was “common knowledge” that you couldn’t solve the two language problem for technical computing. While it seems like the problems for graphics are somewhat different from those for numerical work (although there’s a fair amount of overlap), and at this point less work on Julia has been towards things like minimizing the time to first plot, in general the problems in graphics seem surmountable and overall easier than problems those that have already addressed. In short, it doesn’t seem like there’s any fundamental reason why Julia couldn’t solve the two language problem in graphics with some more time and effort.
9 Likes
No. Firstly: visualization & computation are 2 different things. Secondly: if you read my post about webassembly and understood it you would see that I envision a future where it’s all julia (with possibly some html as a GUI DSL): julia on the server doing the backend number crunching and julia (compiled to webassembly or at least a julia runtime in webassembly) running in the browser for the GUI & interactivity & visualizations. So one language (and one compilation target= webassembly) to rule them all. No 2 language problem. Webassembly is native code (kind of) so nothing like Python (and BTW all the heavy lifting in Python is done in native code (C/C++/Fortran) and gives good performance (OK maybe not C or fortran but close enough for GUI & viz).
My comment about good enough speed was about WebGL (which is also kind of native and much more performant compared to the other web vector graphics tech: SVG). WebGL is certainly good enough for showing a 3D model and rotating it interactively without a hitch (see for example 3D plots on the plotly site). Maybe not for a hi res VR simulation at 50 FPS but that’s a very specific niche and I don’ t see that coming anytime so to browsers or even mainstream. I’ve seen the VR hype more than 20 years ago. Didn’t go anywhere and it seems the same VR hype we had a couple years ago is dead already again. So what 's the problem with standard scientific plotting in the browser? Nothing at all. Totally doable and in fact it has been done: See D3.js, plotly.js and that’s (mostly) in javascript. Probably could be faster when webassembly analogs appear.
3 Likes
Uh, VR of 20 years ago is not today’s VR. Like I said, try it out. Minimum spec for the better HMDs is about 90 Hz with displays soon to be 4K resolution per eye - so two different buffers at 90 Hz. That’s a lot of graphics needing to be shuffled around.
Take a fluid simulation “experience” like Chroma Lab, Chroma Lab on Steam. With my i7 6700K, 16 GB RAM, and 1080 GTX I can simulate about 800,000-1,000,000 particles with good response. Going higher, things start to slow down.
Another demo of a force-directed graph, DevUp: Force-Directed Graph in VR | by Zach Kinstner | Medium, shows off really nicely what you could expect with a VR UI. I’d like to see it being able to handle millions of nodes.
As far as VR being a hype tech like 3D on TVs, I think you’re getting bad info. Once AR/VR glasses/HMDs become a bit more comfortable (and wireless) soon, you’ll see adoption soar, especially for commercial use. You will not be able to compete at your job with with people that choose to use AR/VR instead of limiting themselves to a 2D world and traditional HCI. Remember, in AR/VR you can have the 2D and a lot more.
2 Likes
@asmpal I agree. Look at the current Microsoft standard for mixed reality headsets. Several manufacturers make them - hP, Dell, Asus. They do not need a room set up with sensors, just the headset plus two controllers. Fro Windows users, there is an app which tests if your system meets the minumum specifications. My HP Spectre laptop meets them - just. That’s why I bought it. OR you coudl have a higher end ‘gaming’ laptop.
Just think - you can carry around a VR system in a laptop bag. (HP actually have a backpack mounted VR setup, which basically means carrying a PC on your back… hmmm)
I agree there will eb a big takeup of VR for visualisation and design work using these portable setups.
That’s me, whenever I leave the house - always have my laptop with me on my back!
This is the HP backpack
http://www8.hp.com/us/en/campaigns/vrbackpack/overview.html
I see it having applications, but only in training or simulation facilities which have big budgets.
You could build a functional VR rig using a laptop with a Thunderbolt output plus an e-GPU enclosure (a thunderbolt cabled enclosure with a more powerful GPU)
1 Like
That’s pretty damn cool!
There are a few of these backpack type units, but IMO they’ll be redundant soon if they’re not already. Wireless modules like TPCast (available now for the Vive and soon for Oculus), but also from Intel and others which will be out later this year make tethering to the system unnecessary.
While reading “game developer”, “Gui framework”, “AR/VR” on a topic containing “two language problem”… I can’t resists to ask if you know Godot game engine (under MIT) ? if adding julia as scripting language for godot is doable (due to inheritance) and may be worth the effort of implementing it.
2 Likes
I’ve been developing a game with Godot and indeed it has good support for UI design. Their own scripting language (GDScript) is quite good, but I understand that it’d be interesting to see if one could call Julia from Godot. Godot has bindings with a few other languages using a thing called GDNative: see here for a few examples: D, C++, Rust, etc… Not sure what it’d take to have Julia bindings.
1 Like
It may also be possible to develop a julia module like the C# one or GDScript one: https://github.com/godotengine/godot/tree/master/modules
This way you develop something in julia without UI and the day you want to have an UI you can do it with importing your libs inside godot and develop it with the tools it provides.
As a game engine performance is important, so you avoid pitfalls of html…
@piever with your knowledge of godot, do you see blockers due to the strong object oriented facets of it (nodes, inheritance) ?
I’m just messing around with Godot, I’m not an expert on the implementation. In terms of inheritance, one could try to use abstract types, say:
abstract type Node; end
struct Sprite <: Node
image
....
end
Though it may be limiting that you can only define one supertype for your struct
From what I understand the design is more “encapsulation” than inheritance based (at least as far as the nodes are concerned). So, you would basically add the node you want to use function from in the parent node.
struct Player
ship
bullets
lifebar
end
end then use methods on p.ship. With dot field overloading maybe you could get somewhere, I don’t actually know.
1 Like
Thank you 
FYI here’s a demo video of a 3D game running in the browser on webassembly +webgl EpicZenGarden, WebAssembly and WebGL 2 in Firefox 52 - YouTube
Looks pretty smooth to me!