Any touchscreen APIs for Julia?

General Usage
1

Are there any packages that measure touchscreen gestures? I want to make something like this virtual piano (Virtual Piano Keyboard | Online Piano at Apronus.com) but in an offline interactive window. So far I’m imagining that I just want to measure if and where there are touches at regular intervals, not anything like swipes or double-taps.

1 Like
2

I just looked up for one of the cross-platform GUI toolkits:

The most basic input devices that every computer user has interacted with are keyboards and mice; beyond these, GTK+ supports touchpads, touchscreens and more exotic input devices such as graphics tablets.

Qt or QML may also have this, and are maybe better options in general (I believe many think so, for windows/mouse GUIs). That said, I’m not sure the Julia wrappers for those, Gtkl.jl, QML.jl and Qt.jl support. I’m even skeptical of that, but please look into that (and extend if not, or file an issue).

The web APIs may also support, not sure you want to use program that way with Julia. And Android support for Julia is I think not at that level yet.

1 Like
3

Yeah I’m not sure, either. I read on stackoverflow that multitouch support was added to Gtk some 7+ years ago for version 3.4, so I’m guessing Gtk.jl is using a later version than that. However, I can’t find anything in the Gtk.jl docs explicitly about touchscreen events. Gtk apparently has a GdkEventTouch struct for touchscreen actions, but I can’t find it in Gtk.jl’s github repo (though I should mention I never use github). I did find mentions of the GDK_TOUCH_xxxx event types, though. I’ll try to read up on how Gtk and Gtk.jl work, but boy do I wish I had some ready-made multitouch examples.

4

I did find QTouchEvent in QTouchEvent Class | Qt GUI 5.15.10

but it doesn’t seem to be supported in QML.jl, I didn’t see it exported. @barche might know if easy to support, it seems you’re out of luck with all the Julia wrappers for GUI toolkits. You might want to look into Kivy with Python, or some Qt library with Python or other lanuage. I mention Python as you should be able to use Python libraries with Julia, even frameworks, but I’ve not seen anyone actually use Python (web) frameworks (maybe in the past GUI frameworks), while I know of no one using touch related with Julia (or possibly in conjunction with Python).

5

I did actually find “touch device” mentioned in this GUI wrapper:
https://github.com/Gnimuc/CImGui.jl/blob/e43324e0fd36aeb7709dc653f2252d065ee341fa/src/wrapper.jl#L284

6

I have never tried it (due to the lack of a Julia-running touchscreen device), but I think this might work for touch input:

Conceptually, all these things are handled on the QML side, so it should all just work.

1 Like
7

Also for CImGui, another package I just found that might be usuful (or not, as searching for touch got me nothing, but at least might be helpful for non-touch GUI, maybe simplifying a lot): JuliaHub

8

Nuklear has touch support, and seems the wrapper may have too: https://github.com/Gnimuc/Nuklear.jl/search?q=touch&unscoped_q=touch

9

QML.jl doesn’t have any ready-made examples for this either, but a possible starting point could be the board example:

In the QML part, you can try to replace the MouseArea with a MultiPointTouchArea and follow the QML examples from the Qt documentation from there: Qt Quick Examples - Touch Interaction | Qt Quick 5.15.16

3 Likes
10

I never tried Dear ImGui with a touch screen device, but it looks like the config flags have a IsTouchScreen entry:

https://github.com/ocornut/imgui/blob/13f718337ac4df2c66727e85174d48e9a91eabcf/imgui.h#L1121