# Any touchscreen APIs for Julia?

**URL:** <https://discourse.julialang.org/t/any-touchscreen-apis-for-julia/45614>\
**Category:** General Usage\
**Created:** [August 27, 2020, 7:00am UTC](https://discourse.julialang.org/t/any-touchscreen-apis-for-julia/45614 "2020-08-27T07:00:52Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![Benny](https://avatars.discourse-cdn.com/v4/letter/b/49beb7/32.png) [@Benny](https://discourse.julialang.org/u/Benny)\
**Post date:** [August 27, 2020, 7:00am UTC](https://discourse.julialang.org/t/any-touchscreen-apis-for-julia/45614/1 "2020-08-27T07:00:52Z")

</div>

Are there any packages that measure touchscreen gestures? I want to make something like this virtual piano ([Virtual Piano Keyboard | Online Piano at Apronus.com](https://www.apronus.com/music/flashpiano.htm)) but in an offline interactive window. So far I’m imagining that I just want to measure if and where there are touches at regular intervals, not anything like swipes or double-taps.

---

<div class="post-metadata">

**Author:** ![Palli](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/palli/32/3380_2.png) [@Palli](https://discourse.julialang.org/u/Palli)\
**Post date:** [August 27, 2020, 8:14am UTC](https://discourse.julialang.org/t/any-touchscreen-apis-for-julia/45614/2 "2020-08-27T08:14:01Z")

</div>

I just looked up for one of the cross-platform GUI toolkits:

> **[Gtk-3.0](https://docs.gtk.org/gtk3/)**
>
> Reference for Gtk-3.0

> The most basic input devices that every computer user has interacted with are keyboards and mice; beyond these, GTK+ supports touchpads, touchscreens and more exotic input devices such as graphics tablets.

Qt or QML may also have this, and are maybe better options in general (I believe many think so, for windows/mouse GUIs). That said, I’m not sure the Julia wrappers for those, Gtkl.jl, QML.jl and Qt.jl support. I’m even skeptical of that, but please look into that (and extend if not, or file an issue).

The web APIs may also support, not sure you want to use program that way with Julia. And Android support for Julia is I think not at that level yet.

---

<div class="post-metadata">

**Author:** ![Benny](https://avatars.discourse-cdn.com/v4/letter/b/49beb7/32.png) [@Benny](https://discourse.julialang.org/u/Benny)\
**Post date:** [August 31, 2020, 12:16am UTC](https://discourse.julialang.org/t/any-touchscreen-apis-for-julia/45614/3 "2020-08-31T00:16:36Z")

</div>

Yeah I’m not sure, either. I read on stackoverflow that multitouch support was added to Gtk some 7+ years ago for version 3.4, so I’m guessing Gtk.jl is using a later version than that. However, I can’t find anything in the Gtk.jl docs explicitly about touchscreen events. Gtk apparently has a GdkEventTouch struct for touchscreen actions, but I can’t find it in Gtk.jl’s github repo (though I should mention I never use github). I did find mentions of the GDK\_TOUCH\_xxxx event types, though. I’ll try to read up on how Gtk and Gtk.jl work, but boy do I wish I had some ready-made multitouch examples.

---

<div class="post-metadata">

**Author:** ![Palli](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/palli/32/3380_2.png) [@Palli](https://discourse.julialang.org/u/Palli)\
**Post date:** [August 31, 2020, 12:16pm UTC](https://discourse.julialang.org/t/any-touchscreen-apis-for-julia/45614/4 "2020-08-31T12:16:14Z")

</div>

I did find QTouchEvent in [QTouchEvent Class | Qt GUI 5.15.10](https://doc.qt.io/qt-5/qtouchevent.html)

but it doesn’t seem to be supported in QML.jl, I didn’t see it exported. @barche might know if easy to support, it seems you’re out of luck with all the Julia wrappers for GUI toolkits. You might want to look into Kivy with Python, or some Qt library with Python or other lanuage. I mention Python as you should be able to use Python libraries with Julia, even frameworks, but I’ve not seen anyone actually use Python (web) frameworks (maybe in the past GUI frameworks), while I know of no one using touch related with Julia (or possibly in conjunction with Python).

---

<div class="post-metadata">

**Author:** ![Palli](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/palli/32/3380_2.png) [@Palli](https://discourse.julialang.org/u/Palli)\
**Post date:** [August 31, 2020, 12:24pm UTC](https://discourse.julialang.org/t/any-touchscreen-apis-for-julia/45614/5 "2020-08-31T12:24:02Z")

</div>

I did actually find “touch device” mentioned in this GUI wrapper:  
[https://github.com/Gnimuc/CImGui.jl/blob/e43324e0fd36aeb7709dc653f2252d065ee341fa/src/wrapper.jl#L284](https://github.com/Gnimuc/CImGui.jl/blob/e43324e0fd36aeb7709dc653f2252d065ee341fa/src/wrapper.jl#L284)

---

<div class="post-metadata">

**Author:** ![barche](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/barche/32/79_2.png) [@barche](https://discourse.julialang.org/u/barche)\
**Post date:** [August 31, 2020, 12:30pm UTC](https://discourse.julialang.org/t/any-touchscreen-apis-for-julia/45614/6 "2020-08-31T12:30:30Z")

</div>

> [@Palli](#):
>
> but it doesn’t seem to be supported in QML.jl,

I have never tried it (due to the lack of a Julia-running touchscreen device), but I think this might work for touch input:

> **[MultiPointTouchArea QML Type | Qt Quick 6.6.1](https://doc.qt.io/qt-5/qml-qtquick-multipointtoucharea.html)**
>
> Enables handling of multiple touch points.

Conceptually, all these things are handled on the QML side, so it should all just work.

---

<div class="post-metadata">

**Author:** ![Palli](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/palli/32/3380_2.png) [@Palli](https://discourse.julialang.org/u/Palli)\
**Post date:** [August 31, 2020, 12:35pm UTC](https://discourse.julialang.org/t/any-touchscreen-apis-for-julia/45614/7 "2020-08-31T12:35:34Z")

</div>

Also for CImGui, another package I just found that might be usuful (or not, as searching for touch got me nothing, but at least might be helpful for non-touch GUI, maybe simplifying a lot): [JuliaHub](https://juliahub.com/ui/Packages/Poptart/HOpB5/0.3.0)

> [@\[ANN\] CImGui.jl - A Wrapper for Bloat-free Immediate Mode Graphical User interface(Dear ImGui)](https://discourse.julialang.org/t/ann-cimgui-jl-a-wrapper-for-bloat-free-immediate-mode-graphical-user-interface-dear-imgui/21268/9):
>
> That’s probably because Dear ImGui is not for creating UIs for the average end-user. As explained in the Dear ImGui’s [FAQs](https://github.com/ocornut/imgui/blob/f02705fbaa22cad03f0b033d901f8b232dfb1ce5/imgui.cpp#L569-L653), it’s a renderer-agnostic GUI lib and does not provide any 2D/3D rendering utilities by default. People need to build their own 2D texture renderer for drawing images. Although this can be done in a couple of lines using quite a few knowledge in 3D graphics, the learning curve may look steep for those who don’t have any relevant background. Maybe I should add a default 2D im…

---

<div class="post-metadata">

**Author:** ![Palli](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/palli/32/3380_2.png) [@Palli](https://discourse.julialang.org/u/Palli)\
**Post date:** [August 31, 2020, 12:44pm UTC](https://discourse.julialang.org/t/any-touchscreen-apis-for-julia/45614/8 "2020-08-31T12:44:32Z")

</div>

Nuklear has touch support, and seems the wrapper may have too: [https://github.com/Gnimuc/Nuklear.jl/search?q=touch&unscoped\_q=touch](https://github.com/Gnimuc/Nuklear.jl/search?q=touch&unscoped_q=touch)

---

<div class="post-metadata">

**Author:** ![barche](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/barche/32/79_2.png) [@barche](https://discourse.julialang.org/u/barche)\
**Post date:** [August 31, 2020, 12:51pm UTC](https://discourse.julialang.org/t/any-touchscreen-apis-for-julia/45614/9 "2020-08-31T12:51:20Z")

</div>

> [@Benny](#):
>
> but boy do I wish I had some ready-made multitouch examples.

QML.jl doesn’t have any ready-made examples for this either, but a possible starting point could be the board example:

- Julia part:[https://github.com/barche/QML.jl/blob/master/example/board.jl](https://github.com/barche/QML.jl/blob/master/example/board.jl)
- QML part: [https://github.com/barche/QML.jl/blob/master/example/qml/board.qml](https://github.com/barche/QML.jl/blob/master/example/qml/board.qml)

In the QML part, you can try to replace the `MouseArea` with a `MultiPointTouchArea` and follow the QML examples from the Qt documentation from there: [Qt Quick Examples - Touch Interaction | Qt Quick 5.15.16](https://doc.qt.io/qt-5/qtquick-touchinteraction-example.html)

---

<div class="post-metadata">

**Author:** ![Gnimuc](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gnimuc/32/2194_2.png) [@Gnimuc](https://discourse.julialang.org/u/Gnimuc)\
**Post date:** [August 31, 2020, 3:28pm UTC](https://discourse.julialang.org/t/any-touchscreen-apis-for-julia/45614/10 "2020-08-31T15:28:31Z")

</div>

I never tried Dear ImGui with a touch screen device, but it looks like the config flags have a `IsTouchScreen` entry:

[https://github.com/ocornut/imgui/blob/13f718337ac4df2c66727e85174d48e9a91eabcf/imgui.h#L1121](https://github.com/ocornut/imgui/blob/13f718337ac4df2c66727e85174d48e9a91eabcf/imgui.h#L1121)