Is it possible to use Makie in CImGui?

SteffenPL · August 31, 2020, 3:13pm

Is there a way to render with Makie.jl directly into an existing OpenGL context?
My goal is to render into a CImGui.jl widget using Makie.

Gnimuc · August 31, 2020, 4:03pm

I made a working example by hacking into GLMakie’s rendering loop last year, but it was buried in the Slack chat history. I just tried to cook up a new one, but Makie has evolved a lot since last year, it has too many abstraction layers, I don’t know where to start now.

If someone know how to get the GLFW.Window handle and corresponding OpenGL context from Makie’s scene object and how to insert custom functions into GLMakie’s rendering loop, then rendering CImGui’s widgets would be as simple as this snippet.

Gnimuc · August 31, 2020, 4:11pm

Or the other way around, let Makie do the off-screen rendering, then re-render the output of framebuffer as an CImGui.Image object. This way is probably more feasible since Makie has some recording APIs around there.

SteffenPL · September 5, 2020, 12:56pm

Thanks for the replies @Gnimuc!

I just had to replace one line in your gist file:

GLMakie.opengl_renderloop[] = imgui_renderloop

by

screen = AbstractPlotting.backend_display(GLMakie.GLBackend(), scene)
set_window_config!(renderloop=imgui_renderloop)

A few things are not working, but both display in one window!
The mouse events are partially ignored by Makie or CImGui. (Makie ignores: Right mouse button; CImGui ignores: Mouse wheel, Keyboard, window close event)

(Here is the complete example: Modification of Gnimuc's example: https://gist.github.com/Gnimuc/5a1e4942784cfc3558e2ee50b7bee84d · GitHub)

Gnimuc · September 5, 2020, 2:08pm

Looks like there are some GLFW callback collisions. For example, Makie uses scroll callback to zoom in/out, but CImGui uses it for scrolling. CImGui’s callbacks can be overwritten via the following functions, not sure how to make them compatible with Makie’s callbacks though.

https://github.com/Gnimuc/CImGui.jl/blob/master/src/backend/GLFW/callbacks.jl#L2-L5

SteffenPL · September 5, 2020, 2:25pm

Thanks for the hints and also for the correction in the gist.

Do you know how to give CImGui priority? Can I let CImGui register the callbacks later than Makie?
(For my case that would be sufficient, since I don’t need to interact with Makie too much.)

Gnimuc · September 5, 2020, 3:17pm

I think it should work as long as CImGui registers callback functions later than Makie, the code linked below shows setting callbacks in GLFW is just a pointer swapping:

github.com

glfw/glfw/blob/0cc13ceafc233cb0bc95c8b3ff8152ee75893bf9/src/input.c#L875


      
              _GLFW_SWAP_POINTERS(window->callbacks.character, cbfun);
              return cbfun;
          }
          
          GLFWAPI GLFWcharmodsfun glfwSetCharModsCallback(GLFWwindow* handle, GLFWcharmodsfun cbfun)
          {
              _GLFWwindow* window = (_GLFWwindow*) handle;
              assert(window != NULL);
          
              _GLFW_REQUIRE_INIT_OR_RETURN(NULL);
              _GLFW_SWAP_POINTERS(window->callbacks.charmods, cbfun);
              return cbfun;
          }
          
          GLFWAPI GLFWmousebuttonfun glfwSetMouseButtonCallback(GLFWwindow* handle,
                                                                GLFWmousebuttonfun cbfun)
          {
              _GLFWwindow* window = (_GLFWwindow*) handle;
              assert(window != NULL);
          
              _GLFW_REQUIRE_INIT_OR_RETURN(NULL);

I don’t know when Makie registers its callbacks, so can’t tell much about that.

SteffenPL · September 5, 2020, 3:26pm

Thanks!

Yes, it seems to work. If I create the CImGui context after setting up the Makie scene, then scrolling etc works which is all I need.