Discussion: OpenGL

Specific Domains Visualization
1

Hi everyone,

I would like to open up a discussion with the topic OpenGL, following some considerations made in GLAbstraction.jl issue #87.

The situation with all but the core of the julia OpenGL stack is rather non-ideal. The interplay between
ModernGL.jl, GLFW.jl, GLWindow.jl and GLAbstraction.jl, is hard to figure out coming from outside with no prior knowledge of the structure.

The first two packages are pretty much perfect by themselves, all basic functionality is there and it’s clear what each of them does. This becomes a bigger problem when the latter two packages come into consideration, for example if someone would like to use some abstractions rather than the very bare functionality.

Some functions in GLWindow.jl provide constructors/wrappers around the types in GLFW.jl, also providing some extended functionality which could be used as quality of life improvements over GLFW.jl. The trouble is that it also contains a lot of other functionality that has nothing to do with basic OpenGL or windows/contexts as a whole.

A similar story goes for GLAbstraction.jl in regards to the ModernGL.jl package. There are certain functions/constructors/wrappers, providing very useful abstractions over the ModernGL package, which could be perfectly bundled into an API. Again there is a lot of functionality that wouldn’t really belong in an API, combined with redefinitions of other structures/functions/naming schemes in GLWindow.jl which happens the other way around too.

Then there is the Reactive.jl package, which is very useful for implementing higher level functionality, but doesn’t belong on the level of an API. It is however very much used in both GLWindow.jl and GLAbstraction.jl.

This all makes it very hard for someone to understand what he can use from what package and where the API elements are hiding, compared with higher-level functionality.

I came to the conclusion that it would be very nice if we could bundle all the Core and API kind of abstractions into an OpenGL package (name up for debate ofcourse). The question, of course, is how to exactly structure all this in a clear, useful and most importantly semi-modular way.

This is kind of the starting point that I had in mind for this discussion. To provide some starting point I have fabricobbled up a very crude structure of how I would see all of this happening. The core/basic functionality is all put into OpenGL.zip, where the things that are not there are put into a higher level package, which I didn’t have time to at least get working. Also, please understand that this is not in any way final, and I’m not in any way trying to push anything onto anyone. Aside from very very minor changes, most of the code is a 1 to 1 copy paste, meaning that I spent 0 time on the actual implementation, and praises belong to the original implementers of the aforementioned packages. It’s merely a restructure.

The idea is that the submodules are kind of stand alone, and provide some of the basic functionality. Want to use only the core OpenGL? import OpenGL.CoreGL, want to also use basic windowing? add import OpenGL.GLFW, want to use abstractions, add import OpenGL.Abstractions, want to use higher level stuff, use the high level package.

There is the point of modularity, which is valid. Namely, it shouldn’t be necessary for someone to be able to compile GLFW.jl to be able to use OpenGL.CoreGL. On the other hand, as of now it seems that GLFW.jl is the only cross-os way of providing OpenGL contexts, making it as far as I see it the most probable tool to use. I fully favour mix-n-match functionality similar to different backends of Plots.jl, provided there are actual working alternatives to just using GLFW.jl. Sidenote: for the tests in both GLFW.jl and ModernGL.jl they depend on eachother, so why not merge them.

However, as was mentioned in GLAbstraction.jl issue #87, the ModernGL.jl and GLFW.jl packages are perfect standalone packages as it is at the moment. The reason why I would put them into the same package as the API layer, is that to me it seems nobody would not use the abstractions, if they are there, together with them. Especially when the core functionality is very directly accessible. But again, deciding upon this is exactly why this discussion exisits.

In general I lean towards more inclusive packages rather than a lot of small fragmented ones.

So to summarise the topics:

  • Is an OpenGL package useful?
  • Should the API-layer be included in the same package as the Core functionality?
  • Should GLFW.jl be standard?
  • What should the general structure of the package be?
  • If we would have more of a Core, Middle, High level structure, what should be put where?
  • Is it really useful to seperate the API-layer from the Core package, or make them two submodules?

@sdanisch @jayschwa @tim.holy @aaalexandrov @tkelman

Please @ other people that might be concerned with the evolution of the JuliaGL stack/ OpenGL in julia.

And ofcourse Happy holidays to everyone!

EDIT1: For people interested in the refactor I provided, I annotated it using the following keys:
#Came from packageName/file.jl, where the following block of code came from.
#change: , small changes I made, mainly namingschemes, not many of those,
#question:, questions/observations/comments about certain code (why, is this used, what does it do)

7 Likes
2

I’ve also had trouble wrapping my brain around which GL package is used for what, but I don’t understand enough about the different pieces to know whether a big restructuring is necessary or if it’s mostly a documentation issue. One thing that would be helpful would be some high-level overview documentation (maybe a diagram) that maps out the GL ecosystem and defines which packages are implementing orthogonal behavior, and which ones are abstraction layers on top of lower-level packages.

@louisponet that might be a good place for you to start if you want to get input from other folks, and would probably help inform a discussion about which packages make sense to be merged, or if functionality should move from one package to another.

3

Given the current state this is very hard, and the main reason I gave up on understanding/finding the structure there is at the moment, and decided that a restructure might be useful. This is in no small part because certain code doesn’t work anymore (e.g. half of the exports of GLWindow.jl aren’t implemented anymore).

I could try to document my findings more clearly than I did going through the code during the restructure I did. However as far as trying to understand how everything is intertwined right now, I think just looking at the structure of the package I linked + reading the #Came from s will give you kind of an idea. I tried putting all logically connected code in logical places.

At this point I’m convinced documenation will be very hard to make, and I’m not even sure it would make things a lot clearer.

4

I have come up with a maybe less disruptive way of doing things, largely preserving the structure that is supposed to be there.

ModernGL => GLAbstraction 
                             => GLToolkit
GLFW                                 
GTK     => GLWindow     
QML    
STL

GLWindow would manage the possible backends, providing a unified outward API to construct a glContext or whatever. GLAbstraction would do it’s task of providing quality of life API on top of the ModernGL functionality, and the two things would then be brought together in the GLToolkit packages, where things can be implemented that relate to both contexts/windows and abstractions.

This should sort out and sanitize the stack, and on the way documentation can be written.

5

I think packages that wrap a C library should remain standalone. I believe that includes GLFW and ModernGL. If there are bits of sugar in GLAbstraction and GLWindow that would make GLFW and ModernGL nicer to use (and wouldn’t add dependencies), perhaps those could be moved over. I would also be in favor of renaming ModernGL to “OpenGL” or just “GL”.

It sounds like the confusion stems from the higher-level packages. I can’t really weigh in on those since I haven’t used them, but I will say that I have found three.js pleasant to use. It might help to look at a library like that as a role model.

6

Yes what you say makes perfect sense, I believe that wrappers around libraries need to stay dependency-less (aside from their respective library), that said some added functionality on top might not be bad. I implemented my idea of what would get moved, with a small commentary, in pullrequest.

About the package name, I’m pretty on the fence on what to do with that. It seems “OpenGL” is clearest, but maybe this would be confused as also offering legacy OpenGL functionality, on the other hand, I’m sure people can read the README…

Thanks a lot for the recommendation on the Three.js, I will for sure have a look at it!

7

It seems “OpenGL” is clearest, but maybe this would be confused as also offering legacy OpenGL functionality

I don’t think we should bother making accommodations for legacy OpenGL. If someone really needs it, it can go into a “LegacyGL” package. :slight_smile:

1 Like
8

This is also possible using QML.jl:
https://github.com/barche/QML.jl/blob/master/example/gltriangle.jl

I suppose GTK.jl has similar functionality, so definitely it would be nice to make the coupling to GLFW as loose and optional as possible.

9

You’re right. I kind of got short sighted from my own experiences trying to get CxxWrap to work.

The structure I’m trying to get going now would use GLWindow.jl as a ‘backend manager’, providing windowing/glcontext functionalities, no matter what backends are installed/want to be used. I envision something like how Plots.jl manages their backends. It would not be connected directly to the base OpenGL and GLAbstraction packages. That would be ideal, it might take a while to get that going perfectly.

1 Like
10

Hi , I represent the user side as I currently use and extend glabstraction GLWindow glvisualize geometrytypes and ColorTypes in a commercial AR app.

I have another stake in the rewrite as I would like to promote my own Signals.jl as a replacement for Reactive signals.

  1. GLAbstraction is good in many subtle ways , basically it enables an easy definition and interaction of vertex arrays buffers textures shaders … everything that consists a render object.

Actually I think anything else should not be part of GLAbstraction. For example Camera.
Another thing that annoys me while developing is the dependency on types such as RGBA Point3f and such … I think @sdanisch feels the same way and this is the basis for the auto converting nodes in Visualize.jl right?

The default values generation of glvisualize is also user friendly to use … but I think it belongs in GLAbstraction.

For GLWindow my wishlist is to be able to render into an offscreen buffer each window and to compose them into the screen in a later stage possibly after some post process … per-window Framebuffer … I don’t know if performance wise this is feasible though.

Another thing is to have some hierarchy of render objects so they can “inherit” uniforms from their parent render object or parent window. Similar to the way Camera works now (render objects inherits the uniforms view,projection from the Window)

I have more things to say on various aspects , but let’s see how this discussion develops.

11

I think this is kind of the idea that I would want to go with as well. I’m going to move everything that isn’t purely a wrapper or qol into a separate package, so that GLAbstraction is really just about abstractions over ModernGL/OpenGL. Things like Framebuffers, vertexarrays etc will remain in GLAbstraction, but the other things would be put into the higher level package which brings together both GLWindow and GLAbstraction. This would ideally result in something which is on the same level as other api’s such as Three.js, of course not with the same amount of features, but similar scope.

I will have a look into the Signals.jl! Next week I will begin work on doing the restructuring of GLWindow GLAbstraction etc, but it will take a while, and it will be hard to clearly understand where everything belongs. I think a clear structure is very necessary though, so in the end it should be worth it.

2 Likes
12

Definitely