[ANN] CodeGlass – see what your code is actually doing

Hi all,

Over the past few years, we’ve been building runtime analysis tools for different languages and use cases, including Julia. Today, we would like to officially announce the release of our Julia profiling toolbox: CodeGlass for Julia.

With CodeGlass we try to make profiling and finding issues as easy as possible. We do this by combining all the data in one central place and showing this in a visual and clicky way.

CodeGlass collects everything that happens in your application in real-time. This allows you to see exactly what your code is doing, as it is happening. No changes needed.

Features

CodeGlass comes with a lot of features; some are generic between our language implementations, but some are also Julia specific. A full list of all features and views can be found here on our docs. Some of our most important features are:

Code Statistics

Information such as durations, call count but also memory usage across all modules and functions used in your application.

Memory Statistics

Information such as allocations, deallocations and which function allocates them.

Runtime Function Information

Runtime information about how functions behaved and got called (multiple and dynamic dispatch behavior) to give you up to the line of code information of your functions.


Recordings

Focus on specific parts of your program, by recording parts of execution without needing to restart or modify it, and directly compare recordings to identify differences.

Garbage Collections

Full garbage collection information such as each object that got deallocated and which function originally allocated that object. It also shows you where execution was interrupted of each thread/task.

Why CodeGlass

Understanding performance or memory issues in Julia often means isolating parts of your application, turning them into a minimum working example, and running them repeatedly with different tools.

This is time-consuming, and it often does not reflect what actually happens in the full application. Some issues only show up during real runs and disappear once the code is isolated.

With CodeGlass you don’t have to change your application, just run it like you are used to and see exactly what is happening during execution. And it is tracing in real-time, not sampling. This allows you to drill down to the root cause of your issue, whether that is performance, memory or GC related.

Getting started

You can download CodeGlass from our GitHub page. Our documentation explains how to get it up and running quickly. Although CodeGlass is not a free product, but we do offer a 14-day trial license so you can explore it freely (without entering any payment information​:blush:).

We would love your feedback. Feel free to ask questions here or reach out to us directly.

FAQ

Who are we?

We are a developer driven company that helps software developers by providing specialized development tools for various languages and use cases. So far, this community has been very welcoming to us, but doing B2C instead of B2B is still new for us, on the product side but company side as well. This is something we will have to grow into, and we hope you will help us to do so!

Meet myself and the team, from left to right, back to front:

Anthony, Joost, William, Stan, Terence, Tyrone

How does this work?

Julia already gives users and package developers quite a bit of information to build tools and get insight into what your application does. But this was not enough for us to get the level of detail we wanted. To solve this, we build our own instrumentation layer directly into Julia. For now, this means that you will have to use the Julia version that we ship with your CodeGlass installation (which is v1.12.6 at the moment). Once we are closer to finalizing our instrumentation spec, we will look into upstreaming these changes.

Why not open source?

We have seen that many tools in Julia are open source, and that this is something important within the community. But open sourcing CodeGlass would also mean that we would need to make our other tools available, which we cannot do right now due to agreements we made with other parties.

However, we will look into upstreaming our instrumentation layer so others can make tools on this as well.

Performance impact?

CodeGlass collects all required data by tracing your application. This will of course have an impact on the application’s performance. By configuring what you want to collect, this overhead can range from 2x up to 20x slower, but in most cases, you should expect around 5x. The timings shown in CodeGlass have this overhead removed, so your application’s statistics closely reflect the actual execution time.

What is on the roadmap?

This release is only the first of many. With each update we hope to include many new features to make it easier to solve your problems with your Julia application. If you have any features you would like to see in CodeGlass, please join our Discord and let us know. We cannot guarantee that we will include this in the next version, but your feedback is valuable to us, so please let us know.

For now, we have the following list of features on our roadmap (in no particular order):

  • Stepping functionalities, allowing you to replay parts of your application
  • Integration with Visual Studio Code
  • Systems to make updating CodeGlass and CodeGlass Julia easier, e.g. using Juliaup for distributing our Julia versions.
  • More views to help make finding issues easier, e.g. flame graphs
  • Improved garbage collection overview
  • .NET language support
What are caveats?

One thing we should mention is that CodeGlass can provide you all the information you need, but it will also give you all the information you don’t need. For example, when you run your application, it will not only collect information about your code but also about all the packages you use. This data is then also visible in CodeGlass, which makes it harder than we’d like to find your data and filter out the rest. That is why we are currently working on views and features that will make this task easier.

Another problem is that we have to add our instrumentation layer directly into Julia. This means that we do not support every Julia version currently. At the moment, we only support the latest (at writing v1.12.6), but we can implement support for older versions if the demand is high enough.

I used CodeGlass and it was helpful to solve a tricky memory issue in one of our critical systems. Well done CodeGlass team! You achieve getting a very useful tool to analyse behavior of runtime systems.

I was invited to try an early demo of CodeGlass and I was very impressed! They are also nice people :slight_smile: Congrats with your public release :tada:

What I liked most was the ability to just use the REPL as normal to start any task (like an HTTP server), and then click “Record” and “Stop” at any time to profile my code. That lets me analyse my problem without writing an MWE first.

Congrats on the release and thanks for working on this. It’s really nice to have more tooling in Julia, and this is a really exciting tool!

I’ve had the chance to watch a demo applied to SpineInterface (with some results already), and it looks very promising. I’m excited to try it out and compare it to my usual profiling strategy in the following days.

This looks really cool, but I was pretty bummed when I saw the monthly price. Like as a student, I’ll never be able to afford this :cry:. €35 monthly for the cheapest plan, which doesn’t allow for commercial use, CSV exports, or file dumping, is a lot of money for someone still in university. Are there any plans to introduce an affordable student/academic tier, because, as it stands, I think this software is inaccessible for a pretty large population of potential users? Still, it’s really useful and wish you luck in your endeavor.

Thanks @Alexandre_Bergel , @fonsp and @abelsiqueira for the kind words. It means a lot to see CodeGlass being tried on real Julia use cases already :smiley:

The REPL + Record/Stop workflow, memory analysis, and comparison with existing profiling strategies are exactly the kinds of things we hope to learn more about from the community, so please keep the feedback coming!

Thanks for bringing this up. We get what you are saying, and it is something we recognize from our own university days as well.

Our pricing for individual users is still under consideration, including student and academic options for non-commercial use.

We definitely do not want students or academic users to feel that CodeGlass is out of reach. At the same time, CodeGlass is currently developed by a very small team, so we also need to find a model that lets us keep developing and maintaining the tool sustainably.

In the meantime, the 14-day trial is available without payment information. If you are willing to try it, we would really appreciate your feedback on what kind of license, feature set, and price point would fit your needs.

Thanks again for saying this openly and constructively. This community has been very welcoming so far, and feedback like this helps us a lot while we figure this out. :smiley:

Out of curiosity, how this relates to CodeGlass for other languages, and what does come closest in the python world ? I see that a generic CodeGlass.io doesn’t exists, neither e.g. rust.codeglass.io or python.codeglass.io, only julia.codeglass.io

We made various tools on request for other languages such as for Python, Rust, Java and .NET but for a narrower use case. We never made those tools more generic and publicly available as the problem we encountered in those communities is that there are already many different development tools available, and people usually aren’t that eager to switch up their current tooling.

When we entered the Julia community, we noticed that people we much more open to new tooling. That’s why we decided to make our tool publicly available, instead of only helping companies solve problems with our internal tools.

The reason that we don’t support Python (or other languages) at the same level as Julia at this moment is simply that we do not have the capacity to implement and maintain the same level of tooling we currently have for Julia.

We hope that in the future we can support languages like Python and Rust, as we not only enjoy helping developers spend more time developing rather than debugging, but also love diving deep into the compilers of different languages to achieve the same results we are currently getting with Julia.

It looks fantastic, I wish I had it about 4 years ago when I was writing a large suite of programs for modelling and real-time control. However, now I have retired, it makes no sense to pay for a tool, however sophisticated it might be, when there are other tools around. And that’s bad for you – the more users you get, the more you’ll improve your code base.

Are you giving attribution to the coders whose code you have used, or is every line of code your own?

Thanks Roger, and that makes a lot of sense. A non-commercial / hobbyist / retired-user situation is another good example of where our current pricing may not fit well yet. We are still learning what the right model should be for individual users, and you are right that more users and more feedback would help us improve the tool.

On the attribution question: with the exception of some third-party packages listed in the docs, CodeGlass itself is built by us.
For the Julia version we ship, we build on Julia itself and maintain our own instrumentation changes on top of it. Julia is of course the work of the Julia contributors.

Separately from license attribution, we also want to acknowledge people in the Julia community whose feedback, examples, bug reports, or ideas help shape CodeGlass. We are still figuring out what the right form for that is, but we agree that this matters.

We now also have Linux-arm64 support :slight_smile:

No other changes yet, but I do have something fun to share that @TheLazyTurtle figured out and already shared on our discord:


I was wondering whether it would be possible to run CodeGlass fully from my Visual Studio Code. I was editing my code there anyway, and it would make showing the relationship between code and data very easy. At first, I thought we would need to build a complete plugin to make this possible, but it turned out to be very simple. I used this script which we made to allow CodeGlass Julia right from VSCode.

After that, I had to find a way to display all the data in VSCode. It turns out you can simply open a browser (F1 → Browser: Open Integrated Browser) and navigate to http://127.0.0.1:30340/ to open the CodeGlass Web Client. I thought I would share this in case it ends up being useful for anyone else. (Keep in mind that not all features might work due to limitations in the VSCode browser.)


P.S. we are also working on a full integration and extension for VSCode, this is not it :slight_smile:

Congratulations on this public release! :tada: It seems that I am a bit late to the party.
For some time now we have been collaborating with the CodeGlass team to help them polish their product and make it the best tool for troubleshooting in julia.

I like CodeGlass approach on “debuggers / profilers”, it seems compatible with the thinking outside-of-the-box that is very common among the julia community. With a bit more of polish this could be very useful and help julia break into corporate environments.

We are still collaborating with the CodeGlass team to improve the user experience and I am very eager to see the results!

Hi all,

I’m doing some self-promotion here; I hope that’s fine.

I’ve spent some more time testing CodeGlass, tried a few examples, and created a video (not sponsored) with my experience: https://youtu.be/slGTmtYWgLs
It captures both nice and bad user experiences, so I’m curious what other users think about it. Thanks for watching, and thanks again to the CodeGlass team for making the trial available.

Best,
Abel

Thank you Abel for making this video. It was very interesting and you covered all the elements of CodeGlass very well. It was also very interesting and educational for us to see how you used CodeGlass and how you navigated through the UI. It was different than we have seen so far and resulted in very good feedback. We also noted all the issues you came across and are working hard on solving them.

New Release: CodeGlass V2.1.0

Hey everyone,

Since our last release about two months ago, we have gotten a lot of positive feedback for which we are very grateful.
We have also been working hard with further improving CodeGlass. For this update, we focused on two specific topics: improving garbage collection profiling and continuing to merge existing tools.

Garbage Collection Improvements

One of the places we (and some of our users) felt like CodeGlass lacked was the information we could provide about a garbage collection.
Information about what objects got deallocated in each GC already helped a few people solve their issues, but many other issues were still hard to solve.
To help with finding garbage collection related issues, we added GC markers to the Code Body Reconstruction screens, so it is easier to find places in your code where a GC got triggered.
We also added the full call stack of each active task at the point where the GC happened to the Garbage Collection Details screens. This helps with getting a clear view of where each task got interrupted by the GC.
Lastly, we also updated the Garbage Collection Explorer screen. This screen used to just show a list of all GCs, but now it is a table that includes some (potentially) important statistics, so it is easier to find a specific GC that is interesting.
If you are interested in how to use these improved screens, you can read our new How To about this.

Added .NET support

As we have mentioned before, we have made tooling for multiple different languages. With our 2.0 release, we started making a system that allows us to merge all our tools into one application.
For this update, we started merging our existing .NET tooling back into this version of CodeGlass. For now, the .NET profiler supports all the basic features of CodeGlass, but the more specialized screens and features will be added back in later updates.

Other Features and Fixes

We have worked on solving many bugs and improving other features of CodeGlass. There are too many items to all mention here but here are some of the larger items worth mentioning:
We made it easier for users to access CodeGlass that is running on a different device. This is an opt-in feature, and even when enabled, all collected data is still stored on the device that is running CodeGlass.
On the Code Body Statistics screen you can now toggle to only see functions that were dynamically dispatched.
The Dynamic Dispatch and Distribution Graph screens now support switching between Data Sources. These screens now show the latest data of a recording instead of showing all data collected in the instance.
We also made some small improvements to the UI/UX, but this is still an ongoing thing we are continuously working on improving. If you have any feedback, please let us know!
Additionally, we added some settings to make CodeGlass more accessible for everyone. The first item we added is the option to change the colors that rendering screens, like the Heat Map, use. This allows colorblind people to choose colors that allow them to better read the values. We will continue to keep improving our accessibility, and if you have any issues, please let us know.
The second option we added allows you to configure things like the decimal character to better support your local standards. You can change options like the decimal character (ex. 123,45 or 123.45) and the thousands separator (ex. 1_000 or 1.000).

Pricing updates

After our initial release, we have seen and heard people’s concern about the price of CodeGlass. We are still working on figuring out a better price for every kind of user.
In the meantime, we would like to offer CodeGlass with all features for 15 euros for 3 months. This offer will be available until the end of JuliaCon (15th of august).

If this does not work for you, please message one of us.

JuliaCon

We will also be at JuliaCon this year​:blush: and will be giving a few talks including a workshop and a Birds of a Feather session where we won’t be discussing CodeGlass but talking about tooling needs in general

*FYI: Anyone can participate in the workshop, you don’t need to have a license.

Like always, if you have any feedback, or feature requests on what you would like to see in CodeGlass, please let us know.