@ChrisRackauckas once told me that Julia v1.0’s built in logger would support package level logging. Is this true? If so do any examples exist?
You mean, standard logging systems for packages to tie into? Here’s an example of the underlying logging command Juno uses a Base logging command to get information, and a suggestion to standardize it:
1 Like
What I am getting at is fine grained control of messages produced by calls to @info and @warn, within and across packages.
Using the standard logging infrastructure is part of the question. The second part is how to build different logging streams, so that a users have finer grained control over the messages that are reporting. For example they may wish to turn off logging for just one package they are using.
I don’t follow the referenced example, but it feels orthogonal to what I am inquiring about.
The standard logging lets you put attributes in it, and you can assign meaning to it. Here there’s an attribute for progress, and that’s then used as an info stream for the current progress, and any package (Juno, ProgressMeter, etc.) can access that stream for updating their display. For filtering such messages, maybe should_log is what’s used?
https://docs.julialang.org/en/v1/stdlib/Logging/index.html#Early-filtering-and-message-handling-1
The details on this are still quite new to me so someone else should probably chime in.
The default logger (called ConsoleLogger and defined in the Logging standard library package) doesn’t have this functionality, but it should be possible to create a logger that can do this. The one issue that I foresee is with packages that organize themselves with submodules. Does anyone know if there is a way to get the ‘parent’ of module?
EDIT: parentmodule does what I want. So then the answer is yes, this is possible and just waiting on someone to implement it.
This is possible because every logging macro harvests information about where it was used including the file, line, Module and a couple of other things. All of these are passed on to the Logger backend which you can implement yourself and install using global_logger. You should implement shouldlog for early filtering and handle_message for full filtering, and this will give you complete control.
However, stdlib/Logging is very minimalistic right now, providing only just enough functionality to see messages in the terminal and some extremely simplistic filtering. This is not by design, but simply because I didn’t have time to make something fully featured for 1.0 (the effort mostly went into getting the frontend macros in fairly reasonable shape instead).
The main design challenge which we face in the logging backend is how to make loggers which compose correctly so that packages can use logging backends while also leaving ultimate logging configuration up to the larger application. I don’t have a solution to this yet, but the shape of the problem is becoming clearer.
I hope to have a separate repo JuliaLang/Logging reasonably soon in which we can start to flesh out the API in a way which is somewhat decoupled from the main julia release cycle.
2 Likes
Thanks for the update Chris. The new logging macros are super great, thanks for those!
Our group has a clear use case for package level logging, so we would be happy to test this functionally as it is built up. At this point what we need to get started is a little example of building a custom logger and/or some light documentation on how to do so. Maybe the hints you give in this post are sufficient for now, but I won’t be sure until we try.
In any case, thanks for your efforts on this!
1 Like
It’s great that you’re finding those useful. I’m interested in use cases, particularly ones which teach us what the backends must look like in order for them to be composable. For example, package A calls package B which calls package C and all want to somehow capture and display logs in a way which lets them cooperate with each other. I think the current design succeeds in being composable for generating log events but not for harvesting them.
By the way, I think I’ve just figured out how to get progress logging back in there so I’m pretty happy about that (frontend and prototype terminal backend here). I had to remove it in 1.0 because the design felt a bit off. But I now think this can be made to work even in the presence of exceptions.