Does anyone know if there’s a rich formatter for showing the progress of training in Pluto when training models using Flux or FluxTraining?
I’d quite like to use this if it exists. The current UX for training inside a notebook is pretty bad.
3 Likes
I found the progress bar support since asking the question, but it’s not really what I am looking for. I’m looking for something that can display live metrics inside Pluto. For example, a progress bar, live loss plot, example data at the end of each epoch, etc.
I assume most people are simply using Tensorboard via Flux’s Tensorboard logger. That’s OK, but for experimenting with visualizations of my model and data I’d love to have the visualizations within Pluto.
1 Like
Maybe PlutoUI with_terminal can be of use?
https://docs.juliahub.com/PlutoUI/abXFp/0.7.38/#Terminal-addiction-1
with_terminal also only displays the printout after the cell has returned.
A quick look (not that I have any substantial understanding of the code base) at the changes introduced to facilitate live display of logging messages suggests that the developers had to jump through a few hoops to make that happen. This suggests to me, that updating the output of a cell while it is running is far from trivial. But I may be mistaken here.
It certainly would be a very nice feature to have.
2 Likes
You can actually do it already using the logging functionality and the @info macro for example. You can also put whatever showable julia object inside logs which is pretty neat specifically for adding plots to them:
Only problem is that each log is separate and you can’t just update the previous log, so things can become quite cluttered if you are planning on printing a lot of updates within the same celle execution.
7 Likes
Thanks! That’s almost it. Do you think that’s the limit for what Pluto can do right now? Are there custom components which handle rich log messages?
what do you mean by rich? You can display there whatever you can display in Pluto, which in my opinion makes it very rich
I find the printing behavior in Pluto dissatisfying as well. I just want code like
for epoch in epochs
...
@info (; epoch, loss, accuracy)
end
to be displayed the same way as in the repl and in jupyter notebook, vertically, not in boxes overlapping with each other.
with_terminal() is not required anymore in the most recent Pluto version for showing stdin/stderr in the notebook
1 Like
Hey,
What I mean by rich was exactly that yes, and I agree. I didn’t realize you could have anything in the log messages.
However I also had something else in mind, which was a JavaScript components to collect the log messages as they arrive and display them in a custom way. For example, a component that shows a progress bar, a frequently updating table of the latest values for a selection of statistics, as well as a scrolling vertical list of rich log messages. The progress bar and statistics should remain visible when scrolling.
I have had a look at the code now and I can see that there’s no blessed way to implement this right at the moment. I’m not necessarily suggesting you implement it, but what I’m looking for is a way to create a Julia package that integrates with Pluto including the ability to include a custom log message handler frontend component.
I originally assumed this already existed and was looking for whether someone already created a component for Flux.
In terms of custom printing, maybe https://github.com/FedeClaudi/Term.jl is your best bet. No idea how well it works in Pluto though
Could the PlutoLinks package help? You could run your model using @use_task and generate your dashboard using cells which depend on the output data. @use_file might also work, and you could build in some kind of caching so that training progress is saved.