I need to make a plot/graphic of my neural network. I know there’s stuff out there for plotting graphs but I can’t seem to find anything that has the typical structure with the vertices lined up and connected by edges to subsequent layers. Like so…
I’ve found a lot of generic things to make this work but I need to customize edges. Is there any method of visualizing neural networks others here are using? At this point I can only think of learning some visualization software like photoshop or exporting my Flux model to Tensorflow and using tensorboards.
That should be fairly easy IMHO - if you can show me how to generate the columns and vertices I might give it a shot.
But @sdanisch has also talked about doing this for Makie, interactively, so you can see the network learn…
@oxinabox, these resources are absolutely awesome!
@juliohm and @mkborregaard, this was the sort of thing I was originally hoping for. And having something that updates throughout training would be an incredibly useful way of monitoring a network instead of having to dig through to weights and gradients between epochs
I feel that in general this kind of diagram is not useful.
It is good for explaining right as a very intro.
But in general this is not a good diagram for a network,
since if you ha ve 300 neurons on a layer it s unreadable.
And it encourages implmentation with neurons as objects,
rather than just matrix math.
Much more useful is a block diagram, showing layer or matrix opperations.
The exception, which I guess is what @Zach_Christensen is doing is if you have some kind of special connections, like sparse connections, or connections that have nonnegative weights.
Then it is a bit useful for showing structure
I’m thinking of two additional ways this type of diagram would be useful throughout training.
I’m working on a method that could theoretically be useful in identifying image features in the first few layers of a network. It would be great to see how much these feature contribute to a model very early on in the training process, or even while constructing a model.
If I could easily visualize exploding gradients, vanishing weights, etc. it would really speed up creation of novel models.
Both of these things can be done in the code without visualization, but I think it would help my productivity and really sell people on Flux/Julia to have pretty pictures. It may seem a bit pandering at that point though.
I agree that showing all neurons is hardly useful. More useful would be to just draw a schematic like it is done in papers: a convolutional layer represented as a box of right “shape” with depth representing number of filters and so on; a fully connected layer shown as just a box with bunch of nodes…
Each of these visual representations could be standardized for the good of everyone. It is a pain to read NN papers where authors have poor drawing skills. It is often the case that something is left out in the drawing.
I would think that the most reliable way to get that information is to follow the same path of information as the gradients do. I think that’s going to be switching over to Zygote soon. Otherwise you have to find a way to get that info from each type of layer).
