Should work as well. The basic confusion matrix needs just two vectors ( references and predictions). Btw, I believe MLBase is informally deprecated in favour of LearnBase…

The model you pointed to is just the training. Applying it i.e. inference, should output a vector of 10 posterior probabilities for each sample (10 classes, 0 to 9). From there, the predicted label can be extracted (max posterior probability); the predicted labels can then be fed into the confusion matrix …

Thanks. It seems that some testing is performed throughout the training process, isn’t it? Instead of accuracy, it would be useful to be able to plug-in a full confusion matrix computation:

Couldn’t say, I’m really not familiar with Flux. Neural models backpropagate errors at sample level, not class level. In practice, they do optimize the confusion matrix by trying to learn the correct classes

Sorry for reviving this, is MLBase still maintained? I noticed there were some discussions on the github page of merging it into StatsBase.
I was also searching for a confusion matrix implementation.