Since I am trying to classify it into as many classes as the data points, there should be one data point in each class; however, we see that nothing gets assigned to classes 1, 2, 9, and 10. What’s the issue here?
PS: The above example is just to illustrate the issue. I am obviously trying to classify it in way less number of classes, but I see the same issue every time.
Presumably whatever optimisation algorithm it’s using is stuck in a local optimum. Note that there are very few ML algorithms which can guarantee the optimal output - but nevertheless tend to work very well on real-world data.
You should try the algorithm on the data you actually care about and see if there is still a problem.
The issue here is that the number of clusters and the number of data points is close. Depending on how the medoids are initialized and move with each iteration, you may terminate with the medoids in a position where no points in the dataset are closest to a particular one, in which case you have an empty cluster. If you increase the data size (say, by a factor of 10), then all clusters are nonempty:
Does BetaML.jl use the PAM algorithm for K-medoids? I don’t see it mentioned in the docs. I was previously using Clustering.jl and they do mention in the docs that they don’t use the PAM algorithm and thus might give poor loss (which is why I am more inclined to use BetaML.jl)
Also, is there an inbuilt function that returns the final loss? Something like model.loss or model.totalcost? I don’t see anything mentioned in the docs, or I probably missed it.
I never see this issue of some classes not getting assigned when I use kmedoids function from Clustering.jl. Any pointers on what could be the reason for this?
We also use k-means-style iterations when on each iteration we first find the constituency of each cluster and then independently on each cluster we compare the pairwise distances to find the “next” medoids of the specific cluster as the record with the shorter intra-cluster distance. This has a complexity on each iteration at best O(n²/k) when clusters have all the same number of records.
If you need the PAM algorithm, or the distances eventually cached, I could implement it, let’s say in January… or you could try by yourself to add it… the nice thing in Julia is that it is relatively easy to code the algorithms… In the first case, please open a issue so I can track it, in the second case you could open directly a pull request…
On master I have added an av_distance_last_fit record to the info dictionary that you can retrieve with info(model)
Still on master, I have decoupled the hyperparameters object between k-means and k-medoids and I have now set the default initialisation to k-medoids on "shuffle", addinga warning that in some circumstances using different initialisations may lead to some rapresentatives not being actual training records.
This by the way has also the advantage that we could add a further hyper-parameter “algorithm” to allow PAM and others variants.