Dear ML community, we released LossFunctions.jl v0.9 with a few important breaking changes:
Reversed order of arguments to match other ecosystems, loss(yhat, y) is now the order.
Removed the ObsDim business to support a more general interface with iterables of observations.
Removed OrdinalMarginLoss to support a more general interface with CategoricalArrays.jl].
Dropped unnecessary dependencies, the only dependency is CategoricalArrays.jl now.
These changes will allows us to sync the widely tested loss functions in LossFunctions.jl with loss functions defined in other ecosystems such as Flux.jl. I would like to invite every ML contributor in Julia to try to move their own loss function implementations to LossFunctions.jl and join efforts maintaining the package.
Our next goal is to formalize the support for general datasets with known observation dimension (e.g. Tables.jl). Previously datasets were assumed to be n-dimensional arrays, which is quite limiting and low-level.
In this case we want to preserve the vectorized version in order to optimize aggregation methods. Check the AggMode examples where the result is usually summed up.
Optimizations can dispatch on different types of dataset to aggregate the scalar version more efficiently.
Aggregation is a different story, I was only talking about the value method that takes and returns arrays. Julia specifically avoids defining âimplicitly broadcastedâ methods, as there is no reason to do this.
Aggregation is completely orthogonal to that. Although, instead of
one of the following would be cleaner and not require learning new symbols/objects (AggMode, Sum):
# minimal change from yours:
value(L1DistLoss(), [1,2,3], [2,5,-2], sum)
# like Julia Base map(f, [...], [...]), but aggregate:
sum(L1DistLoss(), [1,2,3], [2,5,-2])
I like the proposed alternatives and we could consider them in the next breaking release. The only thing I donât see yet is how to incorporate weights in these summations? It is easy when the losses are summed up without weights as we can simply rely on sum and mean from Base.
Nice, so that must be a docs issue? Iâve been looking at the âlatestâ docpage at Efficient Sum and Mean ¡ LossFunctions.jl, it should be there right? As the release already happened:
Most machine learning metrics do both a mapping and a reduction in order to calculate a scalar metric. I think itâs reasonable to encapsulate that in one function with a signature like mymetric(yĚ, y), where yĚ and y are iterables. And in fact that is what both MLJ and Flux do:
Personally, I donât really see a need for a value function. It seems like regular old explicit functions like mae(yĚ, y) are good enough and a lot easier to read.
We are refactoring the package as a whole, it is likely that the next release will get rid of the AggMode submodule and will use a more Julian approach.
A big advantage of the AggMode currently implemented in LossFunctions.jl is that it doesnât allocate intermediate arrays. We can dispatch on specific aggregation methods and reduce the terms of the aggregation without broadcasting a big vector of losses to be summed up later. Am I missing something?
Thanks for pointing these out @CameronBieganek. The main difference I see between MLJ and Flux is that MLJ decided to place weight vectors as the third argument of the loss(yĚ, y, w), whereas Flux decided to place the weights in the aggregation function loss(yĚ, y, agg=x->mean(w .* x)). Moreover, MLJ defaults to no aggregation whereas Flux defauls to mean.
In terms of performance, the MLJ approach gives more opportunity to avoid memory allocations. For example, if the aggregation is sum we can simply
sum(wi * loss(yĚi, yi) for (yĚi, yi, wi) in zip(yĚ, y, w))
without ever allocating intermediate arrays.
I am tempted to implement the MLJ approach, but am open to mroe input before proceeding.
I am starting to consider that maybe the AggMode.Sum, AggMode.Mean and AggMode.WeightedSum should be preserved as types in order to dispatch more efficient implementations for the different types of aggregations. Would be happy to be convinced otherwise.
Thanks for clarifying. We will certainly not follow this pattern. We want the same behavior always and extra options to change the behavior to something else if necessary.
I think the âimplicit broadcastingâ methods (like L1DistLoss()([1,2,3], [2,5,-2])) should just be removed, and users directed to regular Julia broadcasting.
This change doesnât affect aggregations at all!
Regarding aggregations:
Big advantage with respect to what?
Surely you can dispatch stuff like
I think we can assume that these are the only two useful aggregation functions, i.e. sum and mean with optional weights, and then get rid of all aggregation types in LossFunctions.jl. Thanks for the suggestion.
