Anyone developing multinomial logistic regression?

proposal
#1

Hi all,

I’m looking to write a multinomial logistic regression function (including conditional logit). From what I can see, there is no extant package that can do this. Given the popularity of such a model, I’m surprised that it doesn’t exist yet in Julia, but would also be interested in collaborating. A good model for such a package would be the R mlogit package.

@Nosferican @mcreel

0 Likes

#2

Not sure what mlogit is but I have one in my package, Haven’t fully tested it yet I’d love feedback

I also have classification measures for multiclass classifications, etc.

If there’s anything missing you’d like to see let me know and I probably have the wherewithall to add it.

Edit - Ah sorry I have a multinomial softmax regression. Sorry thats a different bird.

0 Likes

#3

I will be registering my package in the next few days which has ologit and mlogit. The implementation for mlogit is similar to Stata’s mlogit rather than the R package. The R package allows for outcome specific features (e.g., price of y0, price of y1, etc) while Stata’s and mine only handle features for the unit of observation (e.g., age, education, etc.).

2 Likes

#4

Awesome! That’s definitely a great help. I am basically looking for the equivalent of Stata’s asclogit command but I suspect that I can implement that using your package as a starting point.

I have implemented this in Matlab but will be curious to see how you handle the optimization.

0 Likes

#5

I think softmax regression is the same as multinomial logistic regression. This package is really cool and I will take a look! Thanks for responding.

0 Likes

#6

For nominal response models, it would be nice to add the generalization for conditional multinomial logistic (PR welcome). For ordinal logistic regression, I implemented the proportional odds logistic regression (POLR), but the generalization would also be welcomed as a PR. I am waiting for StatsModels to tag a new release for the Terms 2.0 era and for me to finish up the last final touches.

In terms of the optimization approach, (1) the random effects (Swamy-Arora) is fitted through a partial demeaning framework, (2) absorption of categorical fixed effects is done trough the method of alternating projections, (3) multinomial logistic regression is optimized through Fisher Scoring in a Vector Generalized Linear Model (VGLM) framework, and (4) ordinal logistic regression uses Optim.jl with a Newtonian with AD for the Hessian because I lost so much time trying to write up the analytical Hessian and failed.

1 Like

#7

Awesome, thanks! My Matlab implementation of Stata’s asclogit command is basically the equivalent of calling Optim.jl. I suspect Fisher scoring is faster, but I’ve never programmed that up before.

I had a peek at your source code this morning and will be in touch with a possible PR in the coming weeks.

1 Like

#8

It might still need a tweak or two (e.g., StatsModels changes in master)… I want to get the beta test out in the coming weeks and then start working on optimizing the code and more robustness checks. That was a chapter in my defense (two weeks ago) so probably would be good to release it soon and have it stable by JuliaCon.

1 Like