Help with updating fortran wrapper

JackDunnNZ · August 23, 2022, 8:15pm

I maintain the GLMNet.jl package, which wraps the fortran internals of the glmnet R package. This fortran dependency is shipped as glmnet_jll, and currently uses the version built here

This version of the fortran code dates from 2014, and there have been a number of updates since, so I would like to update the JLL to build the latest version of the code, however I am running into problems with the resulting binary. I am not the original developer of the package and know next to nothing about fortran so am not sure what is going wrong.

I had already updated the JLL to use a more recent source a while ago (here) which builds successfully, but when the resulting library is used from the existing GLMNet.jl, it runs into a segfault:

(GLMNet) pkg> test
...
     Testing Running tests...

signal (11): Segmentation fault: 11
in expression starting at /private/tmp/GLMNet.jl/test/runtests.jl:16
getindex at ./array.jl:924 [inlined]
#glmnet!#10 at /private/tmp/GLMNet.jl/src/GLMNet.jl:264
glmnet! at /private/tmp/GLMNet.jl/src/GLMNet.jl:271
#glmnet#40 at /private/tmp/GLMNet.jl/src/GLMNet.jl:485 [inlined]
glmnet at /private/tmp/GLMNet.jl/src/GLMNet.jl:485 [inlined]
#glmnet#41 at /private/tmp/GLMNet.jl/src/GLMNet.jl:487
glmnet at /private/tmp/GLMNet.jl/src/GLMNet.jl:487 [inlined]
glmnet at /private/tmp/GLMNet.jl/src/GLMNet.jl:487

The line it is crashing on is this one, where it tries to access alm to get results after calling the fortran code. Right above this, it accessed the value of lmu_ref which was also passed into the fortran code, so some parts seem to be working. I am wondering if there is something about the ccall that needs to be updated, but don’t know enough about fortran to know what to try next.

To help narrow the problem, I bisected the fortran code inside the glmnet R package to find which changes led to this error, and found that the last working version was 2.0-13 and was broken in 2.0-16. I added the fortran files and a cleaned diff to a gist here:

gist.github.com

https://gist.github.com/JackDunnNZ/f551fe378aa2c5186255cd936817981b

2.0.13.orig.f

c
c                          newGLMnet (5/12/14)
c
c
c                 Elastic net with squared-error loss
c
c dense predictor matrix:
c
c call elnet(ka,parm,no,ni,x,y,w,jd,vp,cl,ne,nx,nlam,flmin,ulam,thr,isd,
c            intr,maxit,lmu,a0,ca,ia,nin,rsq,alm,nlp,jerr)

This file has been truncated. show original

2.0.16.orig.f

c     mortran 2.0     (version of 7/04/75 mod 7/4/87 (ajc))
      subroutine get_int_parms(sml,eps,big,mnlam,rsqmax,pmin,exmx)          772
      implicit double precision(a-h,o-z)                                    773
      data sml0,eps0,big0,mnlam0,rsqmax0,pmin0,exmx0  /1.0d-5,1.0d-6,9.9    775
     *d35,5,0.999,1.0d-9,250.0/
      sml=sml0                                                              775
      eps=eps0                                                              775
      big=big0                                                              775
      mnlam=mnlam0                                                          775
      rsqmax=rsqmax0                                                        776

This file has been truncated. show original

changes.diff

Most of the changes seem to be updating the type declarations from real to double precision, which I think is supposed to be a no-op (all the relevant variables were/are passed as Ref{Float64})

I would appreciate any guidance or help in getting this working!

viralbshah · August 23, 2022, 9:55pm

Is there a branch with these changes?

-viral

JackDunnNZ · August 23, 2022, 10:16pm

I just pushed a branch to GitHub - JuliaStats/GLMNet.jl at jd/jll

The only change is altering the compat for glmnet_jll