GLM/FixedEffectModels Error: expansion cannot be computed

I recently updated my Julia setup to 1.7.2, along with updating all the packages. I am getting a strange error when running a linear model under both GLM and FixedEffectModels. I get this error only on some specifications. I also tried the same code and data on a different computer (which still has the old version (1.7.1) and packages) and things work fine.

The specifications are below. Test data to replicate the error can be found at [link].

model = reg(df_test, @formula(var1 ~ var2))
model = lm(@formula(var1 ~ var2), df_test)

The error I get is:

ERROR: expansion can't be computed
  [1] error(s::String)
    @ Base .\error.jl:33
  [2] beta_inc_asymptotic_asymmetric(a::Float64, b::Float64, x::Float64, y::Float64, w::Float64, epps::Float64)
    @ SpecialFunctions C:\Users\shres\.julia\packages\SpecialFunctions\rNt4H\src\beta_inc.jl:384
  [3] _beta_inc(a::Float64, b::Float64, x::Float64, y::Float64)
    @ SpecialFunctions C:\Users\shres\.julia\packages\SpecialFunctions\rNt4H\src\beta_inc.jl:865
  [4] _beta_inc
    @ C:\Users\shres\.julia\packages\SpecialFunctions\rNt4H\src\beta_inc.jl:736 [inlined]
  [5] beta_inc
    @ C:\Users\shres\.julia\packages\SpecialFunctions\rNt4H\src\beta_inc.jl:729 [inlined]
  [6] betaccdf
    @ C:\Users\shres\.julia\packages\StatsFuns\dTYga\src\distrs\beta.jl:44 [inlined]
  [7] fdistccdf(Ξ½1::Float64, Ξ½2::Float64, x::Float64)
    @ StatsFuns C:\Users\shres\.julia\packages\StatsFuns\dTYga\src\distrs\fdist.jl:18
  [8] ccdf
    @ C:\Users\shres\.julia\packages\Distributions\T2SAc\src\univariates.jl:624 [inlined]
  [9] _broadcast_getindex_evalf
    @ .\broadcast.jl:670 [inlined]
 [10] _broadcast_getindex
    @ .\broadcast.jl:643 [inlined]
 [11] getindex
    @ .\broadcast.jl:597 [inlined]
 [12] macro expansion
    @ .\broadcast.jl:961 [inlined]
 [13] macro expansion
    @ .\simdloop.jl:77 [inlined]
 [14] copyto!
    @ .\broadcast.jl:960 [inlined]
 [15] copyto!
    @ .\broadcast.jl:913 [inlined]
 [16] copy
    @ .\broadcast.jl:885 [inlined]
 [17] materialize
    @ .\broadcast.jl:860 [inlined]
 [18] coeftable(mm::LinearModel{GLM.LmResp{Vector{Float64}}, GLM.DensePredChol{Float64, LinearAlgebra.CholeskyPivoted{Float64, Matrix{Float64}}}}; level::Float64)
    @ GLM C:\Users\shres\.julia\packages\GLM\gt3bb\src\lm.jl:225
 [19] coeftable
    @ C:\Users\shres\.julia\packages\GLM\gt3bb\src\lm.jl:222 [inlined]
 [20] coeftable(model::StatsModels.TableRegressionModel{LinearModel{GLM.LmResp{Vector{Float64}}, GLM.DensePredChol{Float64, LinearAlgebra.CholeskyPivoted{Float64, Matrix{Float64}}}}, Matrix{Float64}}; kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), 
    @ StatsModels C:\Users\shres\.julia\packages\StatsModels\m1jYD\src\statsmodel.jl:188
 [21] coeftable
    @ C:\Users\shres\.julia\packages\StatsModels\m1jYD\src\statsmodel.jl:188 [inlined]
 [22] show(io::IOContext{Base.TTY}, model::StatsModels.TableRegressionModel{LinearModel{GLM.LmResp{Vector{Float64}}, GLM.DensePredChol{Float64, LinearAlgebra.CholeskyPivoted{Float64, Matrix{Float64}}}}, Matrix{Float64}})
    @ StatsModels C:\Users\shres\.julia\packages\StatsModels\m1jYD\src\statsmodel.jl:199
 [23] show(io::IOContext{Base.TTY}, #unused#::MIME{Symbol("text/plain")}, x::StatsModels.TableRegressionModel{LinearModel{GLM.LmResp{Vector{Float64}}, GLM.DensePredChol{Float64, LinearAlgebra.CholeskyPivoted{Float64, Matrix{Float64}}}}, Matrix{Float64}})
    @ Base.Multimedia .\multimedia.jl:47
 [24] (::REPL.var"#43#44"{REPL.REPLDisplay{REPL.LineEditREPL}, MIME{Symbol("text/plain")}, Base.RefValue{Any}})(io::Any)
    @ REPL C:\Users\shres\AppData\Local\Programs\Julia-1.7.2\share\julia\stdlib\v1.7\REPL\src\REPL.jl:266
 [25] with_repl_linfo(f::Any, repl::REPL.LineEditREPL)
    @ REPL C:\Users\shres\AppData\Local\Programs\Julia-1.7.2\share\julia\stdlib\v1.7\REPL\src\REPL.jl:510
 [26] display(d::REPL.REPLDisplay, mime::MIME{Symbol("text/plain")}, x::Any)
    @ REPL C:\Users\shres\AppData\Local\Programs\Julia-1.7.2\share\julia\stdlib\v1.7\REPL\src\REPL.jl:259
 [27] display(d::REPL.REPLDisplay, x::Any)
    @ REPL C:\Users\shres\AppData\Local\Programs\Julia-1.7.2\share\julia\stdlib\v1.7\REPL\src\REPL.jl:271
 [28] display(x::Any)
    @ Base.Multimedia .\multimedia.jl:328
 [29] #invokelatest#2
    @ .\essentials.jl:716 [inlined]
 [30] invokelatest
    @ .\essentials.jl:714 [inlined]
 [31] print_response(errio::IO, response::Any, show_value::Bool, have_color::Bool, specialdisplay::Union{Nothing, AbstractDisplay})
    @ REPL C:\Users\shres\AppData\Local\Programs\Julia-1.7.2\share\julia\stdlib\v1.7\REPL\src\REPL.jl:293
 [32] (::REPL.var"#45#46"{REPL.LineEditREPL, Pair{Any, Bool}, Bool, Bool})(io::Any)
    @ REPL C:\Users\shres\AppData\Local\Programs\Julia-1.7.2\share\julia\stdlib\v1.7\REPL\src\REPL.jl:277
 [33] with_repl_linfo(f::Any, repl::REPL.LineEditREPL)
    @ REPL C:\Users\shres\AppData\Local\Programs\Julia-1.7.2\share\julia\stdlib\v1.7\REPL\src\REPL.jl:510
 [34] print_response(repl::REPL.AbstractREPL, response::Any, show_value::Bool, have_color::Bool)
    @ REPL C:\Users\shres\AppData\Local\Programs\Julia-1.7.2\share\julia\stdlib\v1.7\REPL\src\REPL.jl:275
 [35] (::REPL.var"#do_respond#66"{Bool, Bool, REPL.var"#77#87"{REPL.LineEditREPL, REPL.REPLHistoryProvider}, REPL.LineEditREPL, REPL.LineEdit.Prompt})(s::REPL.LineEdit.MIState, buf::Any, ok::Bool)
    @ REPL C:\Users\shres\AppData\Local\Programs\Julia-1.7.2\share\julia\stdlib\v1.7\REPL\src\REPL.jl:846
 [36] #invokelatest#2
    @ .\essentials.jl:716 [inlined]
 [37] invokelatest
    @ .\essentials.jl:714 [inlined]
 [38] run_interface(terminal::REPL.Terminals.TextTerminal, m::REPL.LineEdit.ModalInterface, s::REPL.LineEdit.MIState)
    @ REPL.LineEdit C:\Users\shres\AppData\Local\Programs\Julia-1.7.2\share\julia\stdlib\v1.7\REPL\src\LineEdit.jl:2493
 [39] run_frontend(repl::REPL.LineEditREPL, backend::REPL.REPLBackendRef)
    @ REPL C:\Users\shres\AppData\Local\Programs\Julia-1.7.2\share\julia\stdlib\v1.7\REPL\src\REPL.jl:1232
 [40] (::REPL.var"#49#54"{REPL.LineEditREPL, REPL.REPLBackendRef})()
    @ REPL .\task.jl:423

Any suggestions would be appreciated!

I can’t replicate your error. Both regressions work fine for me with 1.7.2 and a fresh environment.

Can you share with us the output of ] st?

Thanks for looking into it!

Here’s what I get (note that it’s sparse because I tried deleting everything and starting from scratch):

(@v1.7) pkg> st
      Status `C:\Users\shres\.julia\environments\v1.7\Project.toml`
  [336ed68f] CSV v0.10.2
  [a93c6f00] DataFrames v1.3.2
  [9d5cd8c9] FixedEffectModels v1.6.4
  [38e38edf] GLM v1.6.1

My hunch is that this has something to do with CSV.jl. Can you try doing something like

test_df = identity.(test_df)

and see if that improves things?

I still can’t replicate with CSV.jl but maybe your csv file is different.

julia> using DataFrames, CSV, FixedEffectModels, GLM;

julia> df = DataFrame(var1 = rand(100), var2 = rand(100));

julia> CSV.write("test_df.csv", df);

julia> test_df ="test_df.csv", DataFrame);

julia> model = lm(@formula(var1 ~ var2), df_test)
StatsModels.TableRegressionModel{LinearModel{GLM.LmResp{Vector{Float64}}, GLM.DensePredChol{Float64, LinearAlgebra.CholeskyPivoted{Float64, Matrix{Float64}}}}, Matrix{Float64}}

var1 ~ 1 + var2

                 Coef.  Std. Error     t  Pr(>|t|)  Lower 95%  Upper 95%
(Intercept)  0.395065    0.0658136  6.00    <1e-07   0.26446    0.52567
var2         0.0682516   0.112477   0.61    0.5454  -0.154956   0.291459

julia> model = reg(df_test, @formula(var1 ~ var2))
                             Linear Model                             
Number of obs:                 100   Degrees of freedom:              2
R2:                          0.004   R2 Adjusted:                -0.006
F-Stat:                   0.368209   p-value:                     0.545
var1        |  Estimate Std.Error  t value Pr(>|t|) Lower 95% Upper 95%
var2        | 0.0682516  0.112477 0.606803    0.545 -0.154956  0.291459
(Intercept) |  0.395065 0.0658136  6.00279    0.000   0.26446   0.52567

Doing the identity.(test_df) does not help - I still get the same error.

I can successfully run the code snippet in your reply. Even in my main data set, a lot of the specifications work, however some fail. I haven’t been able to find a pattern under which things fail, so can’t come up with an example which does not involve reading in a file from CSV.

Try doing


and see what it looks like. Maybe some of your columns have weird types without realizing.

Both of the cols are vectors of Float64:

2-element Vector{DataType}:
 Vector{Float64} (alias for Array{Float64, 1})
 Vector{Float64} (alias for Array{Float64, 1})

I also tried reinstalling 1.6.5 (LTS) from scratch but somehow get the same error.

Sorry I failed to mention this, but I was using the master branch of FixedEffectModels (by running ] add FixedEffectModels#master) when I first got the error. There was a PR fix that I wanted to use, and I decided to update my setup before pulling in that version. I went back to the published version in the fresh reinstall but the error persists.

Very weird.

Please file an issue with StatsModels.jl and with CSV.jl to see if either of those are the problem.

Thanks - will do!

If others are facing similar issue, I was able to fix it by downgrading the StatsFuns package from 0.9.16 (latest) to 0.9.15. I will file an issue there.

Edit: Nevermind, there’s already a thread there.

1 Like

Should be fixed now that SpecialFunctions v2.1.4 is tagged

How did you actually downgrade the package? I’m having the same issue but I cannot pin StatsFun because it’s a second order dependency.

I was getting the same issue. Here is was my Julia environment:

(@v1.5) pkg> st
Status `~/.julia/environments/v1.5/Project.toml`
  [fbe9abb3] AWS v1.74.0
  [1c724243] AWSS3 v0.9.5
  [c7e460c6] ArgParse v1.1.4
  [6e4b80f9] BenchmarkTools v1.3.1
  [336ed68f] CSV v0.10.2
  [a93c6f00] DataFrames v1.3.2
  [1313f7d8] DataFramesMeta v0.10.0
  [38e38edf] GLM v1.6.1 ⚲
  [9b87118b] PackageCompiler v1.4.1
  [626c502c] Parquet v0.8.3
  [49802e3a] ProgressBars v1.4.0
  [438e738f] PyCall v1.93.0
  [1277b4bf] ShiftedArrays v1.0.0

Simply running β€œupdate” in the Julia package manner seems to have fixed the issue. I noticed that SpecialFunctions v2.1.4 was picked up, which lends credence to what @matthieu pointed out about v2.1.4 being the solution.