Change of type results in totally different error

New to Julia
1

Hello im new to the julia, start just a 2weeks ago im trying to write down mle fitting code with given reference code
At first , usually

LoadError: TypeError: in typeassert, expected Vector{Any}, got a value of type Vector{ForwardDiff.Dual{ForwardDiff.Tag{var"#69#74"{Model, F_t_Data}, Float64}, Float64, 8}}
Stacktrace:

this kind of error(about ForwardDiff.Dual) happend, at this code

function Loglikelihood(L::AbstractVector)
    L = Float.64
    logL = -sum(log.(L))
    return logL
end;

So i changed it

function logLikelihood(L::Union{Vector{Float64}, Vector{ForwardDiff.Dual{ForwardDiff.Tag{var"#69#74"{Model, F_t_Data}, Float64}, Float64, 8}}})

    logL = -sum(log.(L))
end

so it can process both float data and Dual data form ForwardDiff which likely operated in OnceDifferentiable or Optimize.
but whenever i change it
this error occured

<LoadError: UndefVarError: mle_fit_log not defined in `Main’>

eventhough i changed nothing else except that.
i want to know why this happened and how dealing it, or, moreover, i should fix my entire code to operate mle

https://github.com/adfdfdfk/MLE_code_currentlyworkingon.git
Here is full code, if you don`t mind

2

Hi, welcome to the Discourse! It is a bit hard to help with your specific problem without a minimal working example posted here (the github link is not accessible, and typically posting code directly here makes it likelier for you to get a useful response), but let me start by pointing you in what I hope is the right direction.

Float64 and ForwardDiff.Dual{...} are both subtypes of Real, so you can (and for ForwardDiff to work nicely, should!) write

function logLikelihood(L::Vector{<:Real})

    logL = -sum(log.(L))
end

instead, which should also work well if you are computing higher order derivatives.
You could even use

function logLikelihood(L::AbstractVector{<:Real})

    logL = -sum(log.(L))
end

such that you can also use e.g. views to compute loglikelihoods for subsets of your sample!

3

Thank you for your advice
i changed it as you stated

function mle_Loglikelihood(p::Array, data::F_t_Data, M::Model)

    L = []::Vector{<:Real}

    for (F_i, t_i) in zip(data.F, data.t)
        M.F = F_i
....
logL = Loglikelihood(L::AbstractVector{<:Real})
    #@show logL
    return logL
end


function Loglikelihood(L::AbstractVector{<:Real})
    logL = -sum(log.(L))
    return logL
end

but this error happened

ERROR: LoadError: TypeError: in typeassert, expected Vector{<:Real}, got a value of type Vector{Any}
Stacktrace:
 [1] mle_Loglikelihood(p::Vector{Float64}, data::F_t_Data, M::Model)
   @ Main C:\Users\Administrator\OneDrive\Desktop\CD40code\code\src\mle_fitting.jl:36
 [2] (::var"#15#20"{Model, F_t_Data, var"#14#19"})(p::Vector{Float64})
   @ Main C:\Users\Administrator\OneDrive\Desktop\CD40code\code\src\mle_fitting.jl:64
 [3] mle_fit_log(p::Vector{Float64}, M::Model, data::F_t_Data, bounds::Vector{Tuple{Float64, Float64}})
   @ Main C:\Users\Administrator\OneDrive\Desktop\CD40code\code\src\mle_fitting.jl:68
 [4] top-level scope
   @ C:\Users\Administrator\OneDrive\Desktop\CD40code\code\excution_MLE.jl:50
in expression starting at C:\Users\Administrator\OneDrive\Desktop\CD40code\code\excution_MLE.jl:50

code for line 36 is

L = []::Vector{<:Real}

The full code are down here

using DifferentialEquations
using Optim
using BlackBoxOptim
using Zygote
using ForwardDiff
using LinearAlgebra

#______________________________________________________________

mutable struct F_t_Data
    F
    t
end

mutable struct Model
    model
    u_0::Vector
    u_0_type::String
    tspan::Tuple
    cons::Vector #using when MTP 
    F :: Float64
    solver ::Type #using when MTP
end

mutable struct Optpar
    rates
    Loglikelihood
    hes
end

#_____________________________________________________________________


function mle_Loglikelihood(p::Array, data::F_t_Data, M::Model)

    L = []::Vector{<:Real}

    for (F_i, t_i) in zip(data.F, data.t)
        M.F = F_i

        L_i = M.model(p, M, t_i ; F=F_i)
        if L_i > 0
            push!(L, L_i)
        else
        end
    end


    logL = Loglikelihood(L::AbstractVector{<:Real})
    #@show logL
    return logL
end


function Loglikelihood(L::AbstractVector{<:Real})
    logL = -sum(log.(L))
    return logL
end


function mle_fit_log(p::Vector{Float64}, M::Model, data::F_t_Data, bounds::Vector{Tuple{Float64,Float64}}) #log parameter estimating
    p2lnp = (p) -> log10.(p)
    lnp2p = (lnp::Vector{Float64}) -> 10 .^ lnp
    loss_in_est = (p) -> mle_Loglikelihood(lnp2p(p)::Vector{Float64}, data, M)


    lnp = p2lnp(p)
    loss_in_est(lnp)
    log_bounds = map(x -> (log10(x[1] + 1e-7), log(x[2])), bounds)

    res_bbo = bboptimize(loss_in_est; SearchRange=log_bounds,
        NumDimensions=length(lnp),
        NThreads=Threads.nthreads() - 1,
        MaxSteps = 5000)
    
    log_p_0_bbo = best_candidate(res_bbo)
    p_0_bbo = 10 .^ log_p_0_bbo
    @show p_0_bbo

    
    #function loss_in_opt(p::Union{Vector{Float64}, Vector{ForwardDiff.Tag{…}, Float64, 8}})
    #    if typeof(p) == Vector{Float64}
    #        loss_in = mle_Loglikelihood(p, data::F_t_Data, M::Model)
    #    elseif typeof(p) == Vector{ForwardDiff.Dual}
    #        loss_in = ForwardDiff.value.(p)[1]
    #    end
    #    return loss_in
    #end
    loss_in_opt = (p) -> mle_Loglikelihood(p, data, M)




    lb, ub = boundaries(bounds)

    od = OnceDifferentiable(loss_in_opt, p_0_bbo; autodiff = :forward)
    res_opt = optimize(od, lb, ub, p_0_bbo, Fminbox(BFGS()),
             Optim.Options(  show_trace = false,
                              iterations = 100,
                              outer_iterations =2))

    rates = res_opt.minimizer
    l = res_opt.minimum
    hes = ForwardDiff.hessian(loss_in_opt, rates)

    println("Model used == ", M.model)
    println("Loss (MLE) == ", l)
    println("Rates == ", rates)
    
    res = OptRes(rates, l, hes)
    
    return res
end

function boundaries(bounds)
    lb = [b[1] for b in bounds]
    ub = [b[2] for b in bounds]
    return lb,ub
end

By the way, thank you for your advice on question post
I`ll keep that in mind

4

The error here is fairly straightforward. First you instantiated [], an empty vector with type Vector{Any}. Then you asserted it was an instance of Vector{<:Real}. [] isn’t, so an error was thrown. If you really need L to be an instance of Vector{<:Real}, then the assertion is doing its job and stopping your program from wasting time.

If you want to instantiate an empty vector with a particular type T, the syntax is T[]. You’ll need to compute a suitable T for your program, preferably from the types of p, data, or M. Float64 is a workable temporary measure because most other builtin numeric types can be automatically converted to it when being assigned to Vector{Float64} indices.

1 Like
5

See the above reply for a solution to the problem. I noticed that Model is a so-called Abstract Container type, which may hurt performance; this problem and other obstacles to performance are described in the Performance Tips · The Julia Language. It may be (more than) worth your while to give them a good read, if you haven’t already!

6

Thank you for replying

with your comment, i have changed code like this

function mle_Loglikelihood(p::Array, data::F_t_Data, M::Model)

    L = Real[]

    for (F_i, t_i) in zip(data.F, data.t)
        M.F = F_i

        L_i = M.model(p, M, t_i ; F=F_i) 
.....

and the error has changed into

ERROR: LoadError: MethodError: no method matching zero(::Type{Any})
This error has been manually thrown, explicitly, so the method may exist but be intentionally marked as unimplemented.

Closest candidates are:
  zero(::Type{Union{Missing, T}}) where T
   @ Base missing.jl:105
  zero(::Type{Union{}}, Any...)
   @ Base number.jl:310
  zero(::Type{Missing})
   @ Base missing.jl:104
  ...

Stacktrace:
  [1] zero(::Type{Any})
    @ Base .\missing.jl:106
  [2] reduce_empty(::typeof(+), ::Type{Any})
    @ Base .\reduce.jl:343
  [3] reduce_empty(::typeof(Base.add_sum), ::Type{Any})
    @ Base .\reduce.jl:350
  [4] mapreduce_empty(::typeof(identity), op::Function, T::Type)
    @ Base .\reduce.jl:369
  [5] reduce_empty(op::Base.MappingRF{typeof(identity), typeof(Base.add_sum)}, ::Type{Any})
    @ Base .\reduce.jl:358
  [6] reduce_empty_iter
    @ .\reduce.jl:381 [inlined]
  [7] mapreduce_empty_iter(f::Function, op::Function, itr::Vector{Any}, ItrEltype::Base.HasEltype)        
    @ Base .\reduce.jl:377
  [8] _mapreduce(f::typeof(identity), op::typeof(Base.add_sum), ::IndexLinear, A::Vector{Any})
    @ Base .\reduce.jl:429
  [9] _mapreduce_dim
    @ .\reducedim.jl:337 [inlined]
 [10] mapreduce
    @ .\reducedim.jl:329 [inlined]
 [11] _sum
    @ .\reducedim.jl:987 [inlined]
 [12] _sum
    @ .\reducedim.jl:986 [inlined]
 [13] sum(a::Vector{Any})
    @ Base .\reducedim.jl:982
 [14] Loglikelihood(L::Vector{Real})
    @ Main C:\Users\Administrator\OneDrive\Desktop\CD40code\code\src\mle_fitting.jl:56
 [15] mle_Loglikelihood(p::Vector{Float64}, data::F_t_Data, M::Model)
    @ Main C:\Users\Administrator\OneDrive\Desktop\CD40code\code\src\mle_fitting.jl:49
 [16] #15
    @ C:\Users\Administrator\OneDrive\Desktop\CD40code\code\src\mle_fitting.jl:64 [inlined]
 [17] fitness(x::Vector{…}, p::FunctionBasedProblem{…})
    @ BlackBoxOptim C:\Users\Administrator\.julia\packages\BlackBoxOptim\lZtsr\src\problem.jl:61
 [18] setup_problem(func::Function, parameters::ParamsDictChain)
    @ BlackBoxOptim C:\Users\Administrator\.julia\packages\BlackBoxOptim\lZtsr\src\bboptimize.jl:37       
 [19] bbsetup(functionOrProblem::Function, parameters::Dict{…}; kwargs::@Kwargs{…})
    @ BlackBoxOptim C:\Users\Administrator\.julia\packages\BlackBoxOptim\lZtsr\src\bboptimize.jl:111      
 [20] bbsetup
    @ C:\Users\Administrator\.julia\packages\BlackBoxOptim\lZtsr\src\bboptimize.jl:109 [inlined]
 [21] bboptimize(functionOrProblem::Function, parameters::Dict{…}; kwargs::@Kwargs{…})
    @ BlackBoxOptim C:\Users\Administrator\.julia\packages\BlackBoxOptim\lZtsr\src\bboptimize.jl:92       
 [22] bboptimize (repeats 2 times)
    @ C:\Users\Administrator\.julia\packages\BlackBoxOptim\lZtsr\src\bboptimize.jl:91 [inlined]
 [23] mle_fit_log(p::Vector{Float64}, M::Model, data::F_t_Data, bounds::Vector{Tuple{Float64, Float64}})  
    @ Main C:\Users\Administrator\OneDrive\Desktop\CD40code\code\src\mle_fitting.jl:71
 [24] top-level scope
    @ C:\Users\Administrator\OneDrive\Desktop\CD40code\code\excution_MLE.jl:50
in expression starting at C:\Users\Administrator\OneDrive\Desktop\CD40code\code\excution_MLE.jl:50        
Some type information was truncated. Use `show(err)` to see complete types.

the message “manually thrown” is really ambigous to interpret…

7

Real is an abstract type which means that the vector L will be able to hold any subtype of Real. This will severely hurt performance and may have led to the error you’re seeing due to type instability (I’m on mobile so haven’t checked thoroughly).

You can probably declare the L like this: Vector{eltype(p)}(). This will make an empty vector for holding the same type of element as the array p in your mle_Loglikelihood function.

I would recommend reading the section on performance tips in the docs, specifically the part on abstract containers: Performance Tips · The Julia Language

2 Likes
8

Thank you so much
after reading performance tips, i changed some part like Model you had mentioned and dealt with p and L data type definition
and it worked pretty well!

1 Like
9

Or you can try the aptly named empty(p).

2 Likes