Smoothing tracks with a Kalman filter

Of course!
Go to GitHub - yakir12/CoffeeBeetles.jl: Code base for "A dung beetle that homes without the use of landmarks" (install but you don’t need to run main()), and then if you run:

data = CoffeeBeetles.deserialize(CoffeeBeetles.datafile)
df = CoffeeBeetles.getdf(data)

that df is a data frame with tons of tracks. To access the raw coordinates of a track in the first row of that data frame, you can do this:

t = df.track[1]
t.rawcoords

The smoothed ones (again via a smoothing spline) are in t.coords.

Have at it!!!

(and any comments on anything are more than welcomed!)

I get a

julia> main()
ERROR: UndefVarError: Common not defined
Stacktrace:
 [1] deserialize_datatype(::Serialization.Serializer{IOStream}, ::Bool) at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.3/Serialization/src/Serialization.jl:1193

on Julia 1.3.1. A clue?

yea, there was a warning when you added the package that you need julia 1.4 (see the info here).

Hi @yakir12,

I might not be able to help you with the modeling part, but we have an implementation of the Kalman filter in our state space models package GitHub - LAMPSPUC/StateSpaceModels.jl: StateSpaceModels.jl is a Julia package for time-series analysis using state-space models. that might interest you. It is flexible enough to work with any user-defined linear model.

The example in GitHub - LAMPSPUC/StateSpaceModels.jl: StateSpaceModels.jl is a Julia package for time-series analysis using state-space models. looks very very relevant and cool, but not as smooth as I had hoped (also, it would be better to have an aspect ratio of 1 on that plot, but that’s minor). Any ways to get it to be smoother?

Hmm, when you say smoother you mean increase the accuracy of the model? That example is a toy problem we created just to illustrate the functionalities of the package, so the error variances are completely arbitrary (you can check the code of the example here https://github.com/LAMPSPUC/StateSpaceModels.jl/blob/master/examples/vehicle_tracking.jl).

The accuracy of the filtering and smoothing is directly related to the error variances and how well the defined model adheres to the actual stochastic process it is trying to represent. In some particular cases the square-root filter might perform better too, although it is more computationally expensive.

I should be more clear: I try to imagine how it would look like if I compared a line drawn between each consecutive measured position and the line of the estimated position. They’d be different, for sure, and I’m sure that the error to the true position would be smaller for the estimated than for the measured one, but… It’d “look” kind of similar. I should try and see how it performs with a lot more aggressive smoothing.

The problem seems to persist

julia> data = CoffeeBeetles.deserialize(CoffeeBeetles.datafile)
ERROR: UndefVarError: Common not defined
Stacktrace:
 [1] deserialize_datatype(::Serialization.Serializer{IOStream}, ::Bool) at /Users/julia/buildbot/worker/package

If you are on 1.4 or higher, and you did

import Pkg
Pkg.Registry.add("General") # you may skip this line if this is not a fresh instalation of Julia and you've updated/added a packge before
Pkg.Registry.add(Pkg.RegistrySpec(url = "https://github.com/yakir12/DackeLab")) # you need to do this only once for each instalation of Julia
Pkg.add(Pkg.PackageSpec(url = "https://github.com/yakir12/CoffeeBeetles.jl")) # you need to do this only once for each environment

It should work. I did update the instructions lately so maybe that’s where the issue is from…

Thanks, it’s working now. It was first a too old, then a too new Julia version

1 Like

After having met @yakir12 again during a recent Julia meetup, I decided to package our experiments discussed in this thread as a tutorial on using some slightly more advanced features available in particle filtering
https://baggepinnen.github.io/LowLevelParticleFilters.jl/dev/beetle_example/
Particle filters may not always be the most computationally efficient way of reaching the answer to a filtering problem, but they are incredible flexible!

2 Likes

@yakir12, is this data still available? When I follow the instructions, I get the following error:

julia> main()
ERROR: UndefRefError: access to undefined reference
Stacktrace:
  [1] getindex
    @ ./array.jl:861 [inlined]
  [2] getindex
    @ ./abstractarray.jl:1221 [inlined]
  [3] desertag
    @ ~/Scratch/julia-1.7.1/share/julia/stdlib/v1.7/Serialization/src/Serialization.jl:100 [inlined]
  [4] handle_deserialize(s::Serializer{IOStream}, b::Int32)
    @ Serialization ~/Scratch/julia-1.7.1/share/julia/stdlib/v1.7/Serialization/src/Serialization.jl:839
[...]
 [20] open(f::typeof(deserialize), args::String; kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ Base ./io.jl:330
 [21] open
    @ ./io.jl:328 [inlined]
 [22] deserialize(filename::String)
    @ Serialization ~/Scratch/julia-1.7.1/share/julia/stdlib/v1.7/Serialization/src/Serialization.jl:798
 [23] main()
    @ CoffeeBeetles ~/.julia/packages/CoffeeBeetles/snoun/src/CoffeeBeetles.jl:23
 [24] top-level scope
    @ REPL[11]:1

No sweat if you’re kind of done with this and don’t have the time to debug this for curious randos. Asking just in case. It looks like a fun dataset to tinker with.

It’s most definitely still relevant. I’m just swamped right now with preparations for more field work. @baggepinnen and I are going to make sure it’s reproducible and working ASAP.

Thanks for the interest!

1 Like

Awesome, I’ll keep an eye on the thread, thanks! I see that the blog posts just loads a vanilla csv, which would also be great. Best wishes for the field work.

1 Like

The file is here if you’d like to experiement
https://github.com/baggepinnen/LowLevelParticleFilters.jl/blob/master/docs/track.csv

1 Like

Thanks so much!