Matmul for large data

New to Julia
1

Say I have a very large vector or matrix M and I want to do M*M’ to obtain say a time by time matrix. This doesn’t work for huge matrices. I guess the issue is memory? Are there any common ways to solve this?

2

The answer depends on how you need it,

  • if its to build an operator better build a SciMLOperator.jl matrix free operator (M M^T x) and solve what you need with Kyrlov
  • if you need the matrix you will have to store it in disk and play one chunk at a time ie Mmap.jl.
  • if you need some elements of the matrix you can go with LazyArrays.jl
  • If you want something more “high-lever” i guess Dagger.jl + DiskArrays.jl / DiskArrayEngine.jl could do the trick but its very young and I cant comment on it.