# Functors instead of arrays, matrices etc

**URL:** <https://discourse.julialang.org/t/functors-instead-of-arrays-matrices-etc/109282>\
**Category:** Performance\
**Tags:** question, memory-allocation\
**Created:** [January 25, 2024, 10:38pm UTC](https://discourse.julialang.org/t/functors-instead-of-arrays-matrices-etc/109282 "2024-01-25T22:38:31Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![Enlil50](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/enlil50/32/206539_2.png) [@Enlil50](https://discourse.julialang.org/u/Enlil50)\
**Post date:** [January 25, 2024, 10:38pm UTC](https://discourse.julialang.org/t/functors-instead-of-arrays-matrices-etc/109282/1 "2024-01-25T22:38:31Z")

</div>

So, there is this C++ eigen feature which I really like: CWiseNullaryOp.

To sum it up, you can write a functor, which stores some values and returns some algorithms, and use this as a matrix.

This permits you to not store an entire “whatever you want” if you know its structure.

However they don’t really use this feature well.

An example would be a simple identity matrix written in csc or csr sparse format, which simply returns the index you ask, for row\_arr and col\_arr, and simply returns 1 for each nnz\_arr value.

Is there something like that here?

I’m really generic cause I hope it’s a feature which can use the same api of “whatever you want”, so it can be used everywhere. In particular my usecase is sparse matrices and tensors.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [January 25, 2024, 10:52pm UTC](https://discourse.julialang.org/t/functors-instead-of-arrays-matrices-etc/109282/2 "2024-01-25T22:52:05Z")

</div>

> [@Enlil50](#):
>
> Is there something like that here?

MappedArrays.jl lets you write a function and use it as a matrix. For example, this creates a 3 \times 5 matrix `A` with entries A\_{ij} = i + j that is not actually stored, but is computed on the fly:

```julia
julia> using MappedArrays

julia> A = mappedarray(i -> i[1] + i[2], CartesianIndices((1:3, 1:5)))
3×5 mappedarray(var"#1#2"(), ::CartesianIndices{2, Tuple{UnitRange{Int64}, UnitRange{Int64}}}) with eltype Int64:
 2 3 4 5 6
 3 4 5 6 7
 4 5 6 7 8

```

Of course, you can always define your own matrix-like object, e.g.:

```julia
struct MyMatrix <: AbstractMatrix{Int}
    m::Int
    n::Int
end
Base.size(A::MyMatrix) = (A.m, A.n)
Base.getindex(A::MyMatrix, i::Integer, j::Integer) = Int(i) + Int(j)

```

gives

```julia
julia> MyMatrix(3,5)
3×5 MyMatrix:
 2 3 4 5 6
 3 4 5 6 7
 4 5 6 7 8

```

Indeed, the built-in “range” types (e.g. `1:5`) as well as the `CartesianIndices` types are examples of “array” types whose entries are computed on the fly rather than being stored.

If you just need to _apply_ the matrix to vectors as a linear operator, e.g. for use in iterative algorithms, you probably want LinearMaps.jl or LinearOperators.jl instead.

> [@Enlil50](#):
>
> In particular my usecase is sparse matrices and tensors.

For sparse matrices, Julia ships with [SparseArrays](https://docs.julialang.org/en/v1/stdlib/SparseArrays/), which uses CSC format. Is there some reason that this doesn’t work for you?

---

<div class="post-metadata">

**Author:** ![Enlil50](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/enlil50/32/206539_2.png) [@Enlil50](https://discourse.julialang.org/u/Enlil50)\
**Post date:** [January 25, 2024, 11:14pm UTC](https://discourse.julialang.org/t/functors-instead-of-arrays-matrices-etc/109282/3 "2024-01-25T23:14:18Z")

</div>

If I can use both this ways in functions which require matrices, arrays or tensors in general, just like matrix multiplication or sparse kronecker products, than I think you solved it.

I didn’t know of these libraries.

Thanks

---

<div class="post-metadata">

**Author:** ![Enlil50](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/enlil50/32/206539_2.png) [@Enlil50](https://discourse.julialang.org/u/Enlil50)\
**Post date:** [January 29, 2024, 8:31pm UTC](https://discourse.julialang.org/t/functors-instead-of-arrays-matrices-etc/109282/4 "2024-01-29T20:31:19Z")

</div>

If I want, for example, to store my matrix in an array, and define the usual algorithm to map it into a matrix, can i use mappedarray or some struct? I need to give it to libraries such KrilovKit and be sure it is recognized as matrix with the algorithm I defined.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [January 29, 2024, 8:46pm UTC](https://discourse.julialang.org/t/functors-instead-of-arrays-matrices-etc/109282/5 "2024-01-29T20:46:16Z")

</div>

> [@Enlil50](#):
>
> If I want, for example, to store my matrix in an array, and define the usual algorithm to map it into a matrix, can i use mappedarray or some struct?

I guess by “store in an array” you mean some data structure other than `Matrix`.

Sure, if you have a datastructure `d` and function `getindex(d, i, j)` that retrieves the `(i,j)` element of your matrix, you can do either

1. `A = mappedarray(i -> getindex(d, i[1], i[2]), CartesianIndices((1:m, 1:n)))` to wrap this in an m \times n matrix, or …
2. Define your data structure (or some wrapper around it) to be a subtype of `AbstractMatrix`, with methods like `getindex` and `size` etcetera.

> [@Enlil50](#):
>
> I need to give it to libraries such KrilovKit and be sure it is recognized as matrix with the algorithm I defined.

In principle, a mapped array should work here, but is probably inefficient.

For an iterative solver like KrylovKit.jl, you don’t necessarily just want to expose a `getindex` function, so that it uses a fallback matrix–vector multiplication by the naive rows-times-column algorithm. Instead, you should ideally implement a _fast_ way to **multiply matrices by vectors** (a `mul!` method) that exploits the structure of your matrix. (Presumably it has some special structure, as otherwise you should just use `Matrix`). The LinearMaps.jl and LinearOperators.jl packages can help with this (as opposed to MappedArrays.jl, which isn’t geared towards that application … if you have a MappedArray and want to multiply vectors by it over and over, it is probably faster just to convert it to a `Matrix`).

In general, what operations you want to define will depend on _what library_ and what algorithms you want to use it with.

---

<div class="post-metadata">

**Author:** ![Enlil50](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/enlil50/32/206539_2.png) [@Enlil50](https://discourse.julialang.org/u/Enlil50)\
**Post date:** [January 29, 2024, 10:08pm UTC](https://discourse.julialang.org/t/functors-instead-of-arrays-matrices-etc/109282/6 "2024-01-29T22:08:58Z")

</div>

The fact is that if I have to build various matrices, even big ones (not like in the following example), i thought preallocating the biggest matrix and viewing inside it would be great:

```julia
N::Int64 = 100
m::Int64 = 10

matrix = Array{Float64}(undef, N, N)

for i in range 1:10
    matrix_view = @view matrix[0:m, 0:m]
    "Assign value to matrix view, find eigenvalues"
    m += 10

```

However i expect, if the matrix is implemented as an array with algorithm, that the last element of the i column is very far from the first element of the i+1 column. Thta’s why I thought of storing it as an array, to solve this issue. But from what you say it’s not always a trivial array which is stored under the matrix representation.

---

<div class="post-metadata">

**Author:** ![Enlil50](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/enlil50/32/206539_2.png) [@Enlil50](https://discourse.julialang.org/u/Enlil50)\
**Post date:** [January 29, 2024, 7:04pm UTC](https://discourse.julialang.org/t/functors-instead-of-arrays-matrices-etc/109282/7 "2024-01-29T19:04:51Z")

</div>

(following messages were merged from another post I made, related to previous question)

I’m new to julia and the concept of AbstractType and AbstractMatrices.  
I wanted to use krilovkit.jl and intead of preallocating a matrix and viewing inside it like this:

```julia
N::Int64 = 100
m::Int64 = 10

matrix = Array{Float64}(undef, N, N)

for i in range 1:10
    matrix_view = @view matrix[0:m, 0:m]
    "Assign value to matrix view, find eigenvalues"
    m += 10

```

I wanted to preallocate a simple array and give to him an algorithm, with the adequate matrix api for krilov.jl and other libraries, like kronecker.jl and even tensorial, with different algorithms.

```julia
N::Int64 = 100
m::Int64 = 10

matrix = Array{Float64}(undef, N*N)

```

Where matrix\_view(i,j) = matrix(i + j\*m)

The reason of why I’m doing this is to put the values I’m going to use one next to the other in the allocated space.  
I expect, in the first example, that the last element of a column of matrix\_view and the first element of the next column of matrix\_view, are far apart, as there are all the other undefined bytes preallocated between them.

Is there a way? And can something use this example to let me understand better how to use AbstractMatrices?

---

<div class="post-metadata">

**Author:** ![ufechner7](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ufechner7/32/51363_2.png) [@ufechner7](https://discourse.julialang.org/u/ufechner7)\
**Post date:** [January 29, 2024, 7:11pm UTC](https://discourse.julialang.org/t/functors-instead-of-arrays-matrices-etc/109282/8 "2024-01-29T19:11:18Z")

</div>

It would be nice if you would enclose your code examples in triple backticks, like ``` .  
Makes it easier to read.

---

<div class="post-metadata">

**Author:** ![GHTaarn](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ghtaarn/32/216007_2.png) [@GHTaarn](https://discourse.julialang.org/u/GHTaarn)\
**Post date:** [January 29, 2024, 7:46pm UTC](https://discourse.julialang.org/t/functors-instead-of-arrays-matrices-etc/109282/9 "2024-01-29T19:46:36Z")

</div>

I don’t think that I understand your question fully, if you need help specifically about krilov.jl it would be wise to put it in the title of your post (I and probably many others don’t know this package).

I do know about `AbstractMatrix` and it is normally used to specify function arguments as illustrated in this example:

```julia
function f(a::Matrix)
    a
end

function g(a::AbstractMatrix)
    a
end

x = ones(2,2)

try
    display(f(x'))
catch e
    println("error encountered ", e)
end

display(g(x'))

```

The function `g` is better than `f` because it will accept things that look like a `Matrix` but aren’t such as transposed matrices.

---

<div class="post-metadata">

**Author:** ![juthohaegeman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juthohaegeman/32/8620_2.png) [@juthohaegeman](https://discourse.julialang.org/u/juthohaegeman)\
**Post date:** [January 29, 2024, 7:46pm UTC](https://discourse.julialang.org/t/functors-instead-of-arrays-matrices-etc/109282/10 "2024-01-29T19:46:50Z")

</div>

I don’t really understand the question here, but one first remark is that in Julia, indexing in standard vectors, matrices, arrays starts at 1.

If you want a view where `matrix_view[i,j] = matrix[i + j * m]`, I think you want to do just allocate a

`storage= Vector{Float64}(undef, N*N)`

and then do something like

`reshape(view(storage, offset + (1:m^2)), (m,m))`

---

<div class="post-metadata">

**Author:** ![Enlil50](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/enlil50/32/206539_2.png) [@Enlil50](https://discourse.julialang.org/u/Enlil50)\
**Post date:** [January 29, 2024, 8:00pm UTC](https://discourse.julialang.org/t/functors-instead-of-arrays-matrices-etc/109282/11 "2024-01-29T20:00:34Z")

</div>

The fact is that i want to write a struct which is recognized as a matrix, even if it’s an array with an algorithm which picks 2 indices as input and gives as output the index, of the matrix elements, in the array.

About the indexing you are right. my fault: bad habit.

I don’t know if reshape is costly.

---

<div class="post-metadata">

**Author:** ![abraemer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/abraemer/32/51403_2.png) [@abraemer](https://discourse.julialang.org/u/abraemer)\
**Post date:** [January 29, 2024, 8:29pm UTC](https://discourse.julialang.org/t/functors-instead-of-arrays-matrices-etc/109282/12 "2024-01-29T20:29:37Z")

</div>

KrylovKit.jl actually doesn’t require your input to be a subtype of `AbstractMatrix`. You can actually just pass a function that computes the action of the linear operator on a vector. See [here](https://jutho.github.io/KrylovKit.jl/stable/#Package-features-and-alternatives)

It is quite unclear to me what your desired linear operator actually should do. It appears to me that you try to optimize something based on some idea that the default approach is too slow. This is usually a bad idea as 1) guessing the performance of code usually goes wrong and 2) you don’t have a baseline to compare to. Also you seem quite new to Julia and it’s performance characteristics are quite a bit different to most other programming languages.

I would recommend just writing the straight-forward code you want to write and if that turns out to be too slow for whatever your usecase is, then you come back and post your code here. Usually a lot of people here enjoy optimizing random code snippets 🙂

---

<div class="post-metadata">

**Author:** ![Enlil50](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/enlil50/32/206539_2.png) [@Enlil50](https://discourse.julialang.org/u/Enlil50)\
**Post date:** [January 29, 2024, 8:36pm UTC](https://discourse.julialang.org/t/functors-instead-of-arrays-matrices-etc/109282/13 "2024-01-29T20:36:48Z")

</div>

mapping an array to a matrix is the usual way of working on them and it’s how they are all implemented under the hood.

I’m really bad at browsing through documentation and, even if I already tried to see examples of matrices not defined as standard julia matrices, I can’t still find them (really serious. I don’t know why i always miss tables and the right examples when i look for them). Do you know what chapter they are in? (like the one that appear on the left)

---

<div class="post-metadata">

**Author:** ![abraemer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/abraemer/32/51403_2.png) [@abraemer](https://discourse.julialang.org/u/abraemer)\
**Post date:** [January 29, 2024, 8:45pm UTC](https://discourse.julialang.org/t/functors-instead-of-arrays-matrices-etc/109282/14 "2024-01-29T20:45:34Z")

</div>

I am not sure why you talk about mapping an array to a matrix? Yes I agree that the underlying storage is linear. That makes reshapes essentially free. On top of that every array in Julia supports linear indexing so you can treat it as vector if you wish.

I am also not sure whether all this hassle is even worth it. If you can afford to allocate a giant vector/matrix with all your data at once then I doubt the trouble is even worth it over allocating the matrices separately.

The point I made about KrylovKit.jl taking functions as inputs just means you can write a callable struct or a closure that implements the index transformation you require.

---

<div class="post-metadata">

**Author:** ![abraemer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/abraemer/32/51403_2.png) [@abraemer](https://discourse.julialang.org/u/abraemer)\
**Post date:** [January 29, 2024, 8:48pm UTC](https://discourse.julialang.org/t/functors-instead-of-arrays-matrices-etc/109282/15 "2024-01-29T20:48:25Z")

</div>

Regarding your edit: Are you looking for this page?  
[https://docs.julialang.org/en/v1/manual/interfaces/#Indexing](https://docs.julialang.org/en/v1/manual/interfaces/#Indexing)

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [January 29, 2024, 8:48pm UTC](https://discourse.julialang.org/t/functors-instead-of-arrays-matrices-etc/109282/16 "2024-01-29T20:48:26Z")

</div>

> [@Enlil50](#):
>
> Is there a way? And can something use this example to let me understand better how to use AbstractMatrices?

Isn’t this a duplicate of the other thread you opened?

> [@Functors instead of arrays, matrices etc](https://discourse.julialang.org/t/functors-instead-of-arrays-matrices-etc/109282):
>
> So, there is this C++ eigen feature which I really like: CWiseNullaryOp. To sum it up, you can write a functor, which stores some values and returns some algorithms, and use this as a matrix. This permits you to not store an entire “whatever you want” if you know its structure. However they don’t really use this feature well. An example would be a simple identity matrix written in csc or csr sparse format, which simply returns the index you ask, for row\_arr and col\_arr, and simply returns 1 …

---

<div class="post-metadata">

**Author:** ![Enlil50](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/enlil50/32/206539_2.png) [@Enlil50](https://discourse.julialang.org/u/Enlil50)\
**Post date:** [January 29, 2024, 9:49pm UTC](https://discourse.julialang.org/t/functors-instead-of-arrays-matrices-etc/109282/17 "2024-01-29T21:49:58Z")

</div>

It wasn’t a duplicate, because it was totally different, but I’m thinking on moving there this thread as well. They can be pretty similar

---

<div class="post-metadata">

**Author:** ![Enlil50](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/enlil50/32/206539_2.png) [@Enlil50](https://discourse.julialang.org/u/Enlil50)\
**Post date:** [January 29, 2024, 10:03pm UTC](https://discourse.julialang.org/t/functors-instead-of-arrays-matrices-etc/109282/18 "2024-01-29T22:03:01Z")

</div>

Thanks, I missed that part. I was read the concise documentation i found somewhere and that part is not there

---

<div class="post-metadata">

**Author:** ![Enlil50](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/enlil50/32/206539_2.png) [@Enlil50](https://discourse.julialang.org/u/Enlil50)\
**Post date:** [January 29, 2024, 10:11pm UTC](https://discourse.julialang.org/t/functors-instead-of-arrays-matrices-etc/109282/19 "2024-01-29T22:11:06Z")

</div>

@stevengj I’m going to delete (if possible) this one in a few minutes, if the issue is solved in the other one, honestly. Merging the thread there could be useful, even if the initial intent was to split them. Informations are a lot related

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [January 29, 2024, 11:40pm UTC](https://discourse.julialang.org/t/functors-instead-of-arrays-matrices-etc/109282/20 "2024-01-29T23:40:52Z")

</div>

> [@Enlil50](#):
>
> Merging the thread there could be useful, even if the initial intent was to split them. Informations are a lot related

No problem, I’ve merged the threads.

[Next page](https://discourse.julialang.org/t/functors-instead-of-arrays-matrices-etc/109282.md?page=2)