# Generic mapping over abstractarrays

**URL:** <https://discourse.julialang.org/t/generic-mapping-over-abstractarrays/17527>\
**Category:** General Usage\
**Created:** [November 14, 2018, 7:47pm UTC](https://discourse.julialang.org/t/generic-mapping-over-abstractarrays/17527 "2018-11-14T19:47:44Z")\
**Posts on this page:** 18\
**Page:** 1

<div class="post-metadata">

**Author:** ![cstjean](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cstjean/32/1444_2.png) [@cstjean](https://discourse.julialang.org/u/cstjean)\
**Post date:** [November 14, 2018, 7:47pm UTC](https://discourse.julialang.org/t/generic-mapping-over-abstractarrays/17527/1 "2018-11-14T19:47:44Z")

</div>

Problem: write `f(a::AbstractArray) = ...` such that

- the output is an abstractarray of the same type as `a`
- `out[i, j, ...] = a[i, j, ...] + i*j*k*...`
- `f` is non-allocating with StaticArrays. But you’re not allowed to import StaticArrays.

Is there a solution?

---

<div class="post-metadata">

**Author:** ![Evizero](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/evizero/32/10118_2.png) [@Evizero](https://discourse.julialang.org/u/Evizero)\
**Post date:** [November 14, 2018, 8:01pm UTC](https://discourse.julialang.org/t/generic-mapping-over-abstractarrays/17527/2 "2018-11-14T20:01:59Z")

</div>

you are not allowed to allocate `out`?

---

<div class="post-metadata">

**Author:** ![cstjean](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cstjean/32/1444_2.png) [@cstjean](https://discourse.julialang.org/u/cstjean)\
**Post date:** [November 14, 2018, 8:06pm UTC](https://discourse.julialang.org/t/generic-mapping-over-abstractarrays/17527/3 "2018-11-14T20:06:36Z")

</div>

With a normal `Array`, of course there will be an allocation. But with `SMatrix`, there shouldn’t be.

The point is that with just regular arrays, this is easy:

`f(a::AbstractMatrix) = [a[i, j]+i*j for i in 1:size(a, 1), for j in 1:size(a, 2)]`

but this will return a `Matrix` when passed an `SMatrix`, which is not what I want. And, of course, it doesn’t support arbitrarily-many dimensions.

---

<div class="post-metadata">

**Author:** ![cstjean](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cstjean/32/1444_2.png) [@cstjean](https://discourse.julialang.org/u/cstjean)\
**Post date:** [November 14, 2018, 8:20pm UTC](https://discourse.julialang.org/t/generic-mapping-over-abstractarrays/17527/4 "2018-11-14T20:20:59Z")

</div>

There’s `map(x->x+2, arr)`, which works nicely with both `Array` and static arrays. Is there a `map_indices(ind->f(ind), arr)`, which constructs a similar array to `arr`?

---

<div class="post-metadata">

**Author:** ![carstenbauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carstenbauer/32/4981_2.png) [@carstenbauer](https://discourse.julialang.org/u/carstenbauer)\
**Post date:** [November 14, 2018, 8:30pm UTC](https://discourse.julialang.org/t/generic-mapping-over-abstractarrays/17527/5 "2018-11-14T20:30:17Z")

</div>

What about `map!(f, similar(a), arr)`?

---

<div class="post-metadata">

**Author:** ![cstjean](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cstjean/32/1444_2.png) [@cstjean](https://discourse.julialang.org/u/cstjean)\
**Post date:** [November 14, 2018, 8:34pm UTC](https://discourse.julialang.org/t/generic-mapping-over-abstractarrays/17527/6 "2018-11-14T20:34:21Z")

</div>

That turns a stack-allocated `SVector` input into a heap-allocated `MVector` output.

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [November 14, 2018, 8:58pm UTC](https://discourse.julialang.org/t/generic-mapping-over-abstractarrays/17527/7 "2018-11-14T20:58:09Z")

</div>

You’re asking to create a fixed-size array similar to a given array (but distinct from the given array) at runtime, if I understand correctly? Without preallocating or having already created a SArray and passing that into the function, I don’t think it’s possible.

---

<div class="post-metadata">

**Author:** ![cstjean](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cstjean/32/1444_2.png) [@cstjean](https://discourse.julialang.org/u/cstjean)\
**Post date:** [November 14, 2018, 9:12pm UTC](https://discourse.julialang.org/t/generic-mapping-over-abstractarrays/17527/8 "2018-11-14T21:12:54Z")

</div>

> [@Sukera](#):
>
> Without having already created a SArray and passing that into the function,

But that’s fine. I want `f([1 2; 3 4]) == [2 4; 5 8]` and `f(@SMatrix([1 2; 3 4])) === @SMatrix [2 4; 5 8]`. Without having a dependency on `StaticArrays`

I just want a function that performs generic mapping over arrays (which `map` does), but also gives me the cartesian index of each element.

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [November 14, 2018, 10:24pm UTC](https://discourse.julialang.org/t/generic-mapping-over-abstractarrays/17527/9 "2018-11-14T22:24:23Z")

</div>

There is not a solution. The solution is to treat static arrays and scalars similarly (the immutables), and other AbstractArrays similarly (the mutables). If you then have one generic code for immutable and one for mutables then you can cover everything I’ve ran into so far. This is how DiffEq supports both BTW: it doubles each internal time step algorithm for these two forms.

---

<div class="post-metadata">

**Author:** ![sairus7](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sairus7/32/10816_2.png) [@sairus7](https://discourse.julialang.org/u/sairus7)\
**Post date:** [November 14, 2018, 11:03pm UTC](https://discourse.julialang.org/t/generic-mapping-over-abstractarrays/17527/11 "2018-11-14T23:03:51Z")

</div>

I don’t clearly understand, if scalars are immutables, yet I can reassign it with another value. If an (fixed-sized) array is immutable, why i cannot reassign its elements with another values? Is there some confusion between immutability in terms of fized size, and in terms of array elements? And is there some link between heap /stack and mutable/immutable?

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [November 14, 2018, 11:16pm UTC](https://discourse.julialang.org/t/generic-mapping-over-abstractarrays/17527/12 "2018-11-14T23:16:27Z")

</div>

> [@sairus7](#):
>
> I don’t clearly understand, if scalars are immutables, yet I can reassign it with another value. If an (fixed-sized) array is immutable, why i cannot reassign its elements with another values? Is there some confusion between immutability in terms of fized size, and in terms of array elements? And is there some link between heap /stack and mutable/immutable?

You can reassign it but not mutate it. With scalars, you cannot write a function `change!(a)` and expect it to change the value of `a` since it passes the value. Instead, you’d need to do `a = change(a)` for immutables. While for mutables in many cases `change!(a)` is more performant, so if you want to optimize both cases you need both forms. You could do things like ensure your mutating functions always return the value, so then the calling code could do `a = change!(a)`, but then `change!` itself still needs at least two code paths to handle the very different behaviors that’s going on between the mutable and immutable arrays.

---

<div class="post-metadata">

**Author:** ![sairus7](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sairus7/32/10816_2.png) [@sairus7](https://discourse.julialang.org/u/sairus7)\
**Post date:** [November 15, 2018, 1:56am UTC](https://discourse.julialang.org/t/generic-mapping-over-abstractarrays/17527/13 "2018-11-15T01:56:29Z")

</div>

Thanks! So, is there any problems with the absence of mutable, fixed-sized, stack-allocated array?  
Also, is it correct that mutable = heap-allocated?

---

<div class="post-metadata">

**Author:** ![Evizero](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/evizero/32/10118_2.png) [@Evizero](https://discourse.julialang.org/u/Evizero)\
**Post date:** [November 15, 2018, 7:22am UTC](https://discourse.julialang.org/t/generic-mapping-over-abstractarrays/17527/14 "2018-11-15T07:22:37Z")

</div>

```julia
julia> using BenchmarkTools

julia> f(X) = map((x,I) -> x + prod(Tuple(I)), X, CartesianIndices(X))
f (generic function with 1 method)

julia> @btime f($([1. 2; 3 4]))
  81.636 ns (3 allocations: 160 bytes)
2×2 Array{Float64,2}:
 2.0 4.0
 5.0 8.0

julia> using StaticArrays

julia> @btime f($(@SArray [1. 2; 3 4]))
  4.287 ns (0 allocations: 0 bytes)
2×2 SArray{Tuple{2,2},Float64,2,4}:
 2.0 4.0
 5.0 8.0

```

---

<div class="post-metadata">

**Author:** ![Evizero](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/evizero/32/10118_2.png) [@Evizero](https://discourse.julialang.org/u/Evizero)\
**Post date:** [November 15, 2018, 8:07am UTC](https://discourse.julialang.org/t/generic-mapping-over-abstractarrays/17527/15 "2018-11-15T08:07:12Z")

</div>

This is mindblowing to me:

```julia
julia> @code_llvm f(@SArray [1. 2; 3 4])

define void @julia_f_35288({ [4 x double] }* noalias nocapture sret, { [4 x double] } addrspace(11)* nocapture nonnull readonly dereferenceable(32)) {
top:
 %2 = bitcast { [4 x double] } addrspace(11)* %1 to <4 x double> addrspace(11)*
 %3 = load <4 x double>, <4 x double> addrspace(11)* %2, align 8
 %4 = fadd <4 x double> %3, <double 1.000000e+00, double 2.000000e+00, double 2.000000e+00, double 4.000000e+00>
 %5 = bitcast { [4 x double] }* %0 to <4 x double>*
 store <4 x double> %4, <4 x double>* %5, align 8
 ret void
}

```

---

<div class="post-metadata">

**Author:** ![jling](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jling/32/212909_2.png) [@jling](https://discourse.julialang.org/u/jling)\
**Post date:** [November 15, 2018, 8:51am UTC](https://discourse.julialang.org/t/generic-mapping-over-abstractarrays/17527/16 "2018-11-15T08:51:58Z")

</div>

can you explain a bit what does this mean? StaticArray is smart/hacky?

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [November 15, 2018, 9:27am UTC](https://discourse.julialang.org/t/generic-mapping-over-abstractarrays/17527/17 "2018-11-15T09:27:23Z")

</div>

> [@sairus7](#):
>
> So, is there any problems with the absence of mutable, fixed-sized, stack-allocated array?

No, this exists. It’s `MArray` in StaticArrays.jl.

> [@sairus7](#):
>
> Also, is it correct that mutable = heap-allocated?

In Julia, yes, at least for now. That’s not always true in general. C and Fortran allow for stack-allocated mutable buffers for example. It’s just not available in Julia quite yet. And it’s not always true in Julia too, since some escape analysis can optimize some cases IIRC. And as the compiler continues to get smarter, this connection between mutability and heap allocations will only lesson.

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [November 15, 2018, 9:30am UTC](https://discourse.julialang.org/t/generic-mapping-over-abstractarrays/17527/18 "2018-11-15T09:30:13Z")

</div>

> [@jling](#):
>
> can you explain a bit what does this mean? StaticArray is smart/hacky?

No, it’s that the `<4 x double>` operations are SIMD operations computing 4 floating points at a time. If you wanted to hand write the assembly and knew to do SIMD, you’d write pretty much that, yet Julia does it automatically. That doesn’t mean that it’s the most optimized LLVM IR that’s possible, but it does mean it generates better LLVM IR than I could write myself, which makes me bow down to the compiler gods.

---

<div class="post-metadata">

**Author:** ![cstjean](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cstjean/32/1444_2.png) [@cstjean](https://discourse.julialang.org/u/cstjean)\
**Post date:** [November 15, 2018, 12:47pm UTC](https://discourse.julialang.org/t/generic-mapping-over-abstractarrays/17527/19 "2018-11-15T12:47:03Z")

</div>

Fantastic! Thank you.