# Broadcast assignment for a custom array type without linear indices fails

**URL:** <https://discourse.julialang.org/t/broadcast-assignment-for-a-custom-array-type-without-linear-indices-fails/45361>\
**Category:** General Usage\
**Tags:** question\
**Created:** [August 22, 2020, 11:37am UTC](https://discourse.julialang.org/t/broadcast-assignment-for-a-custom-array-type-without-linear-indices-fails/45361 "2020-08-22T11:37:43Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![hwjsnc](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/hwjsnc/32/15178_2.png) [@hwjsnc](https://discourse.julialang.org/u/hwjsnc)\
**Post date:** [August 22, 2020, 11:37am UTC](https://discourse.julialang.org/t/broadcast-assignment-for-a-custom-array-type-without-linear-indices-fails/45361/1 "2020-08-22T11:37:43Z")

</div>

I just did some housekeeping for one of my packages that defines a custom array type. In doing so, I ran my test suite and it fails for julia 1.5 where it worked for 1.3. I haven’t done a full detailed analysis of the problem but here are my findings/noteworthy info so far:

- My array type (`NpyArray`, [~quf/Npy.jl: / - sourcehut git](https://git.sr.ht/~quf/Npy.jl/tree)) doesn’t allow linear indices (I overload LinearIndices and throw an error to make sure of this), because the arrays aren’t necessarily linearised in Fortran-contiguous order and not being aware of this may lead to hard-to-diagnose bugs.

- Here’s the failed build log: [build #285247 - failed](https://builds.sr.ht/~quf/job/285247) The stacktrace for this bug is:

```julia
  Stacktrace:
   [1] LinearIndices(::NpyArray{Bool,1}) at /home/build/Npy.jl/src/Npy.jl:277
   [2] copyto_unaliased! at ./abstractarray.jl:851 [inlined]
   [3] copyto! at ./abstractarray.jl:840 [inlined]
   [4] copyto! at ./broadcast.jl:927 [inlined]
   [5] copyto! at ./broadcast.jl:886 [inlined]
   [6] materialize! at ./broadcast.jl:848 [inlined]
   [7] materialize! at ./broadcast.jl:845 [inlined]
   [8] NpyArray(::IOStream, ::Array{Bool,1}) at /home/build/Npy.jl/src/Npy.jl:166
   [9] #8 at /home/build/Npy.jl/src/Npy.jl:182 [inlined]
   [10] open(::Npy.var"#8#9"{Tuple{Array{Bool,1}}}, ::String; kwargs::Base.Iterators.Pairs{Symbol,Bool,Tuple{Symbol,Symbol,Symbol},NamedTuple{(:read, :write, :create),Tuple{Bool,Bool,Bool}}}) at ./io.jl:325
   [11] NpyArray(::String, ::Array{Bool,1}) at /home/build/Npy.jl/src/Npy.jl:181
   [12] NpyArray(::var"#2#5", ::String, ::Vararg{Any,N} where N; kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at /home/build/Npy.jl/src/Npy.jl:189
   [13] NpyArray at /home/build/Npy.jl/src/Npy.jl:189 [inlined]
   [14] (::var"#1#4"{DataType,Tuple{Int64},Array{Bool,1}})(::String) at /home/build/Npy.jl/test/runtests.jl:102
   [15] mktempdir(::var"#1#4"{DataType,Tuple{Int64},Array{Bool,1}}, ::String; prefix::String) at ./file.jl:682
   [16] mktempdir(::Function, ::String) at ./file.jl:680 (repeats 2 times)
   [17] top-level scope at /home/build/Npy.jl/test/runtests.jl:101
   [18] top-level scope at /build/julia/src/julia-1.5.0/usr/share/julia/stdlib/v1.5/Test/src/Test.jl:1115
   [19] top-level scope at /home/build/Npy.jl/test/runtests.jl:98
   [20] include(::String) at ./client.jl:457
   [21] top-level scope at none:6

```

The problem starts at `[8]`, where I set `npy_arr .= arr` ([~quf/Npy.jl: src/Npy.jl - sourcehut git](https://git.sr.ht/~quf/Npy.jl/tree/trunk/src/Npy.jl#L166)), i.e. a simple assignment broadcast. This broadcast delegates to a couple of other functions, until `copyto_unaliased!` calls `LinearIndices` for my array type and fails.

- I tried to identify a likely commit in Julia that caused/surfaced this bug (I hesitate to call this a regression in Julia since my package relies partly on undocumented behaviour - that an array with `IndexStyle() = IndexCartesian()` is never indexed with linear indices), but was unsuccessful (it doesn’t help that for some reason the line numbers in the stacktrace don’t line up with the line numbers in the v.1.5.0 tree).

- I can fix this particular problem easily by replacing this broadcast with an explicit loop, but that doesn’t fix the problem anywhere else, and I want it to “just work”.

On to my questions: What is the best way to proceed in order to fix the problem now and make sure it stays fixed: Am I looking at a Julia regression that needs to be fixed upstream (linear indexing an `IndexCartesian` array isn’t expressly forbidden, but seems to go against the spirit of the index types)? Should I overload `copyto!` for my type, and if so which function signature do I choose? Do I have to worry about general broadcasting breaking in the future as well (it seems to work at the moment)?

---

<div class="post-metadata">

**Author:** ![tim.holy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tim.holy/32/52_2.png) [@tim.holy](https://discourse.julialang.org/u/tim.holy)\
**Post date:** [August 23, 2020, 1:34pm UTC](https://discourse.julialang.org/t/broadcast-assignment-for-a-custom-array-type-without-linear-indices-fails/45361/2 "2020-08-23T13:34:46Z")

</div>

Sorry to hear about the breakage. However, I do think you’re slightly mis-using this trait: the intention behind it is to signal the most efficient indexing style, not the only one that is supported. In particular the documentation for `IndexStyle` says

> This allows users to access elements of your array using any indexing style, even when explicit methods have not been provided.

and being able to access with a linear index requires (I believe) that `LinearIndices(A)` works.

That said, if you identify the change that turned this into an error and engage in a dialog in a GitHub issue, there’s always a chance that we’d make changes to avoid the error. (No promises, though.)

---

<div class="post-metadata">

**Author:** ![hwjsnc](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/hwjsnc/32/15178_2.png) [@hwjsnc](https://discourse.julialang.org/u/hwjsnc)\
**Post date:** [August 24, 2020, 11:28am UTC](https://discourse.julialang.org/t/broadcast-assignment-for-a-custom-array-type-without-linear-indices-fails/45361/3 "2020-08-24T11:28:44Z")

</div>

Thanks for the reply. I’ve missed that line as I was only focused on the `AbstractArray` interface documentation.

Unfortunately, I am unable to effectively identify the change that surfaced this issue at the moment. As a workaround, I have decided to emulate column major linearisation for row major arrays for now.