Using BitArray to index into another Array

New to Julia
1

Hello, I am confused by the error I receive on a simple piece of code involving a BitArray. Here is a MWE:

a = [1 2 3];
B = [1 2 3; 4 5 6; 7 8 9];
ids = all(a. == B, dims=2);

Everything works as intended up to here and I obtain a BitArray that behaves as expected:

julia> all(a.==B,dims=2)
3×1 BitArray{2}:
 1
 0
 0

The problem is trying to use this BitArray to index any given array:

C = B[ids,:];

will throw an out of bounds error even though ids and B have the same maximum dimension. What am I doing wrong?

1 Like
2

ids is a matrix, not a vector. Maybe you are looking for something like

C = B[vec(ids), :]
3 Likes
3

Thank you, this was easier than I thought!

4

It is easy, but I must say that, coming from MATLAB, this feels a bit annoying. I don’t understand why I need to put vec() in between for rather essential operations involving any() and all().

1 Like
5

Are you sure indexing with the output of any or all is a good idea in the first place? It may be common in matlab, which has different semantics, but in Julia you don’t do that very often, a loop may be more convenient in many cases.

6

MATLAB variables are arrays, and the lowest dimension array is a matrix: scalars are 1x1, column vectors are Nx1, row vectors are 1xN. In a translation of this example, B(ids, :) would treat the logical matrix ids as specifying row 1 of the 1st dimension. B(ids) on the other hand would treat ids as specifying index (1,1) of both dimensions. In other words, MATLAB needs to check the number of inputs then conditionally check that a matrix object is a vector specifying 1 dimension.

Julia doesn’t work the same way. Variables can be assigned to any objects, and not all objects are arrays, let alone the parametric Array type instantiated by array literals. Array types are strictly delineated by the number of dimensions N; a vector is a different type from a matrix, and they won’t be interchangeable by default. In this example of logical indexing, a boolean matrix strictly specifies 2 dimensions, so B[ids, :] specified 3 dimensions and errored. B[ids] would also not work because the size 3x1 of the logical index does not match the 2 specified dimensions 3x3, which Julia considers a mistake. There are many ways to get an acceptable boolean vector from ids, vec(ids) is just one of them.

There’s no cross-language standard, but for comparison:

  • NumPy also demands matching dimensions.
  • R “recycles” the linear buffer to make an array with matching dimensions.
  • I can’t find official MATLAB documentation for how it works, though in practice it seems to linearize the indices instead of aligning Cartesian indices. B(ids) isn’t a good demonstration.

The non-matching dimensions behaviors can be manually done in any language, and since people seem to expect the straightforward boolean masks in practice, it’s probably better to do those manually.

1 Like
7

Not sure whether I should be starting a new thread at this point, but could you give me an example of that kind of loop? Thanks!

8 
julia> using Random

julia> Random.seed!(42);

julia> a = randn(10)
10-element Vector{Float64}:
 -0.36335748145177754
  0.2517372155742292
 -0.31498797116895605
 -0.31125240132442067
  0.8163067649323273
  0.47673837983187795
 -0.8595553820616212
 -1.4692882055065464
 -0.20661311448119266
 -0.31074387308373413

julia> a[a .< 0] .= 0;

julia> a
10-element Vector{Float64}:
 0.0
 0.2517372155742292
 0.0
 0.0
 0.8163067649323273
 0.47673837983187795
 0.0
 0.0
 0.0
 0.0

julia> Random.seed!(42);

julia> a = randn(10)
10-element Vector{Float64}:
 -0.36335748145177754
  0.2517372155742292
 -0.31498797116895605
 -0.31125240132442067
  0.8163067649323273
  0.47673837983187795
 -0.8595553820616212
 -1.4692882055065464
 -0.20661311448119266
 -0.31074387308373413

julia> for idx in eachindex(a)
           if a[idx] < 0
               a[idx] = 0
           end
       end

julia> a
10-element Vector{Float64}:
 0.0
 0.2517372155742292
 0.0
 0.0
 0.8163067649323273
 0.47673837983187795
 0.0
 0.0
 0.0
 0.0
9

Thanks, I meant this kind of situation:
a = randn(10,10)
a[6] = NaN
vec(a[vec(any(isnan.(a),dims=2)),:])

Is there any way to avoid at least one of the conversions to vec?

10

Yeah, the trouble with reductions dropping reductions is that there are two incompatible use-cases that you may want to do with the result:

  • Sometimes you want to think of the result of a vector for that particular axis — like in the case of indexing here. In this case, you want to drop the dimensions.
  • Sometimes you want to use the result with respect to the original array — for example to normalize A ./ mean(A, dims=2). This only works if dimensions aren’t dropped.

There’ve been many thoughts on how to simplify this, but as long as these two use-cases use the same functionality, one will always need some sort of conversion.

One way to detangle this is to use eachslice or eachrow. For example:

a[any.(isnan, eachrow(a)), :]
2 Likes
11

It’s implied already, but you don’t want the outermost/last vec call. You’re selecting the rows with any NaN in an input matrix, so not only is it nice to keep it as a row vector, you’d want separate rows in an output matrix in the case where there are multiple fitting rows.

In the rarer cases where dropping specified 1-length dimensions can’t be done with eachrow/eachcol/eachslice and the output isn’t a vector, use dropdims. Be aware that these functions make views, in other words they alias the underlying buffer of the original array, so collect into a fresh Array or copy certain types.

1 Like