# Instatiate a tracked array? or dynamically store slices of a tracked array? (Not sure what to call this)

**URL:** <https://discourse.julialang.org/t/instatiate-a-tracked-array-or-dynamically-store-slices-of-a-tracked-array-not-sure-what-to-call-this/27101>\
**Category:** Machine Learning\
**Created:** [August 2, 2019, 4:17am UTC](https://discourse.julialang.org/t/instatiate-a-tracked-array-or-dynamically-store-slices-of-a-tracked-array-not-sure-what-to-call-this/27101 "2019-08-02T04:17:13Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![tue](https://avatars.discourse-cdn.com/v4/letter/t/4491bb/32.png) [@tue](https://discourse.julialang.org/u/tue)\
**Post date:** [August 2, 2019, 4:17am UTC](https://discourse.julialang.org/t/instatiate-a-tracked-array-or-dynamically-store-slices-of-a-tracked-array-not-sure-what-to-call-this/27101/1 "2019-08-02T04:17:13Z")

</div>

I have some code where I compute what essentially boils down to the following:

```julia
di = zeros(n,knn)
for i=1:n
    di[i,:] = d[idx[i]]
end

```

Now the problem is that d is a tracked array, and hence it cannot be put inside di which is not tracked.  
But how would I go about actually doing such an operation?

---

<div class="post-metadata">

**Author:** ![johnh](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/johnh/32/3615_2.png) [@johnh](https://discourse.julialang.org/u/johnh)\
**Post date:** [August 2, 2019, 7:53am UTC](https://discourse.julialang.org/t/instatiate-a-tracked-array-or-dynamically-store-slices-of-a-tracked-array-not-sure-what-to-call-this/27101/2 "2019-08-02T07:53:22Z")

</div>

I seem to remember in Mike Innes talk at Juliacon that tracked arrays are no longer needed in the latest branch of FLuxML and/or Zygote. I could very well be blathering.

Talk here: [JuliaCon 2019 | Differentiate All The Things! | Mike Innes - YouTube](https://youtu.be/OcUXjk7DFvU)

---

<div class="post-metadata">

**Author:** ![freemint](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/freemint/32/20013_2.png) [@freemint](https://discourse.julialang.org/u/freemint)\
**Post date:** [August 2, 2019, 8:35am UTC](https://discourse.julialang.org/t/instatiate-a-tracked-array-or-dynamically-store-slices-of-a-tracked-array-not-sure-what-to-call-this/27101/3 "2019-08-02T08:35:49Z")

</div>

When you look at the GitHub page. There are still performance regressions so it is not save to pick Zygote under all circumstances for all i know.

---

<div class="post-metadata">

**Author:** ![mcabbott](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mcabbott/32/6603_2.png) [@mcabbott](https://discourse.julialang.org/u/mcabbott)\
**Post date:** [August 2, 2019, 10:59am UTC](https://discourse.julialang.org/t/instatiate-a-tracked-array-or-dynamically-store-slices-of-a-tracked-array-not-sure-what-to-call-this/27101/4 "2019-08-02T10:59:46Z")

</div>

What is `d`? Here’s one possible interpretation, you can’t mutate a tracked array but you can construct one by indexing:

```julia
d = param(rand(2,3)) # TrackedMatrix

idx = [1,3,2,1];
di2 = d[:,idx] # Tracked

di1 = zeros(2,4); # almost your code
for i=1:4
    di1[:,i] = d[:,idx[i]].data
end
di1 # same numbers, but not tracked
di3 = d.data[:,idx] # ditto

```

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [August 2, 2019, 11:11am UTC](https://discourse.julialang.org/t/instatiate-a-tracked-array-or-dynamically-store-slices-of-a-tracked-array-not-sure-what-to-call-this/27101/5 "2019-08-02T11:11:49Z")

</div>

> [@freemint](#):
>
> it is not save to pick Zygote under all circumstances for all i know

AFAIK Zygote is still considered experimental. Flux is a very robust alternative in the meantime.

---

<div class="post-metadata">

**Author:** ![tue](https://avatars.discourse-cdn.com/v4/letter/t/4491bb/32.png) [@tue](https://discourse.julialang.org/u/tue)\
**Post date:** [August 2, 2019, 6:37pm UTC](https://discourse.julialang.org/t/instatiate-a-tracked-array-or-dynamically-store-slices-of-a-tracked-array-not-sure-what-to-call-this/27101/6 "2019-08-02T18:37:56Z")

</div>

x is the input data #cifar10 images for instance  
y = F(x,theta) #F is the neural network, and y is the output of x through the network  
d is built from the output data, hence:  
d=g(y)  
and is used to create my regularization.

I need to be able to track all steps in the creation of the regularization, since this is one of the things we are testing in the research article I’m currently writing.

Regarding your suggestion for the tracking mcabbott, I’m just worried about two things:

1. will it preserve the chain of tracking?, it is fundamental that the tracking goes all the way back to the data.
2. What exactly happens when you just overwrite a tracked parameter like what you are suggesting? is the rand value in some way influential or is that no longer tracked?

---

<div class="post-metadata">

**Author:** ![mcabbott](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mcabbott/32/6603_2.png) [@mcabbott](https://discourse.julialang.org/u/mcabbott)\
**Post date:** [August 2, 2019, 7:30pm UTC](https://discourse.julialang.org/t/instatiate-a-tracked-array-or-dynamically-store-slices-of-a-tracked-array-not-sure-what-to-call-this/27101/7 "2019-08-02T19:30:34Z")

</div>

> [@tue](#):
>
> 1. will it preserve the chain of tracking?, it is fundamental that the tracking goes all the way back to the data.

Yes, `di2` is tracked. You can check it’s working by calling `back!(di2[1,1])` and seeing that `d.grad` is nonzero. (Or by wrapping this up in a function and calling `gradient`.)

> [@tue](#):
>
> 1. What exactly happens when you just overwrite a tracked parameter like what you are suggesting? is the rand value in some way influential or is that no longer tracked?

`di1` is not tracked, it reads just the `.data` part, which is an ordinary array. But it should have the same numbers as `di2`. If you were ever to write into the `.data` part of a tracked array, then strange things will happen, don’t do this! You have to find ways to work without writing into a fresh array with a loop, i.e. like `di2` instead. (Or else you have to write a gradient for this step yourself.)

`d=rand(...)` is just a convenient way of making some numbers to try out. Really this should be your `g(y)`.