Pushing elements into a view

General Usage Performance
,
1

I’m working on a clustering model, and to compute cluster weights I have to compare the cohesions of the current cluster k to the one of cluster k plus unit j (respectively old (o) and new (n) in the code). To do so I thought about using views for the current cluster k, but I haven’ figured out a way to avoid the copy in order to also add the unit j’s information.

Is there a way to define also the new variables (s1n, s2n, etc) in terms of views? since I guess all the copies are affecting a bit the performance, since also I am doing this computations thousands of times during the fit. But maybe it’s impossible since those new variables are obtained merging the information of two different arrays (eg sp1_red and sp1 for s1n).

## stuff for minimal working example
Si_red = rand((-2:2),10)
j = 1; k=1
sp1_red = rand(10); sp2_red = rand(10)
sp1 = rand(10); sp2 = rand(10)
spatial_cohesion(s1,s2) = sum(s1)+sum(s2)
covariate_similarity(X) = sum(X)
sPPM = true
cl_xPPM = true
p_cl = 2
t=1
Xcl_covariates_red = rand(10,2)
Xcl_covariates = rand(10,2,10)

aux_idxs = findall(Si_red .== k)
if sPPM
	s1o = sp1_red[aux_idxs]
	s2o = sp2_red[aux_idxs]
	s1n = copy(s1o); push!(s1n, sp1[j])
	s2n = copy(s2o); push!(s2n, sp2[j])
	lCo = spatial_cohesion(s1o, s2o)
	lCn = spatial_cohesion(s1n, s2n)
end
if cl_xPPM
	lSo = 0.; lSn = 0.
	for p in 1:p_cl
		Xo = Xcl_covariates_red[aux_idxs,p]
		Xn = copy(Xo); push!(Xn,Xcl_covariates[j,p,t])
		lSo += covariate_similarity(Xo)
		lSn += covariate_similarity(Xn)
	end
end
2

Maybe you could keep around another dataset of size n+1 and only replace the last element dynamically?

3

Haven’t looked close in your code logic. Just noticed you were talking about views but were not using them. So know you can create a view in the very easy following way:

ulia> a = [1,2,3,4,5]
5-element Vector{Int64}:
 1
 2
 3
 4
 5

julia> view(a, 2:4)
3-element view(::Vector{Int64}, 2:4) with eltype Int64:
 2
 3
 4

Pushing however should happen in the full vector a. So maybe keep some indices around and use insert! on the parent vector ?

4

Ah yes sorry, I forgot the part where I define the reduced indexes. Before that code I wrote in the other post I had this:

for j in 1:n
    # indexes = ...
	# get also the reduced spatial info if sPPM model
	if sPPM
		sp1_red = @view sp1[indexes]
		sp2_red = @view sp2[indexes]
	end
	# and the reduced covariates info if cl_xPPM model
	if cl_xPPM
		Xcl_covariates_red = @view Xcl_covariates[indexes,:,t]
	end
    # other stuff and the code I included before
5

Yes I think this is the only option. I will try it.

1 Like