Hi, sorry this is kind of a beginner/performance question. I want to generate a matrix of all of the pairwise differences among the rows of a matrix. The code below works, but gets slow as matrices get large (e.g., 1000 x 200 ) c = [1 2 3 ; 4 5 6 ; 7 8 9 ; 10 11 12]; cdist = [0 0 0]; for j = 1:…

yes, thank you very much. you read my mind

To answer your actual question, there are a lot of things we can do to improve this code. The first one is to, as you mentioned, avoid using global variables. This is easy if you put your code into a function, which is always a good idea: julia> function distance_matrix(c) cdist = [0 0 0…

yes! this is the result I’m looking for

Ok, then we should measure the performance so we know if we’re having an effect: julia> using Pkg julia> Pkg.add("BenchmarkTools") julia> using BenchmarkTools julia> @btime distance_matrix($c) 1.134 μs (44 allocations: 4.36 KiB) 1 microsecond is indeed pretty slow for an operation on such a s…

Of course we can’t just stop here, because it wouldn’t be Julia if we didn’t try to make it perfect. The next issue I see with the code is that the inner-most loop is over each row of the matrix c. Iterating over the rows of a matrix in Julia is, in general, slower than iterating over the columns. T…

Next, you might notice that we’re creating cdist as a matrix of zeros and then setting every element. That means we’re wasting time filling the initial array with zeros. We can avoid that by constructing a matrix with undef, which tells Julia to just allocate memory but not fill it with anything: f…

Wow, thank you so much for all of your work on this! This is a huge help!

This final code brought things from not finishing after > 1hr to finishing in about a second for a 132 x 533 matrix (on a laptop from 2012). I’m amazed.

you lucky bastard. what a quality performance debug walk through (on a fortunately concise enough problem)

Improving performance of a nested for loop

New to Julia

rdeits October 10, 2019, 1:12am 2

Welcome! Since I saw you edited your post a couple of times to try to fix the formatting, you might want to take a look at PSA: how to quote code with backticks which explains how to quote your code properly.

Topic		Replies	Views
How to improve the performance while do matrix operation Performance	13	915	August 27, 2018
Matrix shortest distance problem - Need suggestions to improve the performance Optimization (Mathematical) array	27	1742	March 16, 2021
Alternative to computation in a loop for distances for fast performance Performance	9	1121	December 7, 2021
Something faster than for loops General Usage	27	6595	May 8, 2019
Fast offset matrix subtraction Performance question	5	812	April 21, 2022

Improving performance of a nested for loop

Related topics