# Performance

**URL:** https://discourse.julialang.org/c/usage/perf/37.md?page=101

[Latest](https://discourse.julialang.org/latest.md) · [Categories](https://discourse.julialang.org/categories.md) · [Tags](https://discourse.julialang.org/tags.md)

**Page:** 102

---

## [Looking for code to solve (surely common) 'embarrassing parallelism' multithreading use-case](https://discourse.julialang.org/t/looking-for-code-to-solve-surely-common-embarrassing-parallelism-multithreading-use-case/53527)

<div class="topic-metadata">

**Author:** [@Sean\_Matthews](https://discourse.julialang.org/u/Sean_Matthews)\
**Replies:** 16\
**Last updated:** [January 20, 2021, 6:12am UTC](https://discourse.julialang.org/t/looking-for-code-to-solve-surely-common-embarrassing-parallelism-multithreading-use-case/53527 "2021-01-20T06:12:06Z")

</div>

I am looking for code that can solve the following (multi-core, shared memory) parallelisation problem, which is surely so common that a well implemented solution must exist out there. I have N cores. I have a list (…

---

## [Meta-programming an if-else statement of user-defined length](https://discourse.julialang.org/t/meta-programming-an-if-else-statement-of-user-defined-length/53525)

<div class="topic-metadata">

**Author:** [@MilesCranmer](https://discourse.julialang.org/u/MilesCranmer)\
**Replies:** 17\
**Last updated:** [January 19, 2021, 7:07am UTC](https://discourse.julialang.org/t/meta-programming-an-if-else-statement-of-user-defined-length/53525 "2021-01-19T07:07:20Z")

</div>

I am trying to optimize the way my package SymbolicRegression.jl (the pure-Julia version of PySR) calls user-passed operators. The user passes a list of operators like so: SymbolicRegression.Options( binary\_operato…

---

## [Low rank SVD](https://discourse.julialang.org/t/low-rank-svd/53529)

<div class="topic-metadata">

**Author:** [@gdkrmr](https://discourse.julialang.org/u/gdkrmr)\
**Replies:** 3\
**Last updated:** [January 18, 2021, 8:53pm UTC](https://discourse.julialang.org/t/low-rank-svd/53529 "2021-01-18T20:53:18Z")

</div>

I need to calculate a low rank SVD of a large matrix for a pseudoinverse, but I also need the individual matrices. I currently do I like this: X\_svd = svd(X) r = 20 U = X\_svd.U\[:, 1:r\] Σ = Diagonal(X\_svd.S\[1:r\]) Vt = X…

---

## [JSON list of lists to Julia matrix, preferably fast and with low memory overhead](https://discourse.julialang.org/t/json-list-of-lists-to-julia-matrix-preferably-fast-and-with-low-memory-overhead/53410)

<div class="topic-metadata">

**Author:** [@evanfields](https://discourse.julialang.org/u/evanfields)\
**Replies:** 5\
**Last updated:** [January 18, 2021, 8:01pm UTC](https://discourse.julialang.org/t/json-list-of-lists-to-julia-matrix-preferably-fast-and-with-low-memory-overhead/53410 "2021-01-18T20:01:51Z")

</div>

I have JSON files that contain a list of square matrices. Let’s say each JSON has a single key "mats" which is a list of list of lists. Each element in the outer list—i.e. a list of lists—is a matrix in row major order. …

---

## [Associating arguments with partial functions to simplify nesting](https://discourse.julialang.org/t/associating-arguments-with-partial-functions-to-simplify-nesting/53457)

<div class="topic-metadata">

**Author:** [@billykalfus](https://discourse.julialang.org/u/billykalfus)\
**Replies:** 12\
**Last updated:** [January 18, 2021, 1:28pm UTC](https://discourse.julialang.org/t/associating-arguments-with-partial-functions-to-simplify-nesting/53457 "2021-01-18T13:28:51Z")

</div>

Suppose I have some nested functions, and the lowest level function has some arguments which are caches and/or outputs whose values need to be written to and retained between calls: function a!(arg1, arg2) b!(arg1, …

---

## [Creating a macro to broadcast in parallel](https://discourse.julialang.org/t/creating-a-macro-to-broadcast-in-parallel/53508)

<div class="topic-metadata">

**Author:** [@s-kap](https://discourse.julialang.org/u/s-kap)\
**Replies:** 5\
**Last updated:** [January 18, 2021, 9:57am UTC](https://discourse.julialang.org/t/creating-a-macro-to-broadcast-in-parallel/53508 "2021-01-18T09:57:06Z")

</div>

Hey all, I’m trying to create a macro to broadcast in parallel. I have a large dataframe and I frequently create new columns based on values from old columns. Let’s take string concatenation for example. The non paral…

---

## [Inference on field types?](https://discourse.julialang.org/t/inference-on-field-types/53448)

<div class="topic-metadata">

**Author:** [@tamasgal](https://discourse.julialang.org/u/tamasgal)\
**Replies:** 10\
**Last updated:** [January 18, 2021, 12:47am UTC](https://discourse.julialang.org/t/inference-on-field-types/53448 "2021-01-18T00:47:38Z")

</div>

I often have to “categorise” a vector of struct elements by a specific field and I wanted to get rid of some duplicated logic and generalise the categorisation, however I hit the penalty of runtime (dynamic) lookup of th…

---

## [Understanding what -t does](https://discourse.julialang.org/t/understanding-what-t-does/53329)

<div class="topic-metadata">

**Author:** [@fa-bien](https://discourse.julialang.org/u/fa-bien)\
**Replies:** 18\
**Last updated:** [January 15, 2021, 11:43am UTC](https://discourse.julialang.org/t/understanding-what-t-does/53329 "2021-01-15T11:43:46Z")

</div>

I am running a single-threaded program and it executes ~10% faster when I run it with -t 2 compared to when I run with -t 1, i.e. when I allow two threads instead of one. The program does not explicitly refer to anything…

---

## [How to get persistence on distributed workers](https://discourse.julialang.org/t/how-to-get-persistence-on-distributed-workers/53098)

<div class="topic-metadata">

**Author:** [@mkarikom](https://discourse.julialang.org/u/mkarikom)\
**Replies:** 7\
**Last updated:** [January 14, 2021, 11:11pm UTC](https://discourse.julialang.org/t/how-to-get-persistence-on-distributed-workers/53098 "2021-01-14T23:11:28Z")

</div>

The following example attempts to define A on several workers, which then update their local copies. How can I make this work? using Distributed,Test addprocs(2) @everywhere function incA() A += myid() # this is th…

---

## [Faster updates in a priority queue](https://discourse.julialang.org/t/faster-updates-in-a-priority-queue/53346)

<div class="topic-metadata">

**Author:** [@shawnfrost](https://discourse.julialang.org/u/shawnfrost)\
**Replies:** 4\
**Last updated:** [January 14, 2021, 10:24pm UTC](https://discourse.julialang.org/t/faster-updates-in-a-priority-queue/53346 "2021-01-14T22:24:16Z")

</div>

Hi. I’m currently experiencing a bottleneck in my simulations where a relatively large amount of time is spent on the setindex! function from DataStructures.jl’s implementation of a priority queue. I have a representatio…

---

## [How to improve the speed of matrix multiplication?](https://discourse.julialang.org/t/how-to-improve-the-speed-of-matrix-multiplication/53293)

<div class="topic-metadata">

**Author:** [@zxjroger](https://discourse.julialang.org/u/zxjroger)\
**Replies:** 10\
**Last updated:** [January 14, 2021, 4:34pm UTC](https://discourse.julialang.org/t/how-to-improve-the-speed-of-matrix-multiplication/53293 "2021-01-14T16:34:39Z")

</div>

I have a code about large matrix multiplication in a for loop which currently takes longer time than I expected. I wonder if it is possible to improve the following MWE code: temp1 = randn(1600, 1600) temp2 = randn(1600…

---

## [Improve performance for multiple For-loops](https://discourse.julialang.org/t/improve-performance-for-multiple-for-loops/53289)

<div class="topic-metadata">

**Author:** [@EuRoXy](https://discourse.julialang.org/u/EuRoXy)\
**Replies:** 19\
**Last updated:** [January 14, 2021, 4:09pm UTC](https://discourse.julialang.org/t/improve-performance-for-multiple-for-loops/53289 "2021-01-14T16:09:40Z")

</div>

I am trying to optimize a calculation of correlation on a geo-grid of size 16x16: using NetCDF function corr1vall(s, lon1, lat1) # scenario, longitude 1, latitude 1 pathPv = myPath \* "pv/" \* ssps\[s\] \* "/day/EU/regr/…

---

## [SparseArray types over distributed reduction](https://discourse.julialang.org/t/sparsearray-types-over-distributed-reduction/53116)

<div class="topic-metadata">

**Author:** [@tblz](https://discourse.julialang.org/u/tblz)\
**Replies:** 2\
**Last updated:** [January 14, 2021, 8:21am UTC](https://discourse.julialang.org/t/sparsearray-types-over-distributed-reduction/53116 "2021-01-14T08:21:44Z")

</div>

I have a use-case of large sparse arrays that can be built in parallel. A minimum working example similar to my code is: using Distributed @everywhere using SparseArrays @everywhere function build\_array() return spar…

---

## [I want to write a better performance array, don't know if there is a ready-made one?](https://discourse.julialang.org/t/i-want-to-write-a-better-performance-array-dont-know-if-there-is-a-ready-made-one/53317)

<div class="topic-metadata">

**Author:** [@Zq\_F](https://discourse.julialang.org/u/Zq_F)\
**Replies:** 3\
**Last updated:** [January 14, 2021, 4:06am UTC](https://discourse.julialang.org/t/i-want-to-write-a-better-performance-array-dont-know-if-there-is-a-ready-made-one/53317 "2021-01-14T04:06:10Z")

</div>

I want to write a better performance array, don’t know if there is a ready-made one? I wrote a prototype struct Arr{T, N} \<: AbstractArray{T, N} ilength::Int64 data::Array{T, N} end Base.length(A::Arr) = A.ilength …

---

## [Excessive allocations when using Zygote.gradient](https://discourse.julialang.org/t/excessive-allocations-when-using-zygote-gradient/53296)

<div class="topic-metadata">

**Author:** [@MrRobot](https://discourse.julialang.org/u/MrRobot)\
**Replies:** 4\
**Last updated:** [January 13, 2021, 7:16pm UTC](https://discourse.julialang.org/t/excessive-allocations-when-using-zygote-gradient/53296 "2021-01-13T19:16:23Z")

</div>

I have as a MWE a lennard-Jones force calculation. I would like to define only energy functions, and then use automatic differentiation to calculate the derivative, and thus, force. However, for the LJ calculation, the …

---

## [DArray local part via global index range?](https://discourse.julialang.org/t/darray-local-part-via-global-index-range/53298)

<div class="topic-metadata">

**Author:** [@mkarikom](https://discourse.julialang.org/u/mkarikom)\
**Replies:** 0\
**Last updated:** [January 13, 2021, 6:22pm UTC](https://discourse.julialang.org/t/darray-local-part-via-global-index-range/53298 "2021-01-13T18:22:30Z")

</div>

I’d like to be able to get the local part via global indexing (without resorting to :L) in order to facilitate portability with SharedArrays. How can I fix the following: using Test,Distributed,DistributedArrays nwork…

---

## [Performance differences of contiguous vs non-contiguous column indexing?](https://discourse.julialang.org/t/performance-differences-of-contiguous-vs-non-contiguous-column-indexing/53243)

<div class="topic-metadata">

**Author:** [@mkarikom](https://discourse.julialang.org/u/mkarikom)\
**Replies:** 2\
**Last updated:** [January 12, 2021, 9:23pm UTC](https://discourse.julialang.org/t/performance-differences-of-contiguous-vs-non-contiguous-column-indexing/53243 "2021-01-12T21:23:38Z")

</div>

A quick test shows a 15% decrease in mean run time for the non-contiguous case, which seems huge. Why would this happen? using Random,BenchmarkTools,LinearAlgebra rng = MersenneTwister(1234); # data a = rand(rng,1000,…

---

## [Explicit distribution of chunks in SharedArray?](https://discourse.julialang.org/t/explicit-distribution-of-chunks-in-sharedarray/53249)

<div class="topic-metadata">

**Author:** [@mkarikom](https://discourse.julialang.org/u/mkarikom)\
**Replies:** 2\
**Last updated:** [January 12, 2021, 8:23pm UTC](https://discourse.julialang.org/t/explicit-distribution-of-chunks-in-sharedarray/53249 "2021-01-12T20:23:24Z")

</div>

How can I construct a SharedArray with a specific chunk distribution across workers? For instance: julia -p 2 using SharedArrays,Distributed,Test @everywhere using SharedArrays a = SharedArray{Float64,2}((3,5),init = …

---

## [Help with bottleneck in dynamic programming](https://discourse.julialang.org/t/help-with-bottleneck-in-dynamic-programming/53066)

<div class="topic-metadata">

**Author:** [@jmatheusgpa](https://discourse.julialang.org/u/jmatheusgpa)\
**Replies:** 4\
**Last updated:** [January 12, 2021, 6:14pm UTC](https://discourse.julialang.org/t/help-with-bottleneck-in-dynamic-programming/53066 "2021-01-12T18:14:14Z")

</div>

My goal is to calculate all the possible outcomes in a finite dynamic programming problem. The grid size is 100, there are three state variables and T periods. A bottleneck is the following function, which I’ve started …

---

## [Orthonormalize a matrix in place](https://discourse.julialang.org/t/orthonormalize-a-matrix-in-place/52887)

<div class="topic-metadata">

**Author:** [@matthiasbe](https://discourse.julialang.org/u/matthiasbe)\
**Replies:** 10\
**Last updated:** [January 11, 2021, 6:21pm UTC](https://discourse.julialang.org/t/orthonormalize-a-matrix-in-place/52887 "2021-01-11T18:21:32Z")

</div>

Hi, For a given matrix M \\in \\mathbb{R}^{m \\times n}, m \\geq n , I would like to orthonormalize the columns of M in place. M \\leftarrow \\texttt{qr}(M)\\texttt{.Q} I see that the function LinearAlgebra.qr! aims at some…

---

## [Sections of strings or arrays and concrete types](https://discourse.julialang.org/t/sections-of-strings-or-arrays-and-concrete-types/53041)

<div class="topic-metadata">

**Author:** [@tlienart](https://discourse.julialang.org/u/tlienart)\
**Replies:** 9\
**Last updated:** [January 8, 2021, 8:42pm UTC](https://discourse.julialang.org/t/sections-of-strings-or-arrays-and-concrete-types/53041 "2021-01-08T20:42:12Z")

</div>

I’m starting to wade into the wonderful world of SnoopCompile and the many insights it gives (and realising along the way that my understanding of types, inference etc, is shallow to say the least). One thing I got from …

---

## [Slow performance for \`rand(ComplexF64, ...)\`](https://discourse.julialang.org/t/slow-performance-for-rand-complexf64/53032)

<div class="topic-metadata">

**Author:** [@DNF](https://discourse.julialang.org/u/DNF)\
**Replies:** 5\
**Last updated:** [January 8, 2021, 4:12pm UTC](https://discourse.julialang.org/t/slow-performance-for-rand-complexf64/53032 "2021-01-08T16:12:48Z")

</div>

I was quite surprised to see this performance difference: julia\> using BenchmarkTools julia\> N = 1000; julia\> @btime complex.(rand($N, $N), rand($N, $N)); 4.852 ms (6 allocations: 30.52 MiB) julia\> @btime rand(Comp…

---

## [Why is deepcopy() -ing Sets slower than Arrays of the same size?](https://discourse.julialang.org/t/why-is-deepcopy-ing-sets-slower-than-arrays-of-the-same-size/53020)

<div class="topic-metadata">

**Author:** [@bsuwal](https://discourse.julialang.org/u/bsuwal)\
**Replies:** 4\
**Last updated:** [January 8, 2021, 3:23am UTC](https://discourse.julialang.org/t/why-is-deepcopy-ing-sets-slower-than-arrays-of-the-same-size/53020 "2021-01-08T03:23:39Z")

</div>

I have the following code: function deepcopy\_n\_times(obj::Set{Int}, n::Int) for i in 1:n deepcopy(obj) end end function deepcopy\_n\_times(obj::Array{Int, 1}, n::Int) for i in 1:n deepcopy(obj…

---

## [Launching with JULIA\_NUM\_THREADS=\<max\>](https://discourse.julialang.org/t/launching-with-julia-num-threads-max/52964)

<div class="topic-metadata">

**Author:** [@mkarikom](https://discourse.julialang.org/u/mkarikom)\
**Replies:** 2\
**Last updated:** [January 7, 2021, 6:17pm UTC](https://discourse.julialang.org/t/launching-with-julia-num-threads-max/52964 "2021-01-07T18:17:11Z")

</div>

Is there any drawback to always launching julia with the number of existing cpu cores? Suppose my cpu has 4 cores, and I need to do some non-production stuff like debugging, etc: # cpu with 4 cores firefox & export JUL…

---

## [Adding rows to a matrix dynamically](https://discourse.julialang.org/t/adding-rows-to-a-matrix-dynamically/52984)

<div class="topic-metadata">

**Author:** [@Rajesh-Nakka](https://discourse.julialang.org/u/Rajesh-Nakka)\
**Replies:** 4\
**Last updated:** [January 7, 2021, 2:00pm UTC](https://discourse.julialang.org/t/adding-rows-to-a-matrix-dynamically/52984 "2021-01-07T14:00:08Z")

</div>

I am trying to get:= Initialize a matrix of Float64 then adding rows to it dynamically in a loop! I tried with push! and append! but not able to find the right way to do it. I am currently using a not so efficient metho…

---

## [Cannot add any packages under Julia 1.5.3 (Windows 10)](https://discourse.julialang.org/t/cannot-add-any-packages-under-julia-1-5-3-windows-10/52961)

<div class="topic-metadata">

**Author:** [@neophytedave](https://discourse.julialang.org/u/neophytedave)\
**Replies:** 6\
**Last updated:** [January 6, 2021, 11:34pm UTC](https://discourse.julialang.org/t/cannot-add-any-packages-under-julia-1-5-3-windows-10/52961 "2021-01-06T23:34:21Z")

</div>

I am using Julia 1.5.3 under Windows 10, and cannot add any packages. The problem is not a misspelt package name. I have been able to add packages with all previous versions. I notice that, when attempting to add packa…

---

## [Flux usage: Sending callback function to CPU on a separate thread](https://discourse.julialang.org/t/flux-usage-sending-callback-function-to-cpu-on-a-separate-thread/52925)

<div class="topic-metadata">

**Author:** [@groovyda](https://discourse.julialang.org/u/groovyda)\
**Replies:** 2\
**Last updated:** [January 6, 2021, 5:26pm UTC](https://discourse.julialang.org/t/flux-usage-sending-callback-function-to-cpu-on-a-separate-thread/52925 "2021-01-06T17:26:07Z")

</div>

Hi there I was trying to run the Vision vgg16 model in the model-zoo in Flux. https://github.com/FluxML/model-zoo/blob/master/vision/cifar10/cifar10.jl I found that the performance on the GPU was quite slow - mainly du…

---

## [The quasi-best MaxN function](https://discourse.julialang.org/t/the-quasi-best-maxn-function/52813)

<div class="topic-metadata">

**Author:** [@rocco\_sprmnt21](https://discourse.julialang.org/u/rocco_sprmnt21)\
**Replies:** 12\
**Last updated:** [January 6, 2021, 5:26pm UTC](https://discourse.julialang.org/t/the-quasi-best-maxn-function/52813 "2021-01-06T17:26:31Z")

</div>

I have not found among the library functions a function that finds the first N maximum values of a list. I tried to define some functions and make performance comparisons. I found two kinds of problems: one was to find a…

---

## [Type instability when indexing](https://discourse.julialang.org/t/type-instability-when-indexing/52918)

<div class="topic-metadata">

**Author:** [@jlchan](https://discourse.julialang.org/u/jlchan)\
**Replies:** 8\
**Last updated:** [January 6, 2021, 6:08am UTC](https://discourse.julialang.org/t/type-instability-when-indexing/52918 "2021-01-06T06:08:45Z")

</div>

I’m trying to understand why type instability appears for certain indexing operations. Suppose I define three functions (MWE for a longer code where type parameter d is used to index into part of a tuple/array) struct b…

---

## [Vector function representation](https://discourse.julialang.org/t/vector-function-representation/52779)

<div class="topic-metadata">

**Author:** [@shawnfrost](https://discourse.julialang.org/u/shawnfrost)\
**Replies:** 1\
**Last updated:** [January 4, 2021, 12:52pm UTC](https://discourse.julialang.org/t/vector-function-representation/52779 "2021-01-04T12:52:47Z")

</div>

Hi, Some background: I’m new to writing distributed programs and I’m trying to write a variation of the SSA algorithm in Julia. My system has a lot of reactions, and I need to calculate the propensity of these reactions…

[Previous page](https://discourse.julialang.org/c/usage/perf/37.md?page=100)

[Next page](https://discourse.julialang.org/c/usage/perf/37.md?page=102)