# Performance

**URL:** https://discourse.julialang.org/c/usage/perf/37.md?page=24

[Latest](https://discourse.julialang.org/latest.md) · [Categories](https://discourse.julialang.org/categories.md) · [Tags](https://discourse.julialang.org/tags.md)

**Page:** 25

---

## [Refactoring Julia Modules for Shape Representation and Canvas Drawing](https://discourse.julialang.org/t/refactoring-julia-modules-for-shape-representation-and-canvas-drawing/114095)

<div class="topic-metadata">

**Author:** [@Akhil\_Akkapelli](https://discourse.julialang.org/u/Akhil_Akkapelli)\
**Replies:** 1\
**Last updated:** [May 10, 2024, 11:21am UTC](https://discourse.julialang.org/t/refactoring-julia-modules-for-shape-representation-and-canvas-drawing/114095 "2024-05-10T11:21:53Z")

</div>

I have two Julia modules, “Shape.jl” and “Canvas.jl”, which define shapes and a canvas for drawing them. The code works as intended, but I’m wondering if there’s a cleaner and more efficient way to structure these module…

---

## [Can I speed up this recursive set implementation?](https://discourse.julialang.org/t/can-i-speed-up-this-recursive-set-implementation/113917)

<div class="topic-metadata">

**Author:** [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Replies:** 38\
**Last updated:** [May 10, 2024, 7:37am UTC](https://discourse.julialang.org/t/can-i-speed-up-this-recursive-set-implementation/113917 "2024-05-10T07:37:37Z")

</div>

Hey there! I come to you with another performance nerdsnipe, whose solution could have a big impact on the autodiff ecosystem (via SparseConnectivityTracer.jl and DifferentiationInterface.jl). You may remember my first…

---

## [Acceleration of Intel MKL on AMD Ryzen CPU's](https://discourse.julialang.org/t/acceleration-of-intel-mkl-on-amd-ryzen-cpus/31287)

<div class="topic-metadata">

**Author:** [@RoyiAvital](https://discourse.julialang.org/u/RoyiAvital)\
**Replies:** 34\
**Last updated:** [May 9, 2024, 12:07pm UTC](https://discourse.julialang.org/t/acceleration-of-intel-mkl-on-amd-ryzen-cpus/31287 "2024-05-09T12:07:11Z")

</div>

Some of us use Intel MKL with Julia for improved performance. Intel MKL is composed of few code paths for different features of the CPU (SSE2, SSE4, AVX2, AVX512, etc…). One of the issues of MKL is it discriminate non …

---

## [Efficient weighted sum for arbitrary data types](https://discourse.julialang.org/t/efficient-weighted-sum-for-arbitrary-data-types/113992)

<div class="topic-metadata">

**Author:** [@baxmittens](https://discourse.julialang.org/u/baxmittens)\
**Replies:** 7\
**Last updated:** [May 9, 2024, 11:26am UTC](https://discourse.julialang.org/t/efficient-weighted-sum-for-arbitrary-data-types/113992 "2024-05-09T11:26:58Z")

</div>

Hello all, I have some old Julia code, e.g., DistributedSparseGrids.jl, written in 2016, which uses in-place math operations on custom data types implemented by proprietary in-place operator functions. The task I alw…

---

## [Writing Arrow files by column](https://discourse.julialang.org/t/writing-arrow-files-by-column/113955)

<div class="topic-metadata">

**Author:** [@djholiver](https://discourse.julialang.org/u/djholiver)\
**Replies:** 1\
**Last updated:** [May 8, 2024, 2:54pm UTC](https://discourse.julialang.org/t/writing-arrow-files-by-column/113955 "2024-05-08T14:54:04Z")

</div>

Hi, It appears (and I have observed) that Arrow.jl requires the entire data to be held in memory before flushing to disk. This is prohibitively big for my use case (500GB+) and the use of Chained Vectors for batch writi…

---

## [Avoiding allocations when normalizing a vector](https://discourse.julialang.org/t/avoiding-allocations-when-normalizing-a-vector/113913)

<div class="topic-metadata">

**Author:** [@Domenico\_Lahaye](https://discourse.julialang.org/u/Domenico_Lahaye)\
**Replies:** 11\
**Last updated:** [May 6, 2024, 7:08pm UTC](https://discourse.julialang.org/t/avoiding-allocations-when-normalizing-a-vector/113913 "2024-05-06T19:08:52Z")

</div>

Hello. Given that calling function function integrand(r,rp) diff = MArray(r-rp) return normalize!(diff) end as in r = Point3D(0.,0,0); rp = Point3D(5.,0,0) @btime integrand(r,rp) results in one (single) al…

---

## [Best data structure for fast unions of large sets of integers](https://discourse.julialang.org/t/best-data-structure-for-fast-unions-of-large-sets-of-integers/113785)

<div class="topic-metadata">

**Author:** [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Replies:** 56\
**Last updated:** [May 6, 2024, 6:23pm UTC](https://discourse.julialang.org/t/best-data-structure-for-fast-unions-of-large-sets-of-integers/113785 "2024-05-06T18:23:10Z")

</div>

Nerdsnipe alert In the SparseConnectivityTracer.jl project, @hill and I need to efficiencly compute unions of non-disjoint sets of integers, which can be very large (thousands, even millions). The default Set{Int} is …

---

## [How to: High-performance differentiable programming with broad AD-library support?](https://discourse.julialang.org/t/how-to-high-performance-differentiable-programming-with-broad-ad-library-support/113884)

<div class="topic-metadata">

**Author:** [@ThummeTo](https://discourse.julialang.org/u/ThummeTo)\
**Replies:** 13\
**Last updated:** [May 6, 2024, 3:35pm UTC](https://discourse.julialang.org/t/how-to-high-performance-differentiable-programming-with-broad-ad-library-support/113884 "2024-05-06T15:35:28Z")

</div>

Hi at all, I am currently struggling with three (competing?) goals on code design. I want code, that: (1) is fast, (b) is differentiable and (c) differentiable by current (and future) AD-libraries (like at least Forward…

---

## [Is this the right way of distributing the system of differential equations over multiple nodes on a cluster?](https://discourse.julialang.org/t/is-this-the-right-way-of-distributing-the-system-of-differential-equations-over-multiple-nodes-on-a-cluster/113612)

<div class="topic-metadata">

**Author:** [@Shashank](https://discourse.julialang.org/u/Shashank)\
**Replies:** 2\
**Last updated:** [May 6, 2024, 11:51am UTC](https://discourse.julialang.org/t/is-this-the-right-way-of-distributing-the-system-of-differential-equations-over-multiple-nodes-on-a-cluster/113612 "2024-05-06T11:51:40Z")

</div>

I have a large system of ordinary differential equations, and the right-hand side of the equation is coupled and time-consuming. I have written code that parallelizes the evaluation of the right-hand side over multiple t…

---

## [Could Someone Guide me on Issue with Array Indexing in Julia?](https://discourse.julialang.org/t/could-someone-guide-me-on-issue-with-array-indexing-in-julia/113897)

<div class="topic-metadata">

**Author:** [@rich20509](https://discourse.julialang.org/u/rich20509)\
**Replies:** 1\
**Last updated:** [May 6, 2024, 11:13am UTC](https://discourse.julialang.org/t/could-someone-guide-me-on-issue-with-array-indexing-in-julia/113897 "2024-05-06T11:13:41Z")

</div>

Hello there, This is my code: Julia code demonstrating the issue x = \[1, 2, 3, 4, 5\] y = x\[1:3\] Now let’s try to modify y y\[1\] = 10 println("x: ", x) println("y: ", y) I am encountering an unexpected behavior wh…

---

## [Why is \`getindex\` slower the second time in a vector of vectors?](https://discourse.julialang.org/t/why-is-getindex-slower-the-second-time-in-a-vector-of-vectors/113819)

<div class="topic-metadata">

**Author:** [@alex180500](https://discourse.julialang.org/u/alex180500)\
**Replies:** 7\
**Last updated:** [May 5, 2024, 1:56pm UTC](https://discourse.julialang.org/t/why-is-getindex-slower-the-second-time-in-a-vector-of-vectors/113819 "2024-05-05T13:56:15Z")

</div>

Why is in this function calling the getindex method the second time slower? This function (the smallest working example I could make) is a snippet of a bigger project I’m making on agent based models where I choose a ra…

---

## [Custom optimized hash tables with UInt64 keys, lower level than Dict for ultra speed](https://discourse.julialang.org/t/custom-optimized-hash-tables-with-uint64-keys-lower-level-than-dict-for-ultra-speed/113831)

<div class="topic-metadata">

**Author:** [@0x0f0f0f](https://discourse.julialang.org/u/0x0f0f0f)\
**Replies:** 5\
**Last updated:** [May 4, 2024, 2:40pm UTC](https://discourse.julialang.org/t/custom-optimized-hash-tables-with-uint64-keys-lower-level-than-dict-for-ultra-speed/113831 "2024-05-04T14:40:57Z")

</div>

In Metatheory.jl we have e-graphs, which are basically made of some Dicts. We are encountering some performance bottlenecks because of the implementation of Dict, and we realized that optimized data structures could go a…

---

## [Faster way to find all bit arrays of weight n](https://discourse.julialang.org/t/faster-way-to-find-all-bit-arrays-of-weight-n/113658)

<div class="topic-metadata">

**Author:** [@AwesomeQuest](https://discourse.julialang.org/u/AwesomeQuest)\
**Replies:** 14\
**Last updated:** [May 2, 2024, 11:46pm UTC](https://discourse.julialang.org/t/faster-way-to-find-all-bit-arrays-of-weight-n/113658 "2024-05-02T23:46:04Z")

</div>

The obvious way to generate these bit arrays is something like: function findbcn2s(b,n) bitstrs = digits.(2^n-1:2^b-1, base=2, pad=b) .|\> BitVector filter(x-\>sum(x)==n, bitstrs) end I feel like this is horribly…

---

## [Neural ODEs: Fitting a cosine curve](https://discourse.julialang.org/t/neural-odes-fitting-a-cosine-curve/113702)

<div class="topic-metadata">

**Author:** [@bhaveshshrimali](https://discourse.julialang.org/u/bhaveshshrimali)\
**Replies:** 4\
**Last updated:** [May 2, 2024, 3:39pm UTC](https://discourse.julialang.org/t/neural-odes-fitting-a-cosine-curve/113702 "2024-05-02T15:39:51Z")

</div>

Hi everyone, I am trying to experiment with NeuralODEs and wanted to fit a simple cosine curve as an implementation exercise. I have tried experimenting the same with the solvers in torchdiffeq and diffrax but both seem…

---

## [UPDsocket recv without allocations?](https://discourse.julialang.org/t/updsocket-recv-without-allocations/93481)

<div class="topic-metadata">

**Author:** [@nhardy](https://discourse.julialang.org/u/nhardy)\
**Replies:** 3\
**Last updated:** [May 1, 2024, 8:32am UTC](https://discourse.julialang.org/t/updsocket-recv-without-allocations/93481 "2024-05-01T08:32:20Z")

</div>

I have an application where i am receiving / synthesizing data from a number of sources. One source is UDP packets. From what I can find, the only way to receive data from a sock = UDPSocket() is with recv(sock)… which r…

---

## [\[Nerdsnipe warning\] Speed up short vector comparisons to beat R](https://discourse.julialang.org/t/nerdsnipe-warning-speed-up-short-vector-comparisons-to-beat-r/113400)

<div class="topic-metadata">

**Author:** [@nilshg](https://discourse.julialang.org/u/nilshg)\
**Replies:** 37\
**Last updated:** [April 30, 2024, 8:54am UTC](https://discourse.julialang.org/t/nerdsnipe-warning-speed-up-short-vector-comparisons-to-beat-r/113400 "2024-04-30T08:54:02Z")

</div>

Here’s a problem a colleague came to me with; they’ve got an R implementation that takes a day to run something and I said “just re-write it in Julia, I’m sure it’ll take minutes”. A relatively simple re-write was indee…

---

## [Subtracting mean from DataFrame column: Why so many allocations?](https://discourse.julialang.org/t/subtracting-mean-from-dataframe-column-why-so-many-allocations/113607)

<div class="topic-metadata">

**Author:** [@JWMeer](https://discourse.julialang.org/u/JWMeer)\
**Replies:** 5\
**Last updated:** [April 29, 2024, 3:09pm UTC](https://discourse.julialang.org/t/subtracting-mean-from-dataframe-column-why-so-many-allocations/113607 "2024-04-29T15:09:57Z")

</div>

Consider the following DataFrame: df = DataFrame() df.a = repeat(1:1000, outer = 1000) df.b = rand(1000 \* 1000) This DataFrame consists of 1000 groups (grouped by a) of 1000 values each (stored in b). For each group, I…

---

## [Debugging type-instability allocations with ForwardDiff](https://discourse.julialang.org/t/debugging-type-instability-allocations-with-forwarddiff/113591)

<div class="topic-metadata">

**Author:** [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Replies:** 12\
**Last updated:** [April 29, 2024, 10:58am UTC](https://discourse.julialang.org/t/debugging-type-instability-allocations-with-forwarddiff/113591 "2024-04-29T10:58:21Z")

</div>

I’m really struggling with my ForwardDiff.jl bindings for DifferentiationInterface.jl. The goal is to achieve an allocation-free Jacobian-vector product (aka pushforward) for functions of the form f!(y, x). I have succe…

---

## [Performance discrepancy with multiple dispatch](https://discourse.julialang.org/t/performance-discrepancy-with-multiple-dispatch/113556)

<div class="topic-metadata">

**Author:** [@Albert\_de\_montserrat](https://discourse.julialang.org/u/Albert_de_montserrat)\
**Replies:** 4\
**Last updated:** [April 27, 2024, 11:07pm UTC](https://discourse.julialang.org/t/performance-discrepancy-with-multiple-dispatch/113556 "2024-04-27T23:07:26Z")

</div>

I am having problems trying to understand the difference in performance between the two foo methods described below. using Chairmarks abstract type AbstractTrait end struct ConcreteTrait \<: AbstractTrait end foo(::Arra…

---

## [Getting ForwardDiff jacobian! to execute with zero allocations](https://discourse.julialang.org/t/getting-forwarddiff-jacobian-to-execute-with-zero-allocations/72503)

<div class="topic-metadata">

**Author:** [@Anthony](https://discourse.julialang.org/u/Anthony)\
**Replies:** 9\
**Last updated:** [April 26, 2024, 12:35pm UTC](https://discourse.julialang.org/t/getting-forwarddiff-jacobian-to-execute-with-zero-allocations/72503 "2024-04-26T12:35:35Z")

</div>

I’ve been trying (so far unsuccessfully) to figure out how to execute ForwardDiff’s jacobian! function such that it will not perform additional allocations once compiled. However, I’ve been unable to figure out how to ac…

---

## [How to take the gradient of an ODE system with respect to many data points?](https://discourse.julialang.org/t/how-to-take-the-gradient-of-an-ode-system-with-respect-to-many-data-points/113368)

<div class="topic-metadata">

**Author:** [@ctessum](https://discourse.julialang.org/u/ctessum)\
**Replies:** 17\
**Last updated:** [April 25, 2024, 4:58pm UTC](https://discourse.julialang.org/t/how-to-take-the-gradient-of-an-ode-system-with-respect-to-many-data-points/113368 "2024-04-25T16:58:41Z")

</div>

Hello! I have an ode system (dx = f(x, ξ, k)), and I would like to find optimal values of ξ so that: \\mathrm{argmin}\_\\xi \\left( f(x, \\xi, k) - dx \\right)^2 where \\xi \\in R^{n\\xi} are the parameters I want to tune, x…

---

## [Replacing components with indices](https://discourse.julialang.org/t/replacing-components-with-indices/113412)

<div class="topic-metadata">

**Author:** [@KeepLearning](https://discourse.julialang.org/u/KeepLearning)\
**Replies:** 12\
**Last updated:** [April 25, 2024, 4:00pm UTC](https://discourse.julialang.org/t/replacing-components-with-indices/113412 "2024-04-25T16:00:55Z")

</div>

I am wondering if the following can be done in a better way (without for loop, perhaps): given a vector and a vector of vectors, e.g. v = \[1, 2, 3, 3, 4, 4, 5\] u = \[\[1,2\], \[3, 5\], \[4\]\] I like to get \[1, 1, 2, 2, 3, 3,…

---

## [Custom reshape() function for specific usage](https://discourse.julialang.org/t/custom-reshape-function-for-specific-usage/113382)

<div class="topic-metadata">

**Author:** [@Harrykjg-physics](https://discourse.julialang.org/u/Harrykjg-physics)\
**Replies:** 3\
**Last updated:** [April 25, 2024, 9:14am UTC](https://discourse.julialang.org/t/custom-reshape-function-for-specific-usage/113382 "2024-04-25T09:14:10Z")

</div>

Hi there ! Is there any way to minimize the memory usage of the following custom function graded\_reshape() ( Or even achieve the same performance as the Julia reshape function ) ? Suppose we have a usual Julia reshape…

---

## [Native Windows vs WSL2](https://discourse.julialang.org/t/native-windows-vs-wsl2/113468)

<div class="topic-metadata">

**Author:** [@hros](https://discourse.julialang.org/u/hros)\
**Replies:** 0\
**Last updated:** [April 24, 2024, 4:24pm UTC](https://discourse.julialang.org/t/native-windows-vs-wsl2/113468 "2024-04-24T16:24:00Z")

</div>

There is a comparison of Julia running either natively in Windows vs in WSL2 (Ubuntu) from approximately four years ago (link1, link2) Has the situation changed? or is there still a significant performance gap in favor …

---

## [ERROR: Method overwriting is not permitted during Module precompilation. Use \`\_\_precompile\_\_(false)\` to opt-out of precompilation](https://discourse.julialang.org/t/error-method-overwriting-is-not-permitted-during-module-precompilation-use-precompile-false-to-opt-out-of-precompilation/109191)

<div class="topic-metadata">

**Author:** [@schlichtanders](https://discourse.julialang.org/u/schlichtanders)\
**Replies:** 8\
**Last updated:** [April 24, 2024, 2:39pm UTC](https://discourse.julialang.org/t/error-method-overwriting-is-not-permitted-during-module-precompilation-use-precompile-false-to-opt-out-of-precompilation/109191 "2024-04-24T14:39:45Z")

</div>

Hello, I switched to julia 1.10 and see that one of my packages now shows an error during precompilation ERROR: Method overwriting is not permitted during Module precompilation. Use \`\_\_precompile\_\_(false)\` to opt-out o…

---

## [Change parameters of array components in ModelingToolkit.jl](https://discourse.julialang.org/t/change-parameters-of-array-components-in-modelingtoolkit-jl/113358)

<div class="topic-metadata">

**Author:** [@Antomek](https://discourse.julialang.org/u/Antomek)\
**Replies:** 5\
**Last updated:** [April 23, 2024, 7:00pm UTC](https://discourse.julialang.org/t/change-parameters-of-array-components-in-modelingtoolkit-jl/113358 "2024-04-23T19:00:35Z")

</div>

Hello all, I am looking for a way of setting the individual parameters of an array of components in ModelingToolkit. Consider the following example (inspired by the ModelingToolkit docs): using ModelingToolkit using M…

---

## [Avoiding runtime dispatch randomly selecting an algorithm in a loop](https://discourse.julialang.org/t/avoiding-runtime-dispatch-randomly-selecting-an-algorithm-in-a-loop/113389)

<div class="topic-metadata">

**Author:** [@f.ij](https://discourse.julialang.org/u/f.ij)\
**Replies:** 1\
**Last updated:** [April 23, 2024, 2:55pm UTC](https://discourse.julialang.org/t/avoiding-runtime-dispatch-randomly-selecting-an-algorithm-in-a-loop/113389 "2024-04-23T14:55:15Z")

</div>

I have a scenario where I’m switching between different methods of one function in a loop that has a long lifetime. I have a tuple of structs, where I’m selecting one randomly to be put into a function like so: functio…

---

## [How to encourage compiler optimizations?](https://discourse.julialang.org/t/how-to-encourage-compiler-optimizations/113325)

<div class="topic-metadata">

**Author:** [@DanDoe](https://discourse.julialang.org/u/DanDoe)\
**Replies:** 12\
**Last updated:** [April 23, 2024, 1:32pm UTC](https://discourse.julialang.org/t/how-to-encourage-compiler-optimizations/113325 "2024-04-23T13:32:18Z")

</div>

I have two functions which contain an identical block of code foo followed by different blocks of code bar1, bar2. I time these functions using our timing tool which is essentially a derivate of \_timer\_expr from TimerOut…

---

## [Why does JET seemingly not flag dynamic dispatch in this string interpolation?](https://discourse.julialang.org/t/why-does-jet-seemingly-not-flag-dynamic-dispatch-in-this-string-interpolation/112603)

<div class="topic-metadata">

**Author:** [@jishnub](https://discourse.julialang.org/u/jishnub)\
**Replies:** 4\
**Last updated:** [April 23, 2024, 1:13pm UTC](https://discourse.julialang.org/t/why-does-jet-seemingly-not-flag-dynamic-dispatch-in-this-string-interpolation/112603 "2024-04-23T13:13:54Z")

</div>

julia\> using LinearAlgebra, Cthulhu, JET julia\> D = Diagonal(rand(2)); julia\> @descend setindex!(D, 2, 1, 2) # Cthulhu setindex!(D::Diagonal, v, i::Int64, j::Int64) @ LinearAlgebra ~/.julia/juliaup/julia-1.11.0-alpha2+…

---

## [BLAS vs Threads on a cluster](https://discourse.julialang.org/t/blas-vs-threads-on-a-cluster/113332)

<div class="topic-metadata">

**Author:** [@SpuriousEigenstate](https://discourse.julialang.org/u/SpuriousEigenstate)\
**Replies:** 6\
**Last updated:** [April 23, 2024, 8:06am UTC](https://discourse.julialang.org/t/blas-vs-threads-on-a-cluster/113332 "2024-04-23T08:06:29Z")

</div>

I’m developing a script which, at its core, is mainly diagonalizing a large number of Hermitian matrices, i.e., eigen(Hermitian(A)). I want to run the code on a cluster (on a single machine, 96 threads) and am looking in…

[Previous page](https://discourse.julialang.org/c/usage/perf/37.md?page=23)

[Next page](https://discourse.julialang.org/c/usage/perf/37.md?page=25)