# \#benchmarktools

**URL:** https://discourse.julialang.org/tag/benchmarktools/161.md

[Latest](https://discourse.julialang.org/latest.md) · [Categories](https://discourse.julialang.org/categories.md) · [Tags](https://discourse.julialang.org/tags.md)

---

## [Benchmarking and threads](https://discourse.julialang.org/t/benchmarking-and-threads/138593)

<div class="topic-metadata">

**Author:** [@phma](https://discourse.julialang.org/u/phma)\
**Replies:** 9\
**Last updated:** [August 9, 2026, 9:14am UTC](https://discourse.julialang.org/t/benchmarking-and-threads/138593 "2026-08-09T09:14:02Z")

</div>

In the bench workspace, I have this function: function kaftorTime(textLen::Integer,keyLen::Integer) # in nanoseconds text=fill(0x69,textLen) key=fill(0x96,keyLen) trial=@benchmark kaftorEncrypt!($text,$key) me…

---

## [Benchmarking and Pkg.test()](https://discourse.julialang.org/t/benchmarking-and-pkg-test/27719)

<div class="topic-metadata">

**Author:** [@ffevotte](https://discourse.julialang.org/u/ffevotte)\
**Replies:** 5\
**Last updated:** [February 19, 2026, 2:18pm UTC](https://discourse.julialang.org/t/benchmarking-and-pkg-test/27719 "2026-02-19T14:18:28Z")

</div>

This post is an offspring of the following discussion, where I reported inconsistencies in the benchmarked performances of Base.sum between machines. To summarize the issue (which is illustrated a bit more clearly bel…

---

## [Identical method redefinition suspiciously optimizes runtime and allocations](https://discourse.julialang.org/t/identical-method-redefinition-suspiciously-optimizes-runtime-and-allocations/132156)

<div class="topic-metadata">

**Author:** [@afleming](https://discourse.julialang.org/u/afleming)\
**Replies:** 25\
**Last updated:** [September 15, 2025, 12:39am UTC](https://discourse.julialang.org/t/identical-method-redefinition-suspiciously-optimizes-runtime-and-allocations/132156 "2025-09-15T00:39:19Z")

</div>

I am using Revise.jl to work on a package for my research, and am noticing that “revising” a file to identical code with different variable names causes a function to stop allocating. I am not sure how many details othe…

---

## [How to extract histogram from @benchmark results](https://discourse.julialang.org/t/how-to-extract-histogram-from-benchmark-results/131245)

<div class="topic-metadata">

**Author:** [@JJMerelo](https://discourse.julialang.org/u/JJMerelo)\
**Replies:** 2\
**Last updated:** [July 31, 2025, 11:23am UTC](https://discourse.julialang.org/t/how-to-extract-histogram-from-benchmark-results/131245 "2025-07-31T11:23:35Z")

</div>

How can we see the histogram results as shown in from the Julia prompt?

---

## [Why mem allocation may surge when function reads single scalar global variable?](https://discourse.julialang.org/t/why-mem-allocation-may-surge-when-function-reads-single-scalar-global-variable/128394)

<div class="topic-metadata">

**Author:** [@brownlight](https://discourse.julialang.org/u/brownlight)\
**Replies:** 4\
**Last updated:** [May 2, 2025, 7:39pm UTC](https://discourse.julialang.org/t/why-mem-allocation-may-surge-when-function-reads-single-scalar-global-variable/128394 "2025-05-02T19:39:36Z")

</div>

Two functions: pi\_1( ) sums N terms in for loop; N is defined in global scope. pi\_2(nterms) same as pi\_1( ) except that N is passed as a parameter. Output (copied from notebook cells): N = 10\_000\_000 # First run @ti…

---

## [What does 0 allocations: #=nonzero=# bytes mean?](https://discourse.julialang.org/t/what-does-0-allocations-nonzero-bytes-mean/127997)

<div class="topic-metadata">

**Author:** [@Benny](https://discourse.julialang.org/u/Benny)\
**Replies:** 6\
**Last updated:** [April 12, 2025, 6:03pm UTC](https://discourse.julialang.org/t/what-does-0-allocations-nonzero-bytes-mean/127997 "2025-04-12T18:03:36Z")

</div>

Sometimes BenchmarkTools reports 0 allocations but the number of bytes isn’t 0. It can vary between runs. What does that mean? julia\> using BenchmarkTools julia\> foo() = rand((1.3, 1)) foo (generic function with 1 meth…

---

## [Performance in broadcasting vs function preallocation?](https://discourse.julialang.org/t/performance-in-broadcasting-vs-function-preallocation/126211)

<div class="topic-metadata">

**Author:** [@lepton01](https://discourse.julialang.org/u/lepton01)\
**Replies:** 6\
**Last updated:** [February 23, 2025, 2:00pm UTC](https://discourse.julialang.org/t/performance-in-broadcasting-vs-function-preallocation/126211 "2025-02-23T14:00:28Z")

</div>

Hello there. I have created a couple of functions with the same purpose, evaluating: f(x, y) = (x - 3)^2 + (y + 15)^2 over a matrix (really two vectors), the first column for x and the second one for y. The first func…

---

## [Extremely high memory consumption on CPU](https://discourse.julialang.org/t/extremely-high-memory-consumption-on-cpu/124733)

<div class="topic-metadata">

**Author:** [@cirobr](https://discourse.julialang.org/u/cirobr)\
**Replies:** 2\
**Last updated:** [January 13, 2025, 6:20pm UTC](https://discourse.julialang.org/t/extremely-high-memory-consumption-on-cpu/124733 "2025-01-13T18:20:22Z")

</div>

Cheers. I have developed a model for semantic segmentation in Flux. It has around 7M parameters. Have checked its inference performance for segmenting two classes, when an array with size (512,512,3,1) is at the input. T…

---

## [How to get BenchmarkTools times in the same unit?](https://discourse.julialang.org/t/how-to-get-benchmarktools-times-in-the-same-unit/121395)

<div class="topic-metadata">

**Author:** [@Brinkhuis](https://discourse.julialang.org/u/Brinkhuis)\
**Replies:** 3\
**Last updated:** [October 16, 2024, 7:30pm UTC](https://discourse.julialang.org/t/how-to-get-benchmarktools-times-in-the-same-unit/121395 "2024-10-16T19:30:56Z")

</div>

I use BenchmarkTools like this to save and compare performance: using BenchmarkTools b = @benchmark sum(0:2:1\_000\_000) t = sum(b.times)/length(b.times) println(t) The issue is that for some functions it returns times i…

---

## [Performance issue with Vectors vs. 1 Column Matrices](https://discourse.julialang.org/t/performance-issue-with-vectors-vs-1-column-matrices/121240)

<div class="topic-metadata">

**Author:** [@gideonsimpson](https://discourse.julialang.org/u/gideonsimpson)\
**Replies:** 8\
**Last updated:** [October 13, 2024, 7:36pm UTC](https://discourse.julialang.org/t/performance-issue-with-vectors-vs-1-column-matrices/121240 "2024-10-13T19:36:38Z")

</div>

Buried inside a for loop of mine, I have a function which, morally, looks like: function f!(a,b) @. a = b; a end It gets used inside the for loop like: # construct vector x of length nx # construct vector y of len…

---

## [Correct way to benchmark StaticVectors](https://discourse.julialang.org/t/correct-way-to-benchmark-staticvectors/119357)

<div class="topic-metadata">

**Author:** [@bthorne93](https://discourse.julialang.org/u/bthorne93)\
**Replies:** 4\
**Last updated:** [September 13, 2024, 5:01pm UTC](https://discourse.julialang.org/t/correct-way-to-benchmark-staticvectors/119357 "2024-09-13T17:01:22Z")

</div>

I am writing some code to do geometric intersection tests using StaticArrays.jl. I would like to keep some benchmarks as I am developing and would like to know if I am benchmarking static arrays correctly. The documenta…

---

## [SVectors + @reset from Accessors: Strange Benchmarks](https://discourse.julialang.org/t/svectors-reset-from-accessors-strange-benchmarks/119201)

<div class="topic-metadata">

**Author:** [@Tetrakai](https://discourse.julialang.org/u/Tetrakai)\
**Replies:** 31\
**Last updated:** [September 13, 2024, 3:00pm UTC](https://discourse.julialang.org/t/svectors-reset-from-accessors-strange-benchmarks/119201 "2024-09-13T15:00:53Z")

</div>

Full MWE: MWEusing StaticArrays, Parameters, Accessors, BenchmarkTools # Test Functions function loop1(dat) for t in 1:10 for i in eachindex(dat) @reset dat\[i\].A .+= dat\[i\].B end end…

---

## [Benchmarking array closures](https://discourse.julialang.org/t/benchmarking-array-closures/75689)

<div class="topic-metadata">

**Author:** [@lewis](https://discourse.julialang.org/u/lewis)\
**Replies:** 3\
**Last updated:** [September 5, 2024, 6:24pm UTC](https://discourse.julialang.org/t/benchmarking-array-closures/75689 "2024-09-05T18:24:37Z")

</div>

Want to verify that I’ve done this in a way that gives valid results. In the code below dostuff! updates 2 arrays in place. clos! creates a closure that encloses the arrays and accepts arguments for the change factors. …

---

## [How to make sure a task interrupts when it takes too long](https://discourse.julialang.org/t/how-to-make-sure-a-task-interrupts-when-it-takes-too-long/115939)

<div class="topic-metadata">

**Author:** [@arnerob](https://discourse.julialang.org/u/arnerob)\
**Replies:** 1\
**Last updated:** [June 20, 2024, 4:42pm UTC](https://discourse.julialang.org/t/how-to-make-sure-a-task-interrupts-when-it-takes-too-long/115939 "2024-06-20T16:42:41Z")

</div>

So I’m trying to do some experiments on how long some algorithms take. Whenever an experiment takes longer than some set time-out I want to stop it and go to another one. The algorithms I’m trying to time are pretty in…

---

## [BenchmarkTools needs a maintainer](https://discourse.julialang.org/t/benchmarktools-needs-a-maintainer/111178)

<div class="topic-metadata">

**Author:** [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Replies:** 10\
**Last updated:** [May 2, 2024, 3:42pm UTC](https://discourse.julialang.org/t/benchmarktools-needs-a-maintainer/111178 "2024-05-02T15:42:16Z")

</div>

Hi everyone, I have been maintaining BenchmarkTools.jl for about a year. I didn’t particularly want to do it, nor did the person in charge before me, but we did our best to keep it afloat. Now I am stepping down, and lo…

---

## [IterativeSolvers.jl not working as expected with mutating and allocation](https://discourse.julialang.org/t/iterativesolvers-jl-not-working-as-expected-with-mutating-and-allocation/110277)

<div class="topic-metadata">

**Author:** [@erny123](https://discourse.julialang.org/u/erny123)\
**Replies:** 9\
**Last updated:** [March 5, 2024, 9:49pm UTC](https://discourse.julialang.org/t/iterativesolvers-jl-not-working-as-expected-with-mutating-and-allocation/110277 "2024-03-05T21:49:00Z")

</div>

I’m trying to solve a simple linear system. However, it seems like the preallocated versions of any IterativeSolvers.jl functions are either much slower or allocating more memory. Here’s the example: using IterativeSol…

---

## [@btime changes the result](https://discourse.julialang.org/t/btime-changes-the-result/110425)

<div class="topic-metadata">

**Author:** [@Enlil50](https://discourse.julialang.org/u/Enlil50)\
**Replies:** 2\
**Last updated:** [February 19, 2024, 10:15pm UTC](https://discourse.julialang.org/t/btime-changes-the-result/110425 "2024-02-19T22:15:52Z")

</div>

I will write a difficult to read program because I know its analytical solution. What you just need to to is putting @btime in front of the function call, at the end. Read the last result of the 4th column: With @btim…

---

## [Benchmarking function in local scope](https://discourse.julialang.org/t/benchmarking-function-in-local-scope/110228)

<div class="topic-metadata">

**Author:** [@leespen1](https://discourse.julialang.org/u/leespen1)\
**Replies:** 4\
**Last updated:** [February 16, 2024, 7:48pm UTC](https://discourse.julialang.org/t/benchmarking-function-in-local-scope/110228 "2024-02-16T19:48:56Z")

</div>

I have a function that I would like to benchmark. To that end, I would like to time the function for several different argument values, and plot how the runtime changes with the argument values. I can do this using the @…

---

## [How to asychronously build a benchmark group](https://discourse.julialang.org/t/how-to-asychronously-build-a-benchmark-group/106335)

<div class="topic-metadata">

**Author:** [@filchristou](https://discourse.julialang.org/u/filchristou)\
**Replies:** 3\
**Last updated:** [November 17, 2023, 8:35pm UTC](https://discourse.julialang.org/t/how-to-asychronously-build-a-benchmark-group/106335 "2023-11-17T20:35:04Z")

</div>

Can I build several inter-depending benchmarks from a serial procedure in “one go” using BenchmarkTools.jl ? Say I have the following functionality laid out in a function function mymutatingfunc!(x) mutfunc1!(x) …

---

## [Use tags, setup, teardown with the \`@benchmarkset\` macro](https://discourse.julialang.org/t/use-tags-setup-teardown-with-the-benchmarkset-macro/98936)

<div class="topic-metadata">

**Author:** [@AlexanderNenninger](https://discourse.julialang.org/u/AlexanderNenninger)\
**Replies:** 2\
**Last updated:** [November 17, 2023, 10:17am UTC](https://discourse.julialang.org/t/use-tags-setup-teardown-with-the-benchmarkset-macro/98936 "2023-11-17T10:17:07Z")

</div>

Is there a way to use tags with the @benchmarkset macro? What about setup and teardown? I’ve been searching the docsumentation but to no avail.

---

## [Making @benchmark outputs statistically meaningful, and actionable](https://discourse.julialang.org/t/making-benchmark-outputs-statistically-meaningful-and-actionable/104256)

<div class="topic-metadata">

**Author:** [@user664303](https://discourse.julialang.org/u/user664303)\
**Replies:** 123\
**Last updated:** [November 15, 2023, 1:01pm UTC](https://discourse.julialang.org/t/making-benchmark-outputs-statistically-meaningful-and-actionable/104256 "2023-11-15T13:01:44Z")

</div>

I see a huge disparity in the output of different runs of @benchmark, for the same code. See the following two outputs, made consecutively, with zero code changes and no restarting Julia: BenchmarkTools.Trial: 161 sampl…

---

## [How to compare commits with PkgBenchmark](https://discourse.julialang.org/t/how-to-compare-commits-with-pkgbenchmark/104468)

<div class="topic-metadata">

**Author:** [@Christopher\_Fisher](https://discourse.julialang.org/u/Christopher_Fisher)\
**Replies:** 2\
**Last updated:** [October 1, 2023, 5:25pm UTC](https://discourse.julialang.org/t/how-to-compare-commits-with-pkgbenchmark/104468 "2023-10-01T17:25:54Z")

</div>

Hi, I would like to compare benchmarks for two commits. I’m trying to follow the documentation, but I receive an error indicating that the input is not a repo. I tried a few different approach, but with the same result.…

---

## [Should we enable storage of function outputs in BenchmarkTools?](https://discourse.julialang.org/t/should-we-enable-storage-of-function-outputs-in-benchmarktools/103997)

<div class="topic-metadata">

**Author:** [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Replies:** 13\
**Last updated:** [September 26, 2023, 3:01pm UTC](https://discourse.julialang.org/t/should-we-enable-storage-of-function-outputs-in-benchmarktools/103997 "2023-09-26T15:01:11Z")

</div>

Currently doing some maintenance on BenchmarkTools.jl before the upcoming 2.0 release (I’m aiming for October), and I stumbled upon a PR I’m unsure about. The author @AlexanderNenninger wants to record the values return…

---

## [How to plot benchmarks](https://discourse.julialang.org/t/how-to-plot-benchmarks/104243)

<div class="topic-metadata">

**Author:** [@Christopher\_Fisher](https://discourse.julialang.org/u/Christopher_Fisher)\
**Replies:** 7\
**Last updated:** [September 26, 2023, 10:11am UTC](https://discourse.julialang.org/t/how-to-plot-benchmarks/104243 "2023-09-26T10:11:24Z")

</div>

Hi all, I would like to plot some benchmarks, but its not clear to me what approach to take. As a simple example, suppose I have: using BenchmarkTools using DataFrames using Statistics SUITE = BenchmarkGroup() SUITE…

---

## [Using Benchmarktools with in-place functions and a default mutable struct](https://discourse.julialang.org/t/using-benchmarktools-with-in-place-functions-and-a-default-mutable-struct/103545)

<div class="topic-metadata">

**Author:** [@arnerob](https://discourse.julialang.org/u/arnerob)\
**Replies:** 2\
**Last updated:** [September 5, 2023, 7:11pm UTC](https://discourse.julialang.org/t/using-benchmarktools-with-in-place-functions-and-a-default-mutable-struct/103545 "2023-09-05T19:11:10Z")

</div>

I want to benchmark a function by every time using the default value of a mutable struct. I tried to recreate a minimal working example of my code and what goes wrong: mutable struct myNum{T\<:Real} n::T; myN…

---

## [Get unit with @benchmark or set unit to "s"](https://discourse.julialang.org/t/get-unit-with-benchmark-or-set-unit-to-s/101664)

<div class="topic-metadata">

**Author:** [@Sisisi](https://discourse.julialang.org/u/Sisisi)\
**Replies:** 1\
**Last updated:** [July 16, 2023, 1:13am UTC](https://discourse.julialang.org/t/get-unit-with-benchmark-or-set-unit-to-s/101664 "2023-07-16T01:13:47Z")

</div>

Hello, I know how to get the mean, median… about @benchmark, nonetheless, those values are given in the suitable unit (ms, µs, ns) but I need to know with Julia what is this unit. Or, else, set the unit to “second”. I …

---

## [Benchmarking Fallible Code](https://discourse.julialang.org/t/benchmarking-fallible-code/97845)

<div class="topic-metadata">

**Author:** [@AlexanderNenninger](https://discourse.julialang.org/u/AlexanderNenninger)\
**Replies:** 3\
**Last updated:** [June 14, 2023, 8:33am UTC](https://discourse.julialang.org/t/benchmarking-fallible-code/97845 "2023-06-14T08:33:25Z")

</div>

Is there a way to indicate failure in benchmarked code using BenchmarkTools? Say we have the following function using Random function mayfail() if rand() \< 0.1 throw(ErrorException("┻━┻︵ \\(°□°)/ ︵ ┻━┻")) …

---

## [Minimizing PythonCall overhead with BenchmarkTools](https://discourse.julialang.org/t/minimizing-pythoncall-overhead-with-benchmarktools/100067)

<div class="topic-metadata">

**Author:** [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Replies:** 4\
**Last updated:** [June 9, 2023, 7:57am UTC](https://discourse.julialang.org/t/minimizing-pythoncall-overhead-with-benchmarktools/100067 "2023-06-09T07:57:32Z")

</div>

Hi there! I’m generating benchmarks with BenchmarkTools.jl, where a Python package is called with PythonCall.jl. My question is the following: how can I minimize the overhead of calling a method meth from a Python objec…

---

## [Equivalent of BenchmarkTools in Python?](https://discourse.julialang.org/t/equivalent-of-benchmarktools-in-python/97011)

<div class="topic-metadata">

**Author:** [@sylvaticus](https://discourse.julialang.org/u/sylvaticus)\
**Replies:** 8\
**Last updated:** [April 3, 2023, 3:09pm UTC](https://discourse.julialang.org/t/equivalent-of-benchmarktools-in-python/97011 "2023-04-03T15:09:42Z")

</div>

Which is the equivalent of @btime/ @benchmark in Python? I would like something to interactively benchmark a single expression (function call) and where the number of run adapt to the running cost of the expression to b…

---

## [Save @btime output](https://discourse.julialang.org/t/save-btime-output/53595)

<div class="topic-metadata">

**Author:** [@mkarikom](https://discourse.julialang.org/u/mkarikom)\
**Replies:** 7\
**Last updated:** [April 1, 2023, 9:52am UTC](https://discourse.julialang.org/t/save-btime-output/53595 "2023-04-01T09:52:57Z")

</div>

Is there a simple way to do this? I just want to save the time n in milliseconds e.g. n = @btime foo for some benchmark like: check = nothing function checkaaa() rand(10) \* rand(10)' return nothing end check …

[Next page](https://discourse.julialang.org/tag/benchmarktools/161.md?match_all_tags=true&page=1&tags%5B%5D=benchmarktools)