Equivalent of BenchmarkTools in Python?

Which is the equivalent of @btime/ @benchmark in Python?
I would like something to interactively benchmark a single expression (function call) and where the number of run adapt to the running cost of the expression to benchmark.

I can’t get timeit/pyperf for this :frowning:

1 Like

yes, but I am looking to something similar in Python…

omg sorry… I guess it’s time for a break…

4 Likes

It’s crazy how little there is about this in Python… I was convinced to find something like this in seconds, instead the best out there seems to get the current time before and after and subtract, or profile a whole script using command line tools…
Th beauty of @BenchmarkTools is that it is so simple, just @btime ...code.. and you are done, no need to create a struct for the number of times to run the code, the name of the benchmark… sometimes it’s ok, but most of the time I just need a quick benchmark, and that’s what BenchmarkTools provides, but crazy I can’t find something similar in Python…

Crosspost on SO: benchmarking - How to benchmark a single function call in Python? - Stack Overflow
(end of course, starting having downvotes there, I don’t know why, as I did search before…)

there is GitHub - plasma-umass/scalene: Scalene: a high-performance, high-precision CPU, GPU, and memory profiler for Python with AI-powered optimization proposals, but I’ve never used it myself before

2 Likes

Looking at the docs for timeit, it does seem like you can get it to behave as @benchmark does.

See autorange, for example.

2 Likes

We’ve been slowly setting up cross-language benchmarks. I agree the Python options are slightly weird, but you can see what we ended up with here: image_benchmarks/benchmarks.py at main · JuliaImages/image_benchmarks · GitHub

Surprising that even simple stuff like dividing by the number of repeats appears to be something you have to do manually.

2 Likes

If using IPython the %timeit macro is probably the closest you can get.