# Comparing Julia Performance to CLI Program

**URL:** <https://discourse.julialang.org/t/comparing-julia-performance-to-cli-program/42280>\
**Category:** Performance\
**Created:** [June 30, 2020, 2:22am UTC](https://discourse.julialang.org/t/comparing-julia-performance-to-cli-program/42280 "2020-06-30T02:22:01Z")\
**Posts on this page:** 1\
**Showing post:** 1

<div class="post-metadata">

**Author:** ![mtsch](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mtsch/32/2072_2.png) [@mtsch](https://discourse.julialang.org/u/mtsch)\
**Post date:** [June 30, 2020, 2:22am UTC](https://discourse.julialang.org/t/comparing-julia-performance-to-cli-program/42280/1 "2020-06-30T02:22:01Z")

</div>

Hi,

I have written a Julia package that is based on an algorithm that is used by a CLI program. I’m trying to perform some benchmarks to see where I stand. I want to eventually publish the results, so I want to make sure I’m not misleading anyone.

For refrence, the CLI program reads a file and spits the results to `stdout`.

The solution I have right now is creating two `BenchmarkGroup`s with the same name, one running my code and the other benchmarking `read` on the CLI program with the appropriate switches. I use `judge` to compare the timings.

The problem I’m running into is that the `read` beanchmark is very slow when the output is large. The program has an option to turn off output, but a user of the program will need to read the results eventually, so I don’t think that’s a completely fair way to perform benchmarks either.

The other concern is the overhead of running the program from Julia. I’m guessing that’s not a big problem, since the overhead should be small and constant. The benchmarks take at least a couple of seconds to complete.

Is there a standard way of handling benchmarks like these? Is there a repo with this kind of benchmarking I can look at?

The second question I have is if there is reliable a way to benchmark the total memory use of a program or Julia function. The problem I’m solving is memory-intensive, so the total memory used is an important metric.

Thanks for the help!

---

_[View the full topic](https://discourse.julialang.org/t/comparing-julia-performance-to-cli-program/42280)._