# How to compare commits with PkgBenchmark

**URL:** https://discourse.julialang.org/t/how-to-compare-commits-with-pkgbenchmark/104468
**Category:** General Usage
**Tags:** question, benchmarktools
**Created:** [October 1, 2023, 2:55pm UTC](https://discourse.julialang.org/t/how-to-compare-commits-with-pkgbenchmark/104468 "2023-10-01T14:55:03Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![Christopher\_Fisher](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/christopher_fisher/32/26132_2.png) [@Christopher\_Fisher](https://discourse.julialang.org/u/Christopher_Fisher)
#### Post date: [October 1, 2023, 2:55pm UTC](https://discourse.julialang.org/t/how-to-compare-commits-with-pkgbenchmark/104468/1 "2023-10-01T14:55:03Z")

</div>

Hi,

I would like to compare benchmarks for two commits. I’m trying to follow the documentation, but I receive an error indicating that the input is not a repo. I tried a few different approach, but with the same result. Can someone please tell me how to setup the benchmark?

Thanks!

```julia
using PkgBenchmark
import SequentialSamplingModels

# https://github.com/itsdfish/SequentialSamplingModels.jl/commits/master
#target_id = "a45db69"
target_id = "a45db6920287bd6442be175dc9bc690eb5ce57d3"
target = benchmarkpkg(SequentialSamplingModels, target_id)

```

**Error**

```julia
ERROR: /home/dfish/.julia/packages/SequentialSamplingModels/zeGhr/ is not a git repo, cannot benchmark at 0e32c56acea48fd722cd8176cd7e3d1d21d8b93c
Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:35
 [2] benchmarkpkg(pkg::String, target::String; script::Nothing, postprocess::Nothing, resultfile::Nothing, retune::Bool, verbose::Bool, logger_factory::Nothing, progressoptions::Nothing, custom_loadpath::String)
   @ PkgBenchmark ~/.julia/packages/PkgBenchmark/nuPay/src/runbenchmark.jl:134
 [3] benchmarkpkg(pkg::String, target::String)
   @ PkgBenchmark ~/.julia/packages/PkgBenchmark/nuPay/src/runbenchmark.jl:44
 [4] top-level scope
   @ ~/.julia/dev/sandbox/benchmarks/ssm_benchmark.jl:16

julia> 

```

---

<div class="post-metadata">

### Author: ![adamslc](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/adamslc/32/3452_2.png) [@adamslc](https://discourse.julialang.org/u/adamslc)
#### Post date: [October 1, 2023, 4:40pm UTC](https://discourse.julialang.org/t/how-to-compare-commits-with-pkgbenchmark/104468/2 "2023-10-01T16:40:00Z")

</div>

You should `dev` the package in question, which will clone a full git repo into `.julia/dev`. You might then also need to check that the commit you are trying to benchmark is accessible in the repo.

---

<div class="post-metadata">

### Author: ![Christopher\_Fisher](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/christopher_fisher/32/26132_2.png) [@Christopher\_Fisher](https://discourse.julialang.org/u/Christopher_Fisher)
#### Post date: [October 1, 2023, 5:25pm UTC](https://discourse.julialang.org/t/how-to-compare-commits-with-pkgbenchmark/104468/3 "2023-10-01T17:25:54Z")

</div>

Thank you. The problem was that the package was not in dev mode. I did not see that requirement in the documentation. I will submit an issue so that it is more clear to users.