How to compare commits with PkgBenchmark

General Usage
,
1

Hi,

I would like to compare benchmarks for two commits. I’m trying to follow the documentation, but I receive an error indicating that the input is not a repo. I tried a few different approach, but with the same result. Can someone please tell me how to setup the benchmark?

Thanks!

using PkgBenchmark
import SequentialSamplingModels

# https://github.com/itsdfish/SequentialSamplingModels.jl/commits/master
#target_id = "a45db69"
target_id = "a45db6920287bd6442be175dc9bc690eb5ce57d3"
target = benchmarkpkg(SequentialSamplingModels, target_id)

Error

ERROR: /home/dfish/.julia/packages/SequentialSamplingModels/zeGhr/ is not a git repo, cannot benchmark at 0e32c56acea48fd722cd8176cd7e3d1d21d8b93c
Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:35
 [2] benchmarkpkg(pkg::String, target::String; script::Nothing, postprocess::Nothing, resultfile::Nothing, retune::Bool, verbose::Bool, logger_factory::Nothing, progressoptions::Nothing, custom_loadpath::String)
   @ PkgBenchmark ~/.julia/packages/PkgBenchmark/nuPay/src/runbenchmark.jl:134
 [3] benchmarkpkg(pkg::String, target::String)
   @ PkgBenchmark ~/.julia/packages/PkgBenchmark/nuPay/src/runbenchmark.jl:44
 [4] top-level scope
   @ ~/.julia/dev/sandbox/benchmarks/ssm_benchmark.jl:16

julia>
2

You should dev the package in question, which will clone a full git repo into .julia/dev. You might then also need to check that the commit you are trying to benchmark is accessible in the repo.

3

Thank you. The problem was that the package was not in dev mode. I did not see that requirement in the documentation. I will submit an issue so that it is more clear to users.