# Testing a package extension?

**URL:** <https://discourse.julialang.org/t/testing-a-package-extension/107846>\
**Category:** General Usage\
**Tags:** testing, package-extensions\
**Created:** [December 20, 2023, 10:50am UTC](https://discourse.julialang.org/t/testing-a-package-extension/107846 "2023-12-20T10:50:53Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![johnomotani](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/johnomotani/32/26753_2.png) [@johnomotani](https://discourse.julialang.org/u/johnomotani)\
**Post date:** [December 20, 2023, 10:50am UTC](https://discourse.julialang.org/t/testing-a-package-extension/107846/1 "2023-12-20T10:50:53Z")

</div>

General question is: what is the recommended way to run CI tests on functionality provided in package extensions?

One option would be I guess to make the tests require all the possible extension-triggering weak-dependencies. I do not want to do this, because I want users to be able to run tests without installing dependencies that they do not want (I’m happy to add some code to skip tests if the extension is not loaded - if I have trouble with that, it’s a separate issue/question…).

Suppose I have a package which has an extension to provide some extra functionality (e.g. output to NetCDF if NCDatasets is loaded). I run CI tests with a workflow like

```julia
name: Run tests

on: [push, pull_request, workflow_dispatch]

jobs:
  test:
    runs-on: ${{ matrix.os }}
    strategy:
      matrix:
        os: [ubuntu-latest, macOS-latest]
      fail-fast: false
    timeout-minutes: 50

    steps:
      - uses: actions/checkout@v4
      - uses: julia-actions/setup-julia@v1
        with:
          version: '1.8'
          arch: x64
      - uses: julia-actions/julia-buildpkg@v1
      - uses: julia-actions/julia-runtest@v1

```

What would be a good way to get the extension-triggering weak-dependencies loaded? I tried modifying the workflow to install `NCDatasets` into the project, but that seems to fail (i.e. the extension is not loaded), I guess because `NCDatasets` is already in the `[weakdeps]` section? [I’m testing this with julia-1.9.4, in case the version is relevant.]

It seems like maybe the only thing to do is to create a separate project environment, `dev` my package into that environment, add the extension-triggering weak-dependency too, and then run the tests there? I guess that is doable, but more complicated than I would like for something that seems like any package using extensions probably wants to do?

---

<div class="post-metadata">

**Author:** ![johnomotani](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/johnomotani/32/26753_2.png) [@johnomotani](https://discourse.julialang.org/u/johnomotani)\
**Post date:** [December 20, 2023, 11:18am UTC](https://discourse.julialang.org/t/testing-a-package-extension/107846/2 "2023-12-20T11:18:10Z")

</div>

My current thought for a workaround is:

- add `NCDatasets` to the test dependencies, so it will be installed when using `Pkg.test()`.
- for users who want to run the tests without extra dependencies, they can run `test/runtests.jl` directly, which (I think?) does not know about the ‘test dependencies’ from the Project.toml.

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [December 20, 2023, 1:31pm UTC](https://discourse.julialang.org/t/testing-a-package-extension/107846/3 "2023-12-20T13:31:58Z")

</div>

Another workaround would be to add the triggering dependencies in the CI script before the tests are run, but that would be rather dirty since Pkg also does env manipulations for testing

---

<div class="post-metadata">

**Author:** ![johnomotani](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/johnomotani/32/26753_2.png) [@johnomotani](https://discourse.julialang.org/u/johnomotani)\
**Post date:** [December 20, 2023, 2:01pm UTC](https://discourse.julialang.org/t/testing-a-package-extension/107846/4 "2023-12-20T14:01:15Z")

</div>

> Another workaround would be to add the triggering dependencies in the CI script before the tests are run, but that would be rather dirty since Pkg also does env manipulations for testing

I did try that, but it seemed not to work. I don’t understand why, maybe something to do with the triggering dependencies being included in the `[weakdeps]` section of Project.toml? Edit to add what didn’t work:

```julia
julia --project -e 'import Pkg; Pkg.add("NCDatasets"); Pkg.test()

```

failed because the extension was not loaded.

Edit to add:  
Sorry, unless you mean something like

```julia
julia --project=temp -e 'import Pkg; Pkg.add("NCDatasets"); using NCDatasets; include("test/runtests.jl")

```

that might be a decent option acually…

---

<div class="post-metadata">

**Author:** ![thofma](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/thofma/32/1691_2.png) [@thofma](https://discourse.julialang.org/u/thofma)\
**Post date:** [December 20, 2023, 2:24pm UTC](https://discourse.julialang.org/t/testing-a-package-extension/107846/5 "2023-12-20T14:24:30Z")

</div>

If have `push!(Base.LOAD_PATH, "@v#.#")` inside my `test/runtests.jl`, to make this work on CI (together with the `Pkg.add` step in the global environment). It is not pretty, but hey, if it works, it works.

PS: I also have a `try using NCDatasets ...` in the `test/runtests.jl`.

---

<div class="post-metadata">

**Author:** ![ericphanson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ericphanson/32/215186_2.png) [@ericphanson](https://discourse.julialang.org/u/ericphanson)\
**Post date:** [December 20, 2023, 2:40pm UTC](https://discourse.julialang.org/t/testing-a-package-extension/107846/6 "2023-12-20T14:40:22Z")

</div>

> [@johnomotani](#):
>
> I do not want to do this, because I want users to be able to run tests without installing dependencies that they do not want

I don’t really understand this point, what is wrong with users installing more dependencies if they want to test the package? To me it’s a fairly uncommon thing for users to run tests directly (as opposed to just relying on the package’s CI) and it seems fine to me in that case for them to install a bit more to run the tests manually.

---

<div class="post-metadata">

**Author:** ![johnomotani](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/johnomotani/32/26753_2.png) [@johnomotani](https://discourse.julialang.org/u/johnomotani)\
**Post date:** [December 20, 2023, 2:47pm UTC](https://discourse.julialang.org/t/testing-a-package-extension/107846/7 "2023-12-20T14:47:49Z")

</div>

> I don’t really understand this point, what is wrong with users installing more dependencies if they want to test the package? To me it’s a fairly uncommon thing for users to run tests directly (as opposed to just relying on the package’s CI) and it seems fine to me in that case for them to install a bit more to run the tests manually.

In my case (scientific simulation software) essentially all users are also developers who will modify the source code. They then should run the tests to check that existing behaviour is not broken, but might not want to load all dependencies (for example because heavy dependencies make PackageCompiler.jl really slow, but we more-or-less have to use it to be able to submit MPI-parallelised jobs to a batch queue).